انسان در واقع اشیاء را بدون ناظر یاد میگیرد و بعد اینکه مثلا مدتی یک شی را دید و یاد گرفت، بلافاصله پس از اینکه نام آن شی را شنید برچسب آن را نیز یاد میگیرد.
در حال حاضر بهترین مدلهای بینایی ماشین که در سالهای اخیر، خصوصا بعد از الکس‌نت سال 2012 ارائه شده اند با ناظر هستند. خیلی خوب عمل می‌کنند اما به داده ی برچسب گذاری شده ی زیادی نیاز دارند.
اگر به نحوی بتوانیم از داده های بدون برچسب استفاده کنیم و مدل را آموزش دهیم، سپس در فاز کوتاهی با داده های اندک اشیائی که مدل یاد گرفته است را به صورت با ناظرآموزش دهیم تحول بزرگی در یادگیری مدل ها ایجاد خواهد شد. در این صورت میتوان به سادگی میلیون ها ساعت ویدیو را مثلا با استفاده از یوتیوب به مدل آموزش داد و پس از آموزش مدل، شروع به آموزش نام اشیاء یادگرفته شده به مدل پرداخت روندی که در انسان هم مشاهده میشود! در واقع کودک از بدو تولد اشیاء مختلف را میبیند و آن ها را یاد میگیرد اما با یک یا چند بارشنیدن نام آن به آن دسته یا شئی که قبلا فراگرفته نام اختصاص میدهد.
The Next Frontier in AI: Unsupervised Learning
#Yann_LeCun
Director of AI Research at Facebook, Professor of Computer Science, New York University
November 18, 2016
https://www.youtube.com/watch?v=IbjF5VjniVE
Abstract
The rapid progress of #AI in the last few years are largely the result of advances in #deep_learning and neural nets, combined with the availability of large datasets and fast GPUs. We now have systems that can #recognize images with an accuracy that rivals that of humans. This will lead to revolutions in several domains such as autonomous transportation and #medical #image understanding. But all of these systems currently use #supervised learning in which the machine is trained with inputs labeled by humans. The challenge of the next several years is to let machines learn from raw, #unlabeled_data, such as #video or #text. This is known as #unsupervised learning. AI systems today do not possess "common sense", which humans and animals acquire by observing the world, acting in it, and understanding the physical constraints of it. Some of us see unsupervised learning as the key towards machines with common sense. Approaches to unsupervised learning will be reviewed. This presentation assumes some familiarity with the basic concepts of deep learning.
در حال حاضر بهترین مدلهای بینایی ماشین که در سالهای اخیر، خصوصا بعد از الکس‌نت سال 2012 ارائه شده اند با ناظر هستند. خیلی خوب عمل می‌کنند اما به داده ی برچسب گذاری شده ی زیادی نیاز دارند.
اگر به نحوی بتوانیم از داده های بدون برچسب استفاده کنیم و مدل را آموزش دهیم، سپس در فاز کوتاهی با داده های اندک اشیائی که مدل یاد گرفته است را به صورت با ناظرآموزش دهیم تحول بزرگی در یادگیری مدل ها ایجاد خواهد شد. در این صورت میتوان به سادگی میلیون ها ساعت ویدیو را مثلا با استفاده از یوتیوب به مدل آموزش داد و پس از آموزش مدل، شروع به آموزش نام اشیاء یادگرفته شده به مدل پرداخت روندی که در انسان هم مشاهده میشود! در واقع کودک از بدو تولد اشیاء مختلف را میبیند و آن ها را یاد میگیرد اما با یک یا چند بارشنیدن نام آن به آن دسته یا شئی که قبلا فراگرفته نام اختصاص میدهد.
The Next Frontier in AI: Unsupervised Learning
#Yann_LeCun
Director of AI Research at Facebook, Professor of Computer Science, New York University
November 18, 2016
https://www.youtube.com/watch?v=IbjF5VjniVE
Abstract
The rapid progress of #AI in the last few years are largely the result of advances in #deep_learning and neural nets, combined with the availability of large datasets and fast GPUs. We now have systems that can #recognize images with an accuracy that rivals that of humans. This will lead to revolutions in several domains such as autonomous transportation and #medical #image understanding. But all of these systems currently use #supervised learning in which the machine is trained with inputs labeled by humans. The challenge of the next several years is to let machines learn from raw, #unlabeled_data, such as #video or #text. This is known as #unsupervised learning. AI systems today do not possess "common sense", which humans and animals acquire by observing the world, acting in it, and understanding the physical constraints of it. Some of us see unsupervised learning as the key towards machines with common sense. Approaches to unsupervised learning will be reviewed. This presentation assumes some familiarity with the basic concepts of deep learning.
YouTube
RI Seminar: Yann LeCun : The Next Frontier in AI: Unsupervised Learning
Yann LeCun
Director of AI Research at Facebook, Professor of Computer Science, New York University
November 18, 2016
Abstract
The rapid progress of AI in the last few years are largely the result of advances in deep learning and neural nets, combined with…
Director of AI Research at Facebook, Professor of Computer Science, New York University
November 18, 2016
Abstract
The rapid progress of AI in the last few years are largely the result of advances in deep learning and neural nets, combined with…
Tensorflow(@CVision)
انسان در واقع اشیاء را بدون ناظر یاد میگیرد و بعد اینکه مثلا مدتی یک شی را دید و یاد گرفت، بلافاصله پس از اینکه نام آن شی را شنید برچسب آن را نیز یاد میگیرد. در حال حاضر بهترین مدلهای بینایی ماشین که در سالهای اخیر، خصوصا بعد از الکس‌نت سال 2012 ارائه شده…
یان لیکان در این سخنرانی
شبکه های رقابتی مولد
یا
Generative Adversarial Networks
را مهم ترین ایده در 20 سال گذشته برای یادگیری ماشین بیان کرده است.
روشی که مدلها را قادر به یادگیری بدون ناظر می‌کند.
The major advancements in Deep Learning in 2016
🔗https://tryolabs.com/blog/2016/12/06/major-advancements-deep-learning-2016/
Generative Adversarial Nets
https://arxiv.org/pdf/1406.2661v1.pdf
این روش برای مسائل با تعداد کم و ناکافی داده ی با برچسب نیز مناسب است.
#autoencoder #unsupervised #unsupervised_learning #Generative #Generative_Models
شبکه های رقابتی مولد
یا
Generative Adversarial Networks
را مهم ترین ایده در 20 سال گذشته برای یادگیری ماشین بیان کرده است.
روشی که مدلها را قادر به یادگیری بدون ناظر می‌کند.
The major advancements in Deep Learning in 2016
🔗https://tryolabs.com/blog/2016/12/06/major-advancements-deep-learning-2016/
Generative Adversarial Nets
https://arxiv.org/pdf/1406.2661v1.pdf
این روش برای مسائل با تعداد کم و ناکافی داده ی با برچسب نیز مناسب است.
#autoencoder #unsupervised #unsupervised_learning #Generative #Generative_Models
Tryolabs
The major advancements in Deep Learning in 2016
✏️Title:
#Unsupervised #Representation Learning with #Deep #Convolutional #Generative #Adversarial Networks
✏️abstract:
In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications. Comparatively, unsupervised learning with CNNs has received less attention. In this work we hope to help bridge the gap between the success of CNNs for supervised learning and unsupervised learning. We introduce a class of CNNs called deep convolutional generative adversarial networks (DCGANs), that have certain architectural constraints, and demonstrate that they are a strong candidate for unsupervised learning. Training on various image datasets, we show convincing evidence that our deep convolutional adversarial pair learns a hierarchy of representations from object parts to scenes in both the generator and discriminator. Additionally, we use the learned features for novel tasks - demonstrating their applicability as general image representations.
🔗https://arxiv.org/pdf/1511.06434v2.pdf
"Under review as a conference paper at ICLR 2016"
#Unsupervised #Representation Learning with #Deep #Convolutional #Generative #Adversarial Networks
✏️abstract:
In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications. Comparatively, unsupervised learning with CNNs has received less attention. In this work we hope to help bridge the gap between the success of CNNs for supervised learning and unsupervised learning. We introduce a class of CNNs called deep convolutional generative adversarial networks (DCGANs), that have certain architectural constraints, and demonstrate that they are a strong candidate for unsupervised learning. Training on various image datasets, we show convincing evidence that our deep convolutional adversarial pair learns a hierarchy of representations from object parts to scenes in both the generator and discriminator. Additionally, we use the learned features for novel tasks - demonstrating their applicability as general image representations.
🔗https://arxiv.org/pdf/1511.06434v2.pdf
"Under review as a conference paper at ICLR 2016"
The talks at the #Deep_Learning #School on September 24/25, 2016
http://www.bayareadlschool.org
Full Day Live Streams:
Day 1: https://youtu.be/eyovmAtoUx0
Day 2: https://youtu.be/9dXiAecyJrY
دو ویدیوی بالا تفکیک نشده اند و هر کدام حدود 10 ساعت سخرانی است
در زیر هر سخرانی جدا شده است:
1. Foundations of Deep Learning (Hugo Larochelle, Twitter) - https://youtu.be/zij_FTbJHsk
2. Deep Learning for Computer Vision (Andrej Karpathy, OpenAI) - https://youtu.be/u6aEYuemt0M
3. Deep Learning for Natural Language Processing (Richard Socher, Salesforce) - https://youtu.be/oGk1v1jQITw
4. TensorFlow Tutorial (Sherry Moore, Google Brain) - https://youtu.be/Ejec3ID_h0w
5. Foundations of Unsupervised Deep Learning (Ruslan Salakhutdinov, CMU) - https://youtu.be/rK6bchqeaN8
6. Nuts and Bolts of Applying Deep Learning (Andrew Ng) - https://youtu.be/F1ka6a13S9I
7. Deep Reinforcement Learning (John Schulman, OpenAI) - https://youtu.be/PtAIh9KSnjo
8. Theano Tutorial (Pascal Lamblin, MILA) - https://youtu.be/OU8I1oJ9HhI
9. Deep Learning for Speech Recognition (Adam Coates, Baidu) - https://youtu.be/g-sndkf7mCs
10. Torch Tutorial (Alex Wiltschko, Twitter) - https://youtu.be/L1sHcj3qDNc
11. Sequence to Sequence Deep Learning (Quoc Le, Google) - https://youtu.be/G5RY_SUJih4
12. Foundations and Challenges of Deep Learning (Yoshua Bengio) - https://youtu.be/11rsu_WwZTc
http://www.bayareadlschool.org
Full Day Live Streams:
Day 1: https://youtu.be/eyovmAtoUx0
Day 2: https://youtu.be/9dXiAecyJrY
دو ویدیوی بالا تفکیک نشده اند و هر کدام حدود 10 ساعت سخرانی است
در زیر هر سخرانی جدا شده است:
1. Foundations of Deep Learning (Hugo Larochelle, Twitter) - https://youtu.be/zij_FTbJHsk
2. Deep Learning for Computer Vision (Andrej Karpathy, OpenAI) - https://youtu.be/u6aEYuemt0M
3. Deep Learning for Natural Language Processing (Richard Socher, Salesforce) - https://youtu.be/oGk1v1jQITw
4. TensorFlow Tutorial (Sherry Moore, Google Brain) - https://youtu.be/Ejec3ID_h0w
5. Foundations of Unsupervised Deep Learning (Ruslan Salakhutdinov, CMU) - https://youtu.be/rK6bchqeaN8
6. Nuts and Bolts of Applying Deep Learning (Andrew Ng) - https://youtu.be/F1ka6a13S9I
7. Deep Reinforcement Learning (John Schulman, OpenAI) - https://youtu.be/PtAIh9KSnjo
8. Theano Tutorial (Pascal Lamblin, MILA) - https://youtu.be/OU8I1oJ9HhI
9. Deep Learning for Speech Recognition (Adam Coates, Baidu) - https://youtu.be/g-sndkf7mCs
10. Torch Tutorial (Alex Wiltschko, Twitter) - https://youtu.be/L1sHcj3qDNc
11. Sequence to Sequence Deep Learning (Quoc Le, Google) - https://youtu.be/G5RY_SUJih4
12. Foundations and Challenges of Deep Learning (Yoshua Bengio) - https://youtu.be/11rsu_WwZTc
YouTube
Bay Area Deep Learning School Day 1 at CEMEX auditorium, Stanford
Day 1 of Bay Area Deep Learning School featuring speakers Hugo Larochelle, Andrej Karpathy, Richard Socher, Sherry Moore, Ruslan Salakhutdinov and Andrew Ng. Detailed schedule is at http://www.bayareadlschool.org/schedule
Tensorflow(@CVision)
Learning from Simulated and Unsupervised Images through Adversarial Training (Apple Inc.)
مقاله‌ی جالب کمپانی اپل!
( Submitted for review to a conference on Nov 15, 2016)
✏️Title:
Learning from Simulated and Unsupervised Images through Adversarial Training
✏️abstract:
With recent progress in graphics, it has become more tractable to train models on #synthetic images, potentially avoiding the need for expensive annotations. However, learning from synthetic images may not achieve the desired performance due to a gap between synthetic and real image distributions. To reduce this gap, we propose Simulated+Unsupervised (S+U) learning, where the task is to learn a model to improve the realism of a simulator's output using #unlabeled real data, while preserving the annotation information from the simulator. We develop a method for S+U learning that uses an #adversarial network similar to #Generative Adversarial Networks (#GANs), but with synthetic images as inputs instead of random vectors. We make several key modifications to the standard GAN algorithm to preserve annotations, avoid artifacts and stabilize training: (i) a 'self-regularization' term, (ii) a local adversarial loss, and (iii) updating the discriminator using a history of refined images. We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study. We quantitatively evaluate the generated images by training models for gaze estimation and hand pose estimation. We show a significant improvement over using synthetic images, and achieve state-of-the-art results on the MPIIGaze dataset without any labeled real data.
🔗https://arxiv.org/abs/1612.07828v1
🔗https://arxiv.org/pdf/1612.07828v1.pdf
#unlabeled_data #unsupervised #unsupervised_learning #Generative #Generative_Models
( Submitted for review to a conference on Nov 15, 2016)
✏️Title:
Learning from Simulated and Unsupervised Images through Adversarial Training
✏️abstract:
With recent progress in graphics, it has become more tractable to train models on #synthetic images, potentially avoiding the need for expensive annotations. However, learning from synthetic images may not achieve the desired performance due to a gap between synthetic and real image distributions. To reduce this gap, we propose Simulated+Unsupervised (S+U) learning, where the task is to learn a model to improve the realism of a simulator's output using #unlabeled real data, while preserving the annotation information from the simulator. We develop a method for S+U learning that uses an #adversarial network similar to #Generative Adversarial Networks (#GANs), but with synthetic images as inputs instead of random vectors. We make several key modifications to the standard GAN algorithm to preserve annotations, avoid artifacts and stabilize training: (i) a 'self-regularization' term, (ii) a local adversarial loss, and (iii) updating the discriminator using a history of refined images. We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study. We quantitatively evaluate the generated images by training models for gaze estimation and hand pose estimation. We show a significant improvement over using synthetic images, and achieve state-of-the-art results on the MPIIGaze dataset without any labeled real data.
🔗https://arxiv.org/abs/1612.07828v1
🔗https://arxiv.org/pdf/1612.07828v1.pdf
#unlabeled_data #unsupervised #unsupervised_learning #Generative #Generative_Models
Tensorflow(@CVision)
مقاله‌ی جالب کمپانی اپل! ( Submitted for review to a conference on Nov 15, 2016) ✏️Title: Learning from Simulated and Unsupervised Images through Adversarial Training ✏️abstract: With recent progress in graphics, it has become more tractable to train models…
Contributions:
1. We propose S+U learning that uses unlabeled real
data to refine the synthetic images generated by a
simulator.
2. We train a refiner network to add realism to synthetic
images using a combination of an adversarial
loss and a self-regularization loss.
3. We make several key modifications to the #GAN
training framework to stabilize training and prevent
the refiner network from producing artifacts.
4. We present qualitative, quantitative, and user study
experiments showing that the proposed framework
significantly improves the realism of the simulator
output. We achieve state-of-the-art results, without
any human annotation effort, by training deep neural
networks on the refined output images.
1. We propose S+U learning that uses unlabeled real
data to refine the synthetic images generated by a
simulator.
2. We train a refiner network to add realism to synthetic
images using a combination of an adversarial
loss and a self-regularization loss.
3. We make several key modifications to the #GAN
training framework to stabilize training and prevent
the refiner network from producing artifacts.
4. We present qualitative, quantitative, and user study
experiments showing that the proposed framework
significantly improves the realism of the simulator
output. We achieve state-of-the-art results, without
any human annotation effort, by training deep neural
networks on the refined output images.
Deep Learning 2016: The Year in Review
http://www.deeplearningweekly.com/blog/deep-learning-2016-the-year-in-review
✔️ #Unsupervised and #Reinforcement Learning
✔️ Deep Reinforcement Learning
✔️ #Generative Models
✔️ Continued Openness in AI development
✔️ Partnerships & Acquisitions
✔️ Hardware & Chips
(by Jan Bussieck on December 31, 2016)
In order to understand trends in the field, I find it helpful to think of developments in #deep_learning as being driven by three major frontiers that limit the success of #artificial_intelligence in general and deep learning in particular. Firstly, there is the available #computing power and #infrastructure, such as fast #GPUs, cloud services providers (have you checked out Amazon's new #EC2 P2 instance ?) and tools (#Tensorflow, #Torch, #Keras etc), secondly, there is the amount and quality of the training data and thirdly, the algorithms (#CNN, #LSTM, #SGD) using the training data and running on the hardware. Invariably behind every new development or advancement, lies an expansion of one of these frontiers.
...
http://www.deeplearningweekly.com/blog/deep-learning-2016-the-year-in-review
✔️ #Unsupervised and #Reinforcement Learning
✔️ Deep Reinforcement Learning
✔️ #Generative Models
✔️ Continued Openness in AI development
✔️ Partnerships & Acquisitions
✔️ Hardware & Chips
(by Jan Bussieck on December 31, 2016)
In order to understand trends in the field, I find it helpful to think of developments in #deep_learning as being driven by three major frontiers that limit the success of #artificial_intelligence in general and deep learning in particular. Firstly, there is the available #computing power and #infrastructure, such as fast #GPUs, cloud services providers (have you checked out Amazon's new #EC2 P2 instance ?) and tools (#Tensorflow, #Torch, #Keras etc), secondly, there is the amount and quality of the training data and thirdly, the algorithms (#CNN, #LSTM, #SGD) using the training data and running on the hardware. Invariably behind every new development or advancement, lies an expansion of one of these frontiers.
...
Deeplearningweekly
Deep Learning 2016: The Year in Review | Deep Learning Weekly
A weekly newsletter about the latest developments in Deep Learning
Generative Adversarial Denoising Autoencoder for #Face Completion
pic: http://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/images/one.png
🔗http://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/
#GAN
#Generative #adversarial #Generative_Models #Autoencoder
pic: http://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/images/one.png
🔗http://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/
#GAN
#Generative #adversarial #Generative_Models #Autoencoder
قلب انسان "مغز" دارد !
http://isna.ir/news/95101207026
تحقیقات اخیر نشان میدهد که قلب، دارای یک "مغز منحصر به خود" است به این معنی که قلب نیز دارای یک مرکز بوده که اطلاعات را حمل و دریافت می‌کند
"مغز قلب"، قلب را قادر می سازد تا عملکردهایی نظیر یادگیری، یادآوری و تصمیم گیری را جدا از قشر مخ انجام دهد
...آنچه باعث شگفتی میشود این است که قلب این اطلاعات را برای اولین بار و حتی زودتر از مغز دریافت می‌کند.
http://isna.ir/news/95101207026
تحقیقات اخیر نشان میدهد که قلب، دارای یک "مغز منحصر به خود" است به این معنی که قلب نیز دارای یک مرکز بوده که اطلاعات را حمل و دریافت می‌کند
"مغز قلب"، قلب را قادر می سازد تا عملکردهایی نظیر یادگیری، یادآوری و تصمیم گیری را جدا از قشر مخ انجام دهد
...آنچه باعث شگفتی میشود این است که قلب این اطلاعات را برای اولین بار و حتی زودتر از مغز دریافت می‌کند.
ایسنا
قلب انسان "مغز" دارد!
یک پژوهش جدید نشان میدهد که بر خلاف باورهای رایج، مغز مرکز هوشیاری کل بدن نبوده و قلب نیز نقش مهمی در روند هوشیاری ایفا می‌کند.
Tensorflow(@CVision)
Photo
Intelligent Perception_463500597.pdf
3.9 MB
#Ali_Eslami
اسلاید ارائه ی دکتر علی اسلامی با موضوع :
Intelligent Perception
(Beyond Supervised #Deep_Learning)
۸ دی 1395، دانشگاه #شریف
#اسلاید #اسلامی #سمینار #زمستانه
اسلاید ارائه ی دکتر علی اسلامی با موضوع :
Intelligent Perception
(Beyond Supervised #Deep_Learning)
۸ دی 1395، دانشگاه #شریف
#اسلاید #اسلامی #سمینار #زمستانه
Modern Artificial Intelligence via Deep Learning.pdf
7.9 MB
#Ali_Eslami
اسلاید ارائه ی دکتر علی اسلامی با موضوع :
Modern Artificial Intelligence via Deep Learning
(Beyond Supervised #Deep_Learning)
شنبه ١١ دى 1395، دانشگاه #تهران
#اسلاید #اسلامی #سمینار
اسلاید ارائه ی دکتر علی اسلامی با موضوع :
Modern Artificial Intelligence via Deep Learning
(Beyond Supervised #Deep_Learning)
شنبه ١١ دى 1395، دانشگاه #تهران
#اسلاید #اسلامی #سمینار
Tensorflow(@CVision)
مقاله‌ی جالب کمپانی اپل! ( Submitted for review to a conference on Nov 15, 2016) ✏️Title: Learning from Simulated and Unsupervised Images through Adversarial Training ✏️abstract: With recent progress in graphics, it has become more tractable to train models…
http://www.kurzweilai.net/apples-first-ai-paper-focuses-on-creating-superrealistic-image-recognition
#apple #GAN
#apple #GAN
www.kurzweilai.net
Apple’s first AI paper focuses on creating ‘superrealistic’ image recognition | KurzweilAI
Apple's first paper on artificial intelligence, published Dec. 22 on arXiv (open access), describes a method for improving the ability of a deep neural network
The Alien Style of Deep Learning #Generative Design
https://medium.com/intuitionmachine/the-alien-look-of-deep-learning-generative-design-5c5f871f7d10
[Dec 25, 2016, 3min read]
طراحی های بیگانه، عجیب و کارا توسط یادگیری ژرف!
با استفاده از generative modelها ایده های جالبی برای طراحی های صنعتی یا حتی ایده ی دکوراسیون منزل خروجی بگیرید!
همچنین توانسته یک شبکه ی lstm بهینه را بسازد (کاربرد متا مدل)
این روش به طراحان اجازه می دهد به اهداف طراحی ورودی خاص، از جمله الزامات عملکردی، نوع مواد، روش ساخت، معیارهای عملکرد و محدودیت های هزینه را به عنوان ورودی بدهند.
سیستم پس از ایجاد طرح های مختلف و جستجو در بین طرح های خلق شده، بر اساس نیازمندی های ذکر شده بهترین طرح های پیشنهادی را خروجی میدهد!
#Alien_Style #GAN
https://medium.com/intuitionmachine/the-alien-look-of-deep-learning-generative-design-5c5f871f7d10
[Dec 25, 2016, 3min read]
طراحی های بیگانه، عجیب و کارا توسط یادگیری ژرف!
با استفاده از generative modelها ایده های جالبی برای طراحی های صنعتی یا حتی ایده ی دکوراسیون منزل خروجی بگیرید!
همچنین توانسته یک شبکه ی lstm بهینه را بسازد (کاربرد متا مدل)
این روش به طراحان اجازه می دهد به اهداف طراحی ورودی خاص، از جمله الزامات عملکردی، نوع مواد، روش ساخت، معیارهای عملکرد و محدودیت های هزینه را به عنوان ورودی بدهند.
سیستم پس از ایجاد طرح های مختلف و جستجو در بین طرح های خلق شده، بر اساس نیازمندی های ذکر شده بهترین طرح های پیشنهادی را خروجی میدهد!
#Alien_Style #GAN
Medium
The Alien Style of Deep Learning Generative Design
What happens when you have Deep Learning begin to generate your designs? The commons misconception would be that a machine’s design would…
چرا استفاده از تابع فعالسازی Relu به جای sigmoid در شبکه های عمیق رایج است؟ چه مزایایی دارد؟
pic: http://jmbeaujour.com/pics/posts_pics/udacity-DL/reLu_function.png
مزایا:
- تنها 50 درصد مواقع این تابع فعال میشود و در نتیجه از لحاظ پردازشی صرفه جویی میشود.
- جلوی انفجار گرادیان یا محو شدن آن را میگیرد. (عجب ترجمه ای!)
- با یافته های بیولوژیکی توسط نوروساینتیست ها مطابقت بیشتری دارد (علوم شناختی)
#ReLU provides some important benefits that might not be obvious at first glance:
✔️during the initialization process of a Neural Network model, weights are distributed at random for each unit. ReLU will only activate approximately 50% of the time, which actually saves some processing
power.
✔️The ReLU structure also solves the 'Vanishing Gradient' and 'Exploding Gradient' problems,both of which are well-known issues with the training process.
✔️ as a marginal benefit, this type of activation achieves 'Biological Plausibility', because it's directly relatable to the common biological model of a neuron.
pic: http://jmbeaujour.com/pics/posts_pics/udacity-DL/reLu_function.png
مزایا:
- تنها 50 درصد مواقع این تابع فعال میشود و در نتیجه از لحاظ پردازشی صرفه جویی میشود.
- جلوی انفجار گرادیان یا محو شدن آن را میگیرد. (عجب ترجمه ای!)
- با یافته های بیولوژیکی توسط نوروساینتیست ها مطابقت بیشتری دارد (علوم شناختی)
#ReLU provides some important benefits that might not be obvious at first glance:
✔️during the initialization process of a Neural Network model, weights are distributed at random for each unit. ReLU will only activate approximately 50% of the time, which actually saves some processing
power.
✔️The ReLU structure also solves the 'Vanishing Gradient' and 'Exploding Gradient' problems,both of which are well-known issues with the training process.
✔️ as a marginal benefit, this type of activation achieves 'Biological Plausibility', because it's directly relatable to the common biological model of a neuron.
Tensorflow(@CVision)
چرا استفاده از تابع فعالسازی Relu به جای sigmoid در شبکه های عمیق رایج است؟ چه مزایایی دارد؟ pic: http://jmbeaujour.com/pics/posts_pics/udacity-DL/reLu_function.png مزایا: - تنها 50 درصد مواقع این تابع فعال میشود و در نتیجه از لحاظ پردازشی صرفه جویی میشود.…
#Relu
[109] Nair, V. and G. E. Hinton. 2010. “Rectified Linear Units Improve Restricted Boltzmann Machines,” Proc. 27th Int. Conf. Mach. Learn., no. 3, pp. 807–814.
[109] Nair, V. and G. E. Hinton. 2010. “Rectified Linear Units Improve Restricted Boltzmann Machines,” Proc. 27th Int. Conf. Mach. Learn., no. 3, pp. 807–814.
Tensorflow(@CVision)
چرا استفاده از تابع فعالسازی Relu به جای sigmoid در شبکه های عمیق رایج است؟ چه مزایایی دارد؟ pic: http://jmbeaujour.com/pics/posts_pics/udacity-DL/reLu_function.png مزایا: - تنها 50 درصد مواقع این تابع فعال میشود و در نتیجه از لحاظ پردازشی صرفه جویی میشود.…
توابع فعالساز دیگری نیز نظیر
Parametric ELU (#PELU)
Exponential Linear Unit (#ELU)
برای شبکه های عمیق مطرح شده اند...
pic: http://deepnn.ir/tensorflow-telegram-files/PELU.PNG
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
[Submitted to International Conference on Learning Representations (ICLR) 2017]
The #activation function is an important component in #Convolutional Neural Networks (CNNs). For instance, recent breakthroughs in Deep Learning can be attributed to the Rectified Linear Unit (ReLU). Another recently proposed activation function, the #Exponential Linear Unit (ELU), has the supplementary property of reducing bias shift without explicitly centering the values at zero. In this paper, we show that learning a parameterization of ELU improves its performance. We analyzed our proposed #Parametric ELU (PELU) in the context of #vanishing gradients and provide a #gradient-based #optimization framework. We conducted several experiments on CIFAR-10/100 and ImageNet with different network architectures, such as NiN, Overfeat, All-CNN and ResNet. Our results show that our PELU has relative error improvements over ELU of 4.45% and 5.68% on CIFAR-10 and 100, and as much as 7.28% with only 0.0003% #parameter increase on ImageNet. We also observed that Vgg using PELU tended to prefer activations saturating closer to zero, as in ReLU, except at the last layer, which saturated near -2. Finally, other presented results suggest that varying the shape of the activations during training along with the other parameters helps controlling vanishing gradients and bias shift, thus facilitating #learning.
🔗https://arxiv.org/abs/1605.09332
🔗https://arxiv.org/pdf/1605.09332v3.pdf
Poster: 🔗http://deepnn.ir/tensorflow-telegram-files/PELU-DLSS_2016.pdf
related: http://www.kdnuggets.com/2016/03/must-know-tips-deep-learning-part-2.html
Parametric ELU (#PELU)
Exponential Linear Unit (#ELU)
برای شبکه های عمیق مطرح شده اند...
pic: http://deepnn.ir/tensorflow-telegram-files/PELU.PNG
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
[Submitted to International Conference on Learning Representations (ICLR) 2017]
The #activation function is an important component in #Convolutional Neural Networks (CNNs). For instance, recent breakthroughs in Deep Learning can be attributed to the Rectified Linear Unit (ReLU). Another recently proposed activation function, the #Exponential Linear Unit (ELU), has the supplementary property of reducing bias shift without explicitly centering the values at zero. In this paper, we show that learning a parameterization of ELU improves its performance. We analyzed our proposed #Parametric ELU (PELU) in the context of #vanishing gradients and provide a #gradient-based #optimization framework. We conducted several experiments on CIFAR-10/100 and ImageNet with different network architectures, such as NiN, Overfeat, All-CNN and ResNet. Our results show that our PELU has relative error improvements over ELU of 4.45% and 5.68% on CIFAR-10 and 100, and as much as 7.28% with only 0.0003% #parameter increase on ImageNet. We also observed that Vgg using PELU tended to prefer activations saturating closer to zero, as in ReLU, except at the last layer, which saturated near -2. Finally, other presented results suggest that varying the shape of the activations during training along with the other parameters helps controlling vanishing gradients and bias shift, thus facilitating #learning.
🔗https://arxiv.org/abs/1605.09332
🔗https://arxiv.org/pdf/1605.09332v3.pdf
Poster: 🔗http://deepnn.ir/tensorflow-telegram-files/PELU-DLSS_2016.pdf
related: http://www.kdnuggets.com/2016/03/must-know-tips-deep-learning-part-2.html
دوره آموزشی آنلاین «یادگیری ژرف (#deep_learning):
🔗URL: http://course.fast.ai/
در این دوره رایگان که مدت آن ۷ هفته و هر هفته 10 ساعت زمان نیاز است، مبانی یادگیری عمیق بر اساس سرفصل‌های دانشگاه سان فرانسیسکو در قالب ویدئو تدریس میشود:
این ویدیوها قسمت اول است و قسمت دوم ویدیوها در تاریخ
May 2017
در همین سایت به صورت آنلاین قرار خواهد گرفت...
پیش نیازها:
- حداقل یکسال سابقه برنامه نویسی
- ریاضی در حد دبیرستان
0—Why deep learning; Intro to convolutions
1—#Recognizing cats and dogs
2—#Convolutional neural networks
3—#Under_fitting and #over_fitting
4—#Collaborative filtering, embeddings, and more
5—Intro to #NLP and RNNs
6—Building RNNs
7—Exotic #CNN architectures; #RNN from scratch
#course
🔗URL: http://course.fast.ai/
در این دوره رایگان که مدت آن ۷ هفته و هر هفته 10 ساعت زمان نیاز است، مبانی یادگیری عمیق بر اساس سرفصل‌های دانشگاه سان فرانسیسکو در قالب ویدئو تدریس میشود:
این ویدیوها قسمت اول است و قسمت دوم ویدیوها در تاریخ
May 2017
در همین سایت به صورت آنلاین قرار خواهد گرفت...
پیش نیازها:
- حداقل یکسال سابقه برنامه نویسی
- ریاضی در حد دبیرستان
0—Why deep learning; Intro to convolutions
1—#Recognizing cats and dogs
2—#Convolutional neural networks
3—#Under_fitting and #over_fitting
4—#Collaborative filtering, embeddings, and more
5—Intro to #NLP and RNNs
6—Building RNNs
7—Exotic #CNN architectures; #RNN from scratch
#course
Practical Deep Learning for Coders
Practical Deep Learning for Coders - Practical Deep Learning
A free course designed for people with some coding experience, who want to learn how to apply deep learning and machine learning to practical problems.
Two blogs with wonderful posts about deep learning
دو وبلاگ با پست‌های عالی و شگفت‌انگیز در مورد یادگیری عمیق
http://distill.pub
http://colah.github.io
#blog #deep_learning
دو وبلاگ با پست‌های عالی و شگفت‌انگیز در مورد یادگیری عمیق
http://distill.pub
http://colah.github.io
#blog #deep_learning