Obviously this channel was created because I get bored of sharing the same (awesome) links to all my friends.
This link contains a lot of useful lectures. Also on the topic of AutoDiff and Transformers.
https://github.com/glouppe/info8010-deep-learning#agenda
https://github.com/glouppe/info8010-deep-learning#agenda
GitHub
GitHub - glouppe/info8010-deep-learning: Lectures for INFO8010 Deep Learning, ULiège
Lectures for INFO8010 Deep Learning, ULiège. Contribute to glouppe/info8010-deep-learning development by creating an account on GitHub.
Why AI is Harder Than We Think
Reading difficulty: 🟩◼️◼️
This video by Yannic Kilcher explains the paper "Why AI is harder than we think", which suggests that something fundamental is still missing to achieve General AI that is capable of reaching the human method of reasoning. I recommend viewing.
https://www.youtube.com/watch?v=uwfVxckuq50
Reading difficulty: 🟩◼️◼️
This video by Yannic Kilcher explains the paper "Why AI is harder than we think", which suggests that something fundamental is still missing to achieve General AI that is capable of reaching the human method of reasoning. I recommend viewing.
https://www.youtube.com/watch?v=uwfVxckuq50
YouTube
Why AI is Harder Than We Think (Machine Learning Research Paper Explained)
#aiwinter #agi #embodiedcognition
The AI community has gone through regular cycles of AI Springs, where rapid progress gave rise to massive overconfidence, high funding, and overpromise, followed by these promises being unfulfilled, subsequently diving into…
The AI community has gone through regular cycles of AI Springs, where rapid progress gave rise to massive overconfidence, high funding, and overpromise, followed by these promises being unfulfilled, subsequently diving into…
For those of you who are working with Transformers, check out this webinar (26/5/2021)
https://picampus.it/events/efficient-transformers/
https://picampus.it/events/efficient-transformers/
Pi Campus Rome
Efficient Transformers: an AI Webinar with Lukasz Kaiser
Lukasz will introduce a new efficient variant of Transformers and show how they address the main problems that previously limited the use of some Transformers.
If we take attention away from transformers, what's left? An old dear and elegant masked ensemble
Do You Even Need Attention? A Stack of Feed-Forward Layers Does
Surprisingly Well on ImageNet
Reading difficulty: 🟧🟧◼️
https://arxiv.org/pdf/2105.02723.pdf
Do You Even Need Attention? A Stack of Feed-Forward Layers Does
Surprisingly Well on ImageNet
Reading difficulty: 🟧🟧◼️
https://arxiv.org/pdf/2105.02723.pdf
MLP-Mixer: An all-MLP Architecture for Vision
Reading difficulty: 🟧🟧◼️
Very similar to the previous paper there is also MLP-Mixer from Google Research: https://arxiv.org/pdf/2105.01601.pdf
Both papers show that the Vision Transformer can be reduced to just using MLPs while achieving competitive performance.
A very straightforward explanation can be found in this video: https://www.youtube.com/watch?v=7K4Z8RqjWIk
Reading difficulty: 🟧🟧◼️
Very similar to the previous paper there is also MLP-Mixer from Google Research: https://arxiv.org/pdf/2105.01601.pdf
Both papers show that the Vision Transformer can be reduced to just using MLPs while achieving competitive performance.
A very straightforward explanation can be found in this video: https://www.youtube.com/watch?v=7K4Z8RqjWIk
YouTube
MLP-Mixer: An all-MLP Architecture for Vision (Machine Learning Research Paper Explained)
#mixer #google #imagenet
Convolutional Neural Networks have dominated computer vision for nearly 10 years, and that might finally come to an end. First, Vision Transformers (ViT) have shown remarkable performance, and now even simple MLP-based models reach…
Convolutional Neural Networks have dominated computer vision for nearly 10 years, and that might finally come to an end. First, Vision Transformers (ViT) have shown remarkable performance, and now even simple MLP-based models reach…
"Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis"
Reading difficulty: 🟧🟧◼️
📚 https://arxiv.org/pdf/1806.04558.pdf 📚 https://google.github.io/tacotron/publications/speaker_adaptation
Something interesting done some years ago in the TTS (Text To Speech) field. Have you ever asked yourself - "How much time is needed, for a machine, to imitate my voice?" Near 5 seconds!
In this paper (first link) the architectural proposal uses 3 main components:
1) Speaker encoder network: it captures, in an embedding space, the characteristics of several and different speakers. It's the most important part of the entire architecture because its work is to influence the Mel spectrogram generated by the synthesizer.
2) Synthesizer network: trained on pairs of text transcript and target audio to produce a synthesized Mel spectrogram.
3) Vocoder (WaveNet): useful to invert spectrograms into time domain waveforms.
A possible application of this technology? Transferring a voice across languages for more natural speech-to-speech translation.
Reading difficulty: 🟧🟧◼️
📚 https://arxiv.org/pdf/1806.04558.pdf 📚 https://google.github.io/tacotron/publications/speaker_adaptation
Something interesting done some years ago in the TTS (Text To Speech) field. Have you ever asked yourself - "How much time is needed, for a machine, to imitate my voice?" Near 5 seconds!
In this paper (first link) the architectural proposal uses 3 main components:
1) Speaker encoder network: it captures, in an embedding space, the characteristics of several and different speakers. It's the most important part of the entire architecture because its work is to influence the Mel spectrogram generated by the synthesizer.
2) Synthesizer network: trained on pairs of text transcript and target audio to produce a synthesized Mel spectrogram.
3) Vocoder (WaveNet): useful to invert spectrograms into time domain waveforms.
A possible application of this technology? Transferring a voice across languages for more natural speech-to-speech translation.
Dataset trick: COCO (Common Objects in Context)
Reading difficulty: 🟩◼️◼️
Hi everyone, this evening we propose you some practical tricks to work and to enjoy with COCO.
What is COCO? COCO is a large Microsoft image dataset designed for many tasks like:
🔎 Object detection
🔎 Segmentation
🔎 Person keypoints detection
🔎 Stuff segmentation
🔎 Caption generation
Obviously COCO has several annotation types, one for each task to perform, stored using JSON format. It may be difficult to use them and easily switch from one to another, so Python helps us in order to do that and visualize what happens.
pycocotools is a Python API that assists in loading, parsing and visualizing the annotations in COCO. A very simple and practical tutorial shows how to use it https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocoDemo.ipynb
The tool is really interesting because due to the popularity of the dataset, the format that COCO uses to store annotations is often the go-to format when creating a new custom object detection dataset or simply find one that follow the same baseline.
We suggest you to open the tutorial and to try to interact with it. It's funny 😆.
COCO homepage -> https://cocodataset.org/#home
COCO annotation format -> https://cocodataset.org/#format-data
COCO paper -> https://arxiv.org/pdf/1405.0312.pdf
COCO download -> https://cocodataset.org/#download
pycocotools -> https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py
Reading difficulty: 🟩◼️◼️
Hi everyone, this evening we propose you some practical tricks to work and to enjoy with COCO.
What is COCO? COCO is a large Microsoft image dataset designed for many tasks like:
🔎 Object detection
🔎 Segmentation
🔎 Person keypoints detection
🔎 Stuff segmentation
🔎 Caption generation
Obviously COCO has several annotation types, one for each task to perform, stored using JSON format. It may be difficult to use them and easily switch from one to another, so Python helps us in order to do that and visualize what happens.
pycocotools is a Python API that assists in loading, parsing and visualizing the annotations in COCO. A very simple and practical tutorial shows how to use it https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocoDemo.ipynb
The tool is really interesting because due to the popularity of the dataset, the format that COCO uses to store annotations is often the go-to format when creating a new custom object detection dataset or simply find one that follow the same baseline.
We suggest you to open the tutorial and to try to interact with it. It's funny 😆.
COCO homepage -> https://cocodataset.org/#home
COCO annotation format -> https://cocodataset.org/#format-data
COCO paper -> https://arxiv.org/pdf/1405.0312.pdf
COCO download -> https://cocodataset.org/#download
pycocotools -> https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py
GitHub
cocoapi/PythonAPI/pycocoDemo.ipynb at master · cocodataset/cocoapi
COCO API - Dataset @ http://cocodataset.org/ . Contribute to cocodataset/cocoapi development by creating an account on GitHub.
AI Experts in 2021
Reading difficulty: 🟩◼️◼️
This roadmap is very useful to understand if we have all the knowledge to become AI experts.
It is interesting to check if we are forgetting something important.
Reading difficulty: 🟩◼️◼️
This roadmap is very useful to understand if we have all the knowledge to become AI experts.
It is interesting to check if we are forgetting something important.
AI Roadmap
Follow these roadmaps to become an Artificial Intelligence expert.
Classifying emotions using audio recordings
Reading difficulty: 🟧🟧◼️
Today we propose an article, precisely a tutorial, that explains how to classify emotions using Python and CNN. Hope you enjoy it!
https://towardsdatascience.com/classifying-emotions-using-audio-recordings-and-python-434e748a95eb
Reading difficulty: 🟧🟧◼️
Today we propose an article, precisely a tutorial, that explains how to classify emotions using Python and CNN. Hope you enjoy it!
https://towardsdatascience.com/classifying-emotions-using-audio-recordings-and-python-434e748a95eb
Medium
Classifying emotions using audio recordings and Python
The theory of Sound and Acoustics could be used for cutting edge Emotion-Recognition technologies. here’s how we can apply it using Python
DDPM - Diffusion Models Beat GANs on Image Synthesis (NEW!)
Reading difficulty: 🟧🟧◼️
GANs have dominated the image generation space for the main part of the last decade.
This paper shows for the first time, how a non-GAN model, a DDPM, can be improved to overtake GANs at standard evaluation metrics for image generation. The produced samples look amazing and other than GANs, the new model has a formal probabilistic foundation. Is there a future for GANs or are Diffusion Models going to overtake them for good?
If you are looking for someone to help you read this paper I recommend Yannic's video, as always.
We suggest you first to read about Diffision Models (also explained in the Yannic's video).
Then the actual paper 📚: https://arxiv.org/pdf/2105.05233
And obviously keep an eye on the code to understand the details 💻: https://github.com/openai/guided-diffusion
Reading difficulty: 🟧🟧◼️
GANs have dominated the image generation space for the main part of the last decade.
This paper shows for the first time, how a non-GAN model, a DDPM, can be improved to overtake GANs at standard evaluation metrics for image generation. The produced samples look amazing and other than GANs, the new model has a formal probabilistic foundation. Is there a future for GANs or are Diffusion Models going to overtake them for good?
If you are looking for someone to help you read this paper I recommend Yannic's video, as always.
We suggest you first to read about Diffision Models (also explained in the Yannic's video).
Then the actual paper 📚: https://arxiv.org/pdf/2105.05233
And obviously keep an eye on the code to understand the details 💻: https://github.com/openai/guided-diffusion
Siamese Neural Networks: an overview
Reading difficulty: 🟧🟧◼️
A general consideration of what Siamese networks are and how useful they might be in dynamic contexts that require what is known as One Shot Learning
https://www.mygreatlearning.com/blog/siamese-networks/
Here you can see a practical example applied to the well-known Omniglot Dataset for the use of Siamese Neural Networks, using Keras as a framework.
https://towardsdatascience.com/one-shot-learning-with-siamese-networks-using-keras-17f34e75bb3d
Reading difficulty: 🟧🟧◼️
A general consideration of what Siamese networks are and how useful they might be in dynamic contexts that require what is known as One Shot Learning
https://www.mygreatlearning.com/blog/siamese-networks/
Here you can see a practical example applied to the well-known Omniglot Dataset for the use of Siamese Neural Networks, using Keras as a framework.
https://towardsdatascience.com/one-shot-learning-with-siamese-networks-using-keras-17f34e75bb3d
Great Learning Blog: Free Resources what Matters to shape your Career!
Similarity Learning with Siamese Networks
Siamese network is an artificial neural network that is used to find out how similar two objects are when comapring them with each other.
gMLP: Pay Attention to MLPs
Reading difficulty: 🟧🟧◼️
🔍 We've been discussing Transformers a lot on this channel, and the latest publications are pointing out more and more that, what's important in Transformers, are more MLPs and not Attention modules.
Here is another study from Google Research that shows that the attention module can be removed from the Transformers architecture.
The proposed architecture is called gMLP, based only on MLPs with gating (that's what the "g" stands for), and it shows that it can perform just like Transformers in language and vision applications.
⚡️ With 66% fewer parameters, a gMLP model is 3% more accurate than MLP-Mixer.
If you'd like to first understand how transformers work here's a video for you: https://www.youtube.com/watch?v=OyFJWRnt_AY
📚 Paper: https://arxiv.org/pdf/2105.08050.pdf
Reading difficulty: 🟧🟧◼️
🔍 We've been discussing Transformers a lot on this channel, and the latest publications are pointing out more and more that, what's important in Transformers, are more MLPs and not Attention modules.
Here is another study from Google Research that shows that the attention module can be removed from the Transformers architecture.
The proposed architecture is called gMLP, based only on MLPs with gating (that's what the "g" stands for), and it shows that it can perform just like Transformers in language and vision applications.
⚡️ With 66% fewer parameters, a gMLP model is 3% more accurate than MLP-Mixer.
If you'd like to first understand how transformers work here's a video for you: https://www.youtube.com/watch?v=OyFJWRnt_AY
📚 Paper: https://arxiv.org/pdf/2105.08050.pdf
CodeNet: IBM releases 14M sample coding dataset for "AI for code"
Reading difficulty: 🟩◼️◼️
IBM Research released Project CodeNet, a dataset of 14 million code samples to train machine learning models for programming tasks.
⚡️ Key highlights:
- Largest coding dataset gathered yet (4,000 problems, 14 million code samples, 50+ languages)
- The dataset has been annotated (problem description, memory/time limit, language, success, errors, etc.)
Possible uses:
- Translation from one programming language to another
- Code recommendation/completion
- Code optimization
💻 GitHub: https://github.com/IBM/Project_CodeNet
🔍 IBM blog: https://research.ibm.com/blog/codenet-ai-for-code
@deeplearning_material
Reading difficulty: 🟩◼️◼️
IBM Research released Project CodeNet, a dataset of 14 million code samples to train machine learning models for programming tasks.
⚡️ Key highlights:
- Largest coding dataset gathered yet (4,000 problems, 14 million code samples, 50+ languages)
- The dataset has been annotated (problem description, memory/time limit, language, success, errors, etc.)
Possible uses:
- Translation from one programming language to another
- Code recommendation/completion
- Code optimization
💻 GitHub: https://github.com/IBM/Project_CodeNet
🔍 IBM blog: https://research.ibm.com/blog/codenet-ai-for-code
@deeplearning_material
AAAI 2020 Keynotes
Reading difficulty: 🟩◼️◼️
The AAAI Conference (Association for the Advancement of Artificial Intelligence) is an important forum to talk about Artificial Intelligence and Deep Learning.
In February 2020 some big bosses of these fields met to discuss about problems, challenges and opportunities of Deep Learning for the next years.
💣 Yan LeCun presented three challenges of Deep Learning:
1. Learning with fewer labeled samples
2. Learning to reason
3. Learning to plan complex action sequences
📚 Geoff Hinton deals with Convolutional Neural Networks:
1. CNNs not good at dealing with rotation or scaling
2. CNNs do not understand images in terms of objects and their parts
3. CNNs are brittle against adversarial examples
🔎 Yoshua Bengio talks about:
1. Should generalize faster from fewer examples
2. Learn better models from the world, like common sense
3. Get better at "System 2" thinking (slower, methodological thinking as opposed to fast recognition)
Remarkable is the participation of Daniel Kahneman; that I suggest you to follow. It's a very interesting talk. I give you the link of the presentation, enjoy it 🤩.
https://aaai.org/Conferences/AAAI-20/livestreamed-talks/
Reading difficulty: 🟩◼️◼️
The AAAI Conference (Association for the Advancement of Artificial Intelligence) is an important forum to talk about Artificial Intelligence and Deep Learning.
In February 2020 some big bosses of these fields met to discuss about problems, challenges and opportunities of Deep Learning for the next years.
💣 Yan LeCun presented three challenges of Deep Learning:
1. Learning with fewer labeled samples
2. Learning to reason
3. Learning to plan complex action sequences
📚 Geoff Hinton deals with Convolutional Neural Networks:
1. CNNs not good at dealing with rotation or scaling
2. CNNs do not understand images in terms of objects and their parts
3. CNNs are brittle against adversarial examples
🔎 Yoshua Bengio talks about:
1. Should generalize faster from fewer examples
2. Learn better models from the world, like common sense
3. Get better at "System 2" thinking (slower, methodological thinking as opposed to fast recognition)
Remarkable is the participation of Daniel Kahneman; that I suggest you to follow. It's a very interesting talk. I give you the link of the presentation, enjoy it 🤩.
https://aaai.org/Conferences/AAAI-20/livestreamed-talks/
What type of content would you like most?
Final Results
50%
Last papers and publications
68%
Blog and tutorial links
45%
News (AI and Deep Learning)
SETR: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Reading difficulty: 🟧🟧◼️
By reformulating the semantic segmentation problem from a sequence-to-sequence learning perspective, SETR proposes to replace the stacked convolution layers-based encoder with a pure transformer to encode an image as a sequence of patches. This transformer-based encoder can be combined with a simple decoder to provide a powerful segmentation model.
📚 Paper (to appear at CVPR 2021): https://arxiv.org/pdf/2012.15840.pdf
🔍 Web Site: https://fudan-zvg.github.io/SETR
💻 GitHub: https://github.com/fudan-zvg/SETR
Reading difficulty: 🟧🟧◼️
By reformulating the semantic segmentation problem from a sequence-to-sequence learning perspective, SETR proposes to replace the stacked convolution layers-based encoder with a pure transformer to encode an image as a sequence of patches. This transformer-based encoder can be combined with a simple decoder to provide a powerful segmentation model.
📚 Paper (to appear at CVPR 2021): https://arxiv.org/pdf/2012.15840.pdf
🔍 Web Site: https://fudan-zvg.github.io/SETR
💻 GitHub: https://github.com/fudan-zvg/SETR
Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysis
Reading difficulty 🟧🟧◼️
The world of XAI is quickly making some steps forward and getting some good results, especially when it comes to visualize the decision of a CNN for classification!
The first good idea to "explain" a CNN for classification is to find a good and interpretable method (named Attribution Method) to give each input pixel its relevance w.r.t the final output (a sort of saliency map of the CNN).
The most common choices to build Attribution Methods are now concerning three approaches:
- Gradient based approach
- Perturbation based approach
- CAM based approach (often combined with one of the previous)
But each of them is giving very different results in terms of conceptual meaning and correctness of the method. Some questions are definitely still open points and need to find an answer.
My collaborators and me have tried to find a more stable and unbiased benchmark to evaluate firstly the CAM based methods and then the others, with the aim of finding a good abstraction of what should be the Ideal Attribution Method, and which properties it should have.
The importance of XAI is getting more and more relevance as the AI breaks through everyone's life!
If you'd like to know more details this paper will appear at CVPR 2021 (RCV workshop). I'll be presenting it personally and I'll be more than glad to talk about it!
📚Paper (to appear at CVPR 2021, RCV workshop): https://arxiv.org/abs/2104.10252
Reading difficulty 🟧🟧◼️
The world of XAI is quickly making some steps forward and getting some good results, especially when it comes to visualize the decision of a CNN for classification!
The first good idea to "explain" a CNN for classification is to find a good and interpretable method (named Attribution Method) to give each input pixel its relevance w.r.t the final output (a sort of saliency map of the CNN).
The most common choices to build Attribution Methods are now concerning three approaches:
- Gradient based approach
- Perturbation based approach
- CAM based approach (often combined with one of the previous)
But each of them is giving very different results in terms of conceptual meaning and correctness of the method. Some questions are definitely still open points and need to find an answer.
My collaborators and me have tried to find a more stable and unbiased benchmark to evaluate firstly the CAM based methods and then the others, with the aim of finding a good abstraction of what should be the Ideal Attribution Method, and which properties it should have.
The importance of XAI is getting more and more relevance as the AI breaks through everyone's life!
If you'd like to know more details this paper will appear at CVPR 2021 (RCV workshop). I'll be presenting it personally and I'll be more than glad to talk about it!
📚Paper (to appear at CVPR 2021, RCV workshop): https://arxiv.org/abs/2104.10252
Is it difficult for you to find the right scientific papers? Not anymore
Reading difficulty: 🟩◼️◼️
Connected Papers is a fantastic tool published one year ago that aims at helping researchers visually find and explore academic papers.
📖 Input: a paper that you like
🔗 Output: a full interactive map of similar papers to explore
So, just open the following link and use it! ⚡️
Tool link: https://www.connectedpapers.com/
Reading difficulty: 🟩◼️◼️
Connected Papers is a fantastic tool published one year ago that aims at helping researchers visually find and explore academic papers.
📖 Input: a paper that you like
🔗 Output: a full interactive map of similar papers to explore
So, just open the following link and use it! ⚡️
Tool link: https://www.connectedpapers.com/
Connectedpapers
Connected Papers | Find and explore academic papers
A unique, visual tool to help researchers and applied scientists find and explore papers relevant to their field of work.