"Linear Algebra"

Instructor : Prof. Gilbert Strang

MIT OpenCourseWare : https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/

#LinearAlgebra #MatrixTheory

Yan lecun:
About to give a keynote at AAAI after Geoff Hinton and just before Yoshua Bengio.

My slides here: https://drive.google.com/open?id=1r-mDL4IX_hzZLDBKp8_e8VZqD7fOzBkF

Forget the old versions of capsule networks, they were all wrong. Here's the correct 2019 version, in which the presence and positions of object parts is detected with set transformers.

EmotioNet Challenge
http://cbcsl.ece.ohio-state.edu/EmotionNetChallenge/index.html

LeCun: 3 challenges for deep learning.

How AI and neuroscience drive each other forwards
https://www.nature.com/articles/d41586-019-02212-4

t-SNE algorithm on MNIST dataset in Kaggle kernels. NVIDIA's Rapids library with GPU acceleration. The algorithm achieves a 2000x speedup as compared to the sklearn version on CPU!
https://www.kaggle.com/tunguz/mnist-2d-t-sne-with-rapids

Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs. http://arxiv.org/abs/2002.02117

Unbalanced GANs: Pre-training the Generator of Generative Adversarial Network using Varia... http://arxiv.org/abs/2002.02112

SADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications
Hamdi et al.: http://arxiv.org/abs/1812.02132
Code http://github.com/ajhamdi/SADA
Video http://youtu.be/clguL24kVG0
#Cryptography #MachineLearning #Robotics

Nobel prize winner Danny Kahneman cited Gary Marcus (see the recent #AIdebate) and the need for hybrid models in science/AI (including reasoning and logic, in addition to learning), when referring to his systems 1 and 2 (see Thinking Fast and Slow). Neural-symbolic computing is a foundation for this line of research. #AAAI2020 congratulations to Francesca Rossi for the panel with Kahneman and the Turing Award winners. https://link.springer.com/book/10.1007/978-3-540-73246-4

PyTorch Wrapper version 1.1 is out!

New Features:

- Samplers for smart batching based on text length for faster training.

- Loss and Evaluation wrappers for token prediction tasks.

- New nn.modules for attention based models.

- Support for multi GPU training / evaluation / prediction.

- Verbose argument in system's methods.

- Examples using Transformer based models like BERT for text classification.

Check it out in the following links:

install with: pip install pytorch-wrapper

GitHub: https://github.com/jkoutsikakis/pytorch-wrapper

docs: https://pytorch-wrapper.readthedocs.io/en/latest/

examples: https://github.com/jkouts…/pytorch-wrapper/…/master/examples

#DeepLearning #PyTorch #NeuralNetworks #MachineLearning #DataScience #python #TensorFlow

Deep Learning: State of the Art (2020). Slides + video

Slides:
https://drive.google.com/file/d/10rZ4ldHUBoDu8bVoqGn-CEqGgX-w-oKm/view?usp=drivesdk

Video :
https://youtu.be/0VH1Lim8gL8

Image Fine-grained Inpainting
Hui et al.: https://arxiv.org/abs/2002.02609
Github: https://github.com/Zheng222/DMFN
#ArtificialIntelligence #DeepLearning #MachineLearning

Connections: Log Likelihood, Cross-Entropy, KL Divergence, Logistic Regression, and Neural Networks
https://towardsdatascience.com/connections-log-likelihood-cross-entropy-kl-divergence-logistic-regression-and-neural-networks-40043dfb6200

"If you really believe in an idea, never give up" - Geoff Hinton

#AAAI2020

An Introduction to Reinforcement Learning - Lex Fridman, MIT
Blog by Luke Kenworthy, RE•WORK : https://blog.re-work.co/an-introduction-to-reinforcement-learning-lex-fridman-mit/
#ReinforcementLearning #reworkAI #reworkDL

Continuous Geodesic Convolutions for Learning on 3D Shapes. http://arxiv.org/abs/2002.02506

The Future of Deep Learning Is Unsupervised, AI Pioneers Say
https://www.wsj.com/articles/the-future-of-deep-learning-is-unsupervised-ai-pioneers-say-11581330600

How to train a model with 10^11 parameters without running out of GPU memory?
Use DeepSpeed from Microsoft Research!
It's PyTorch compatible.
It partitions the network onto multiple processors automatically and efficiently.

https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/