Swift Core ML 3 implementation of BERT for Question answering

Built Julien Chaumond, Lysandre Debut and Thomas Wolf at Hugging Face: https://lnkd.in/ejJabYh

#machinelearning #naturallanguageprocessing #nlp

✴️ @AI_Python_EN

AI Simulates The Universe And Not Even Its Creators Know How It's So Accurate!

For the first time, scientists have used artificial intelligence to create complex, three-dimensional simulations of the Universe.

It's called the Deep Density Displacement Model, or D3M, and it's so fast and so accurate that the astrophysicists who designed it don't even know how it does what it does!

What it does is accurately simulate the way gravity shapes the Universe over billions of years. Each simulation takes just 30 milliseconds - compared to the minutes it takes other simulations.

Paper: https://lnkd.in/dMJgJRb
#deeplearning #machinelearning

✴️ @AI_Python_EN

VideoBERT: A Joint Model for Video and Language Representation Learning

Sun et al.: https://lnkd.in/ek7MYKP

#ComputerVision #PatternRecognition #ArtificialIntelligence

✴️ @AI_Python_EN

Google announced the updated YouTube-8M dataset

Updated set now includes a subset with verified 5-s segment level labels, along with the 3rd Large-Scale Video Understanding Challenge and Workshop at #ICCV19.

Link: https://lnkd.in/f_6Jb7Y

#DL #datasets

✴️ @AI_Python_EN

*Fine-Grained Zero-Shot Recognition with Metric Rescaling*
https://arxiv.org/abs/1906.11892

✴️ @AI_Python_EN

Training an AI agent to play Snake with TensorFlow 2.0.

Code by Paweł Kieliszczyk: https://lnkd.in/edTVYEC

#MachineLearning #ReinforcementLearning #TensorFlow

✴️ @AI_Python_EN

StarAi: Deep Reinforcement Learning Course provides easy to use exercises, with answers, to reinforce our learning.

Link: https://lnkd.in/fYQUVJs

StarAi Starcraft Google Colaboratory IPython Notebook
https://lnkd.in/fv_siQY
Author: Frank He

StarAi tutorial on how to setup your Starcraft machine learning model for Colaboratory
https://lnkd.in/fBEHRPN
Author: Paul Conyngham

StarAi: “Foundations” modular Reinforcement Learning Framework
https://lnkd.in/fH2pAQX
Author: William Xu

✴️ @AI_Python_EN

Generating content using AI and Machine learning

GANs are generating different types of contents and probably we all have seen many examples:
1- Videos: This is how you can do gans yourself
https://lnkd.in/gZeB8sY

2-Music: https://lnkd.in/gsxuaMb
3-Text: https://lnkd.in/gu8fWrh
4-Image: https://lnkd.in/gDTwDNU
Yet, as far as I know, no one is currently getting paid for using GANs (except for deepfake!). Please comment if you know any?

Apart from GANs, AI is generating content using other Machine learning techniques (+ heuristics) e.g. Natural Language Generation.
Presentation slides:
https://lnkd.in/g996v-r


#artificialintelligence #datascience #machinelearning

✴️ @AI_Python_EN

#imbalancedData
What is it?
Ans-> Suppose, you are having a Classification problem with 2M records. The Output variable is having 2 categories (Yes- 500, No- 1.99M or more).

This is the imbalanced data, as one category is far less than the other category in the Output variable.

Examples-> Credit Card fraud, Cancer Detection(or any other disease that is severe), and many more.

How to deal with it?
1) Undersampling
2) Oversampling

#datascience #dataanalysis #learning

✴️ @AI_Python_EN

“We don’t like deep learning, because it is a black box and we cannot explain it.” Really? How many people actually know how their car engine, refrigerator or laptop works? How many people actually know how spaghetti or fried chicken are made? These are common everyday black boxes. And we feel totally comfortable using them every day even though we have no clue how they work. So what is the difference? Why it is okay to drive a car but not okay to use deep learning model? Because EVERYONE drives a car and not EVERYONE uses deep learning. That’s the only difference.

DL can be made less black box when needed. (If we're classifying images of felines, I'm not sure how transparent they need to be except to the developers.) Charu Aggarwal of IBM has suggested that gradients can be used to interpret the size and direction of predictions resulting from a change in an input. Sensitivity analysis, as used by statisticians, can be used in similar ways, as can statistical analysis of the DL's predictions.


Negative. All those other examples you listed have readily available experts that understand how it all works. In Deep Learning, even the brightest people who created a neural network have a difficult time explaining why their creation is working/not working. Neural networks are by definition convoluted.

Also, deep learning is often impractical and has very few applications.

https://link.medium.com/lsmcXVn8WX
✴️ @AI_Python_EN

Here is an exclusive video that highlights the 5 things that any aspirant should consider before choosing a Machine Learning course. Watch the full video here:
https://lnkd.in/fVyW5Uq



#machinelearning #artificialintelligence #datascience #deeplearning #datascientist #ai

✴️ @AI_Python_EN

A simple tutorial on how to train a Transformer model from scratch using TensorFlow GPU in Docker. It's based on the official implementation from the TF GitHub... But using Docker!
https://blog.exxactcorp.com/examining-the-transformer-architecture-part-3-training-a-transformer-network-from-scratch-in-docker/

✴️ @AI_Python_EN

Deep RL agents are data hungry and often learn task-specific representations. Our model learns object-centric abstractions from raw videos. This enables highly data-efficient RL and structured exploration.
https://arxiv.org/abs/1906.11883

✴️ @AI_Python_EN

Quantile Regression Deep Reinforcement Learning

Researchers: Oliver Richter, Roger Wattenhofer
Paper: https://lnkd.in/fnwiYXi
#artificialintelligence #ai #ml #machinelearning #bigdata #deeplearning #technology #datascience

✴️ @AI_Python_EN

Anticipatory Thinking: A Metacognitive Capability
Researchers: Adam Amos-Binks, Dustin Dannenhauer
Paper: http://ow.ly/wEyC50uR9q1

#artificialintelligence #ai #ml #machinelearning #bigdata #deeplearning #technology #datascience

✴️ @AI_Python_EN

#AI/ #DataScience/ #MachineLearning/ #ML:
7 Steps for Data Preparation Using #Python

Link => https://bit.ly/PyDataPrep

#datamining #statistics #bigdata #artificialintelligence

✴️ @AI_Python_EN

Progressively growing the action space creates a great curriculum for learning agents

paper: https://arxiv.org/abs/1906.12266

code: https://github.com/TorchCraft/TorchCraftAI/tree/gas-micro

✴️ @AI_Python_EN

By your walking style, Artificial Intelligence can know if you are happy, sad, angry or neutral Great work on emotion recognition based on deep features learned via LSTM on labeled emotion datasets + psychological characterization for affective features
https://arxiv.org/pdf/1906.11884.pdf

✴️ @AI_Python_EN

How do neural networks see depth in single images?
https://arxiv.org/pdf/1905.07005.pdf

Deep Set Prediction Networks by Yan Zhang et. al.
https://arxiv.org/abs/1906.06565
This paper has received a lot of attention (deservedly) in the short time it has been published.

Encoding sets in neural networks is done using permutation-equivariant (applying a function on every item) and permutation-invariant (sum, average, max) operations. (I think sum works the best, because sum is a gradient distributor and help optimization).

"Deep Sets" paper showed that theoretically this can learn any function on sets (even maximum, and the second maximum, I know it is unintuitive, but it can. theoretically!) But there are some limitations which you can read about it here (https://www.inference.vc/deepsets-modeling-permutation-invariance/).

Similar ideas were also used and shown very effective in relation networks (https://arxiv.org/abs/1706.01427 ) and GQNs from Eslami et. al. But this paper tries to tackle "decoding" or generating sets.
To the best of my knowledge this is one of the very few papers that actually generates "real" sets. The trick is to use a set "encoder" for decoding! Assume you have a target latent vector z for your set that you want to "decode".
Start with an initial set estimate. Encode the set estimate into the latent space. Measure error to the target z. Backpropagate error to the set estimate. Use gradient descent to update your set estimate. Repeat 10 times to arrive at a set that encodes close to your target z.

This means that each "decoding" step is a mini-optimization loop. During training you have your main optimization loop as well. The initial set estimate in fixed and learnable. This paper studies 2 set generation settings. Auto-encoding setting and supervised setting.
In auto-encoding setting you use the set encoder to encode your set into your target z and use the same encoder for "decoding". The "set loss" is Soft-L1 loss, after finding the best assignment between the estimate set and the input set using the Hungarian algorithm.
In the supervised setting, the input (e.g. image) is first mapped into the latent space (e.g. using a resent) creating the target z. An encoder maps the target set into the same latent space. The encoder is used during "decoding" as well.
In the supervised setting, the loss has 2 parts: "set loss" as in auto-encoding setting, and "representation loss" which is the l2 distance between the encoder estimate of z and target z.

Some limitations of this approach: - The Hungarian assignment of your estimated set and your target set O(n^3) has to be calculated each time. (Actually 10 times for each iteration, refer to "practical tricks" in the paper) limiting the scalability of the approach.
The set estimate is real-valued. - The set estimate is not really variable size, you have to use some masking tricks to make it seem variable size. But over all I liked the paper.

The thing I like about Yan is that he has already published his code. Also he reports mean, std over multiple runs in his experiments. This is very important IMHO and is very hard to find these days!

✴️ @AI_Python_EN

Want to play with PyTorch, but prefer OCaml to Python? Here are OCaml bindings to the PyTorch C++ API. With examples for image recognition, GANs, RL, etc. By
Laurent Mazare. https://github.com/LaurentMazare/ocaml-torch