R-Transformer: Recurrent Neural Network Enhanced Transformer "empirical results show that R-Transformer outperforms the state-of-the-art methods by a large margin in most of the tasks"
https://arxiv.org/abs/1907.05572
https://github.com/DSE-MSU/R-transformer
✴️ @AI_Python_EN
https://arxiv.org/abs/1907.05572
https://github.com/DSE-MSU/R-transformer
✴️ @AI_Python_EN
July 15, 2019
Machine learners commonly used in predictive analytics all have at least some roots in statistics.
Some, such as regression, PCA and K-means, have been considered statistical methods since they were first developed.
Newer methods such as random forests and XGBoost will often outperform methods such as linear and binary logistic regression.
However, this depends on how the regression methods are used - statisticians often criticize data scientists for using methods like these incorrectly or for not using newer variations better suited to a particular data set.
In general, it takes more work to get statistical methods right, but this effort may pay off in the long term. Statistical methods can also be automated once the general parameters are known.
When it comes to helping us understand our data and the process(es) which generated them, as well as for what if simulations, statistical methods are better suited.
I think there is also confusion about tasks. Data are not data, and there are some tasks for which newer approaches such as Deep Learning (DL) are better suited. NLP and speech recognition are two examples.
However, because methods such as DL are better suited to "exotic" tasks than statistics, this does not mean they are better for any task (e.g., survey analytics).
✴️ @AI_Python_EN
Some, such as regression, PCA and K-means, have been considered statistical methods since they were first developed.
Newer methods such as random forests and XGBoost will often outperform methods such as linear and binary logistic regression.
However, this depends on how the regression methods are used - statisticians often criticize data scientists for using methods like these incorrectly or for not using newer variations better suited to a particular data set.
In general, it takes more work to get statistical methods right, but this effort may pay off in the long term. Statistical methods can also be automated once the general parameters are known.
When it comes to helping us understand our data and the process(es) which generated them, as well as for what if simulations, statistical methods are better suited.
I think there is also confusion about tasks. Data are not data, and there are some tasks for which newer approaches such as Deep Learning (DL) are better suited. NLP and speech recognition are two examples.
However, because methods such as DL are better suited to "exotic" tasks than statistics, this does not mean they are better for any task (e.g., survey analytics).
✴️ @AI_Python_EN
July 16, 2019
Data scientists typically use principal components analysis (PCA) as a data reduction tool. PCA and a related method, factor analysis (FA), can do much more than this, however.
Either method can also help us understand how the variables in our data interrelate. Rotation of components and factors is especially helpful in this regard.
Unfortunately, many data scientists never use rotation and many marketing researchers habitually use varimax orthogonal rotation.
Here are some other rotation methods, many with oblique variations which allow the components/factors to be correlated:
- GEOMIN
- QUARTIMIN
- QUARTIMAX
- EQUAMAX
- PARSIMAX
- PROMAX
- OBLIMIN
- OBLIMAX
- PROCRUSTEAN
- BI-FACTOR
- BENTLER
- CRAWFORD-FERGUSON
The eigenvalue >= 1 guideline for selecting the number of components or factors is not always best, either.
There are also numerous factor extraction methods that are seldom used even when they would be more appropriate.
Moreover, we are not bound to assume our data are continuous, and I often use variations of factor analysis designed for ordinal or nominal data.
There is also no need to assume one factor solution fits all - factor mixture modeling has rescued me on more than one occasion.
✴️ @AI_Python_EN
Either method can also help us understand how the variables in our data interrelate. Rotation of components and factors is especially helpful in this regard.
Unfortunately, many data scientists never use rotation and many marketing researchers habitually use varimax orthogonal rotation.
Here are some other rotation methods, many with oblique variations which allow the components/factors to be correlated:
- GEOMIN
- QUARTIMIN
- QUARTIMAX
- EQUAMAX
- PARSIMAX
- PROMAX
- OBLIMIN
- OBLIMAX
- PROCRUSTEAN
- BI-FACTOR
- BENTLER
- CRAWFORD-FERGUSON
The eigenvalue >= 1 guideline for selecting the number of components or factors is not always best, either.
There are also numerous factor extraction methods that are seldom used even when they would be more appropriate.
Moreover, we are not bound to assume our data are continuous, and I often use variations of factor analysis designed for ordinal or nominal data.
There is also no need to assume one factor solution fits all - factor mixture modeling has rescued me on more than one occasion.
✴️ @AI_Python_EN
July 16, 2019
"Multi-Label Learning", MLL (https://lnkd.in/ePhb3Fy) is a classification problem where many labels can be assigned to each instance. The more general multiple outputs/targets learning subsumes MLL for categorical targets. Wei Tong, et al, proposed SafeML (safe multi-label) model in [1] for prediction of weakly labeled data, that is when relevant labels of examples are partially known or missing which means the MLL method does not hurt performance when using weakly labeled data. The #Matlab code is in [2].
Other related posts on:
a) #MultiLabelLearning posts are here: https://lnkd.in/g_FhHDq
b) #MultiTargetLearning are here: https://lnkd.in/gxxns3a
Abstract:
Here, the MLL with weakly labeled data, i.e, labels of training examples are incomplete, which commonly occurs in real applications, e.g, image classification, document categorization is studied. This setting includes, e.g, (i) semi-supervised multi-label learning where completely labeled examples are partially known; (ii) weak label learning where relevant labels of examples are partially known; (iii) extended weak label learning where relevant & irrelevant labels of examples are partially known.
[1]"Learning safe multilabel prediction for weakly labeled data"-pdf
https://lnkd.in/g73gWJU
[2]Code
https://lnkd.in/gk4uvcH
✴️ @AI_Python_EN
Other related posts on:
a) #MultiLabelLearning posts are here: https://lnkd.in/g_FhHDq
b) #MultiTargetLearning are here: https://lnkd.in/gxxns3a
Abstract:
Here, the MLL with weakly labeled data, i.e, labels of training examples are incomplete, which commonly occurs in real applications, e.g, image classification, document categorization is studied. This setting includes, e.g, (i) semi-supervised multi-label learning where completely labeled examples are partially known; (ii) weak label learning where relevant labels of examples are partially known; (iii) extended weak label learning where relevant & irrelevant labels of examples are partially known.
[1]"Learning safe multilabel prediction for weakly labeled data"-pdf
https://lnkd.in/g73gWJU
[2]Code
https://lnkd.in/gk4uvcH
✴️ @AI_Python_EN
July 16, 2019
Build Graph Nets in Tensorflow https://arxiv.org/abs/1806.01261
Graph Nets is DeepMind's library for building graph networks in Tensorflow and Sonnet.
Contact graph-nets@google.com for comments and questions.
What are graph networks?
A graph network takes a graph as input and returns a graph as output. The input graph has edge- (E ), node- (V ), and global-level (u) attributes. The output graph has the same structure, but updated attributes. Graph networks are part of the broader family of "graph neural networks" (Scarselli et al., 2009).
https://lnkd.in/dpG9e7g
✴️ @AI_Python_EN
Graph Nets is DeepMind's library for building graph networks in Tensorflow and Sonnet.
Contact graph-nets@google.com for comments and questions.
What are graph networks?
A graph network takes a graph as input and returns a graph as output. The input graph has edge- (E ), node- (V ), and global-level (u) attributes. The output graph has the same structure, but updated attributes. Graph networks are part of the broader family of "graph neural networks" (Scarselli et al., 2009).
https://lnkd.in/dpG9e7g
✴️ @AI_Python_EN
July 16, 2019
Relating sentence representations in deep neural networks with those encoded by the brain
In their investigation, the researchers considered several neural network architectures for word representation, including two recently proposed models called ELMo and BERT.
They compared how these networks process particular sentences with data collected from human subjects using magnetoencephalography (MEG), a functional neuroimaging technique for mapping brain activity, as they read the same sentences.
To begin with, they decided to use sentences with a simple syntax and basic semantics, such as "the bone was eaten by the dog."
Read more below: or watch this Github link:
https://lnkd.in/dvbiJAz
#deeplearning
✴️ @AI_Python_EN
In their investigation, the researchers considered several neural network architectures for word representation, including two recently proposed models called ELMo and BERT.
They compared how these networks process particular sentences with data collected from human subjects using magnetoencephalography (MEG), a functional neuroimaging technique for mapping brain activity, as they read the same sentences.
To begin with, they decided to use sentences with a simple syntax and basic semantics, such as "the bone was eaten by the dog."
Read more below: or watch this Github link:
https://lnkd.in/dvbiJAz
#deeplearning
✴️ @AI_Python_EN
July 16, 2019
This media is not supported in your browser
VIEW IN TELEGRAM
Classifying Legendary Pokemon Birds 🐦🐦🐦
👉👉👉 Try it yourself:
https://lnkd.in/eYhKNAh 👈👈👈
After only the second fastai "Practical Deep Learning for Coders" class I was able to complete an end-to-end deep learning project! 🤖🤖🤖
The main goal is to classify an image as either one of the Legendary Pokemon Birds - Articuno, Moltres or Zapdos - or an alternative class which includes everything else. Needless to say that sometimes my model gets confused about the alternative class since not so many diverse images were feed into it...
Source code:
https://lnkd.in/eRfkBx8
Forked from:
https://lnkd.in/e_k4nqN
#ai #ml #dl #deeplearning #cnn #python
✴️ @AI_Python_EN
👉👉👉 Try it yourself:
https://lnkd.in/eYhKNAh 👈👈👈
After only the second fastai "Practical Deep Learning for Coders" class I was able to complete an end-to-end deep learning project! 🤖🤖🤖
The main goal is to classify an image as either one of the Legendary Pokemon Birds - Articuno, Moltres or Zapdos - or an alternative class which includes everything else. Needless to say that sometimes my model gets confused about the alternative class since not so many diverse images were feed into it...
Source code:
https://lnkd.in/eRfkBx8
Forked from:
https://lnkd.in/e_k4nqN
#ai #ml #dl #deeplearning #cnn #python
✴️ @AI_Python_EN
July 16, 2019
July 16, 2019
One of the best resources for #PyTorch based #pretrained CNN models.
https://lnkd.in/eY87mFf
✴️ @AI_Python_EN
https://lnkd.in/eY87mFf
✴️ @AI_Python_EN
July 16, 2019
July 16, 2019
Have you heard of "R-Transformer?", a Recurrent Neural Network Enhanced Transformer
Recurrent Neural Networks have long been the dominating choice for sequence modeling. However, it severely suffers from two issues: impotent in capturing very long-term dependencies and unable to parallelize the sequential computation procedure.
Therefore, many non-recurrent sequence models that are built on convolution and attention operations have been proposed recently.
Here the authors propose the R-Transformer which enjoys the advantages of both RNNs and the multi-head attention mechanism while avoids their respective drawbacks.
The proposed model can effectively capture both local structures and global long-term dependencies in sequences without any use of position embeddings. They evaluated R-Transformer through extensive experiments with data from a wide range of domains and the empirical results show that R-Transformer outperforms the state-of-the-art methods by a large margin in most of the tasks.
Github code: https://lnkd.in/dpFckix
#research #algorithms #machinelearning #deeplearning #rnn
✴️ @AI_Python_EN
Recurrent Neural Networks have long been the dominating choice for sequence modeling. However, it severely suffers from two issues: impotent in capturing very long-term dependencies and unable to parallelize the sequential computation procedure.
Therefore, many non-recurrent sequence models that are built on convolution and attention operations have been proposed recently.
Here the authors propose the R-Transformer which enjoys the advantages of both RNNs and the multi-head attention mechanism while avoids their respective drawbacks.
The proposed model can effectively capture both local structures and global long-term dependencies in sequences without any use of position embeddings. They evaluated R-Transformer through extensive experiments with data from a wide range of domains and the empirical results show that R-Transformer outperforms the state-of-the-art methods by a large margin in most of the tasks.
Github code: https://lnkd.in/dpFckix
#research #algorithms #machinelearning #deeplearning #rnn
✴️ @AI_Python_EN
July 16, 2019
July 17, 2019
On the “steerability” of generative adversarial networks
pdf: https://arxiv.org/pdf/1907.07171.pdf
abs: https://arxiv.org/abs/1907.07171
github: https://github.com/ali-design/gan_steerability
✴️ @AI_Python_EN
pdf: https://arxiv.org/pdf/1907.07171.pdf
abs: https://arxiv.org/abs/1907.07171
github: https://github.com/ali-design/gan_steerability
✴️ @AI_Python_EN
July 17, 2019
This media is not supported in your browser
VIEW IN TELEGRAM
Sinkhorn iteration for Optimal Transport with TF. Code:
https://colab.research.google.com/github/znah/notebooks/blob/master/mini_sinkhorn.ipynb
✴️ @AI_Python_EN
https://colab.research.google.com/github/znah/notebooks/blob/master/mini_sinkhorn.ipynb
✴️ @AI_Python_EN
July 17, 2019
how weak supervision can train deep learning models using unlabeled cardiac MRI sequences/videos
Key idea: use Snorkel to transform cardiologist domain knowledge into labeling functions --simple rules which capture information about our task-- to take advantage of the massive scale of unlabeled imaging data available in biobank.
This lets us rapidly build training sets for classification tasks such as bicuspid aortic valve, where a Snorkel-based approach improved our end model F1 performance by 64%.
This lets us rapidly build training sets for classification tasks such as bicuspid aortic valve, where a Snorkel-based approach improved our end model F1 performance by 64%.
https://www.nature.com/articles/s41467-019-11012-3
✴️ @AI_Python_EN
Key idea: use Snorkel to transform cardiologist domain knowledge into labeling functions --simple rules which capture information about our task-- to take advantage of the massive scale of unlabeled imaging data available in biobank.
This lets us rapidly build training sets for classification tasks such as bicuspid aortic valve, where a Snorkel-based approach improved our end model F1 performance by 64%.
This lets us rapidly build training sets for classification tasks such as bicuspid aortic valve, where a Snorkel-based approach improved our end model F1 performance by 64%.
https://www.nature.com/articles/s41467-019-11012-3
✴️ @AI_Python_EN
July 17, 2019
How to ship ML in practice:
1/ Write a simple rule based solution to cover 80% of use cases
2/ Write a simple ML algorithm to cover 95% of cases
3/ Write a filtering algorithm to route inputs to the correct method
4/ Add monitoring
5/ Detect drift
...
24/ Deep Learning
✴️ @AI_Python_EN
1/ Write a simple rule based solution to cover 80% of use cases
2/ Write a simple ML algorithm to cover 95% of cases
3/ Write a filtering algorithm to route inputs to the correct method
4/ Add monitoring
5/ Detect drift
...
24/ Deep Learning
✴️ @AI_Python_EN
July 17, 2019
'Artificial intelligence' fit to monitor volcanoes: Platform uses 'machine learning' to analyse satellite data
https://www.sciencedaily.com/releases/2019/07/190715103313.htm
✴️ @AI_Python_EN
https://www.sciencedaily.com/releases/2019/07/190715103313.htm
✴️ @AI_Python_EN
July 17, 2019
EasyGen, a visual programming language for text data pipelines for neural nets.
Colab: https://drive.google.com/open?id=1XNiOuNtMnItl5CPGvRjEvj9C78nDuvXj
Github: https://github.com/markriedl/easygen …
Here’s a program to scrape the web for generating Star Trek/romance book titles.
✴️ @AI_Python_EN
Colab: https://drive.google.com/open?id=1XNiOuNtMnItl5CPGvRjEvj9C78nDuvXj
Github: https://github.com/markriedl/easygen …
Here’s a program to scrape the web for generating Star Trek/romance book titles.
✴️ @AI_Python_EN
July 17, 2019