Music Transformer


arxiv.org/abs/1809.04281

#DeepLearning #Transformer #MachineLearning #SpeechProcessing #Music

MidiMe: Personalizing MusicVAE
Dinculescu et al.: https://magenta.tensorflow.org/midi-me
#ArtificialIntelligence #DeepLearning #Transformer

MegatronLM: Training Billion+ Parameter Language Models Using GPU Model Parallelism
"... training an 8.3 billion parameter transformer language model with 8-way model parallelism and 64-way data parallelism on 512 GPUs, making it the largest transformer based language model ever trained at 24x the size of BERT and 5.6x the size of GPT-2."
Blog by NVIDIA Applied Deep Learning Research : https://nv-adlr.github.io/MegatronLM
Code: https://github.com/nvidia/megatron-lm
#ArtificialIntelligence #DeepLearning #NLP #PyTorch #Transformer

Write With Transformer
See how a modern neural network auto-completes your text 🤗
With the brand new GPT-2 large!
Built by the phenomenal Hugging Face team : https://transformer.huggingface.co
H / T : Lysandre Debut
#GPT2 #NeuralNetwork #Transformer

Write With Transformer
Hugging Face released a new version of their Write With Transformer app, using a language model trained directly on Arxiv to generate Deep Learning and NLP completions!
In addition, they add state-of-the-art NLP models such as GPT, GPT-2 and XLNet completions:

https://transformer.huggingface.co/

H / T : Lysandre Debut
#Transformer #Pytorch #NLP

@ArtificialIntelligenceArticles

On Extractive and Abstractive Neural Document Summarization with Transformer Language Models"
Sandeep Subramanian, Raymond Li, Jonathan Pilault, Christopher Pal : https://arxiv.org/abs/1909.03186
#transformer #naturallanguageprocessing #machinelearning

Forecaster: A Graph Transformer for Forecasting Spatial and Time-Dependent Data
Yang Li, José M. F. Moura : https://arxiv.org/abs/1909.04019v3
#MachineLearning #ArtificialIntelligence #Transformer

Compressive Transformers for Long-Range Sequence Modelling
Anonymous : https://openreview.net/forum?id=SylKikSYDH
#ArtificialIntelligence #MachineLearning #Transformer

The Illustrated GPT-2 (Visualizing Transformer Language Models)
Blog by Jay Alammar : https://jalammar.github.io/illustrated-gpt2/
#BERT #Transformer #ArtificialIntelligence

Q8BERT: Quantized 8Bit BERT
Zafrir et al.: https://arxiv.org/abs/1910.06188
#NaturalLanguageProcessing #NLP #Transformer

CE7454 : Deep Learning for Data Science
Lecture 13: Attention Neural Networks
Xavier Bresson : https://dropbox.com/s/kbrsvhwe2lac1uo/lecture13_attention_neural_networks.pdf?dl=0
Demo :
https://github.com/xbresson/CE7454_2019/blob/master/codes/labs_lecture13/seq2seq_transformers_demo.ipynb
#DeepLearning #DataScience #Transformer

DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
Zhang et al.: https://arxiv.org/abs/1911.00536
#ArtificialIntelligence #MachineLearning #Transformer

Reformer: The Efficient Transformer
Kitaev et al.: https://openreview.net/forum?id=rkgNKkHtvB
#Artificialintelligence #DeepLearning #Transformer

Multi-Graph Transformer for Free-Hand Sketch Recognition
Xu et al.: https://arxiv.org/abs/1912.11258
#ArtificialIntelligence #DeepLearning #Transformer

Text classification with Transformer
Apoorv Nandan, Colab : https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/nlp/ipynb/text_classification_with_transformer.ipynb
#ArtificialIntelligence #DeepLearning #Transformer

Transformer Reinforcement Learning
Leandro von Werra, GitHub : https://github.com/lvwerra/trl
#ProximalPolicyOptimization #Transformer #ReinforcementLearning

Efficient Transformers: A Survey
Tay et al.: https://arxiv.org/abs/2009.06732
#Transformer #DeepLearning #ArtificialIntelligence