Simple, Scalable Adaptation for Neural Machine Translation

Fine-tuning pre-trained Neural Machine Translation (NMT) models is the dominant approach for adapting to new languages and domains. However, fine-tuning requires adapting and maintaining a separate model for each target task. Researchers from Google propose a simple yet efficient approach for adaptation in #NMT. Their proposed approach consists of injecting tiny task specific adapter layers into a pre-trained model. These lightweight adapters, with just a small fraction of the original model size, adapt the model to multiple individual tasks simultaneously.

Guess it can be applied not only in #NMT but in many other #NLP, #NLU and #NLG tasks.

Paper: https://arxiv.org/pdf/1909.08478.pdf

#BERT

❇️ @AI_Python_EN

Communication-based Evaluation for Natural Language Generation (#NLG) that's dramatically out-performed standard n-gram-based methods.

Have you ever think that n-gram overlap measures like #BLEU or #ROUGE is not good enough for #NLG evaluation and human based evaluation is too expensive? Researchers from Stanford University also think so. The main shortcoming of #BLEU or #ROUGE methods is that they fail to take into account the communicative function of language; a speaker's goal is not only to produce well-formed expressions, but also to convey relevant information to a listener.

Researchers propose approach based on color reference game. In this game, a speaker and a listener see a set of three colors. The speaker is told one color is the target and tries to communicate the target to the listener using a natural language utterance. A good utterance is more likely to lead the listener to select the target, while a bad utterance is less likely to do so. In turn, effective metrics should assign high scores to good utterances and low scores to bad ones.

Paper: https://arxiv.org/pdf/1909.07290.pdf
Code: https://github.com/bnewm0609/comm-eval

#NLP #NLU

❇️ @AI_Python_EN

Omid Sarfarzadeh and Maysam Asgari-Chenaghlu , we will have a session on #DeepNLP and it’s applications to #SearchEngine and #Chatbot in #Google’s #DevFest, Istanbul. We will be honored to represent adesso Turkey. Thanks to Tufan K. and all adesso Turkey family to provide this chance for us. More information is provided as follows:
#DeepLearning #DeepNLP #NLP #chatbot #SearchEngine #adesso #adessoTurkey

https://devfest.istanbul
https://dfist19.firebaseapp.com/

❇ @AI_Python_EN

Evaluating the Factual Consistency of Abstractive Text Summarization
https://lnkd.in/ewFMX8T

#ArtificialIntelligence #DeepLearning #NLP #NaturalLanguageProcessing

✴ @AI_Python_EN

News classification using classic Machine Learning tools (TF-IDF) and modern NLP approach based on transfer learning (ULMFIT) deployed on GCP
Github:
https://github.com/imadelh/NLP-news-classification

Blog:
https://imadelhanafi.com/posts/text_classification_ulmfit/

#DeepLearning #MachineLearning #NLP

❇️ @AI_Python_EN

Productionizing #NLP Models

https://bit.ly/2OkdRAD

❇️ @AI_Python_EN

Another nice visual guide by Jay Alammar about how you can use BERT to do text classification. In particular, he’s using DistilBERT to create sentence embeddings which is then used as an input for logistic regression. Code is also provided! Check it out! #deeplearning #machinelearning #NLP
📝 Article:
https://jalammar.github.io/a-visual-guide-to-using-bert-for-the-first-time/

❇️ @AI_Python_EN

FacebookAI: Is the lottery ticket phenomenon a general property of DNNs or merely an artifact of supervised image classification? We show that the lottery ticket phenomenon is a general property which is present in both
#reinforcementlearning #NLP

https://arxiv.org/abs/1906.02768

❇️ @AI_Python_EN

#GraphNeuralNetwork s for Natural Language Processing

#neuralnetwork #NLP

https://bit.ly/33oprRc

❇️ @AI_Python_EN

As it turns out, Wang Ling was way ahead of the curve re NLP's muppet craze (see slides from LxMLS '16 & Oxford #NLP course '17 below).


https://github.com/oxford-cs-deepnlp-2017/lectures

❇️ @AI_Python_EN

Transformers v2.2 is out, with *4* new models and seq2seq capabilities!

ALBERT is released alongside CamemBERT, implemented by the authors, DistilRoBERTa (twice as fast as RoBERTa-base!) and GPT-2 XL!

Encoder-decoder with
⭐Model2Model⭐

Available on

https://github.com/huggingface/transformers/releases/tag/v2.2.0

#NLP

❇️ @AI_Python_EN

📢📢📢 Twitter Cortex is creating a NLP Research team. Brand new #NLP Researcher💫 job posting👇 Please spread the word.
https://careers.twitter.com/en/work-for-twitter/201911/machine-learning-researcher-nlp-cortex-applied-machine-learning.html

❇️ @AI_Python_EN

Single Headed Attention RNN: Stop Thinking With Your Head

https://arxiv.org/abs/1911.11423

#ArtificialIntelligence #NeuralComputing #NLP


❇️ @AI_Python_EN

Lit BERT: NLP Transfer Learning In 3 Steps Blog by William Falcon :
https://towardsdatascience.com/lit-bert-nlp-transfer-learning-in-3-steps-272a866570db


#MachineLearning #ArtificialIntelligence #NLP

❇️ @AI_Python_EN

ever wondered how we translate questions and commands into programs a machine can run? Jonathan Berant gives us an overview of (executable) semantic parsing.
#NLP

https://t.co/Mzvks7f9GR

❇️ @AI_Python_EN

Very interesting use of #AI to tackle bias in the written text by substituting words automatically to more neutral wording. However, one must also consider the challenges and ramifications such technology could mean to the written language as it can not only accidentally change the meaning of what was written, it can also change the tone and expression of the author and neutralize the point-of-view and remove emotion from language.
#NLP
https://arxiv.org/pdf/1911.09709.pdf

❇️ @AI_Python_EN

Named Entity Recognition Benchmark: spaCy, Flair, m-BERT and camemBERT on anonymizing French commercial legal cases

http://bit.ly/2rq1I5H
#DataScience #MachineLearning #ArtificialIntelligence #NLP

❇️ @AI_Python_EN

🔥 As you know ML has proven its importance in many fields, like computer vision, NLP, reinforcement learning, adversarial learning, etc .. Unfortunately, there is a little work to make machine learning accessible for Arabic-speaking people. Arabic language has many complicated features compared to other languages. First, Arabic language is written right to left. Second, it contains many letters that cannot be pronounced by most foreigners like ض ، غ ، ح ، خ، ظ. Moreover, Arabic language contains special characters called Diacritics which are special characters that help readers pronounced words correctly. For instance the statement السَّلامُ عَلَيْكُمْ وَرَحْمَةُ اللَّهِ وَبَرَكَاتُهُ containts special characters after most of the letters. The diactrics follow special rules to be given to a certain character. These rules are construct a complete area called النَّحْوُ الْعَرَبِيُّ. Compared to English, the Arabic language words letters are mostly connected اللغة as making them disconnected is difficult to read ا ل ل غ ة. ArbML helps fixing this by implementing many open-source projects that support Arabic, ML and NLP.

https://github.com/zaidalyafeai/ARBML

#machinelearning #deeplearning #artificialintelligence #nlp

❇️ @AI_Python_EN

Google Research • Representation Learning for Information Extraction from Templatic Documents such as receipts, bills, insurance quotes. We propose a novel approach using representation learning for tackling the problem of extracting structured information from form-like document images.

Blogpost

https://ai.googleblog.com/2020/06/extracting-structured-data-from.html?m=1

Paper

https://research.google/pubs/pub49122/
We propose an extraction system that uses knowledge of the types of the target fields to generate extraction candidates, and a neural network architecture that learns a dense representation of each candidate based on neighboring words in the document. These learned representations are not only useful in solving the extraction task for unseen document templates from two different domains, but are also interpretable, as we show using loss cases. #machinelearning #deeplearning #datascience #dataengineer #nlp

Google Research • Representation Learning for Information Extraction from Templatic Documents such as receipts, bills, insurance quotes. We propose a novel approach using representation learning for tackling the problem of extracting structured information from form-like document images.

Blogpost

https://ai.googleblog.com/2020/06/extracting-structured-data-from.html?m=1

Paper

https://research.google/pubs/pub49122/
We propose an extraction system that uses knowledge of the types of the target fields to generate extraction candidates, and a neural network architecture that learns a dense representation of each candidate based on neighboring words in the document. These learned representations are not only useful in solving the extraction task for unseen document templates from two different domains, but are also interpretable, as we show using loss cases. #machinelearning #deeplearning #datascience #dataengineer #nlp