will-thompson-k/tldr-transformers
The "tl;dr" on a few notable transformer papers.
#nlp #deep_learning #notes #transformers #attention #transfer_learning #language_models #language_model #bert #open_ai #huggingface #huggingface_transformer #gpt_3
Stars: 90 Issues: 2 Forks: 2
https://github.com/will-thompson-k/tldr-transformers

extreme-bert/extreme-bert
ExtremeBERT is a toolkit that accelerates the pretraining of customized language models on customized datasets, described in the paper “ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT”.
Language: Python
#bert #deep_learning #language_model #language_models #machine_learning #natural_language_processing #nlp #python #pytorch #transformer
Stars: 135 Issues: 0 Forks: 5
https://github.com/extreme-bert/extreme-bert

facebookresearch/large_concept_model
Large Concept Models: Language modeling in a sentence representation space
Language: Python
#language_models #nlp #pytorch #seq2seq #sequence_to_sequence
Stars: 340 Issues: 0 Forks: 27
https://github.com/facebookresearch/large_concept_model