​​ByT5: Towards a token-free future with pre-trained byte-to-byte models

Pre-trained language models usually operate on the sequences of tokens, which are based on words or subword units.

Token-free models operate directly on the raw text (characters or bytes) instead. They can work with any language, are more robust to the noise, and don’t require preprocessing.

The authors use a modified mT5 architecture and show that their approach is competitive with token-level models.

Paper: https://arxiv.org/abs/2105.13626
Code: https://github.com/google-research/byt5

A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-byt5

#nlp #deeplearning #transformer #pretraining
@Machine_learn

Reproducible Bioinformatics with Python
How to Write, Document, and Test Programs for
Biology
Ken Youens-Clark
#book #2021 #python #RL
@Machine_learn

Text analytics with sas
#SAS #book #Text_mining
@Machine_learn

Unsupervised Learning in Space and Time
A Modern Approach for Computer Vision using Graph-based Techniques and Deep Neural Networks
#book #2020 #ML #DL

@Machine_learn

🧹 Overview of data cleaning methods

https://habr.com/ru/company/ru_mts/blog/571606/

@Machine_learn

🔍 Contrastive Sensor Fusion

Github: https://github.com/descarteslabs/contrastive_sensor_fusion

Paper: https://arxiv.org/abs/2108.05094v1

@Machine_learn

📹 Internal Video Inpainting by Implicit Long-range Propagation

Github: https://github.com/Tengfei-Wang/Implicit-Internal-Video-Inpainting

Paper: https://arxiv.org/abs/2108.01912v1

4k Data: https://github.com/Tengfei-Wang/Annotated-4K-Videos

Dataset: https://paperswithcode.com/dataset/videoremoval4k

@Machine_learn

Quantum Machine Learning with Python - 2021
#Quantum #book #2021
@Machine_learn

Deep Learning and Physics
#book #2021 #DL #Physics
@Machine_learn

PyTorch Pocket Reference (2021)
#book #2021 #PyTorch
@Machine_learn

Thai Natural Language
Processing
Word Segmentation, Semantic Analysis,
and Application #NLP #Book #2021
@Machine_learn

Formalising Natural
Languages: Applications
to Natural Language
Processing and Digital
Humanities #NLP #Book #2021

@Machine_learn

Hands-on Question
Answering Systems
with BERT
Applications in Neural
Networks and Natural
Language Processing #NLP #BERT #book #2021

@Machine_learn

Synthetic Data for Deep
Learning #DL #book #2021

@Machine_learn

Image Processing
and Capsule Networks
ICIPCN 2020 #CapsuleNet #DL #image #book #2020

@Machine_learn

Beginning Robotics
with Raspberry Pi
and Arduino
Using Python and OpenCV
Second Edition #OpenCv #book #2021

@Machine_learn

Smart Computing
Techniques and Applications
Proceedings of the Fourth International
Conference on Smart Computing
and Informatics, Volume 1 #book #2021

@Machine_learn

🔍 TOOD: Task-aligned One-stage Object Detection

Github: https://github.com/fcjian/TOOD

Paper: https://arxiv.org/abs/2108.07755v2

Dataset: https://paperswithcode.com/dataset/coco

@Machine_learn

🔗 A Unified Objective for Novel Class Discovery

Github: https://github.com/DonkeyShot21/UNO

Paper: https://arxiv.org/abs/2108.08536v2

Dataset: https://paperswithcode.com/dataset/cifar-100

@Machine_learn