Transformer Module Optimization

Article on how to apply different methods to make your transformer network up to 10x smaller and faster:

- Plain model optimization and PyTorch tricks;
- How and why to use FFT instead of self-attention;
- Model Factorization and quantization;

https://habr.com/ru/post/563778/

#deep_learning

DocNLI

Natural Language Inference (NLI) is the task of determining whether a “hypothesis” is true (entailment), false (contradiction), or undetermined (neutral) given a “premise”.

Previously, this task was solved for sentence-level texts. A new work "DOCNLI: A Large-scale Dataset for Document-level Natural Language Inference" to be appeared in ACL 2021 presenting the study for document/paragraph level NLI:
https://arxiv.org/abs/2106.09449v1

In Github repo you can find data and pretrained weights of RoBERTa:
https://github.com/salesforce/DocNLI
For release in HuggingFace we, probably, should wait...

P.S. I am already waiting to test this setup for fake news detection🙃

Article on how to use #XGBoost for #timeseries forcasting

Link: https://machinelearningmastery.com/xgboost-for-time-series-forecasting/

RL + NLP + Minecraft = Awesomeness

The video from Data Fest Online 2021 about IGLU Competition which was accepted at competition track of NeurIPS 2021

Link: https://youtu.be/mbDY8uxk9bs

New Coding Assistant Tool From OpenAI and Microsoft

Github announced new tool for improving coding experience: Github's copilot, developed with Microsoft and OpenAI's help. This looks really promosing, at least from the announce perspective: imaging just typing convert_datetime_to_date and getting function for that. Looking forward to the actual demo.

Project: https://copilot.github.com
Blog entry: https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/
CNBC news post: https://www.cnbc.com/2021/06/29/microsoft-github-copilot-ai-offers-coding-suggestions.html

#OpenAI #microsoft #coding #CS #computerlanguageunderstanding #CLU #Github

GitHub's Copilot #demo

@opendatascience

MMPX Style-Preserving Pixel Art Magnification

Work on #pixel graphics resolution upscale. Hopefully we will get all the classic games auto-remastered someday.

Publication: http://www.jcgt.org/published/0010/02/04/
Article: http://www.jcgt.org/published/0010/02/04/paper.pdf

#CV #superresolution #upscale

Habitat 2.0: Training home assistant robots with faster simulation and new benchmarks

Facebook released a new simulation platform to train robots in. Yeah, virtual robots in virtual environment, which can be a real space replica. This work brings us closer to domestic use of assistive robots.

Project website: https://ai.facebook.com/blog/habitat-20-training-home-assistant-robots-with-faster-simulation-and-new-benchmarks
Paper: https://ai.facebook.com/research/publications/habitat-2.0-training-home-assistants-to-rearrange-their-habitat

#Facebook #DigitalTwin #VR #RL #assistiverobots

Cloud-Native MLOps Framework

In this video, Artem Koval, Big Data and Machine Learning Practice Lead at Clear Scale, will analyse the requirements for modern MLOps and the main trends: Human-Centered AI, Fairness, Explainability, Model Monitoring, Human Augmented AI.

Link: https://youtu.be/K8s6dD7TPH4

FEDOT - AutoML framework for composite pipelines

FEDOT is an open-source framework for automated modeling and machine learning (AutoML). It can build custom modeling pipelines for different real-world processes in an automated way using an evolutionary approach. FEDOT supports classification (binary and multiclass), regression, clustering, and time series prediction tasks, as well as different data types and multi-modal cases. Also, sensitivity analysis of the pipelines, custom pipelines design as the initial assumption of optimization, domain-specific objective functions, and other interesting features are implemented.

Github: https://github.com/nccr-itmo/FEDOT

Preprint: https://arxiv.org/abs/2106.15397

Intro: https://www.youtube.com/watch?v=RjbuV6i6de4