Microsoft released new version of High-Performance, Open-Source, Deep Learning Toolkit

https://news.developer.nvidia.com/microsoft-releases-new-version-of-high-performance-open-source-deep-learning-toolkit/

#microsoft #nvidia #news

Microsoft’s AI course is now open and free

Microsoft Professional Program for Artificial Intelligence — previously being available only to employees is now available at edx.org for free. Program includes 10 courses from basic python programming to deep learning and reinforcement learning disciplines.

https://academy.microsoft.com/en-us/professional-program/tracks/artificial-intelligence/

#mooc #microsoft #course

Microsoft Research announced Open Data project! This single, cloud-hosted location offers datasets representing many years of data curation and research efforts by Microsoft.

https://www.microsoft.com/en-us/research/blog/announcing-microsoft-research-open-data-datasets-by-microsoft-research-now-available-in-the-cloud/

#data #microsoft #opensource

Introducing MASS – A pre-training method that outperforms BERT and GPT in sequence to sequence language generation tasks

Researchers from Microsoft Research Asia have introduced MASS—a new pre-training method that claimed to achieve better results than BERT and GPT.

Link: https://www.microsoft.com/en-us/research/blog/introducing-mass-a-pre-training-method-that-outperforms-bert-and-gpt-in-sequence-to-sequence-language-generation-tasks/

#nlp #microsoft

Microsoft open-sourced scripts and notebooks to pre-train and finetune BERT natural language model with domain-specific texts

Github: https://github.com/microsoft/AzureML-BERT
Earlier: https://t.me/opendatascience/837

#Bert #Microsoft #NLP #dl

​​Deep Learning Image Segmentation for Ecommerce Catalogue Visual Search

Microsoft’s article on image segmentation

Link: https://www.microsoft.com/developerblog/2018/04/18/deep-learning-image-segmentation-for-ecommerce-catalogue-visual-search/

#CV #DL #Segmentation #Microsoft

Open-source library provides explanation for machine learning through diverse counterfactuals

This is a development of #interpretable ML. Library to explore “what-if” scenarios for ML models.

Blog post: https://www.microsoft.com/en-us/research/blog/open-source-library-provides-explanation-for-machine-learning-through-diverse-counterfactuals/
Paper: https://www.microsoft.com/en-us/research/publication/explaining-machine-learning-classifiers-through-diverse-counterfactual-examples/
Github: https://github.com/microsoft/dice

#Microsoft #ML #opensource

Microsoft Research 2019 reflection—a year of progress on technology’s toughest challenges

Highlights:

* MT-DNN — a model for learning universal language embeddings that combines the multi-task learning and the language model pre-training of BERT.
* Guidelines for human-AI interaction design
* AirSim, coming from strong MS background with flight simulations, for AI realisting testing environment.
* Sand Dance, a data visualization tool included in Visual Studio Code
* Icecaps — a toolkit for conversation modeling

Link: https://www.microsoft.com/en-us/research/blog/microsoft-research-2019-reflection-a-year-of-progress-on-technologys-toughest-challenges/

#microsoft #yearinreview

​​ZeRO, DeepSpeed & Turing-NLG
ZeRO: Memory Optimization Towards Training A Trillion Parameter Models
Turing-NLG: A 17-billion-parameter language model by Microsoft

Microsoft is releasing an open-source library called DeepSpeed, which vastly advances large model training by improving scale, speed, cost, and usability, unlocking the ability to train 100-billion-parameter models; compatible with PyTorch.

ZeRO – is a new parallelized optimizer that greatly reduces the resources needed for model and data parallelism while massively increasing the number of parameters that can be trained.

ZeRO has three main optimization stages, which correspond to the partitioning of optimizer states, gradients, and parameters. When enabled cumulatively:
0. Optimizer State Partitioning (P_os_) – 4x memory reduction, same communication volume as data parallelism
1. Add Gradient Partitioning (P_os+g_) – 8x memory reduction, same communication volume as data parallelism
2. Add Parameter Partitioning (P_os+g+p_) – memory reduction is linear with data parallelism degree N_d_

They have used these breakthroughs to create Turing Natural Language Generation (Turing-NLG), the largest publicly known language model at 17 billion parameters, which you can learn more about in this accompanying blog post. Also, the abstract for Turing-NLG had been written by their own model

ZeRO & DeepSpeed: https://www.microsoft.com/en-us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/
paper: https://arxiv.org/abs/1910.02054
github: https://github.com/microsoft/DeepSpeed

Turing-NLG: https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/


#nlp #dl #ml #microsoft #deepspeed #optimization

​​Racial Disparities in Automated Speech Recognition

To no surprise, speech recognition tools have #bias due to the lack of diversity in the datasets. Group of explorers addressed that issue and provided their’s research results as a paper and #reproducible research repo.

Project link: https://fairspeech.stanford.edu
Paper: https://www.pnas.org/cgi/doi/10.1073/pnas.1915768117
Github: https://github.com/stanford-policylab/asr-disparities

#speechrecognition #voice #audiolearning #dl #microsoft #google #apple #ibm #amazon

New Coding Assistant Tool From OpenAI and Microsoft

Github announced new tool for improving coding experience: Github's copilot, developed with Microsoft and OpenAI's help. This looks really promosing, at least from the announce perspective: imaging just typing convert_datetime_to_date and getting function for that. Looking forward to the actual demo.

Project: https://copilot.github.com
Blog entry: https://github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/
CNBC news post: https://www.cnbc.com/2021/06/29/microsoft-github-copilot-ai-offers-coding-suggestions.html

#OpenAI #microsoft #coding #CS #computerlanguageunderstanding #CLU #Github

🦜 Hi!

We are the first Telegram Data Science channel.

Channel was started as a collection of notable papers, news and releases shared for the members of Open Data Science (ODS) community. Through the years of just keeping the thing going we grew to an independent online Media supporting principles of Free and Open access to the information related to Data Science.


Ultimate Posts

* Where to start learning more about Data Science. https://github.com/open-data-science/ultimate_posts/tree/master/where_to_start
* @opendatascience channel audience research. https://github.com/open-data-science/ods_channel_stats_eda


Open Data Science

ODS.ai is an international community of people anyhow related to Data Science.

Website: https://ods.ai



Hashtags

Through the years we accumulated a big collection of materials, most of them accompanied by hashtags.

#deeplearning #DL — post about deep neural networks (> 1 layer)
#cv — posts related to Computer Vision. Pictures and videos
#nlp #nlu — Natural Language Processing and Natural Language Understanding. Texts and sequences
#audiolearning #speechrecognition — related to audio information processing
#ar — augmeneted reality related content
#rl — Reinforcement Learning (agents, bots and neural networks capable of playing games)
#gan #generation #generatinveart #neuralart — about neural artt and image generation
#transformer #vqgan #vae #bert #clip #StyleGAN2 #Unet #resnet #keras #Pytorch #GPT3 #GPT2 — related to special architectures or frameworks
#coding #CS — content related to software engineering sphere
#OpenAI #microsoft #Github #DeepMind #Yandex #Google #Facebook #huggingface — hashtags related to certain companies
#productionml #sota #recommendation #embeddings #selfdriving #dataset #opensource #analytics #statistics #attention #machine #translation #visualization


Chats

- Data Science Chat https://t.me/datascience_chat
- ODS Slack through invite form at website

ODS resources

* Main website: https://ods.ai
* ODS Community Telegram Channel (in Russian): @ods_ru
* ML trainings Telegram Channel: @mltrainings
* ODS Community Twitter: https://twitter.com/ods_ai

Feedback and Contacts

You are welcome to reach administration through telegram bot: @opendatasciencebot

Sparks of Artificial General Intelligence: Early experiments with GPT-4

TLDR: Paper from #Microsoft research about #GPT4 showing something which can be considered signs of #AGI.


ArXiV: https://arxiv.org/abs/2303.12712