Google open sources #Gpipe under #Lingvo library for Sequence Modeling.

What is Lingvo?

Lingvo is the international language Esperanto word for “language”. This naming alludes to the roots of the Lingvo framework — it was developed as a general deep learning framework using TensorFlow with a focus on sequence models for language-related tasks such as machine translation, speech recognition, and speech synthesis.

What is Gpipe?

GPipe is a distributed machine learning library that uses synchronous stochastic gradient descent and pipeline parallelism for training, applicable to any DNN that consists of multiple sequential layers. Importantly, GPipe allows researchers to easily deploy more accelerators to train larger models and to scale the performance without tuning hyperparameters.
Link: https://lnkd.in/ePsTCxw

GitHub: https://lnkd.in/eRwgEZz
Article: https://lnkd.in/e2y4fV2

#deeplearning #machinelearning #tensorflow

✴️ @AI_Python_EN