🌟 Step-by-step implementation of the Transformer architecture

This laptop describes in as much detail as possible each step of implementing a transformer from scratch, with the necessary theoretical minimum
For complete enlightenment, you can combine it with video 3b1b

▶️ Jupyter Notebook

✅️ http://t.me/codeprogrammer ✅️

🖥 Data structures in Python - cheat sheet

Keep a powerful cheat sheet on data structures in Python; Everything is explained here with examples, so it will be crystal clear
Concepts such as mutability, immutability are described, things like list comprehensions and much more are described.

📎 Crib

✅️ http://t.me/codeprogrammer ✅️

⚡️ Big ML cheat sheet

Here you will find the basic theory of Machine Learning and examples of the implementation of specific ML algorithms - in general, this is just the thing to brush up on your knowledge before the interview.

📎 Crib

✅️ http://t.me/codeprogrammer ✅️

⚡️ Neural Networks: Zero to Hero - a block of 8 lectures and practical exercises from Andrey Karpati

This is a course on neural networks from the very basics, perhaps the best on the entire Internet.
The course is a series of YouTube videos in which Karpathy shows how to design and train neural networks.
All these results are written in Jupyter-Notebooks, you can download them and experiment

▶️ Neural Networks: Zero to Hero

✅️ http://t.me/codeprogrammer ✅️

🌟 Megatron-Core - PyTorch library for training Transformers

docker run --ipc=host --shm-size=512m --gpus all -it nvcr.io/nvidia/pytorch:24.02-py3

pip install megatron_core
pip install tensorstore==0.1.45
pip install zarr

Megatron-Core is a self-contained, lightweight PyTorch library that contains everything you need to train Transformers.
Offers a large collection of GPU techniques for optimizing memory and calculations, uses a lot of developments from Megatron-LM and Transformer Engine.

Megatron-Core provides flexibility for developers and makes it easy to develop their own LLM framework on NVIDIA computing infrastructure.

🖥 GitHub
🟡 Docks

✅️ http://t.me/codeprogrammer ✅️