Babe wake up Andrej dropped a video
• youtu.be/l8pRSuU81PU

Or in our files channel
• https://t.me/c/1156511084/935

4 hours long. 🤯

#YouTube #AndrejKarpathy #GPT2
@Dagmawi_Babi

From Andrej:

"The video ended up so long because it is... comprehensive: we start with empty file and end up with a GPT-2 (124M) model:
- first we build the GPT-2 network
- then we optimize it to train very fast
- then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers
- then we bring up model evaluation, and
- then cross our fingers and go to sleep.

In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M) model.

This video builds on the Zero To Hero series and at times references previous videos. You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.

The associated GitHub repo contains the full commit history so you can step through all of the code changes in the video, step by step.
• github.com/karpathy/build-nanogpt"

🔥🔥🔥🔥

#YouTube #AndrejKarpathy #GPT2
@Dagmawi_Babi