Building PyTorch: A Hands-On Guide to the Core Foundations of a Training Framework

Dive into the essentials of neural networks with this hands-on tutorial! Learn how to create a neural network from the ground up, focusing on the Parameter and Module classes, backpropagation, and practical implementation. Explore key concepts like initialization methods (Xavier, He), gradient tracking, and sequential deep learning models - all explained step-by-step with real code examples.

📼 YouTube: https://www.youtube.com/watch?v=b16qKLmp2ro

Building PyTorch: Crafting Linear Layers and Parameter Counting in MicroTorch

In this hands-on tutorial, we refine our tensor operations and introduce linear layers from scratch. Explore how to fix transpose gradients, implement parameter counting for modules, and build efficient linear layers using matrix dot products. We’ll dive into dimensionality checks, Xavier initialization, and bias handling - all with practical code examples.

📼 YouTube: https://www.youtube.com/watch?v=F9GH3nF4nkM

Me

Building PyTorch: Mastering Matrix Multiplication and Linear Layers in MicroTorch

Dive deep into implementing matrix multiplication (dot product) and linear layers from scratch. Learn the calculus behind forward and backward passes, handle multi-dimensional tensors, and build PyTorch-style APIs with real code examples. We’ll cover gradient computation, broadcasting, and operator overloading—key foundations for neural network training.

📼 YouTube: https://www.youtube.com/watch?v=UQIGmdXZd_U

Prepping for your next Machine Learning interview? These 3 resources have you covered:

1️⃣ NeetCode – Master coding interviews: https://neetcode.io/practice

2️⃣ DeepML – Practice ML & Data Science problems: https://www.deep-ml.com/

3️⃣ TensorGym – Sharpen AI skills with unique exercises: https://tensorgym.com/exercises

#MachineLearning #AI #TechInterviews

Building PyTorch: Adding Broadcasting and Addition Operations to MicroTorch

Handle gradient propagation across different tensor shapes, implement backward broadcasting, and enable the plus operator - all with practical code examples. We'll finalize the linear layer and set the stage for a full training loop

https://www.youtube.com/watch?v=pdZij4qj2WQ

Building PyTorch: Overloading Operators for Subtraction and Multiplication in MicroTorch

We'll overload essential tensor operators—subtraction, multiplication, and in-place operations—while ensuring proper gradient tracking and broadcasting. Learn how to implement negation, element-wise multiplication, and right-hand operations, all with real code examples, and see them in action with a mini-experiment.

📼 YouTube: https://www.youtube.com/watch?v=kPRDyKfLYlA

Building PyTorch: Enriching MicroTorch with Logs, Exponents, and Activation Functions

We'll expand the tensor class by adding logarithm, hyperbolic tangent, power, division, and exponential operations, complete with gradient support. Then, we'll build tanh and sigmoid activation functions using these operations - all with practical code examples. See how autograd simplifies backpropagation for free!

📼 YouTube: https://www.youtube.com/watch?v=IM_W-RGMTVc

Building PyTorch: Enhancing MicroTorch with Squeeze, View, and Clip Operations

In this episode of our MicroTorch series, we dive into tensor shape manipulations and gradient clipping! We’ll add squeeze, unsqueeze, view, and clip operations to the tensor class, ensuring proper forward and backward passes for autograd

YouTube: https://youtu.be/9EaSwTdoTag

Thank you all for your support!
300 subs from absolute zero-only because of you! Every comment, repost, like, and view is my best motivation.
See you soon in the new video on Sunday!

Also, check out my X! I'm trying to be more active: https://x.com/datasantaa

If you want to go deeper in Matrix Calculus for DL I think this's the best course: MIT 18.S096 Matrix Calculus For Machine Learning And Beyond, IAP 2023 Instructors: Alan Edelman, Steven G. Johnson

https://www.youtube.com/playlist?list=PLUl4u3cNGP62EaLLH92E_VCN4izBKK6OE

Mastering Tensor Operations: Comparison, Where, and More in MicroTorch

In this episode of our MicroTorch series, we explore essential tensor operations for deep learning! We'll implement comparison operators (less than, greater than, equals) to create boolean tensors, build the "where" method for conditional element-wise selection, and add useful operations like maximum, minimum, threshold, mask_fill, sign, and clip. These methods are key for neural network development, enabling efficient tensor manipulations and gradient computations. Follow along with detailed code examples and tests to see how these operations work in practice.

📼 YouTube: https://www.youtube.com/watch?v=4_kJy6hmv2E

Building MicroTorch: Indexing and Absolute Value Operations for Tensors

In this episode of our MicroTorch series, we dive into tensor indexing and absolute value operations for deep learning! We'll implement the "get item" method to enable flexible tensor indexing, handle gradient computations for indexed tensors, and build the "abs" method to compute the absolute value of tensor elements. These operations are crucial for neural network development, ensuring proper forward and backward passes for autograd. Follow along with detailed code examples and tests to see how these methods work in practice

📼 YouTube: https://www.youtube.com/watch?v=mk2FkpeZOAM

Check my new blog post: MicroTorch - Deep Learning from Scratch!

Implementing deep learning algorithms involves managing data flow in two directions: forward and backward. While the forward pass is typically straightforward, handling the backward pass can be more challenging. As discussed in previous posts, implementing backpropagation requires a strong grasp of calculus, and even minor mistakes can lead to significant issues.

Fortunately, modern frameworks like PyTorch simplify this process with autograd, an automatic differentiation system that dynamically computes gradients during training. This eliminates the need for manually deriving and coding gradient calculations, making development more efficient and less error-prone.

Now, let's build the backbone of such an algorithm - Tensor class!

https://datasanta.net/2025/04/03/microtorch---deep-learning-from-scratch/

Essential Tensor Operations in MicroTorch: Mastering Max with Backward Mode

In this episode of our MicroTorch series, we explore the max operation for deep learning tensors! We'll implement the maximum operation to return the highest value along a specified axis or across the entire tensor, handling duplicate maximums and axis-specific computations. We'll also dive into the backward mode, assigning gradients to maximum value positions and ensuring proper shape alignment with keep_dimensions. Follow along with detailed code examples and tests to see how these operations work in practice.

📼 YouTube: https://youtu.be/olINYduKFJY

The final video on tensor operations - next, we'll switch to the training loop: Tensor Operations in MicroTorch: Min, Max, Sum, Mean, and Activation Functions

In this episode of our MicroTorch series, we'll implement minimum and unify the backward step for min and max. Build the sum and mean methods for efficient tensor aggregation, and explore key activation functions like ReLU, LeakyReLU, and a numerically stable softmax. These operations form the backbone of neural network development, enabling powerful tensor manipulations and automatic gradient computations. Follow along with detailed code examples and tests to see how these methods work in practice.

📼 YouTube: https://www.youtube.com/watch?v=kuElg7VtSII

Building Loss Functions in MicroTorch: Expected Risk, Empirical Risk, and Binary Cross-Entropy

In this episode of our MicroTorch series, we dive into loss functions, the backbone of training neural networks. Learn how to define expected risk and empirical risk to measure model performance against true data distributions. We implement a flexible loss reduction system with mean, sum, and none options, and build a base Loss class with a unified forward method. Then, we create a BCELoss class for binary classification, complete with epsilon clipping for numerical stability. Follow along with detailed code examples to see how these components power MicroTorch's training loop.

📼 YouTube: https://www.youtube.com/watch?v=s7blWKlV3uM

Implementing Cross-Entropy Loss and Log-Sum-Exp in MicroTorch

In this episode of our MicroTorch series, we scale our loss function implementation from binary to multi-class classification by building a numerically stable version of the Cross-Entropy loss using the log-sum-exp trick. We explore the need for softmax in multi-class scenarios and explain how subtracting the max from logits prevents exponential blow-ups. You'll learn how to compute the softmax function, derive log-softmax directly using log-sum-exp, and implement it efficiently inside our Tensor class.

📼 YouTube: https://www.youtube.com/watch?v=ftob7lsheX4

Empirical Risk and Cross-Entropy in MicroTorch

In the previous chapter we prepared the MicroTorch - Deep Learning from Scratch. Now, it's time to dive into creating the loss functions that will guide our model during training. In this session, we're going to focus on building two fundamental loss functions: Binary Cross-Entropy (BCE) and Cross-Entropy (CE), using the Microtorch framework. These functions are essential for training models, especially for classification tasks, and I'll walk you through how to implement them from scratch.

Check my new blog post: https://datasanta.net/2025/04/13/empirical-risk-and-cross-entropy-in-microtorch/

Yes