Day 7 of self-studying Berkeley CS189 — stochastic gradient descent notes 📚📝

🔥 *Stochastic Gradient Descent (SGD)* is a powerful optimization algorithm used to minimize loss functions in machine learning. Unlike batch gradient descent, which uses the entire dataset to compute gradients, SGD updates parameters using a single training example (or a small mini-batch) at a time.

🚀 Key Benefits:
- Faster convergence on large datasets
- Escapes local minima more easily
- Suitable for online learning scenarios

📊 The Update Rule:
θ = θ - α * ∇J(θ; x⁽ⁱ⁾, y⁽ⁱ⁾)
Where α is the learning rate and (x⁽ⁱ⁾, y⁽ⁱ⁾) is a single training example.

📌 Challenges:
- High variance in updates
- Requires careful tuning of the learning rate

🧠 *Tip:* Use momentum or adaptive learning rates (like Adam) to stabilize training!

#MachineLearning #CS189 #SGD #DeepLearning #DataScience #Algorithms

✨ Join Best TG Channels https://t.me/addlist/0f6vfFbEMdAwODBk

⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A