Loss Functions and Optimisers - How AI Learns from Mistakes

Public lesson preview

Discover how loss functions measure a model's errors and how optimisers use gradients to systematically reduce them.

Learning objectives

By the end of this lesson, you will understand:

• ✓What Is a Loss Function?
• ✓Loss Functions for Regression - MSE
• ✓Loss Functions for Classification - Cross-Entropy
• ✓Gradient Descent - Rolling Downhill
• ✓Flavours of Gradient Descent

Lesson outline

1. What Is a Loss Function?
2. Loss Functions for Regression - MSE
3. Loss Functions for Classification - Cross-Entropy
4. Gradient Descent - Rolling Downhill
5. The Learning Rate Dilemma
6. Flavours of Gradient Descent
7. Batch Gradient Descent
8. Stochastic Gradient Descent (SGD)
9. Mini-Batch Gradient Descent
10. Modern Optimisers
11. SGD with Momentum
12. AdaGrad
13. Adam (Adaptive Moment Estimation)
14. Learning Rate Schedules
15. Gradient Clipping - Safety Rails
16. Key Takeaways
17. 📚 Further Reading