Linear Algebra Through Problems: Understanding Ax = b and Its Connection to Machine Learning

From solving mathematical problems to understanding how Machine Learning and Deep Learning models learn.

Figure 1. Linear Algebra is everywhere in Machine Learning and Deep Learning. From matrix multiplication to projections, these mathematical foundations help us understand how models learn, transform, and represent data.

Introduction

In our previous articles, we explored vectors, vector operations, and how mathematical operations such as addition, subtraction, scalar multiplication, and dot products form the foundation of Machine Learning.

Now, let’s take the next step.

Instead of simply learning mathematical concepts and formulas, let’s solve some numerical problems, understand the intuition behind them, and discover where these concepts appear in real-world Machine Learning and Deep Learning models.

For this article, we will follow a problem-solving approach inspired by Gilbert Strang’s Introduction to Linear Algebra.

One of the most fundamental problems in Linear Algebra is:

How do we solve a system of linear equations?

This seemingly simple question leads us to several important concepts:

• Matrix representation
• Linear combinations
• Column space
• Least Squares
• Orthogonal projections