# Category: Linear Algebra

### #010 Linear Algebra – Linear least squares

Highlight: Linear least squares is a very powerful algorithm to find the approximate solutions of overdetermined linear systems of linear equations. Those are systems of linear equations that have more equations than unknowns. The solution to this idea is to minimize the sum of squares of errors in the equation. This method was discovered independently by the mathematicians, Carl Friedrich Gauss, and Adrien-Marie Legendre, around the beginning of the 19th century. So, let’s begin with…

### #011 Linear Algebra – Nonlinear Least Squares

Highlights: In the real-world scenario, models don’t produce linear graphs that often. Most of the time the equation of the model involves higher-order and higher-degree functions. In this post, we will learn how to solve the harder nonlinear equations using a heuristic algorithm of finding the least-squares approximate solution. So let’s begin. Tutorial Overview: Nonlinear Equations Introduction Difficulty Of Solving nonlinear equations Nonlinear Least Squares Optimality condition Gauss-Newton Algorithm Basic Gauss-Newton Algorithm Shortcomings Of The Gauss-Newton Algorithm…

### #007 Linear Algebra – Change of basis

Highlight: So far, we have already talked that it is possible to represent the vector using different basis vectors. In this post we will learn how to go from our standard coordinate system $$\left ( x,y \right )$$ into some other bases. Next, we will also learn why this change of basis can be very useful. For now, we will just say that it’s frequently applied in many signal processing and machine learning methods.… 