Introduction to Linear Algebra

📚 Content Summary

A comprehensive textbook providing a modern introduction to linear algebra, covering vectors, matrices, linear equations, vector spaces, orthogonality, determinants, and eigenvalues, with a focus on both theory and applications.

Master the foundations and applications of linear algebra through Gilbert Strang's intuitive and rigorous framework.

Author: Gilbert Strang

Acknowledgments: Massachusetts Institute of Technology; Wellesley-Cambridge Press

🎯 Learning Objectives

1. Compute linear combinations, dot products, vector lengths, and angles between vectors.
2. Describe the geometric configurations (lines, planes, or volumes) formed by sets of vectors.
3. Solve linear equations using matrix-vector products and interpret the role of inverse and singular matrices.
4. Differentiate between the row picture (intersecting planes) and column picture (linear combinations of vectors) of a system.
5. Execute Gaussian elimination to transform a system into an upper triangular form (U) and solve via back substitution.
6. Formalize elimination steps using elementary matrices (E_{ij}) and permutations (P_{ij}).
7. Identify whether a subset of vectors satisfies the requirements to be a subspace.
8. Perform row reduction to reach the Reduced Row Echelon Form (R) and identify the rank, pivot columns, and free variables.
9. Construct the nullspace matrix N from special solutions and describe the complete solution to linear systems.
10. Identify and prove the orthogonality of the four fundamental subspaces and determine orthogonal complements.

🔹 Lesson 1: Fundamentals of Vectors and Linear Combinations

Overview: This lesson covers the foundational pillars of Linear Algebra: vector operations and their geometric interpretations. It transitions from basic linear combinations and dot products to the algebraic structure of matrices, linear equations (Ax = b), and the critical concepts of vector independence and matrix invertibility. Students will learn to navigate between algebraic calculations and the geometric reality of vectors in \mathbb{R}^3.

Learning Outcomes:

• Compute linear combinations, dot products, vector lengths, and angles between vectors.
• Describe the geometric configurations (lines, planes, or volumes) formed by sets of vectors.
• Solve linear equations using matrix-vector products and interpret the role of inverse and singular matrices.

🔹 Lesson 2: Systems of Linear Equations and Matrix Factorization

Overview: This lesson covers the transition from the geometric interpretation of linear systems to their computational resolution via matrix algebra. It details the mechanics of Gaussian elimination, the formalization of row operations through elementary matrices, and the culmination of these processes into fundamental matrix factorizations (LU, PA=LU, and LDL^T). The material bridges theoretical linearity with practical implementation, including computational costs and software-specific execution.

Learning Outcomes:

• Differentiate between the row picture (intersecting planes) and column picture (linear combinations of vectors) of a system.
• Execute Gaussian elimination to transform a system into an upper triangular form (U) and solve via back substitution.
• Formalize elimination steps using elementary matrices (E_{ij}) and permutations (P_{ij}).

🔹 Lesson 3: Vector Spaces and the Four Fundamental Subspaces

Overview: This lesson explores the structural backbone of linear algebra, focusing on the definition and requirements of vector spaces and subspaces. Students will learn to solve the equation Ax=0 using the Reduced Row Echelon Form (R) to identify pivot and free variables, which lead to the "special solutions" that form a basis for the nullspace. Finally, the lesson culminates in the Fundamental Theorem of Linear Algebra, connecting the dimensions and properties of the four fundamental subspaces: the column space, row space, nullspace, and left nullspace.

Learning Outcomes:

• Identify whether a subset of vectors satisfies the requirements to be a subspace.
• Perform row reduction to reach the Reduced Row Echelon Form (R) and identify the rank, pivot columns, and free variables.
• Construct the nullspace matrix N from special solutions and describe the complete solution to linear systems.

🔹 Lesson 4: Orthogonality and Least Squares Approximations

Overview: This lesson explores the fundamental relationship between the four fundamental subspaces through the lens of orthogonality. Students will learn how to project vectors onto lines and subspaces using projection matrices, solve overdetermined systems via least squares approximations (fitting lines and parabolas), and utilize orthonormal bases and Gram-Schmidt orthogonalization to simplify complex linear algebra problems into A = QR factorizations.

Learning Outcomes:

• Identify and prove the orthogonality of the four fundamental subspaces and determine orthogonal complements.
• Construct projection matrices P and calculate projections of vectors onto lines and higher-dimensional subspaces.
• Apply the Normal Equations (A^T A \hat{x} = A^T b) to find the best-fit line or parabola for a set of data points.

🔹 Lesson 5: Properties and Applications of Determinants

Overview: This lesson explores the algebraic and geometric properties of determinants, transitioning from the pivot-based definition to the "Big Formula" involving permutations and cofactors. Students will apply these concepts to solve linear systems via Cramer's Rule, calculate inverse matrices, and determine areas and volumes in both linear algebra and multivariable calculus (Jacobians).

Learning Outcomes:

• Compute determinants using properties (product rule, transpose), pivot formulas, and cofactor expansion.
• Apply Cramer’s Rule and the cofactor formula to find solutions and matrix inverses.
• Geometrically interpret determinants as areas of triangles/parallelograms and volumes of parallelepipeds, extending to triple products and Jacobians.

🔹 Lesson 6: Eigenvalues, Eigenvectors, and SVD

Overview: This lesson explores the transformation of matrices into their simplest forms to solve complex problems in linear systems and dynamic equations. Students will learn how to decompose matrices using eigenvalues/eigenvectors (A = S\Lambda S^{-1}) for square matrices and the Singular Value Decomposition (A = U\Sigma V^T) for any matrix, providing the foundation for solving differential equations, testing for local minima, and performing image compression.

Learning Outcomes:

• Solve the eigenvalue equation Ax = \lambda x and relate \lambda to the matrix trace and determinant.
• Diagonalize matrices to calculate powers and solve systems of linear differential equations.
• Identify and test for positive definite matrices and calculate Cholesky factorizations.

🔹 Lesson 7: Linear Transformations and Change of Basis

Overview: This lesson explores the fundamental shift from viewing matrices as static data arrays to viewing them as dynamic operators called linear transformations. We will define the rules of linearity, examine how transformations map specific shapes (like the "house") in the plane, and learn to represent calculus operations (derivatives and integrals) as matrices. Finally, we conclude with advanced decompositions—including the Pseudoinverse and Polar Decomposition—which extend our ability to invert and factor transformations even when standard inverses do not exist.

Learning Outcomes:

• Define and identify linear transformations using the principles of addition and scalar multiplication.
• Construct the matrix A for a linear transformation T by mapping basis vectors.
• Perform changes of basis for signals (Wavelets and Fourier) and generalize matrix inversion using the Pseudoinverse A^+.

🔹 Lesson 8: Linear Algebra in Engineering and Statistics

Overview: This lesson explores the practical application of linear algebra across diverse fields including structural engineering, network theory, stochastic processes, optimization, signal processing, and data science. Students will learn how the fundamental framework of A^TCA governs physical systems, how eigenvalues predict long-term population trends, and how orthogonal functions extend vector space concepts to functional analysis and statistics.

Learning Outcomes:

• Model physical systems of springs and masses using the stiffness matrix K = A^TCA.
• Analyze graph connectivity using incidence matrices and verify Euler’s Formula for networks.
• Calculate steady-state vectors for Markov matrices and apply the Perron-Frobenius theorem to population and economic models.

🔹 Lesson 9: Numerical Linear Algebra and Iterative Methods

Overview: This lesson explores the practical implementation of linear algebra on computers, focusing on the transition from theoretical exactness to numerical stability and efficiency. Students will learn how to mitigate roundoff errors through partial pivoting, optimize computations using operation counts for sparse and band matrices, and evaluate system sensitivity using condition numbers. Additionally, the lesson covers iterative techniques for solving large-scale systems and methods for approximating eigenvalues.

Learning Outcomes:

• Analyze the impact of roundoff errors and apply partial pivoting to ensure numerical stability in Gaussian elimination.
• Evaluate the computational efficiency of algorithms by calculating operation counts for full, band, and sparse matrices.
• Measure the sensitivity of linear systems using norms and condition numbers, specifically identifying ill-conditioned cases like the Hilbert matrix.

🔹 Lesson 10: Complex Vectors and Unitary Matrices

Overview: This lesson covers the transition from real to complex numbers, focusing on their arithmetic, geometric representation in the complex plane, and polar forms. It culminates in the study of complex-specific matrix structures—Hermitian and Unitary matrices—which are the complex-valued counterparts to symmetric and orthogonal matrices.

Learning Outcomes:

• Perform arithmetic operations and find the conjugate/modulus of complex numbers.
• Map complex numbers to the complex plane and convert between rectangular (a + bi) and polar (re^{i\theta}) forms.
• Apply Euler’s Formula to simplify products and powers of complex numbers.