Please enable JavaScript.
Coggle requires JavaScript to display documents.
Linear Algebra For Machine Learning - Coggle Diagram
Linear Algebra For Machine Learning
Linear Systems
Linear algebra provides a way to represent and understand the solutions to systems of linear equations. We represent linear equations in the general form of Ax +By = c.
A system of linear equations consists of multiple, related functions with a common set of variables. The point where the equations intersect is known as a solution to the system.
The elimination method involves representing one of our variables in terms of a desired variable
and substituting the equation that is in terms of the desired variable.
A matrix uses rows and columns to represent only the coefficients in a linear system, and it's
similar to the way data is represented in a spreadsheet or a DataFrame.
Gaussian elimination is used to solve systems of equation that are modeled by many variables
and equations.
In an augmented matrix, the coefficients from the left side of the function are on the left side ofthe bar ( | ), while the constants from the right sides of the function are on the right side.
To preserve the relationships in the linear system, we can use the following row operations:
Any rows can be swapped
Any Row Can be multiplied by a nonzero constant
Any row can be added to another row
To solve an augmented matrix, you'll have to rearrange the matrix into echelon form. In this form, the values on the diagonal are all equal to 1 and the values below the diagonal are equal to 0.
Vectors
When referring to matrices, the convention is to specify the number of rows first then the number of columns. Ex: a matrix with
two rows
and
three columns
is known as a
2X3 matrix
A list of numbers in a matrix is known as a vector, a row from a matrix is known as a row vector, and a column from a matrix is known as a column vector.
A vector can be visualized on a coordinate grid when a vector contains two or three elements. Typically, vectors are drawn from an origin(0,0) to the point described by the vector.
Arrows are used to visualize individual vectors because they emphasize two properties of vector - direction, and magnitude. The direction of a vector describes the way it's pointing while the magnitude describes its length.
The
pyplot.quiver()
function takes four required parameters:
X;Y;U
and
V
. X and Y correspond to the (x,y) coordinates we want to vector to start at while U and V correspond the (x,y) coordinate we want to draw the vector.
Similiar to rows in a matrix, vectors can be added or substracted together. To add or subtract vectors, you add the corresponding elements in the same position. Vectors can also by scaled up by multiplying the vector by a real number greater than 1 or less that .-1. Vectors can also be scaled down by multiplying the vector by a number between -1 and 1-
The optional parameters:
angles
,
scale_units
and
scale
always want to be used when plotting vectors. Setting angles to
xy
lets matplotlib know we want the angle of the vector to be between the points we specified. The
scale_units
and
scale
parameters lets us specify custom scaling parameters for the vectors.
To compute the dot product, we need to sum the products of the 2 values in each position in each vector.
A linear combinations are vectors that are scaled up and then added or subtracted.
The arithmetic representation of the matrix equation is Ax = b where where represents the coefficient matrix, represents the solution vector, and represents the constants. Note that can't be a vector containing all zeros, also known as the zero factor and represented using .
Matrix Algebra
Many operations that can be performed on vectors can also be performed on matrices. With
matrices, we can perform the following operations
Add and subtract matrices containing the same number of rows and columns
Multiply a matrix by a scaler value.
Multiply a matrix with a vector and other matrices. To multiply a matrix by a vector or another matrix, the number of columns must match up with the number of rows in the vector or matrix. The order of multiplication does matter when multiplying matrices and vectors.
Taking the transpose of a matrix switches the rows and columns of a matrix. Mathematically, we
use the notation
AT
to specify the transpose operation.
The identity matrix contains along the diagonal and elsewhere. The identity matrix is often
represented using
In
where is the number of rows and columns.
When we multiply with any vector containing two elements, the resulting vector matches the
original vector exactly.
To transform A into the identity I matrix in Ax(v) =b(v), we multiply each side by the inverse of
matrix .
If the determinant of a
2x2
matrix or
ad-bc
is equal to 0, the matrix is not invertible. We can compute the determinant and matrix inverse only for matrices that have the same number of rows in columns. These matrices are known as square matrices.
To compute the determinant of a matrix other than
2x2
a matrix, we can break down the full
matrix into minor matrices.
Solution Sets
An inconsistent system has two or more equations that no solution exists when the augmented
matrix is in reduced echelon form.
When the determinant is equal to zero, we say the matrix is singular or it contains no inverse.
A nonhomogenous system is a system where the constants vector (b) doesn't contain all zeros.
A homogenous system is a system where the constants vector (b) is equal to the zero vector.
A homogenous system always contains the trivial solution: the zero vector
For a
nonhomogenous
system that contains the same number of rows and columns, there are 3
possible solutions:
No Solution
A Single Solution
Infinitely many solutions
For rectangular (nonsquare, nonhomogenous) systems, there are two possible solutions
No Solution
Infinitely many Solutions
If Ax=b is a linear system, then every vector x which satisfies the system is said to be a solution vector of the linear system. The set of solution vectors of the system is called the solution space of the linear system.
When the solution is a solution space (and not just a unique set of values), it's common to
rewrite it into parametric vector form.