From 0dfede6574d3ae2261878033bd719c5e1047b104 Mon Sep 17 00:00:00 2001 From: ErdemOzgen <14043035+ErdemOzgen@users.noreply.github.com> Date: Sun, 10 Nov 2024 14:12:25 +0300 Subject: [PATCH] Add section 2 mit calculus single variable --- content/AI&DATA/MathsForML/Calculus.md | 2 + .../MathsForML/Coursera Maths for ML.md | 108 ++++++++++++++++++ content/Untitled 2.canvas | 0 3 files changed, 110 insertions(+) create mode 100644 content/AI&DATA/MathsForML/Coursera Maths for ML.md create mode 100644 content/Untitled 2.canvas diff --git a/content/AI&DATA/MathsForML/Calculus.md b/content/AI&DATA/MathsForML/Calculus.md index ed830b424..aa67d0ab7 100644 --- a/content/AI&DATA/MathsForML/Calculus.md +++ b/content/AI&DATA/MathsForML/Calculus.md @@ -26,3 +26,5 @@ The Derivative of |x| - [Problem (PDF)](https://ocw.mit.edu/courses/18-01sc-single-variable-calculus-fall-2010/resources/mit18_01scf10_ex02prb/) - [Solution (PDF)](https://ocw.mit.edu/courses/18-01sc-single-variable-calculus-fall-2010/resources/mit18_01scf10_ex02sol/) + + diff --git a/content/AI&DATA/MathsForML/Coursera Maths for ML.md b/content/AI&DATA/MathsForML/Coursera Maths for ML.md new file mode 100644 index 000000000..ab8e64eaa --- /dev/null +++ b/content/AI&DATA/MathsForML/Coursera Maths for ML.md @@ -0,0 +1,108 @@ + +--- + +## Linear Regression Explanation: Weights Vector and Feature Matrix + +### 1. How the Weights Vector Multiplies with the Feature Matrix + +In linear regression, we model the relationship between input features and the target output as a linear combination of the input features. The formula for linear regression is: + +$$ +\mathbf{y} = \mathbf{X} \mathbf{w} + \mathbf{b} +$$ + +where: +- **\(\mathbf{y}\)** is the vector of predicted outputs. +- **\(\mathbf{X}\)** is the feature matrix, where each row represents a data point and each column represents a feature. If there are \(n\) data points and \(p\) features, then \(\mathbf{X}\) has a shape of \(n \times p\). +- **\(\mathbf{w}\)** is the weights vector (also called the coefficient vector), which has a size of \(p \times 1\). Each weight corresponds to the importance or contribution of a specific feature. +- **\(\mathbf{b}\)** is the bias term, which shifts the output up or down. + +#### Explanation of \(\mathbf{X} \mathbf{w}\) + +1. **Matrix-vector multiplication**: In \(\mathbf{X} \mathbf{w}\), each row of \(\mathbf{X}\) (a single data point) is multiplied by the weights vector \(\mathbf{w}\) to produce a prediction for that data point. +2. **Output**: The result \(\mathbf{X} \mathbf{w}\) yields an \(n \times 1\) vector of predictions for all \(n\) data points. The bias term \(\mathbf{b}\) is added to each element in this vector to produce the final prediction vector. + +--- + +### 2. The Weights Vector \(\mathbf{w}\) + +The weights vector, typically denoted as **\(\mathbf{w}\)**, is a column vector that holds the coefficients assigned to each feature in linear regression. It defines the influence or contribution of each feature to the output prediction. + +For a model with \(p\) features, the weights vector **\(\mathbf{w}\)** is of size \(p \times 1\) and can be represented as: + +$$ +\mathbf{w} = \begin{bmatrix} w_1 \\ w_2 \\ \vdots \\ w_p \end{bmatrix} +$$ + +where each element \(w_i\) in the weights vector corresponds to the coefficient for feature \(x_i\): +- \(w_1\) is the weight for the first feature. +- \(w_2\) is the weight for the second feature. +- ... +- \(w_p\) is the weight for the \(p\)-th feature. + +For example, if the model has three features, then the weights vector would be: + +$$ +\mathbf{w} = \begin{bmatrix} w_1 \\ w_2 \\ w_3 \end{bmatrix} +$$ + +In practice, these weights are determined during training by minimizing the error between the predictions and the actual target values, adjusting the weights accordingly. + +--- + +### 3. The Feature Matrix \(\mathbf{X}\) + +The feature matrix, denoted as **\(\mathbf{X}\)**, contains all input data points for a linear regression model. In this matrix: +- Each row represents a single data point (observation). +- Each column represents a feature (variable) of that data point. + +If there are \(n\) data points and \(p\) features, the feature matrix \(\mathbf{X}\) is of size \(n \times p\) and has the following structure: + +$$ +\mathbf{X} = \begin{bmatrix} +x_{11} & x_{12} & \cdots & x_{1p} \\ +x_{21} & x_{22} & \cdots & x_{2p} \\ +\vdots & \vdots & \ddots & \vdots \\ +x_{n1} & x_{n2} & \cdots & x_{np} \\ +\end{bmatrix} +$$ + +where: +- \(x_{ij}\) represents the value of the \(j\)-th feature for the \(i\)-th data point. + +#### Example of a Feature Matrix + +For three data points and two features, the feature matrix would look like this: + +$$ +\mathbf{X} = \begin{bmatrix} +x_{11} & x_{12} \\ +x_{21} & x_{22} \\ +x_{31} & x_{32} \\ +\end{bmatrix} +$$ + +In this example: +- The first row \([x_{11}, x_{12}]\) represents the features for the first data point. +- The second row \([x_{21}, x_{22}]\) represents the features for the second data point. +- The third row \([x_{31}, x_{32}]\) represents the features for the third data point. + +During the linear regression calculation, each row of **\(\mathbf{X}\)** is multiplied by the weights vector **\(\mathbf{w}\)** to generate a prediction for that specific data point. + + +In linear algebra, a **singularity** typically refers to a **singular matrix**—a matrix that does not have an inverse. Here’s what that means: + +1. **Singular Matrix**: + - A matrix \( \mathbf{A} \) is singular if its **determinant** is zero: \( \det(\mathbf{A}) = 0 \). + - A singular matrix cannot be inverted, which implies that there is no matrix \( \mathbf{A}^{-1} \) such that \( \mathbf{A} \mathbf{A}^{-1} = \mathbf{I} \) (where \( \mathbf{I} \) is the identity matrix). + +2. **Implications of Singularity**: + - **Linearly Dependent Rows or Columns**: A matrix is singular if its rows or columns are linearly dependent, meaning that at least one row or column can be written as a linear combination of the others. + - **No Unique Solutions**: When using a singular matrix in a system of linear equations \( \mathbf{A}\mathbf{x} = \mathbf{b} \), the system either has **no solutions** or **infinitely many solutions**, but never a unique solution. + - **Non-Invertible Transformations**: In transformations represented by matrices, a singular matrix corresponds to a transformation that squashes some or all of the space, meaning the transformation is not fully reversible. + +3. **Geometric Interpretation**: + - A singular matrix, when representing a transformation in vector space, will map some vectors onto lower-dimensional space (e.g., a 2D plane in 3D space), causing a loss of dimensionality. This often manifests as "flattening" or "collapsing" parts of the space onto each other. + +In summary, in linear algebra, singularity indicates a loss of invertibility due to linearly dependent vectors, resulting in a matrix that cannot map uniquely back to its original space. + diff --git a/content/Untitled 2.canvas b/content/Untitled 2.canvas new file mode 100644 index 000000000..e69de29bb