Regression Line

A regression line, also known as the line of best fit, is a statistical tool used to describe the relationship between two variables. Typically, it represents a linear equation that best approximates the distribution of data points on a scatter plot. By minimizing the distance between the line and each data point, it predicts the dependent variable based on the value of the independent variable. This line is foundational in simple linear regression analysis, allowing for straightforward visualization and understanding of how changes in one variable could affect the other.

Residual

The residual is the difference between an observed value of the dependent variable ($y$) and the value predicted by the regression line ($\hat{y}$).

$e = y - \hat{y}$

Residuals help in assessing the fit of a regression model. A residual plot, which displays residuals on the vertical axis and the independent variable or predicted values on the horizontal axis, can reveal patterns that suggest non-linearity, outliers, or other anomalies that might affect the model.

Slope

The slope ($\beta_1$) is the estimated change of the dependent variable ($y$) per unit change in the independent variable ($x$).

Intercept

The intercept ($\beta_0$) is the expected value of $y$ when $x$ is zero. It represents the point where the regression line crosses the $y$-axis.

$R^2$ (Coefficient of Determination)

$R^2$ is a statistic that measures the proportion of variance in the dependent variable ($y$) that is predictable from the independent variable ($x$). It provides an indication of goodness of fit and tells us how well the unseen samples are likely to be predicted by the model. \ An $R^2$ value of 1 indicates a perfect fit, meaning that all data points lie exactly on the regression line. An $R^2$ value of 0 indicates that the regression line does not explain any of the variability in the dependent variable. $R^2$ is a key measure used to assess the effectiveness of a model in explaining the relationship between variables.