Regression Calculator

About Regression Calculator

How Linear Regression Works - the Least Squares Method

Simple linear regression finds the straight line that best fits a set of data points. "Best fit" means the line that minimises the total squared distance between each observed Y value and the Y value the line predicts. This approach is called the ordinary least squares (OLS) method, introduced by Adrien-Marie Legendre in 1805 and Carl Friedrich Gauss around the same time.

The formulas are:

slope (b) = Σ((x_i - x̄)(y_i - ȳ)) / Σ(x_i - x̄)²

intercept (a) = ȳ - b × x̄

where x̄ and ȳ are the means of the X and Y values. The slope tells you how much Y changes for each one-unit increase in X. The intercept is where the line crosses the Y-axis.

Worked example: Take five data points: (1, 2), (2, 4), (3, 5), (4, 4), (5, 5). The mean of X is 3, the mean of Y is 4. Computing the sums: Σ(dx × dy) = (1-3)(2-4) + (2-3)(4-4) + (3-3)(5-4) + (4-3)(4-4) + (5-3)(5-4) = 4 + 0 + 0 + 0 + 2 = 6. And Σdx² = 4 + 1 + 0 + 1 + 4 = 10. So slope = 6/10 = 0.6, and intercept = 4 - 0.6 × 3 = 2.2. The regression line is y = 0.6x + 2.2. To predict Y when X = 7, plug in: y = 0.6(7) + 2.2 = 6.4.

You can verify regression results against your scatter plot using the graphing calculator, which lets you plot equations alongside raw data.

Understanding R-Squared and Model Fit

R-squared (R²) is the square of the Pearson correlation coefficient. It tells you what fraction of the total variance in Y is accounted for by the regression model. The formula breaks down like this:

R² = 1 - (SS_res / SS_tot)

where SS_res is the sum of squared residuals and SS_tot is the total sum of squares (the variance of Y around its mean). If the regression line passes through every point, SS_res = 0 and R² = 1. If the line is no better than just guessing the mean of Y, R² = 0.

R² Range	Interpretation	Typical Example
0.90 - 1.00	Excellent fit - model explains most of the variance	Ohm's law (voltage vs current in a resistor)
0.70 - 0.89	Good fit - strong linear pattern with some scatter	Height vs arm span in adults
0.50 - 0.69	Moderate fit - clear trend but substantial unexplained variance	Study hours vs exam score
0.30 - 0.49	Weak fit - some linear tendency, lots of noise	Income vs self-reported happiness
0.00 - 0.29	Poor fit - model barely explains the data	Shoe size vs IQ

Keep in mind that R-squared alone does not tell you if linear regression is the right model. A curved relationship can produce a modest R-squared that would jump dramatically with a polynomial model. Always check the residual plot before trusting R-squared.

The standard error of the estimate (Se) gives the average distance of observed Y values from the regression line, measured in the same units as Y. Smaller Se means predictions are more precise. Together, R-squared and Se give a complete picture of model quality.

Checking Assumptions with Residual Analysis

Linear regression makes four key assumptions. Residual analysis is the primary way to check them:

Linearity. The relationship between X and Y should be roughly linear. If the residual plot shows a U-shape or any systematic curve, the data has a non-linear pattern that a straight line cannot capture. Consider polynomial regression or transforming the variables.

Homoscedasticity (constant variance). The spread of residuals should be roughly the same across all X values. If the residual plot fans out (wider on the right than the left, or vice versa), the model has heteroscedastic errors. This does not bias the slope estimate, but it makes the standard errors and p-values unreliable.

Independence. Each data point should be independent of the others. This assumption is often violated in time-series data, where consecutive observations tend to be correlated. If your residuals show a wave-like pattern when plotted in order, the independence assumption is broken.

Normality of residuals. For small samples, the residuals should be approximately normally distributed. This matters mainly for the accuracy of confidence intervals and p-values. With large samples (over 30 data points), moderate departures from normality have little practical effect thanks to the central limit theorem. You can assess normality using the normal distribution calculator to compare your residuals against a normal curve.

What to look for in a healthy residual plot: random scatter above and below zero, no patterns, no obvious outliers, and roughly equal spread across the range of X values. If you see clear structure, the linear model needs rethinking.

When to Use Linear Regression

Linear regression is one of several approaches to modelling relationships in data. The right method depends on what your data looks like and what you are trying to predict. Here is a comparison:

Method	Best For	Relationship Shape	Outcome Type
Simple linear regression	One predictor, continuous outcome	Straight line	Continuous (e.g. weight, temperature)
Multiple linear regression	Several predictors, continuous outcome	Hyperplane (flat surface)	Continuous
Polynomial regression	Curved relationships	Quadratic, cubic, etc.	Continuous
Logistic regression	Binary classification	S-shaped curve	Binary (yes/no, 0/1)
Correlation analysis	Measuring strength of association	Any monotonic trend	Not predictive

Start with a scatter plot. If the relationship looks like a straight line, simple linear regression is a natural choice. If you see curvature, try adding a polynomial term (x²) or log-transforming one of the variables. If your outcome is categorical (pass/fail, buy/not buy), logistic regression is more appropriate.

One common mistake is forcing linear regression on data that is clearly non-linear. The model will still produce an equation and R-squared, but the predictions will be systematically wrong in parts of the range. The residual plot is your best diagnostic tool - if it shows a pattern, the model is missing something.

For exploring the relationship between variables before fitting a regression, the correlation calculator helps you quantify the strength of association. And if you are testing whether a regression slope is significantly different from zero, that is essentially a t-test on the slope coefficient, which this calculator reports automatically.

Real-World Applications of Linear Regression

Linear regression is one of the most widely used statistical methods across disciplines. Here are some examples of where it appears in practice:

Economics and business. Economists use regression to model relationships like advertising spend versus revenue, or GDP growth versus unemployment. The simplest demand curve - quantity demanded as a function of price - is often modelled as a linear regression. Retail companies use it to forecast sales based on historical trends.

Science and engineering. Hooke's Law (force = spring constant times extension) is a linear relationship. Calibration curves in analytical chemistry plot instrument readings against known concentrations to predict unknown samples. In physics labs, students routinely fit regression lines to experimental data and compare the slope to theoretical predictions.

Health and medicine. Dose-response studies often begin with linear regression to check if a drug's effect increases proportionally with dose. Growth charts that track a child's height over time use regression lines. Epidemiologists regress disease rates against risk factors to estimate effect sizes.

Education. Admissions offices model the relationship between standardised test scores and college GPA. Research on class size and academic performance frequently uses regression to control for confounding variables.

Practical Tips for Better Regression Analysis

Plot your data first. Never run a regression blind. A scatter plot takes seconds and can reveal outliers, non-linearity, or clusters that would invalidate the model. Anscombe's Quartet (1973) famously demonstrated four datasets with identical regression statistics but wildly different scatter plots.

Watch out for extrapolation. Predictions are only reliable within the range of your X data. If your data covers X = 1 to 10, predicting Y at X = 50 assumes the linear trend continues far beyond what you have observed. That assumption is often wrong.

Check for influential points. A single outlier near the edge of your X range can dramatically change the slope and intercept. Look at the residual table for points with unusually large residuals, and consider whether they are data errors or genuine observations.

Report the standard error alongside R-squared. R-squared tells you proportional fit, but the standard error tells you the typical prediction error in the same units as Y. An R-squared of 0.95 is less impressive if the standard error is 20 kg and you are predicting body weight.

Sample size matters. With only 5 data points, your regression coefficients have wide confidence intervals and the p-value may not be trustworthy. A common guideline is at least 10 to 15 observations for simple regression, and at least 10 per predictor for multiple regression. Use the sample size calculator to plan ahead.

Do not confuse statistical significance with practical significance. A large dataset can produce a highly significant slope (tiny p-value) for a relationship that explains very little variance. Always consider both the p-value and R-squared together before drawing conclusions.

Sources

• NIST/SEMATECH e-Handbook of Statistical Methods - Linear Least Squares Regression
• NIST/SEMATECH e-Handbook - Residual Analysis and Model Assumptions
• Anscombe, F. J. (1973) - Graphs in Statistical Analysis, The American Statistician
• Brown University - Seeing Theory: Regression Analysis
• Penn State STAT 501 - Simple Linear Regression
• Carnegie Mellon - Assumptions of Linear Regression