What a statistical model is
A statistical model is a simplified description of how one thing depends on others. You might model monthly electricity use as depending on outdoor temperature, or the time to resolve a support request as depending on how many requests are open. The model will never be exactly right, but a useful one captures the main pattern and leaves the rest as unexplained variation.
Models are used for two broad purposes. One is explanation: understanding how measures relate. The other is prediction: estimating a value for a new case. The same tools serve both, but the cautions differ, and it helps to know which one you are after before you begin.
Start by looking at the data
Before fitting anything, plot the data and check it. A scatter plot of the two measures shows whether a straight line is even plausible, and whether a few points sit far from the rest. Confirm that the data has been through data cleaning, because duplicates, typos and impossible values distort a model quietly. Fitted numbers look precise even when the input was messy.
Simple linear regression
The simplest common model fits a straight line through the points:
outcome = intercept + slope × predictor
- The intercept is the predicted outcome when the predictor is zero. Sometimes that has no real-world meaning.
- The slope is the predicted change in the outcome for each one-unit increase in the predictor.
The line is usually chosen by “least squares”: it minimizes the sum of the squared vertical gaps between the points and the line. Those gaps are called residuals. Each residual is the part of one observation that the model fails to explain.
A hypothetical example
Suppose, purely for illustration, a small office records the number of open tickets and the average hours to resolve a ticket, and a fitted line is: hours = 2 + 0.5 × open tickets. The intercept of 2 says that with no backlog, resolution takes about 2 hours. The slope of 0.5 says that each additional open ticket is associated with about half an hour more. At 10 open tickets the model predicts 2 + 0.5 × 10 = 7 hours. These numbers are invented to show the arithmetic, not real findings.
Adding more predictors
Multiple regression uses several predictors at once, such as open tickets and staffing level. Each slope then describes the association with the outcome while the other predictors are held constant in the model. This is helpful, but it also makes interpretation harder, especially when predictors are closely related to each other. More predictors is not always better: a model with too many can fit the quirks of one dataset and then perform poorly on new data.
Assumptions to check
Linear regression works best when some conditions roughly hold. They are rarely perfect, but big violations deserve attention.
- Linearity. The relationship is roughly straight. A curved pattern in the scatter plot or in the residuals suggests a different form.
- Independence. Observations do not influence each other. Data collected over time, where today resembles yesterday, often breaks this.
- Constant spread. The residuals have a similar spread across the range of predictions. A fan shape is a warning sign.
- Roughly symmetric residuals. Extreme departures can make the uncertainty estimates unreliable, particularly with small samples.
- No overly influential points. One extreme record can tilt the whole line. Check whether results change when it is removed, and investigate why it is unusual before deciding anything.
A residual plot, with predictions on one axis and residuals on the other, is the simplest check. You want a shapeless cloud around zero, not a curve or a funnel.
Reading the results
Software usually reports several numbers. Understanding them in general terms is enough to start.
- Coefficients are the intercept and slopes described above.
- Standard errors, confidence intervals and p-values describe uncertainty about the coefficients. A confidence interval gives a plausible range for the true value; a small p-value suggests the data would be unusual if there were truly no relationship, but it says nothing about how large or important the effect is.
- R-squared is the share of the outcome’s variation the model accounts for, on a scale from 0 to 1. A higher value means a closer fit to the sample, not necessarily a better or more truthful model.
Common pitfalls
- Correlation is not causation. A strong association can come from a third factor or from chance. Only careful study design, not a fitted line, supports causal claims.
- Extrapolation. Predicting far outside the range of the data is risky, because the pattern may not continue.
- Overfitting. A model tuned too closely to one sample will often disappoint on new data. Testing on data that was held back is the usual safeguard.
- Small samples. With few observations, estimates can swing widely.
- Treating predictions as certainties. A prediction is an estimate with a range around it. Reporting only the single number hides that.
From simple models to machine learning
Regression is the base for many more flexible methods. Machine learning builds on similar ideas and is aimed mostly at prediction, often at the cost of easy interpretation. Starting with a simple, transparent model gives you a baseline to compare against, and it is often good enough.
Presenting a model honestly
Show the data and the fitted line together, state the sample and time period, report uncertainty, and describe limits in plain words. A chart that shows the points around the line is more honest than a table of coefficients alone, and the right chart for the job is worth a few minutes of thought.