ML Linear Regression
Linear regression predicts a number by finding the straight line that best fits the data.
What is Linear Regression?
Linear regression is the simplest regression model. It assumes the answer changes in a straight line as a feature changes.
The line follows this formula:
Example
prediction = slope × feature + intercept- The slope says how much the prediction goes up when the feature goes up by 1
- The intercept is the prediction when the feature is 0
"Best fit" means the line that keeps the total squared distance between the line and the real points as small as possible.
One Feature: Study Hours and Marks
Example
from sklearn.linear_model import LinearRegression
# Hours of study per week and marks out of 100
hours = [[2], [4], [5], [7], [8], [10]]
marks = [45, 55, 60, 70, 74, 85]
model = LinearRegression()
model.fit(hours, marks)
print("Slope:", round(model.coef_[0], 2))
print("Intercept:", round(model.intercept_, 2))
print("Marks for 6 hours:", round(model.predict([[6]])[0], 1))Slope: 4.95 Intercept: 35.12 Marks for 6 hours: 64.8
Reading the result
- Each extra hour of study adds about 4.95 marks
- With 0 hours, the line starts at about 35 marks
- A student who studies 6 hours is predicted to get about 65
Check it by hand: 4.95 × 6 + 35.12 = 64.82.
Features must be 2D: a list of rows, even when there is only one feature. That is why we write [[2], [4], ...] and predict([[6]]), not [2, 4, ...].
Several Features
Most real problems have more than one feature. Linear regression then learns one coefficient (slope) for each feature:
Example
from sklearn.linear_model import LinearRegression
# [size in sq ft, rooms, distance to main road in km]
X = [
[600, 2, 1.0], [800, 2, 0.5], [1000, 3, 2.0],
[1200, 3, 1.0], [1500, 4, 0.5], [1800, 4, 3.0],
]
rent = [28000, 36000, 40000, 50000, 65000, 66000] # Rs per month
model = LinearRegression()
model.fit(X, rent)
for name, c in zip(["size", "rooms", "distance"], model.coef_):
print(name, round(c, 1))
print("Prediction:", round(model.predict([[1100, 3, 1.5]])[0]))size 32.8 rooms 2947.0 distance -3536.4 Prediction: 45272
The model says: each extra square foot adds about Rs 33, an extra room adds about Rs 2,900, and each km further from the main road lowers the rent by about Rs 3,500 (a negative coefficient).
With only six examples these numbers are rough. Real projects use hundreds or thousands of rows. Also, compare coefficients only when features are on the same scale.
How Good is the Line?
Two common scores for regression are:
| Score | Meaning | Best value |
|---|---|---|
| MAE (mean absolute error) | Average size of the mistakes, in the same unit as the label | 0 |
| R² (R-squared) | How much of the variation the model explains | 1 |
Example
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, r2_score
hours = [[2], [4], [5], [7], [8], [10]]
marks = [45, 55, 60, 70, 74, 85]
model = LinearRegression().fit(hours, marks)
predicted = model.predict(hours)
print("MAE:", round(mean_absolute_error(marks, predicted), 2))
print("R2:", round(r2_score(marks, predicted), 3))MAE: 0.25 R2: 0.999
The line is very close to the points: on average it is only 0.25 marks off. (Here we scored on the training data to keep it short; in a real project, score on a test set.) You will learn more scores in Evaluating Models.
When Linear Regression Works Well
- The relationship is roughly a straight line
- You want a model that is easy to explain to others
- You need a quick baseline to compare other models against
It struggles when the pattern curves (for example, rent jumps sharply in some areas) or when there are big outliers. Then models like decision trees often do better.
Exercise
Exercise
A linear regression model for flat rent has a coefficient of -3000 for "distance from main road". What does this mean?