ML Data and Features

A model is only as good as the data you give it, so preparing data is the most important step in machine learning.

Features and Labels

Machine learning data is usually a table. Each row is one example. Each column is one piece of information.

• Features are the columns the model uses as input. In code we call them X (capital, because it is a table).
• The label (or target) is the column we want to predict. We call it y.

Here is a small table of flats for rent in Karachi:

import pandas as pd

data = pd.DataFrame({
    "area": ["Gulshan", "Orangi", "DHA", "Orangi", "Gulshan"],
    "rooms": [3, 2, 4, 3, 2],
    "size_sqft": [1200, 700, 2000, 900, 850],
    "rent": [60000, 25000, 150000, 30000, 45000],
})

X = data[["area", "rooms", "size_sqft"]]   # features
y = data["rent"]                            # label (target)

print(X.shape)
print(y.tolist())

(5, 3)
[60000, 25000, 150000, 30000, 45000]

X.shape is (5, 3): 5 rows (flats) and 3 features.

Choosing Good Features

A good feature is information that really affects the answer, and that you will know at prediction time.

Handling Missing Values

Real data often has empty cells. Most scikit-learn models cannot work with them, so you must fix them first. A common choice is to fill them with the median (middle value) of the column:

import pandas as pd

data = pd.DataFrame({
    "rooms": [3, 2, None, 4],
    "size_sqft": [1200, 700, 900, None],
})

print(data.isna().sum())
data = data.fillna(data.median())
print(data)

rooms        1
size_sqft    1
dtype: int64
   rooms  size_sqft
0    3.0     1200.0
1    2.0      700.0
2    3.0      900.0
3    4.0      900.0

Other options are to remove rows with missing values (if there are only a few) or use scikit-learn's SimpleImputer, which does the same job inside a pipeline.

Turning Text into Numbers

Models work with numbers, not words. A column like area must be converted. One-hot encoding makes one new column per category, with 1 or 0:

import pandas as pd

data = pd.DataFrame({
    "area": ["Gulshan", "Orangi", "DHA"],
    "rooms": [3, 2, 4],
})

encoded = pd.get_dummies(data, columns=["area"], dtype=int)
print(encoded)

   rooms  area_DHA  area_Gulshan  area_Orangi
0      3         0             1            0
1      2         0             0            1
2      4         1             0            0

Why not just write DHA = 1, Gulshan = 2, Orangi = 3? Because the model would think Orangi is "bigger" than DHA, which makes no sense. One-hot encoding avoids this false order.

Scaling Numbers

Features can have very different ranges. Rooms go from 1 to 5, but size goes into thousands. Some models (like K-Nearest Neighbors and Logistic Regression) work better when all features are on a similar scale.

StandardScaler changes each column so its average is 0 and most values fall between about -2 and 2:

from sklearn.preprocessing import StandardScaler

X = [[2, 700], [3, 1200], [4, 2000]]   # rooms, size in sq ft

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled.round(2))

[[-1.22 -1.12]
 [ 0.   -0.19]
 [ 1.22  1.31]]

Decision trees do not need scaling, but it rarely hurts.

A Data Checklist

1. Look at the data first: data.head(), data.info(), data.describe()
2. Remove duplicate rows and obvious mistakes (a flat of 5 square feet)
3. Fill or remove missing values
4. Encode text columns as numbers
5. Scale numeric features if your model needs it
6. Separate features X from the label y

Exercise