Skills101

AdvancedPython for Data Science & Machine Learning · Lesson 16 of 21 · Updated ·

Feature Engineering — Encoding Categories, Scaling, Log Transforms, Binning and Date Features

Models only see numbers. Feature engineering is the work of turning raw columns — text categories, dates, rupee amounts, counts — into numbers that carry the right meaning. On tabular data, good features usually improve a model more than switching to a fancier algorithm.

1. Categories: one-hot or ordinal?

A model can't multiply "CSE" by a coefficient. There are two main ways to turn categories into numbers, and choosing the wrong one quietly misleads the model.

  • Ordinal encoding — map categories to 1, 2, 3… only when they have a real order (year of study, small < medium < large, grade C < B < A).
  • One-hot encoding — one 0/1 column per category, for categories with no order (branch, city, payment method).
import pandas as pd

df = pd.DataFrame({
    "name": ["Asha", "Ravi", "Neha", "Arjun"],
    "branch": ["CSE", "ECE", "CSE", "ME"],
    "year": ["2nd", "4th", "3rd", "1st"],
})
year_order = {"1st": 1, "2nd": 2, "3rd": 3, "4th": 4}       # a real order: ordinal is fine
df["year_num"] = df["year"].map(year_order)

encoded = pd.get_dummies(df, columns=["branch"], dtype=int)  # no order: one-hot
print(encoded.drop(columns="year").to_string(index=False))
 name  year_num  branch_CSE  branch_ECE  branch_ME
 Asha         2           1           0          0
 Ravi         4           0           1          0
 Neha         3           1           0          0
Arjun         1           0           0          1

One-hot encoding: a single branch column with values CSE, ECE, CSE and ME becomes three 0 or 1 columns, one per branch, with exactly one 1 in each row

Why not just map CSE = 1, ECE = 2, ME = 3? Because a linear model would then believe ME is "three times" CSE and that ECE lies "between" them. One-hot columns make no such claim.

Two practical warnings:

  • Many categories (5,000 colleges, 20,000 pin codes) explode into thousands of columns. Group rare ones into "Other", or use frequency/target encoding with care.
  • New categories at prediction time (a branch never seen in training) must be handled — scikit-learn's OneHotEncoder(handle_unknown="ignore") does this, as shown in the pipelines lesson.

2. Scaling, written by hand

Features often live on very different scales: CGPA from 0 to 10, an aptitude score from 0 to 100, coding minutes per week from 0 to 1,000. Two common fixes:

MethodFormulaResult
Min-max scaling(x − min) ÷ (max − min)values between 0 and 1
Standardisation (z-score)(x − mean) ÷ stdmean 0, standard deviation 1

The golden rule: learn the min, max, mean and std from the training data only, then apply those same numbers to test data and to new data.

import numpy as np

train_minutes = np.array([120, 300, 450, 800, 200, 650])     # coding minutes per week (training rows)
new_minutes = np.array([500, 1200])                           # two new students

lo, hi = train_minutes.min(), train_minutes.max()
mean, std = train_minutes.mean(), train_minutes.std()
print("min-max  train:", ((train_minutes - lo) / (hi - lo)).round(2))
print("min-max  new:  ", ((new_minutes - lo) / (hi - lo)).round(2))
print("z-score  new:  ", ((new_minutes - mean) / std).round(2))
min-max  train: [0.   0.26 0.49 1.   0.12 0.78]
min-max  new:   [0.56 1.59]
z-score  new:   [0.33 3.22]

Notice the new student with 1,200 minutes gets a min-max value above 1. That's correct behaviour — the training range is not a law. What would be wrong is recomputing min and max using the test data: that leaks test information into training (the leakage section of the pipelines lesson).

3. Why scaling changes the answer: nearest neighbours

k-nearest neighbours (k-NN) classifies a new point by looking at the k most similar training points, using distance. If one feature has much bigger numbers, it decides the distance almost alone.

import numpy as np

# cgpa, aptitude score (0-100), coding minutes per week
students = np.array([[9.1, 62, 300],
                     [6.2, 88, 320],
                     [8.8, 85, 420]])
names = ["Asha", "Ravi", "Sara"]
new = np.array([8.9, 86, 330])

def nearest(data, point):
    dist = np.sqrt(((data - point) ** 2).sum(axis=1))
    return names[dist.argmin()], dist.round(2)

print("raw:   ", nearest(students, new))
mean, std = students.mean(axis=0), students.std(axis=0)
print("scaled:", nearest((students - mean) / std, (new - mean) / std))
raw:    ('Ravi', array([38.42, 10.55, 90.01]))
scaled: ('Sara', array([2.15, 2.09, 1.72]))

On raw numbers, coding minutes dominate the distance and the new student looks most like Ravi; after standardising each feature, CGPA and aptitude count too and the nearest student becomes Sara

On raw numbers, Sara's 90-minute difference in coding time outweighs everything else, even though her CGPA (8.8) and aptitude (85) almost equal the new student's — so the new student (CGPA 8.9) is matched with Ravi (CGPA 6.2). After scaling, each feature counts on a comparable scale and Sara becomes the nearest neighbour. Distance-based and gradient-based models (k-NN, k-means, SVM, logistic regression with regularisation, neural networks) need scaling; decision trees do not, because they only compare values within one feature at a time.

4. Skewed numbers: the log transform

Money, follower counts and time-on-app are usually right-skewed: most values are small and a few are huge. A log transform pulls the long tail in, so a few extreme rows don't dominate:

import numpy as np

monthly_spend = np.array([120, 250, 300, 180, 90, 400, 220, 15000, 260, 30000])   # rupees
logged = np.log1p(monthly_spend)          # log(1 + x): safe when x is 0
print("raw:    mean", monthly_spend.mean(), " median", np.median(monthly_spend))
print("logged: mean", logged.mean().round(2), " median", np.median(logged).round(2))
print("back to rupees:", np.expm1(np.median(logged)).round())
raw:    mean 4682.0  median 255.0
logged: mean 6.26  median 5.54
back to rupees: 255.0

Before the transform the mean (₹4,682) is about 18 times the median (₹255) because of two big spenders; after it, the mean and median are much closer. Use np.log1p (not np.log) so zero values don't become minus infinity, and np.expm1 to convert predictions back.

5. Binning, ratios and dates

Some of the most useful features are made by combining or reshaping columns with domain knowledge:

import pandas as pd

orders = pd.DataFrame({
    "placed_at": pd.to_datetime(["2026-03-06 21:40", "2026-03-07 13:05", "2026-03-09 08:15", "2026-03-08 22:30"]),
    "problems_solved": [45, 3, 120, 60],
    "problems_tried": [50, 20, 150, 61],
    "age": [19, 23, 31, 20],
})
orders["weekday"] = orders["placed_at"].dt.day_name()
orders["is_weekend"] = orders["placed_at"].dt.dayofweek >= 5
orders["hour"] = orders["placed_at"].dt.hour
orders["solve_rate"] = (orders["problems_solved"] / orders["problems_tried"]).round(2)
orders["age_group"] = pd.cut(orders["age"], bins=[0, 21, 25, 100], labels=["≤21", "22-25", "26+"])
print(orders.drop(columns=["placed_at", "problems_solved", "problems_tried"]).to_string(index=False))
 age  weekday  is_weekend  hour  solve_rate age_group
  19   Friday       False    21        0.90       ≤21
  23 Saturday        True    13        0.15     22-25
  31   Monday       False     8        0.80       26+
  20   Sunday        True    22        0.98       ≤21
Feature typeExampleWhy it helps
Date partsweekday, hour, month, is_weekendbehaviour follows the calendar
Ratiossolved ÷ tried, fee paid ÷ fee duecompares fairly across very active and less active users
Binningage groups, CGPA bandscaptures steps, and is easy to explain
Counts / recencyorders in last 30 days, days since last loginrecent behaviour predicts the next action

A raw timestamp like 2026-03-06 21:40 is nearly useless to a model; "Friday, 21:00, weekday" is meaningful.

6. Where feature engineering goes wrong

  • Leakage features — anything known only after the outcome (e.g. "offer_letter_date" when predicting placement). Ask: would I have this value at prediction time?
  • Fitting on all data — scaling or encoding statistics must come from the training split only.
  • Hand-made features in a notebook only — the same code must run in production. Put it in a function or a scikit-learn Pipeline so training and prediction use identical steps.

Common mistakes

  • Ordinal-encoding categories that have no order (branch → 1, 2, 3).
  • Scaling with statistics computed on the whole dataset, including test rows.
  • Forgetting to scale before k-NN or k-means.
  • Using np.log on data containing zeros instead of np.log1p.
  • Feeding raw timestamps or IDs to a model as if they were numbers.

Quick recap

  • One-hot for unordered categories; ordinal mapping only for real orders.
  • Min-max → 0 to 1; z-score → mean 0, std 1; always fit on training data only.
  • Distance-based models need scaling; trees don't.
  • Log-transform right-skewed values; build date parts, ratios and bins from domain knowledge.
  • Every feature must be available at prediction time and computed the same way in production.

Practice what you learned

📝 Quiz: Feature Engineering5 questions · instant answers & explanations · +10 XP · counts towards your free certificateTake the quiz →