# Classification — Logistic Regression, Decision Trees, Confusion Matrix, Precision and Recall

Source: https://skills101.in/learn/data-science-ml/machine-learning-classification — Skills101 free tutorial (Python for Data Science & Machine Learning). Updated 2026-11-09.

Regression predicts numbers; **classification** predicts categories: placed or not, spam or not, which loan is risky. It is the most common kind of ML at work.

## Case study: early placement support

A college wants to find, in the third year, students at risk of **not** being placed, so mentors can help early. A model that flags students is useful only if it finds most at-risk students (they get help) without flagging everyone (mentors have limited time).

## 1. The data and two models

```python
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

rng = np.random.default_rng(7)
n = 400
df = pd.DataFrame({
    "cgpa": rng.normal(7.5, 0.9, n).clip(5, 10).round(2),
    "internships": rng.integers(0, 3, n),
    "projects": rng.integers(0, 5, n),
    "aptitude": rng.normal(65, 12, n).clip(20, 100).round(),
})
score = 1.4 * (df["cgpa"] - 7.5) + 0.8 * df["internships"] + 0.35 * df["projects"] + 0.05 * (df["aptitude"] - 65) + rng.normal(0, 0.8, n)
df["placed"] = (score > 0.3).astype(int)
print(df["placed"].value_counts())

X, y = df[["cgpa", "internships", "projects", "aptitude"]], df["placed"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)

logreg = LogisticRegression(max_iter=1000).fit(X_train, y_train)
tree = DecisionTreeClassifier(max_depth=3, random_state=0).fit(X_train, y_train)
print("logistic regression accuracy:", round(logreg.score(X_test, y_test), 3))
print("decision tree accuracy:      ", round(tree.score(X_test, y_test), 3))

student = pd.DataFrame([{"cgpa": 7.0, "internships": 0, "projects": 1, "aptitude": 55}])
print("chance of placement:", logreg.predict_proba(student)[0, 1].round(2))
```

```text
placed
1    290
0    110
Name: count, dtype: int64
logistic regression accuracy: 0.79
decision tree accuracy:       0.76
chance of placement: 0.12
```

- **Logistic regression** outputs a **probability** (`predict_proba`), then uses 0.5 as the default cut-off. Despite its name it's a classification model.
- A **decision tree** learns if/else questions ("cgpa ≤ 7.2? internships ≤ 0?…"). `max_depth=3` keeps it small and readable.
- `stratify=y` keeps the same placed/not-placed ratio in train and test.

## 2. The confusion matrix

![A confusion matrix has true positives, false negatives, false positives and true negatives; precision is TP over TP plus FP, recall is TP over TP plus FN](https://skills101.in/learn-img/data-science-ml/confusion.svg)

```python
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(7)
n = 400
df = pd.DataFrame({
    "cgpa": rng.normal(7.5, 0.9, n).clip(5, 10).round(2),
    "internships": rng.integers(0, 3, n),
    "projects": rng.integers(0, 5, n),
    "aptitude": rng.normal(65, 12, n).clip(20, 100).round(),
})
score = 1.4 * (df["cgpa"] - 7.5) + 0.8 * df["internships"] + 0.35 * df["projects"] + 0.05 * (df["aptitude"] - 65) + rng.normal(0, 0.8, n)
df["at_risk"] = (score <= 0.3).astype(int)          # 1 = NOT placed: the class we want to find
X, y = df[["cgpa", "internships", "projects", "aptitude"]], df["at_risk"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)

model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
pred = model.predict(X_test)
print(confusion_matrix(y_test, pred))               # rows = actual 0/1, columns = predicted 0/1
print(classification_report(y_test, pred, target_names=["placed", "at risk"], digits=2))
```

```text
[[68  5]
 [13 14]]
              precision    recall  f1-score   support

      placed       0.84      0.93      0.88        73
     at risk       0.74      0.52      0.61        27

    accuracy                           0.82       100
   macro avg       0.79      0.73      0.75       100
weighted avg       0.81      0.82      0.81       100
```

For the **at-risk** row:

- **Precision** — of the students we flagged, how many really were at risk? (Mentor time well spent.)
- **Recall** — of all truly at-risk students, how many did we flag? (Nobody left behind.)
- **F1** — one number balancing both.

## 3. Moving the threshold

```python
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(7)
n = 400
df = pd.DataFrame({
    "cgpa": rng.normal(7.5, 0.9, n).clip(5, 10).round(2),
    "internships": rng.integers(0, 3, n),
    "projects": rng.integers(0, 5, n),
    "aptitude": rng.normal(65, 12, n).clip(20, 100).round(),
})
score = 1.4 * (df["cgpa"] - 7.5) + 0.8 * df["internships"] + 0.35 * df["projects"] + 0.05 * (df["aptitude"] - 65) + rng.normal(0, 0.8, n)
df["at_risk"] = (score <= 0.3).astype(int)
X, y = df[["cgpa", "internships", "projects", "aptitude"]], df["at_risk"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
proba = LogisticRegression(max_iter=1000).fit(X_train, y_train).predict_proba(X_test)[:, 1]

for threshold in (0.7, 0.5, 0.3):
    flagged = (proba >= threshold).astype(int)
    print(f"threshold {threshold}: flagged {flagged.sum():3d}  precision {precision_score(y_test, flagged):.2f}  recall {recall_score(y_test, flagged):.2f}")
```

```text
threshold 0.7: flagged   7  precision 0.86  recall 0.22
threshold 0.5: flagged  19  precision 0.74  recall 0.52
threshold 0.3: flagged  25  precision 0.76  recall 0.70
```

Lowering the threshold flags more students and **recall rises** — from 0.22 to 0.70 here. Precision usually falls as you flag more; in this run it drops from 0.86 at 0.7 and then stays around 0.75, because the extra students flagged at 0.3 are mostly real at-risk cases. The right threshold is a business decision: for early mentoring, missing a student is worse than an extra conversation, so we'd pick a lower threshold. For fraud alerts that freeze a customer's card, false alarms are costly — a higher threshold.

## 4. The accuracy trap (imbalanced data)

If only 2% of transactions are fraud, a model that says "never fraud" is **98% accurate** — and completely useless. With rare classes, look at precision, recall and the confusion matrix, not accuracy. Options: `class_weight="balanced"`, collecting more examples of the rare class, or tuning the threshold as above.

## Common mistakes

- Judging a classifier by accuracy alone on imbalanced data.
- Forgetting `stratify=y` on small datasets.
- Deep, unlimited decision trees that memorise the training data (next lesson).
- Treating 0.5 as a law — choose the threshold from the cost of each kind of mistake.

## Quick recap

- Classification predicts categories; logistic regression gives probabilities; trees give if/else rules.
- Confusion matrix → precision (are flags right?) and recall (did we find them all?), F1 balances them.
- The threshold trades precision against recall — decide it from business costs.
- Accuracy misleads when classes are imbalanced.


Practice quiz and free certificate: https://skills101.in/learn/data-science-ml/machine-learning-classification