• This chapter is really about classification as decision-making under uncertainty, not just “predicting classes.”
  • The strongest ideas:
    • Logistic regression outputs probabilities, not magical class labels. The sigmoid/logistic function squeezes the linear score into the range 0–1.
    • The nice thing about logistic regression is that it gives you both:
      • a prediction,
      • and relatively interpretable coefficients.
    • Linear regression is unbounded, so it is generally unsuitable for probabilities/classes.
  • Then the chapter moves into trees and ensembles:
    • Decision trees are intuitive and useful for exploring relationships between inputs and an outcome.
    • Random forests average many trees.
    • Gradient boosted trees build trees sequentially, with later trees correcting earlier mistakes.
    • That extra predictive power usually comes with a cost: less interpretability.
  • Two highlighted lines here are especially important:
    • Critical thinking about the data and features is something software cannot do for you.
    • and:
    • Testing on the training set is one of the biggest sins in machine learning.
  • Those fit perfectly with earlier chapters: sophisticated algorithms do not rescue bad reasoning or bad validation.
  • The most important part of this chapter to me is the decision threshold:
    • A classification model usually gives you a probability. Turning that probability into a class is a human/business decision.
    • The default 0.5 threshold is not sacred.
    • If false negatives are very costly, you may lower the threshold to catch more positives.
    • If false positives are expensive, you may raise it.
  • That leads directly to the metrics point:
    • Accuracy is often a bad metric, especially with imbalanced classes.
  • You usually care more about things like:
    • Recall / sensitivity — how many true positives you caught
    • Specificity — how many true negatives you correctly rejected
    • Precision — when you predict positive, how often you are right
  • The clean takeaway I’d save is:
    • The model estimates probability; humans decide what action that probability should trigger.
    • That’s a very “Data Head” idea because it puts business cost and context above blindly accepting a model’s default settings.