- This chapter is basically the book gathering all its warnings into one place and saying: most data disasters are predictable.
- The strongest ideas here:
- No model gives “the truth.” Every result depends on assumptions, data, framing, and choices.
- Correlation ≠ causation. Still one of the easiest mistakes to make.
- p-hacking: if you test enough relationships, some will look significant just by chance.
- Non-representative samples produce misleading conclusions, even if the math afterward is flawless.
- Data leakage can make a useless model look amazing.
- Overfitting means memorizing the training data instead of learning patterns that generalize.
- Models only know the data they were trained on. That line is worth keeping.
- Ask a sharp question first. A vague or wrong question poisons everything downstream.
- Adapt when the original question fails. Don’t keep forcing the project just because work has already been spent.
- Data can be available but still not contain the information you need.
- Prototype cheaply first. Don’t spend a fortune on infrastructure before proving the idea has value.
- Timelines should include setbacks. Data projects almost always uncover ugly surprises.
- Don’t oversell value. Inflated expectations create backlash later.
- Sunk cost / effort bias is a major trap: once enough time and money are invested, teams keep going even when they already know the project is wrong.
- That last one is especially important because it connects technical judgment with organizational behavior.
- I’d compress the chapter into:
- A lot of data-science failure comes from refusing to stop, question, or reframe a bad project early enough.
- And one quote here is absolutely final-post material:
- No model’s predictions represent the final truth. All results are products of assumptions.
- That line is basically the whole book in miniature.