Synopsis:

In Weapons of Math Destruction, the author walks us through her own career in big data — first at Harvard and Columbia, then as a hedge fund quant at D.E. Shaw. She explains how she came to be disillusioned by the space after seeing how data models were part of the catalyst to the 2008 financial crisis.

Cathy O’Neil demonstrates real world examples of the ethical and moral dilemmas algorithms create for society’s marginalized and underserved communities. She shows how privilege protects certain groups of people and reinforces inequality, while the vulnerable continue to be penalized by these machines in a pernicious negative feedback loop.

Key Takeaways:

• Dangerous data models are classified by Cathy O’Neil as Weapons of Math Destruction (WMDs)
• WMDs exhibit the following characteristics:
  • Scale. They process large volumes of people without human oversight.
  • Non-iterative. Dangerous data models lack a feedback loop to correct and update the model over time.
  • Biased. WMDs use various proxies to account for the predicted outcome, which needs more precision and can be discriminatory.
  • Transparency. Data models are opaque and not transparent to the people they are profiling.

• WMDs profile individuals (and institutions) based on specific variables which can determine large milestone life events like college rankings, employment application screeners, policing and sentencing algorithms, workplace wellness programs, and the many inappropriate ways credit scores reward the rich and punish the poor.
  • Example: “In Florida, adults with clean driving records and poor credit scores paid an average of $1,552 more than the same drivers with excellent credit and a drunk driving conviction.”

Memorable Quotes:

• The result is that we criminalize poverty, believing all the while that our tools are not only scientific but fair
• We’ve seen time and again that mathematical models can sift through data to locate people who are likely to face great challenges, whether from crime, poverty, or education. It’s up to society whether to use that intelligence to reject and punish them—or to reach out to them with the resources they need.
• The math-powered applications powering the data economy were based on choices made by fallible human beings. Some of these choices were no doubt made with the best intentions. Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives. Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domains: mathematicians and computer scientist. Their verdicts, even when wrong or harmful, are beyond dispute or appeal. They tended to punish the poor and the oppressed in our society, while making the rich richer.

Yina’s Review:

Cathy O’Neil’s writing felt oversimplified given the subject matter at hand. I don’t disagree with her examples of bad data models, but I was disappointed that she did not elaborate further on the examples. Her writing felt very unfinished, lacking focus and clarity.

With her work experience, I expected something that read more in depth and pulled from better sources (example sources: NYTimes, The Guardian, Washington Post, etc.)

It reads oddly trite for such a fascinating subject matter. There were many interesting examples, none of which were looked at in great depth — it read more like a laundry list of issues with blanket high level solutions that were not actionable.

Perhaps Cathy O’Neil was targeting the general audience and I was looking for a deeper dive.

TL;DR: Good book for an introduction into the world of data and shares a strong stance on fallacies behind data models with examples to go with it. It did not fit my needs for more granular substance, unfortunately. But a good introductory book nonetheless for anyone interested in big data and data science.

Hope this review was helpful! You can check out what I’m reading on GoodReads.