"Weapons of Math Destruction": a MUST read
Over the last few years, I have recommended a number of books that form a composite picture of the challenges and opportunities for schools in our "age of accelerations."
As compelling as these books are, there is only one book that I am adding to that list that is relevant, convincing, and...
Important.
Weapons of Math Destruction is the most important book I have read in several years. I don't make that claim lightly. It is a must read.
In fact, I consider this book so important that I would put it in the hands of every high schooler, teacher, and parent, and then initiate a sustained discussion of its ideas and implications.
Weapons of Math Destruction deals with the invisible and pernicious effects of algorithms in so many corners of our lives. O'Neil, a PhD in Math, former hedge fund quant, and now data-scientist-for-good, illustrates her ideas with compelling stories and deep, essential questions. You might think that a book about algorithms would be boring or inaccessible; in fact, O'Neil has written something exciting (and terrifying) and crystal clear.
What is a "weapon of math destruction?" A WMD is an algorithm that is:
opaque (i.e., the people affected by it have no access to the model)
scalable (i.e., it affects large numbers of people)
destructive (i.e., it can harm an individual's financial welfare, erode criminal and civil justice, reinforce structural racism... and more)
Two nights ago I had the good fortune to attend a live interview and Q&A with the author, Cathy O'Neil, at Grand Central Tech. (It was a special joy to watch Grand Central Tech co-founder and my former student Matt Harrigan conduct a masterful interview.)
Matt Harrigan, co-founder of Grand Central Tech, interviews Cathy O'Neil, author of Weapons of Math Destruction
Among O'Neil's many compelling ideas:
How do we pick "winners and losers"? One way to think of a data scientist is that s/he is someone who picks the winners and losers (e.g., Based on the data we have on you, we will offer you a discount, but that person over there gets the high interest rate option...)
What is the role of ethics in big data? We need to insist that data scientists do training in ethics. They need to think about the cost of failure in their algorithms--specifically, what ratio of false positives to false negatives are they willing to tolerate? "Until you have answered that question," O'Neil said, "you cannot build that model."
What does it mean to be an active and effective citizen in a democratic republic that relies on algorithms? As we become a more algorithmic society, citizens need to be involved in a sustained discussion of what we want to optimize for. O'Neil pointed out that "Government isn't optimized for efficiency. That's not its purpose. It's purpose is to optimize fairness for the average person."
If you take seriously the notion of a "21st century education," then I urge you to read this book and start discussing its implications.