how big data increases inequality and threatens democracy
Paperback, 259 pages
English language
Published April 28, 2016 by Allen Lane.
A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life and threaten to rip apart our social fabric We live in the age of the algorithm. Increasingly, the decisions that affect our lives where we go to school, whether we get a loan, how much we pay for insurance are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: everyone is judged according to the same rules, and bias is eliminated. And yet, as Cathy O'Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and incontestable, even when they're wrong. Most troubling, they reinforce discrimination. Tracing the arc of a person's life, O'Neil exposes the black box models that shape our future, both as individuals and as a society. These "weapons of math destruction" score …
A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life and threaten to rip apart our social fabric We live in the age of the algorithm. Increasingly, the decisions that affect our lives where we go to school, whether we get a loan, how much we pay for insurance are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: everyone is judged according to the same rules, and bias is eliminated. And yet, as Cathy O'Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and incontestable, even when they're wrong. Most troubling, they reinforce discrimination. Tracing the arc of a person's life, O'Neil exposes the black box models that shape our future, both as individuals and as a society. These "weapons of math destruction" score teachers and students, sort CVs, grant or deny loans, evaluate workers, target voters, and monitor our health. O'Neil calls on modellers to take more responsibility for their algorithms and on policy makers to regulate their use. But in the end, it's up to us to become more savvy about the models that govern our lives. This important book empowers us to ask the tough questions, uncover the truth, and demand change.
This was an exceptional book. It's not heavy into statistics but gives the rationale for what is a WMD (Weapon of Math Destruction) and WMDs maybe a new term but we have been under the exploitation of WMDs well before we think. It's not a new phenomenon but it is one that we should be aware of.
Take a read and learn how about them so that we can all do better to combat them and use math to not only help describe the world but make it a better place to live in.
Excellent discussion on how the use of algorithms is affecting our education system, how likely are we to be hired, how much we pay for insurance and mortgages. These models have become black boxes that nobody knows exactly how they work but are considered reliable. What few people realize is that these algorithms are reinforcing discrimination and have biases built in them. So, instead of a fair objective system to evaluate whatever (loan approvals, credit scores, job candidates, school teacher's performance, etc), we have opaque models being applied everywhere that cannot be disputed or even understood. It's scary to think that our future life decisions will rely on algorithms.
I, too, was a mathematician once, but I lost my faith. Ms. O'Neil still seems to have much of hers, for an Occupier. I kept thinking she was naive that this stuff is fixable, but I may just be naively paranoid.
That algorithms can be biased was not a surprise to me. Logic itself can be biased because it is dependent on language which, like many WMDs, is a black box. It is full of proxies. Take the term "criminal" which (like "terrorist") brands the one so called as an evil doer. And it's measurable by determining if one has been convicted of anything. You can read [b:The New Jim Crow: Mass Incarceration in the Age of Colorblindness|6792458|The New Jim Crow Mass Incarceration in the Age of Colorblindness|Michelle Alexander|https://images.gr-assets.com/books/1328751532s/6792458.jpg|6996712] and discover that going to jail is part of systemic racism but when you hear the word "criminal" or "convict", you …
I, too, was a mathematician once, but I lost my faith. Ms. O'Neil still seems to have much of hers, for an Occupier. I kept thinking she was naive that this stuff is fixable, but I may just be naively paranoid.
That algorithms can be biased was not a surprise to me. Logic itself can be biased because it is dependent on language which, like many WMDs, is a black box. It is full of proxies. Take the term "criminal" which (like "terrorist") brands the one so called as an evil doer. And it's measurable by determining if one has been convicted of anything. You can read [b:The New Jim Crow: Mass Incarceration in the Age of Colorblindness|6792458|The New Jim Crow Mass Incarceration in the Age of Colorblindness|Michelle Alexander|https://images.gr-assets.com/books/1328751532s/6792458.jpg|6996712] and discover that going to jail is part of systemic racism but when you hear the word "criminal" or "convict", you usually don't think much further.
Or consider how we think about patents. It's supposed to protect inventors and most people think that's what it's doing (and perhaps much of the time it is) but it has become a weapon of big companies to keep small ones from entering a field.
This is all before mathematics enters into it.
When reasoning is done by a computer, it's faster, bigger, and less reflective. You might think the marketplace can fix this. A business that does less error-prone reasoning will make more money and prevail. Ms. O'Neil tries to explain why it doesn't work this way but I don't think she does that good a job of it. At least she tries.
The problem is that even with a sloppy algorithm, you can make up the difference on volume. In the world of actual weapons of mass destruction, you could nuke entire countries and win the war. Most of the people you killed are "innocent" except in the sense of having been born into the wrong nation. Businesses using "approximate" algorithms will succeed and success is self-validating. On an individual level we say "life is unfair" but on a corporate level the winners selected by the marketplace are seen as deserving their success if they haven't broken any laws which anyone noticed. (And if someone noticed? Google "war on whistleblowers.")
Ms. O'Neil thinks that this is just the birth throes of a new technology and in the future, we'll look back at it like we do at sweat shops and child labor, excesses that we have managed to overcome. I am less confident of our rosy future. I'm more like [a:Chris Hedges|15438|Chris Hedges|https://images.gr-assets.com/authors/1273339351p2/15438.jpg] (one of whose books I read right before this one) who sees morality not as a dimension of society that progresses like technology and science, but one which remains at pretty much a constant level. I see the technology as multiplying our moral failings in a way that will be difficult to correct. (I am reminded of those who believe that those who fear global warming are underestimating man's ability to solve problems and some yet unforeseen discovery will come along when we need it, and we can just ignore it for now.)
Currently, outside of dystopian science fiction, people see apps as a universal good and trust our cyber-overlords. Then, some may find themselves among the collateral damage, and if sufficiently hurt, will discover no one will want to listen to them. Isn't it an axiom of capitalism that, with some exceptions we are safe to ignore, the poor deserve to be poor?
