Computers, Anxiety, and Big Data

weapons of math destruction how big data increases inequality and threatens democracy

Computers, Anxiety, and Big Data

Weapons of Math Destruction: How Big Data Increases inequality and threatens democracy was written by Cathy O’Neil recently. It discusses how some big data analytics algorithms are being used today to exacerbate existing inequality. O’ Neil claims that these techniques, which are being used by marketing firms and analytics firms, are not neutral and are not designed to reduce any social equity or freedom. This book examines how these practices threaten the future of democracy and how we can regain the trust and confidence in our institutions. Its core theme is how big data analytics threaten the future of U.S. democracy.

The premise of Weapons of Math Destruction: How Big Data Increases Inequity and Threatens Democracy is that mathematical algorithms, which have been used to analyze consumer shopping trends, are not neutral and cannot be trusted. O’ Neil states that because mathematical algorithms have been formulated to evaluate and rank consumer behavior it has been used to create and enforce policies that discriminate against certain groups of individuals. These policies have in effect created what she calls “price discrimination,” which continues to reduce economic opportunity for many people. O’ Neil argues that because these policies are not designed to discriminate, they violate the equal opportunity clause of the Fair Credit Reporting Act. She contends that these laws were intended to protect the rights of consumers, not empower special interests.

Further, algorithms that reward people who engage in good behavior will inevitably lead to resentment, which increases the likelihood that policies will not be enacted. Furthermore, these algorithms will generate false feedback loops that cause the continuation of the cycle. This means that there will be an increase in political gridlock that hinders policy change. Eventually, if this continues unchecked, the U.S. will enter a “growth recession,” which will exacerbate the already declining unemployment rate.

The above arguments have been advanced by others, but O’ Neil has provided enough detail to make her case. In Weapons of Math Destruction: How Big Data Increases Inequity and Threatens Democracy, she offers several recommendations to help counter these problems. For starters, she recommends instituting a “Congressional Accountability and Ethics oversight” program to ensure that the use of algorithmically based systems is limited. Specifically, she recommends codifying the Fair Credit Reporting Act (FCRA) so that individuals are given the right to dispute inaccurate, misleading, or outdated information on their credit reports. Similarly, she advocates for the FTC to codify rules that require Equifax, Experian, and TransUnion to provide a full disclosure of all negative information on the credit report.

These rules would make it difficult, if not impossible, for companies to use the FCRA to challenge inaccurate, misleading, or outdated information on their reports. Further, the FTC should be made aware that the use of big data to support business decisions is often not well justified. O’ Neil contends that there are far too many unanswered questions regarding the optimal formula for predicting customer demand, how banks should calculate the marginal economic value of new customers, how government agencies should regulate payday lenders, how academic researchers should evaluate new technologies, etc. While each of these scenarios is unique, she maintains that more work needs to be done to determine how the adoption of high-speed Internet and social media will affect society, technology, policies, and our economy. This book is a valuable primer on the methods used by those who seek to manipulate or understand human behavior.

The second half of the book examines three types of algorithmic discrimination. First, there is the intentional targeting of individuals who fall into a protected class based on statistical analysis. Second, there are problems caused by the “asymmetry” of our mathematical models of what causes recidivism. The third category is “feedback loops.” Algorithms are designed to solve problems, but they inadvertently create feedback cycles that lead to more discrimination. The proposed remedies to address both kinds of discrimination.

Algorithms and big data continue to be linked because they are not mutually exclusive. In a recent article in The New York Times, economists Gary Krueger and James A. Cohn noted, “One of the biggest areas where academic researchers are concerned about algorithms is mathematics…The worry is that because many algorithms give great weight to factors such as age and gender, there could be a bias in how people score the scoring game.” The article continues, “That worry has been fueled by a series of research papers and a book published last year by Mr. Cohn and Mr. Krueger as the pair were presenting at a conference on mathematical policies and practices at the American Association for Quantitative Analysis.” In a review of the book, Columbia University economics professor Luigi Zingale noted that “many of the same problems from machine learning and artificial intelligence, are also present in the financial markets, which make up a large portion of Wall Street.” Mr. Cohn and Mr. Zingale also noted, “Machines are not good enough because humans are better at some things than they are at others, and this also applies to technology.”

While the review of the book is helpful, it is too late. The problem is bigger than Professor Cohn and Mr. Krueger recognize, perhaps because it is much deeper. Techno-utopian dreaming may seem cute when it comes to public policy and social programs, but it does not go beyond that. If you want a techno utopia, the future must be based on solutions to the existing problem of extreme wealth inequality, rather than future visions of perfect machines. A society that assumes that all students are able to participate equally in society, regardless of talent or motivation is headed toward a crisis. This means that no matter how reasonable and democratic the policies and social programs are, they will always fail to improve the balance between opportunity and need.