Topline
Calls for policy changes and warnings of a possible extinction-level takeover of essential systems have the public’s fears about an artificial intelligence doomsday at an all-time high, but experts say marginalized communities are already suffering from AI biases that can deny healthcare, influence housing allotments or automate criminal sentencing.
A street protest mural referencing OpenAI and Google’s artificial intelligence technology in San Francisco.
Gado via Getty Images
Key Facts
Joy Buolamwini, founder of the Algorithmic Justice League, argues that centering AI risks around “existential threats” is inherently elitist and shifts focus away from the immediate harms already being inflicted on poorer communities.
Buolamwini’s research centers on what she calls the “coded gaze,” a now-proven idea that biases and the preferences of a system’s creators are encoded directly into software and, because the datasets used to train AI have historically skewed toward white men, so do the models now themselves.
A study called “Gender Shades” found that major facial recognition tools had error rates under 1% for light-skinned men, but up to 35% to 47% for dark-skinned women, and these technical failures have led to wrongful arrests of Black men across the country.
Algorithms used by property management firms to evaluate prospective renters have been found to reject qualified low-income applicants without explanation or recourse.
One tenant-screening company, SafeRent Solutions, was forced to settle a lawsuit for $2.3 million after more than 400 plaintiffs alleged its algorithm was disproportionately assigning lower scores to Black and Hispanic applicants and renters using government housing vouchers.
A landmark study published in Science revealed a commercial healthcare algorithm has been prioritizing white patients for extra medical care over Black patients who were equally sick because it used past healthcare spending as a proxy for healthcare needs. Because less money is historically spent on Black patients due to systemic inequalities, the system concluded Black patients were “healthier.”
CRUCIAL QUOTE
“AI is good at describing the world as it is today with all of its biases, but it does not know how the world should be,” Joanne Chen, a partner at AI investment firm Foundation Capital, said at SXSW in 2018.
SURPRISING FACT
Amazon scrapped an internal AI recruitment engine after realizing it systematically downgraded applications that included the word “women’s” and downgraded graduates from all-women’s colleges. The AI taught itself that male candidates were preferred because it was trained on 10 years of historical hiring data from a male-dominated tech industry.
Key background
The CEOs of top AI companies are calling for a slowdown on the development of frontier models to allow safety and legal regulators to catch up after a series of dire warnings from industry insiders. Evan Hubinger, Anthropic’s alignment science lead, last week said he thinks there is a greater than 10% chance of human extinction from unaligned AI within the next 10 years and Jacob Coxon, a former Anthropic and OpenAI researcher, said the developers building these tools “earnestly believe that it could kill us all by the end of the decade.” A handful of other industry workers have issued similar warnings and more than 1,300 employees of frontier AI companies have signed on to an initiative called “Pacing the Frontier,” aimed at allowing developers’ understanding to catch up to the software that is on a path to self improvement.
CONTRA
Some say the amplified concerns about an AI doomsday are a predictable example of cyclical “technology panic” and others say it’s meant to distract the public and lawmakers from the real-world harm AI is already causing. Timnit Gebru, a computer scientist who left Google after going public about the dangers of AI, said a doomsday prediction overshadows the “really existential” threats of AI, including mass layoffs being blamed on the technology and an already relevant climate catastrophe made worse by massive data centers to power AI. Margaret Mitchell, also formerly of Google, has said focusing on existential threat scenarios focuses lawmakers on writing new regulations for the industry and keeps them from enforcing existing laws against labor exploitation, copyright theft, environmental degradation and algorithmic bias. Others argue that society has a long history of viewing transformative tools as existential threats and billionaire Marc Andreessen has compared AI to past technological evolutions—like fire—and says it will amplify productivity rather than replace human agency.
further reading
Leave a comment