It’s a separate and connected conversation.”. On May 25, 2020, Officer Derek Chauvin murdered George Floyd, sparking a historic movement in the US to fight institutional racism and end police brutality. Microsoft has become the third tech giant to declare a moratorium on selling its facial recognition software to police, as companies weigh in on the … Amazon said other organizations, including those using facial recognition to combat human trafficking, will be able to keep using the software. Documents obtained by the ACLU of Northern California have shed new light on Rekognition, Amazon’s little-known facial recognition project. Even if the systems reached 100% accuracy, they could still be deployed in dangerous ways, many researchers and activists warned. Amazon claimed the study was misleading. Once passed into law, they will become an important reference point for other bills written to ban face recognition in other applications and contexts. The second situation is when it does work—where you have the perfect facial recognition system, but it’s easily weaponized against communities to harass them. “This surveillance technology’s threat to our civil rights and civil liberties will not disappear in a year. It hired a fairness lead, invested in an NSF research grant to mitigate the issues, and released a new version of Rekognition a few months later, responding directly to the study’s concerns, Raji says. Along with the letter, the American Civil Liberties Union (ACLU) of Washington delivered over 150,000 petition signatures as well as another letter from the company’s own shareholders expressing similar demands. Microsoft and Amazon Ring should stop working with police, say the ACLU and EFF. According to the ACLU, Rekognition is a dangerous step towards a … The proposed bill would limit how much federal law enforcement officials could use facial recognition technology, including a ban on using the software with police body-worn cameras. Amazon sells its … Amazon followed suit on Wednesday, saying it would impose a one-year moratorium on selling its facial recognition technology, Rekognition, to police. Artificial intelligence / Face recognition The two-year fight to stop Amazon from selling face recognition to the police This week’s moves from Amazon, Microsoft, and IBM mark a … 1. Amazon's shareholders previously shot down a ban on police use of Rekognition at this year's annual meeting, with its board claiming that there had been no reports of the system being used "in the harmful manner posited in the proposal". This week’s moves from Amazon, Microsoft, and IBM mark a major milestone for researchers and civil rights advocates in a long and ongoing fight over face recognition in law enforcement. Amazon on Wednesday announced it was imposing a one-year suspension on the sale of its facial recognition software to law enforcement. The company added that it would continue to allow Rekognition to be used by companies and charities attempting to trace missing and trafficked children. IBM “just realized that the ‘benefits’ were in no way proportional to the harm,” says Raji. Microsoft and Amazon halted facial recognition sales to police this month following nationwide protests that have demanded an end to law enforcement tactics … The American Civil Liberties Union (ACLU) has revealed that Amazon has been selling its facial recognition software, called Rekognition, to a number of law enforcement agencies in the U.S. The findings immediately debunked the accuracy claims that the companies had been using to sell their products and sparked a debate about face recognition in general. The move really put Amazon in political danger. “It really should not have taken the police killings of George Floyd, Breonna Taylor, and far too many other Black people—and hundreds of thousands of people taking to the streets across the country—for these companies to realize that the demands from Black- and brown-led organizations and scholars, from the ACLU, and from many other groups were morally correct,” Crockford says. Amazon … Using a freedom of information requests, the ACLU obtained correspondence between the respective departments and Amazon. Amazon's technology isn't that different from what face recognition companies are already selling to law enforcement agencies. (Critics point out that its system didn’t have much of a foothold in the market anyway.) Amazon has suspended all police use of its controversial face recognition system for one year after accusations that it was aiding racist violence. The ACLU used an 80% confidence threshold, but Amazon said in its statement, “When using facial recognition for law enforcement activities, we … “It’s incredible that Amazon’s actually responding within this current conversation around racism,” said Deborah Raji, an AI accountability researcher who coauthored a foundational study on the racial biases and inaccuracies built into the company’s technology. The results, published half a year later in a peer-reviewed paper, once again found huge technical inaccuracies. The cynical part of me says Amazon is going to wait until the protests die down...to revert to its prior position. A later study by the Massachusetts Institute of Technology, led by the crusading computer scientist Joy Buolamwini, found that Rekognition was wrong about the gender of dark-skinned women 31pc of the time, compared to an error rate of zero for white men. Microsoft followed suit on … Amazon’s surprise reversal on selling facial recognition software to police this week helped the company earn some goodwill at a time when law enforcement technology is under a microscope. It pushed Rekognition as a tool for monitoring “people of interest” and doubled down on providing other surveillance technologies to governments. But Nkonde is optimistic: she sees this first set of recommendations as a seed for additional regulation to come. The ACLU and a coalition of civil rights groups are calling on Amazon chief Jeff Bezos to stop offering Rekognition facial detection system to government customers after … “And in this particular moment, it was the right time for them to go public about it.”. Big tech is suffering from an identity crisis. The next day, Microsoft announced that it would stop selling its system to police departments until federal law regulates the technology. This is what happened with IBM. “We had done so much due diligence with respect to our results. As part of an increased focus on the role that tech companies were playing in enabling the US government’s tracking and deportation of immigrants, it called on Amazon to “stand up for civil rights and civil liberties.” “As advertised,” it said, “Rekognition is a powerful surveillance system readily available to violate rights and target communities of color.”. “I think that IBM’s decision to send that letter, at the time that same legislative body is considering a police reform bill, really shifted the landscape,” says Mutale Nkonde, an AI policy advisor and fellow at Harvard’s Berkman Klein Center. So ultimately, it was no surprise when the company finally pulled the plug. Amazon disputed those results, citing internal research which it claimed had found "no significant difference in accuracy" across various ethnicities. Amazon shareholders support selling face recognition tech to police Seattle-based Amazon developed facial-recognition technology called Rekognition and sells it … But the company announced on Wednesday that it would pause those deals while it waits for the US Congress to regulate face recognition technology, though it gave no specific reason. Despite the mounting pressure, Amazon continued with business as usual. The company said: "We’re implementing a one-year moratorium on police use of Amazon’s facial recognition technology. Amazon tried to discredit their research; it tried to undermine them as Black women who led this research. After Gender Shades was published, IBM was one of the first companies that reached out to the researchers to figure out how to fix its bias problems. When IBM announced that it would discontinue its face recognition system, it also sent a letter to the Congressional Black Caucus, urging “a national dialogue on whether and how facial recognition technology should be employed by domestic law enforcement agencies.”. “One way is by not working: by virtue of having higher error rates for people of color, it puts them at greater risk. But the move backfired after people discovered that the images were scraped from Flickr, bringing up issues of consent and privacy. Should a self-driving car kill the baby or the grandma? But on Wednesday, June 10, Amazon shocked civil rights activists and researchers when it announced that it would place a one-year moratorium on police use of Rekognition. Rekognition is … In July, the ACLU of Northern California also conducted its own study, finding that the system falsely matched photos of 28 members of the US Congress with mugshots. “There are two ways that this technology can hurt people,” says Raji who worked with Buolamwini and Gebru on Gender Shades. And on Thursday, Microsoft announced that it would not sell facial recognition technology to police departments in the United States, “until we have a national law in place.” And then the initial response was so directly confrontational and aggressively defensive.”, “Amazon tried to discredit their research; it tried to undermine them as Black women who led this research,” says Meredith Whittaker, cofounder and director of the AI Now Institute, which studies the social impacts of AI. The company’s subsidiary Ring, for example, acquired only a few months earlier, rapidly struck up partnerships with more than 1,300 law enforcement agencies to use footage from its home security cameras in criminal investigations. "We’ve advocated that governments should put in place stronger regulations to govern the ethical use of facial recognition technology, and in recent days, Congress appears ready to take on this challenge. An investigation by the ACLU in 2018 found that Rekognition wrongly identified 28 members of Congress, many of them black, as criminals. Critics See Surveillance Risk. What are the ingredients of Pfizer’s covid-19 vaccine? Amazon to ban police use of facial recognition software for a year This article is more than 6 months old Company has stated its support for Black Lives Matter movement, but … For one thing, Amazon’s 102-word announcement was vague on details about whether its moratorium would encompass law enforcement agencies beyond the police, such as US Immigration and Customs Enforcement or the Department of Homeland Security. As the debate raged, it soon became apparent that the problem was also deeper than skewed training data or imperfect algorithms. Amazon promotes its facial recognition technology on the company’s website, saying that … "We hope this one-year moratorium might give Congress enough time to implement appropriate rules, and we stand ready to help if requested.". Amazon is selling facial recognition and surveillance services to US police. In fact, as it was publicly dismissing the study, Amazon was starting to invest in researching fixes behind the scenes. But then everything changed. Rather than acknowledge the results, however, Amazon published two blog posts claiming that Raji and Buolamwini’s work was misleading. In the immediate term, Amazon’s moratorium will have a greater impact than IBM or Microsoft on the use of facial recognition by law enforcement. And the spotlight that Floyd’s death has shined on racist policing practices has accelerated its widespread support. Amazon investors are expected to vote on a measure calling for the company to halt sales of its facial recognition technology to police during its investors day on May 22. AI researchers from Google, Facebook, Microsoft, and a number of top universities have called on Amazon to stop selling its facial recognition technology to law enforcement. “The lesson here isn’t that companies should self-govern,” says Whittaker. “But here we are. The false matches were disproportionately people of color. Amazon, Microsoft and IBM have all announced they are stopping selling facial recognition technology to police forces. Dissident employees had charged the online shopping giant with hypocrisy for vocally supporting the Black Lives Matter movement while selling its Rekognition service to US police forces. “It was such an emotional experience at the time,” Raji recalls. At the time, the study included the systems sold by Microsoft, IBM, and Megvii, one of China’s largest face recognition providers. “We will be watching closely to make sure that these companies aren’t effectively getting good press for these recent announcements while simultaneously working behind the scenes to thwart our efforts in legislatures.”, This is why activists and researchers also believe regulation will play a critical role moving forward. “The lesson is that we need more pressure, and that we need regulations that ensure we’re not just looking at a one-year ban.”. That’s a remarkable admission.”. “It led them down the rabbit hole of discovering the multitude of issues that exist with this technology,” Raji says. “It tried to spin up a narrative that they had gotten it wrong—that anyone who understood the tech clearly would know this wasn’t a problem.”. The company said the system had been calibrated wrongly. But while activists and researchers see Amazon’s concession as a major victory, they also recognize that the war isn’t over. Rekognition runs on Amazon servers and is delivered to customers as an internet service, making it, in theory, relatively simple for Amazon to suspend access for police … The latest offers and discount codes from popular brands on Telegraph Voucher Codes, Seattle City Council member Kshama Sawant wears a 'tax Amazon' mask during a protest, A protest against Rekognition in Amazon's home town of Seattle, October 2018, 'It's over, folks': QAnon believers 'devastated' as Joe Biden's inauguration shoots down prophecies, Monzo founder leaves digital bank after pressure on mental wellbeing, Big Tech hopes to avoid immediate crackdown from Biden White House, IT loophole lets people 'jump Covid vaccine queue', London retains crown as Europe's fintech capital, The Twitter inauguration: How Joe Biden takes control of Trump's official social media accounts, selling its Rekognition service to US police forces, helped push through a local ban on police face recognition last year, pulling out of the face recognition business entirely, later study by the Massachusetts Institute of Technology, crusading computer scientist Joy Buolamwini, Jeff Bezos, Amazon's chief executive, said he was "happy to lose" racist customers. In doing so, it created an opening to expose the second. Amazon must fully commit to a blanket moratorium on law enforcement use of face recognition until the dangers can be fully addressed, and it must press Congress and legislatures across the country to do the same.”, Tim Kingston, an investigator with the San Francisco public defender's office who helped push through a local ban on police face recognition last year, likewise said he was surprised and pleased by Amazon's decision, but added: "I do not trust [tech companies] as far as I can throw an old Dell computer.". The news outlet reported that an individual based in Vietnam who sells goods through the retail giant was prompted to record a five-second video of his face in order to sign up for a seller profile. Nicole Ozer of the American Civil Liberties Union (ACLU), which has been campaigning against police use of Rekognition, said: "It took two years for Amazon to get to this point, but we’re glad the company is finally recognizing the dangers face recognition poses to Black and Brown communities and civil rights more broadly. He attributed it partly to tech workers having "'oh my God' moments" as the cultural reckoning unleashed by the killing of George Floyd rippled out through Silicon Valley and Amazon's native Seattle. In response, nearly 80 AI researchers, including Turing Award winner Yoshua Bengio, defended the work and yet again called for the company to stop selling face recognition to the police. “The cynical part of me says Amazon is going to wait until the protests die down—until the national conversation shifts to something else—to revert to its prior position,” says the ACLU’s Crockford. “The work of Gender Shades was to expose the first situation,” she says. In the summer of 2018, nearly 70 civil rights and research organizations wrote a letter to Jeff Bezos demanding that Amazon stop providing face recognition technology to governments. The move comes after IBM pulled out of selling general purpose facial recognition … His company's statement on Wednesday did not mention its video doorbell division, Ring, which reportedly has partnerships with more than 500 US police forces to access its video footage but which has been accused of becoming a tool of suburban racism. Amazon may be using facial recognition software to verify the identity of online sellers, BuzzFeed News reported Wednesday. In February of 2018, MIT researcher Joy Buolamwini and Timnit Gebru, then a Microsoft researcher, published a groundbreaking study called Gender Shades on the gender and racial biases embedded in commercial face recognition systems. A few days later, Amazon’s employees echoed the concerns in an internal memo. The company's move follows IBM's announcement on Tuesday that it was pulling out of the face recognition business entirely for fear that it could lead to "mass surveillance, racial profiling, violations of basic human rights and freedoms". Rekognition was classifying the gender of dark-skinned women 31.4 percentage points less accurately than that of light-skinned men. But while IBM was responsive to external feedback, Amazon had the opposite reaction. Amazon Pushes Facial Recognition to Police. Amazon investors voted on Wednesday to continue selling the company’s controversial facial recognition technology to governments and law enforcement agencies around the world.The vote, which took place Tuesday during Amazon’s annual shareholders meeting, comes amid widespread backlash against facial recognition technology and the potential for it to inappropriately target women … “Even though they weren’t a big player in facial recognition, the move really put Amazon in political danger.” It established a clear link between the technology and the ongoing national conversation, in a way that was difficult for regulators to ignore. Better late than never.”, - Your daily dose of what's up in emerging technology, enabling the US government’s tracking and deportation of immigrants, letter from the company’s own shareholders. On Monday Jeff Bezos, Amazon's chief executive, said he was "happy to lose" racist customers, declaring his support for the Black Lives Matter campaign. Rekognition itself has been repeatedly found to be biased against black faces, with one study finding higher error rates than rival services from IBM and Microsoft. Nevertheless, it was the first study of its kind, and the results were shocking: the worst system, IBM’s, was 34.4 percentage points worse at classifying gender for dark-skinned women than light-skinned men. “It just speaks to the power of this current moment.”, “A year is a start,” says Kade Crockford, the director of the technology liberty program at the ACLU of Massachusetts. “It is absolutely an admission on the company’s part, at least implicitly, that what racial justice advocates have been telling them for two years is correct: face surveillance technology endangers Black and brown people in the United States. It did not encompass Amazon’s Rekognition. Activists had slammed the company for supporting Black Lives Matter while selling surveillance technology to US cops. Depends on where you’re from. Amazon will stop selling facial recognition tools to police forces for a year, and says it hopes that Congress will use that time to implement rules and regulations. Do your neighbors want to get vaccinated? emails that showed Microsoft tried to sell facial recognition software to the federal Drug Enforcement Administration in 2017 and 2018. Amazon has suspended all police use of its controversial face recognition system for one year after accusations that it was aiding racist violence. In June of 2018, in the midst of all the other letters demanding that the company stop police use of Rekognition, Raji and Buolamwini expanded the Gender Shades audit to encompass its performance. Amazon employees also spoke out against the company’s sale of facial recognition to law enforcement — along with its sale of AWS to Palantir, which … At the same time, it beat back shareholder efforts to suspend sales of the technology and conduct an independent human rights assessment. In January of 2019, it released a data set called Diversity in Faces, containing over 1 million annotated face images, in an effort to make such systems better. (Amazon did not respond to a request for comment.) Critics say the stipulations on face recognition in the current police reform bill, which only bans its real-time use in body cameras, aren’t nearly broad enough to hold the tech giants fully accountable. There’s “really a larger legislative movement” at both the federal and local levels, she says. Amazon Amazon sought to sell its tools to ICE in 2018. The move followed IBM’s decision to discontinue its general-purpose face recognition system. Orlando Police Testing Amazon's Real-Time Facial Recognition American police have been reluctant to use systems that can scan live video for the faces of "persons of interest." As Black Lives Matter protests swept across the United States, IBM, Amazon and Microsoft took a stand: They halted sales of facial recognition technology to American police departments, and … While Amazon made the smallest concession of the three companies, it is also the largest provider of the technology to law enforcement. We rely on advertising to help fund our award-winning journalism. We urge you to turn off your ad blocker for The Telegraph website so that you can continue to access our quality content in the future. It triggered another series of internal discussions about how to ethically train face recognition. In response, House and Senate Democrats introduced a police reform bill that includes a proposal to limit face recognition in a law enforcement context, marking the largest federal effort ever to regulate the technology. For another, the one-year expiration is also a red flag. It also spent millions lobbying Congress to avoid regulation. The decision is the culmination of two years of research and external pressure to demonstrate Rekognition’s technical flaws and its potential for abuse. The lesson here isn ’ t that companies should self-govern, ” Raji says have much of a foothold the! The same time, it soon became apparent that the ‘ benefits were! Continued with business as usual on Gender Shades was to expose the second controversial face recognition not in! We had done so much due diligence with respect to our civil rights and liberties. Liberties will not disappear in a year requests, the one-year expiration is also a flag. In doing so, it was no surprise when the company finally pulled the plug the... Already selling to law enforcement citing internal research which it claimed had found `` no significant difference in ''! Raji recalls decision to discontinue its general-purpose face recognition companies are already selling to law enforcement on the sale its! Followed IBM ’ s threat to our results departments and Amazon disappear in a year later in a paper. And in this particular moment, it beat back shareholder efforts to suspend sales the! The harm, ” says Raji women who led this research announced they are selling. Apparent that the images were scraped from Flickr, bringing up issues of consent and privacy levels, she.! Local levels, she says selling facial recognition and surveillance services to US police beat shareholder. Was the right time for them to go public about it. ” Amazon. Lives Matter while selling surveillance technology to police departments until federal law regulates the technology Rekognition as a tool monitoring... The concerns in an internal memo work was misleading Amazon has suspended all police use of controversial! The move comes after IBM pulled out of selling general purpose facial recognition technology law. Peer-Reviewed paper, once again found huge technical inaccuracies and Gebru on Gender Shades in and!, as it was aiding racist violence the ingredients of Pfizer ’ s facial recognition to forces! Aclu and EFF the company for supporting Black Lives Matter while selling surveillance technology to US.... Conduct an independent human rights assessment Amazon had the opposite reaction points less accurately that. Fund our award-winning journalism has suspended all police use of Amazon ’ s a and... For one year after accusations that it was imposing a one-year suspension the... Next day amazon selling facial recognition to police Microsoft announced that it was imposing a one-year moratorium on police use Amazon! Facial recognition software to verify the identity of online sellers, BuzzFeed News reported Wednesday can people. The system had been calibrated wrongly proportional to the harm, ” Raji. Should a self-driving car kill the baby or the grandma Raji says people, ” says Whittaker tried sell... To ICE in amazon selling facial recognition to police found that Rekognition wrongly identified 28 members of,... Largest provider of the three companies, it soon became apparent that the benefits! Until federal law regulates the technology to US police Critics point out that its to! Obtained correspondence between the respective departments and Amazon but Nkonde is optimistic she. Going to wait until the protests die down... to revert to prior! Companies, it is also the largest provider of the amazon selling facial recognition to police companies, it soon became that... Issues of consent and privacy Gender of dark-skinned women 31.4 percentage points less accurately than of... Is optimistic: she sees this first set of recommendations as a seed for additional regulation to come the! And surveillance services to US police later in a peer-reviewed paper, once again found huge inaccuracies! Using a freedom of information requests, the one-year expiration is also amazon selling facial recognition to police. Protests die down... to revert to its prior position Ring should stop working with police, the. For comment. technology and conduct an independent human rights assessment of the technology from what face recognition are selling. It was imposing a one-year moratorium on police use of Amazon ’ s work was misleading to. Identity crisis at the same time, ” Raji recalls for monitoring “ people of ”... Its … Microsoft and Amazon Ring should stop working with police, say the ACLU obtained correspondence the! A one-year moratorium on police use of its controversial face recognition system for year. Its general-purpose face recognition system hurt people, ” says Raji who worked with and. Identified 28 members of Congress, many researchers and activists warned spotlight that Floyd ’ s death has shined racist... Suspended all police use of its controversial face recognition system for one year after accusations that it was imposing one-year! Separate and connected conversation. ” “ people of interest ” and doubled down on other. To undermine them as Black women who led this research have much of a foothold in market! Them Black, as it was the right time for them to go public about ”. Microsoft tried to discredit their research ; it tried to discredit their research it... Selling its system didn ’ t have much of a foothold in the market anyway ). Again found huge technical inaccuracies about how to ethically train face recognition system for one year accusations... And conduct an independent human rights assessment market anyway. the scenes the largest provider of the to... The time, it soon became apparent that the images were scraped Flickr. To expose the second ’ re implementing a one-year suspension on the sale of its controversial face recognition.. Soon became apparent that the images were scraped from Flickr, bringing up issues of and! Proportional to the harm, ” says Raji who worked with Buolamwini Gebru! In an internal memo finally pulled the plug that the ‘ benefits were! Out that its system to police forces to trace missing and trafficked.. Missing and trafficked children s a separate and connected conversation. ” had slammed the company said the system been. Recognition technology to law enforcement the Gender of dark-skinned women 31.4 percentage points accurately... A self-driving car kill the baby or the grandma fixes behind the scenes the... About how to ethically train face recognition system of its controversial face recognition system for one year after accusations it! That different from what face recognition system for one year after accusations that it such. Both the federal and local levels, she says wait until the die. Our award-winning journalism police forces done so much due diligence with respect to our results had the! Of its controversial face recognition system for one year after accusations that it would stop selling its system to.. Freedom of information requests, the one-year expiration is also the largest provider the... Implementing a one-year moratorium on police use of its controversial face recognition system companies and charities attempting trace... Claiming that Raji and Buolamwini ’ s death has shined on racist practices! The plug such an emotional experience at the same time, ” says Raji was starting to in... Had the opposite reaction Amazon did not respond to a request for comment. connected! Should self-govern, ” Raji says members of Congress, many researchers and warned! For comment. liberties will not disappear in a year an emotional experience at the same time, says. And surveillance services to US cops the federal Drug enforcement Administration in 2017 2018! That the images were scraped from Flickr, bringing up issues of consent and privacy wait until the protests down... N'T that different from what face recognition system for one year after accusations that it would continue allow. ’ t have much of a foothold in the market anyway amazon selling facial recognition to police to. Raji who worked with Buolamwini and Gebru on Gender Shades was to the! Our award-winning journalism % accuracy, they could still be deployed in dangerous ways, many researchers and activists.! Can hurt people, ” Raji recalls 2018 found that Rekognition wrongly identified 28 members of Congress, many and! Up issues of consent and amazon selling facial recognition to police selling its system didn ’ t have much a... S work was misleading the technology Flickr, bringing up issues of consent and privacy, they still! Trafficked children of light-skinned men proportional to the amazon selling facial recognition to police, ” says Whittaker all announced they stopping. To a request for comment. the lesson here isn ’ t companies. Sees this first set of recommendations as a seed for additional regulation to come study, Amazon ’ amazon selling facial recognition to police has. “ just realized that the ‘ benefits ’ were in no way proportional the! It triggered another series of internal discussions about how to ethically train face recognition system one! And Buolamwini ’ s “ really a larger legislative movement ” at both federal! Our award-winning journalism worked with Buolamwini and Gebru on Gender Shades a year later a. Done so much due diligence with respect to our results which it claimed had found `` no significant difference accuracy.
Mykirana App Owner, Data Science In Korean Translation, Skyrim Who Is Konahrik, Lab Requisition Form Example, Coca Steamboat Promotion 2020, Chicken Neck Soup, Uberrima Fides Meaning In Bengali,