REAL-WORLD AI INCIDENT

When AI Gets It Wrong

How a false facial-recognition match contributed to the wrongful arrest of Robert Williams — and what the case teaches us about human oversight, bias, and responsible AI.

Explore the Incident ↓

What Happened?

A facial-recognition search turned a low-quality surveillance image into an investigative lead. The consequences became very real.

Robert Williams, whose wrongful arrest followed an incorrect facial-recognition match

Robert Williams, whose wrongful arrest followed an incorrect facial-recognition match. Photo/Source: American Civil Liberties Union (ACLU)

01 — The Crime

In 2018, several watches were stolen from a Shinola store in Detroit. Investigators obtained surveillance footage showing the suspected shoplifter.

02 — The AI Match

Police used a blurry still image from the surveillance footage in a facial-recognition search. The system returned Robert Williams as a possible match.

03 — The Consequence

Williams was arrested outside his home in January 2020 and detained for approximately 30 hours before the case against him was ultimately dismissed.

Interactive Timeline

Click each event to see what happened.

2018

The Watch Theft

Surveillance footage captured a person suspected of stealing watches from a Shinola store in Detroit. Investigators later used a low-quality still image from that footage for facial recognition.
2019

Facial Recognition Search

Michigan State Police performed a facial-recognition search at the request of Detroit investigators. Robert Williams' driver's-license photograph appeared as a possible match.
2020

Wrongful Arrest

Detroit police arrested Williams outside his home in front of his family. He was detained for approximately 30 hours. Prosecutors later dropped the case because of insufficient evidence.
2021

Civil Rights Lawsuit

Williams, represented by civil-rights advocates, filed a federal lawsuit challenging the investigation and the police department's use of facial-recognition technology.
2024

Major Policy Changes

A settlement established new safeguards governing Detroit police use of facial recognition, including requirements for independent evidence before an arrest can be made based on a facial-recognition lead.

Where Did the Process Fail?

The incident demonstrates why an AI output should be treated as a lead — not as proof.

Technology Failure

The facial-recognition search began with a blurry, low-quality surveillance image. The resulting match was incorrect. Facial-recognition systems can also show differences in performance across demographic groups, making careful evaluation essential.

Human Oversight Failure

Technology alone did not arrest Robert Williams. Investigators made decisions after receiving the algorithmic lead. The case demonstrates the danger of automation bias: people can give computer-generated results more confidence than the evidence deserves.

Lessons Learned

Responsible AI requires more than an accurate algorithm.

01

AI Is Not Evidence

An algorithmic match should generate an investigative lead, not establish guilt.

02

Humans Must Verify

High-impact decisions require independent evidence and meaningful human review.

03

Quality Matters

Poor input data can produce unreliable results. A blurry surveillance image should immediately raise concerns about confidence and accuracy.

04

Bias Must Be Tested

Organizations should evaluate whether AI systems perform differently across demographic groups before deploying them in high-stakes situations.

05

Policies Need to Come First

Organizations need clear rules governing when AI can be used, how outputs are verified, and who remains accountable for the final decision.

06

Accountability Remains Human

AI can assist decision makers, but responsibility cannot simply be transferred to an algorithm.

My Perspective: How Could This Have Been Prevented?

In my opinion, the biggest failure in this incident was not simply that the facial-recognition system produced the wrong match. AI systems can make mistakes. The more serious problem was allowing an uncertain computer-generated result to have such a large influence on a decision that affected a person's freedom.


Facial recognition should only be used as an investigative starting point. Before an arrest, investigators should be required to obtain independent evidence connecting the person to the crime. Agencies should also establish minimum image quality standards, regularly test systems for demographic performance differences, document how AI was used, and train employees to question algorithmic recommendations.


AI can be extremely useful, but the more serious the consequence of a decision becomes, the more important human judgment, transparency, and verification become.

References

Sources used to verify the facts presented in this incident report.

American Civil Liberties Union. (2024). Williams v. City of Detroit.
View Source
Allyn, B. (2020, June 24). 'The computer got it wrong': How facial recognition led to false arrest of Black man. NPR.
View Source
American Civil Liberties Union of Michigan. (2024, June 28). Civil rights advocates achieve the nation's strongest police department policy on facial recognition technology.
View Source