Not just "privacy". I would imagine some judges will issue search or even arrest warrants based on the information ("it's science!"), which might mean in the worst case the police might roll up to your house, shoot your dog, scare your children, take you into custody (even without an arrest warrant if you're somehow deemed "resisting") or even shoot you because they got scared of how you look, you may lose your job because you cannot show up to work while you're in jail, your neighbors will think you're a hardened criminal, etc.
Even if the courts do not convict in the end, a lot of damage might already been done.
The pseudo-scientific hair analysis stuff performed by the FBI showed the dangers of "science" and "tech"[1].
People went to real prison because investigators, judges and juries overestimated the flaky results the sometimes outright negligent pseudo-science produced. I imagine some people were shoot during arrests based on that evidence, died in prison or committed suicide.
There also was the Phantom of Heilbronn here in Germany, where police looked for a master criminal and serial killer for ages (2001 to 2009), but it turned out the materials they used to do DNA swaps at crime scenes were contaminated at the factory by a worker[2]. For years nobody of the many involved in the investigations even considered questioning the DNA results.
So even if the science and tech is sound (which it really isn't in case of facial recognition yet, if ever), wrong application, common mistakes, and misunderstanding the results are real problems.
There's a lot of assumption about future behavior here.
I think there are a couple reasons to be hopeful about court systems taking a more nuanced view of this technology: 1) it cannot be denied that it is imprecise, and 2) it's pretty easy for laity to understand (at least in principle) how it works. DNA evidence is effectively magic by comparison.
I actually think the imprecision is an asset for this technology. I would much rather this tool be 95% reliable than 99.99% reliable. The former inherently requires law enforcement to work harder; the later tempts "oh the machine said so it must be right".
People went to prison over expert testimony from FBI scientists over hair analysis in the past (not just warrants, actual convictions).
Like this guy, who spent 23 years in prison because FBI told the jury some hair they found was his... while it was actually not even human but dog hair.
>An FBI analyst testified that one of the hairs from the stocking mask linked Tribble to the crime and “matched in all microscopic characteristics.”
>Tribble’s attorneys were successful in obtaining mitochondrial DNA testing on the 13 hairs recovered from the stocking mask. None of the hairs—including the alleged match—implicated Tribble or Wright. Further, the analysis revealed FBI analysts’ errors, including mistakenly calling a dog hair human.
https://www.innocenceproject.org/cases/santae-tribble/
As for face recognition, I wonder if you're correct. I'd expect a lot of people think it's very precise. Well, at least the "My Sibling can unlock my iPhone with their face" debacle might have generated some press to combat that misconception.
Even if the courts do not convict in the end, a lot of damage might already been done.
The pseudo-scientific hair analysis stuff performed by the FBI showed the dangers of "science" and "tech"[1]. People went to real prison because investigators, judges and juries overestimated the flaky results the sometimes outright negligent pseudo-science produced. I imagine some people were shoot during arrests based on that evidence, died in prison or committed suicide.
There also was the Phantom of Heilbronn here in Germany, where police looked for a master criminal and serial killer for ages (2001 to 2009), but it turned out the materials they used to do DNA swaps at crime scenes were contaminated at the factory by a worker[2]. For years nobody of the many involved in the investigations even considered questioning the DNA results.
So even if the science and tech is sound (which it really isn't in case of facial recognition yet, if ever), wrong application, common mistakes, and misunderstanding the results are real problems.
[1] https://www.fbi.gov/news/pressrel/press-releases/fbi-testimo...
[2] https://en.wikipedia.org/wiki/Phantom_of_Heilbronn