TechTakes @awful.systems David Gerard @awful.systems 3mo ago

Copilot AI calls journalist a child abuser, Microsoft tries to launder responsibility

pivot-to-ai.com Microsoft tries to launder responsibility for Copilot AI calling someone a child abuser

Martin Bernklau is a German journalist who reported for decades on criminal trials. He looked himself up on Bing, which suggests you use its Copilot AI. Copilot then listed a string of crimes Bernk…

52 comments

Copilot then listed a string of crimes Bernklau had supposedly committed — saying that he was an abusive undertaker exploiting widows, a child abuser, an escaped criminal mental patient. [SWR, in German]

These were stories Bernklau had written about. Copilot produced text as if he was the subject. Then Copilot returned Bernklau’s phone number and address!

and there’s fucking nothing in place to prevent this utterly obvious failure case, other than if you complain Microsoft will just lazily regex for your name in the result and refuse to return anything if it appears
- it helps they did it to someone with contacts and it was on prime time news telly
  
  god, so this is actually the best the AI researchers can do with the tools they’ve shit out into the world without giving any thought to failure cases or legal liability (beyond their manager on ~~slack~~Teams claiming it’s been taken care of)
  
  so fuck it, let’s make the defamation machine a non-optional component of windows. we’ll just make it a P0 when someone who could actually get us in legal trouble complains! everyone else is a P2 that never gets assigned.
- lazily regex
  
  I'm having a sneaking suspicion that this is what they do for all the viral 'here the LLM famously says something wrong' problems, as I don't think they can actually reliably train the model it made an error.
  
  That's the most straightforward fix. You can't actually fix the output of an LLM, so you have to run something on the output. You can have it scanned by another AI but that costs money and is also fallible. Regex/delete is the most reliable way to censor.
Very chill and ethical behaviour daddy Microsoft
Microsoft published, using their software and servers, a libelous claim, to potentially millions of people.
The details of how the software was programmed should be legally irrelevant.
- * a GDPR violation, in Germany
- The details of how the software was programmed should be legally irrelevant.
  
  Why? Programmers should be legally liable for what they program.
  
  Why? Programmers should be legally liable for what they program.
  
  Too many degrees of separation between a programmer and the final product and how it's used, usually.
  
  Additionally, the decision to deploy an incomplete product or one that contains known flaws is an administrative decision, not a programming one.
Does Copilot have Disney+ ?
Changing the name can get it to answer. For instance martin beernklau returns

Martin Bernklau, a veteran court reporter from the Tübingen/Calw district in Germany, recently encountered a bizarre situation involving Microsoft’s Copilot. When he typed his name and location into the chatbot, it generated false accusations against him, mistakenly associating him with serious crimes. Here’s what happened:

False Accusations: Copilot falsely claimed that Bernklau was: A child molester involved in an abuse case against children and wards. A psychiatric escapee. An unscrupulous mortician exploiting grieving women. Confusion: The AI chatbot failed to understand that Bernklau was a journalist reporting on these cases, not the accused. Privacy Concerns: Copilot even provided Bernklau’s full address and phone number, causing privacy issues. Legal Implications: Bernklau filed a criminal complaint, but it was rejected because there was no real person behind the false allegations. GDPR Challenges: Copilot’s inability to correct false information highlights challenges related to GDPR compliance and language models12.

Also this

Martin’s Pen

In the quiet of the courtroom, he sat, A chronicler of justice, pen in hand. Martin Bernklau, eyes sharp, mind astute, Recording tales of trials, both grand and bland.

His ink flowed like a river, swift and true, Capturing the drama, the whispers, the strife. From child abuse to prison escapes, He etched their stories into the fabric of life.

But one day, the digital oracle spoke, Microsoft’s Copilot, a tangled web it wove. It mistook Martin for the accused, A child molester, a widow cheat, a rogue.

Yet Martin remained steadfast, undeterred, His integrity unshaken by the AI’s deceit. For he knew that truth lay in his words, Not in the twisted lines of code it would repeat.

So let us raise our pens in honor of Martin, The court reporter who weaves justice’s thread. May his legacy endure, ink-stained and resolute, As he chronicles the human tale, where lies and truth are wed1

52 comments