AI Generated Police Report Claims Officer Transformed Into Frog

Law enforcement has quickly embraced AI for everything from drafting police reports to facial recognition. 

The results have been predictably dismal. In one particularly glaring – and unintentionally comedic – instance, the police department in Heber City, Utah in the US, were forced to explain why a police report software declared that an officer had somehow shapeshifted into a frog.

The flawed system seems to have picked up on some unrelated background chatter to devise its fantastical fairy tale ending. That's what happens when you give pattern recognition software authority without insisting it understand reality. 

Police sergeant Rick Keel explained: "The body cam software and the AI report writing software picked up on the movie that was playing in the background, which happened to be 'The Princess and the Frog'", referring to Disney’s 2009 musical comedy. "That's when we learned the importance of correcting these AI-generated reports". 

The department had begun testing an AI-powered software called Draft One to automatically generate police reports from body camera footage. The goal was to reduce the amount of paperwork – but considering that immense mistakes are falling through the cracks, results clearly vary. Even a simple mock traffic stop meant to demonstrate what the tool is capable of turned into a disaster. The resulting report required plenty of corrections. 

Despite the drawbacks, Keel maintained that it is saving him “six to eight hours weekly now”. 

“I’m not the most tech-savvy person, so it’s very user-friendly”, he added.

The software makes use of OpenAI’s GPT large language models to generate entire police reports from body camera audio. Experts quickly warned that hallucinations could fall through the cracks in these important documents. 

“I am concerned that automation and the ease of the technology would cause police officers to be sort of less careful with their writing”, American University law professor Andrew Ferguson told the Associated Press last year. 

Others warn that the software could further pre-existing racial and gender biases, a troubling possibility considering law enforcement’s historic role in perpetuating them long before the advent of AI. Generative AI tools have also been shown to perpetuate biases against both women and non-white people.