Deloitte to Refund Australia Over AI Errors in Welfare Report

A report intended to crack down on welfare is now raising questions about oversight. Deloitte will partially refund the Australian government for a report marred by artificial intelligence errors.

Report Sparks Controversy

Deloitte’s Australian branch will repay part of the $290,000 fee for the flawed report. It contained AI-generated inaccuracies, including fake research papers and a fabricated quote from a judge.

The report, initially published in July on the Department of Employment and Workplace Relations website, has been revised. The updated version appeared after researcher Chris Rudge flagged the issues.

Hallucinations Uncovered

Rudge, from Sydney University, alerted media outlets to the numerous fabricated references. Deloitte reviewed the 237-page document and confirmed the presence of incorrect information.

A statement from the department noted some footnotes and references were indeed wrong. Deloitte has since included a disclosure that Azure OpenAI, a generative AI system, was used in creating the report.

Corrections and Revisions

The revised report omits the fake judge quote and nonexistent expert reports. Deloitte stated the updates don't affect the report's main content or recommendations.

The Australian Financial Review first reported on the errors in late August, crediting Rudge with identifying the inaccuracies.

Discovery of Fabricated Work

Rudge noticed the errors when he saw Lisa Burton Crawford, a Sydney University professor, wrongly credited with a book outside her expertise. He immediately suspected AI involvement.

AI Investment and Oversight

Big Four consulting firms like Deloitte have invested heavily in AI to boost efficiency. Deloitte announced a $3 billion investment in generative AI development through 2030.

In September, Anthropic announced a partnership with Deloitte, making Claude available to many professionals. However, watchdogs have warned about monitoring AI's impact on audit quality.

Calls for Full Refund

Despite the partial refund, some, like Senator Barbara Pocock, argue Deloitte should repay the entire $290,000. She criticized the firm for misusing AI and misquoting a judge, errors that would penalize a university student.

A Deloitte Australia spokesperson stated the issue has been resolved directly with the client.