An Australian court has fined audit and consultancy firm Deloitte and ordered it to refund the review fee it paid for submitting an AI-generated report to a government agency. The Department of Employment and Labor Relations (DEWR) said the Deloitte report contained a number of inaccuracies, including missing academic research sources and citations.
The Department of Employment and Labor Relations commissioned a $290,000 “independent assurance review” from Deloitte in December last year. The first version of Deloitte’s review was published in July, and a revised version was published on the DEWR website on Friday. After discovering fabricated sources and citations, as well as factual errors, in the report, Deloitte revised the 237-page report and acknowledged that some of the footnotes and references in the review were incorrect.
According to the department, the audit and consulting firm agreed to pay the amount stipulated in the contract. Deloitte said that “the matter was resolved directly with the client,” but the firm did not say anything about the use of AI in the report. Deloitte also attributes some of the inaccuracies in the report to “hallucinations” of artificial intelligence.
As it was determined, there were up to 20 factual errors in the initial version of the report. According to the investigation, the review was prepared using a generative artificial intelligence language system, Azure OpenAI.
Despite the inaccuracies found, the main part of the report and its recommendations remained unchanged. The review removed quotes from a federal judge, as well as references to nonexistent reports attributed to legal and software engineering experts, which do not actually exist.
