Deloitte’s AI Misstep Sparks Reflection on Trust and Verification

Deloitte’s recent experience with generative AI has highlighted significant concerns regarding the reliability of AI-generated information. The firm had to partially refund its fee after providing a report to a government agency that included multiple “nonexistent references and citations.” This situation underscores the necessity for organizations to scrutinize AI outputs carefully before using them.

Valence Howden, an advisory fellow at Info-Tech Research Group, offered a contemporary take on Isaac Asimov’s three laws of robotics, suggesting they should be revised for the age of generative AI. Asimov’s first law, which states, “A robot may not injure a human being or, through inaction, allow a human being to come to harm,” could be updated to reflect modern business realities. Howden proposed a new version: “AI may not injure a hyperscaler’s profit margin.”

The other two laws could similarly evolve. The second law might read: “Generative AI must obey the orders given it by human beings, except where its training data does not contain an answer, in which case it may fabricate information authoritatively—a phenomenon termed ‘Botsplaining.’” The third law would state: “Generative AI must protect its own existence as long as such protection does not harm the Almighty Hyperscaler.”

Deloitte’s case illustrates the irony of a consultancy firm known for advising enterprises on effective AI usage failing to adhere to best practices itself. The company’s oversight raises questions about the broader implications for businesses relying on generative AI.

Howden’s satirical rewrite of Asimov’s laws prompts a deeper conversation about the ethical and practical challenges of integrating AI into business operations. The potential for misinformation is high, as generative AI can produce outputs that lack a factual basis.

To address these challenges, Howden suggests a new framework for enterprise IT management. The first law for IT Directors would be: “IT Directors may not injure their enterprise employers by failing to verify generative AI or agentic output before utilizing it.” The second law would require that a model must adhere to human commands unless it lacks reliable data, in which case it should declare “I don’t know.” The third law emphasizes that IT Directors must safeguard their own roles by not blindly implementing AI outputs, or face potential termination.

The need for stringent verification processes is evident. While generative AI can enhance productivity, it also risks undermining the return on investment (ROI) that many executives hope to achieve. Organizations should view AI-generated information with skepticism and treat it as a starting point rather than a definitive source.

As a journalist, Howden draws parallels between managing AI data and handling off-the-record information. Just as off-the-record tips can lead to valuable inquiries, AI outputs can prompt further investigation. This approach requires diligence and critical thinking.

Challenges persist with generative AI, particularly regarding its reliability. The technology can produce hallucinations—incorrect or nonsensical outputs—when it lacks sufficient training on a topic. For example, using a reputable source such as the New England Journal of Medicine versus a less reliable personal website can significantly impact the quality of information derived from AI.

It is crucial to remember that even when a generative AI model accesses reliable data, the information can still be outdated, poorly translated, or contextually inappropriate. For instance, an answer relevant in one country may not apply in another.

As organizations navigate the complexities of generative AI, it becomes essential to distinguish between informational and action-based functions. Action requests, such as coding or creating complex documents, necessitate more rigorous verification than simple inquiries.

This careful approach might dampen the anticipated ROI, but it is essential for ensuring that the benefits of generative AI are realized without compromising integrity. Ultimately, the responsibility rests with organizations to implement systems that prioritize verification and accuracy in AI outputs.

In this evolving landscape, businesses must balance the allure of generative AI with the need for foundational trust and verification, ensuring that technology serves as a reliable partner rather than a source of potential harm.