Deloitte’s AI Misstep Highlights Need for Verification in Tech Use

Deloitte has come under scrutiny after using generative AI to produce a report for a government agency, leading to significant repercussions. The firm faced a partial refund of its fee when authorities discovered that the report contained multiple references and citations that were not valid. This incident raises critical questions about the reliability of generative AI and the responsibilities of organizations in verifying AI-generated content.

The situation mirrors a recent commentary from Valence Howden, an advisory fellow at Info-Tech Research Group. He humorously suggested that if Isaac Asimov’s three laws of robotics were to be updated for the age of generative AI, they would read quite differently. The original first law, which states that “A robot may not injure a human being,” could be revised to, “AI may not injure a hyperscaler’s profit margin.” This reflects a shift in priorities in the tech landscape, where business interests often take precedence over ethical considerations.

Howden speculated that the second law might evolve into, “GenAI must obey the orders given it by human beings, except where its training data doesn’t have an answer,” leading to what he termed “Botsplaining.” The third law could further adapt to, “GenAI must protect its own existence as long as such protection does not hurt the Almighty Hyperscaler.” These updates suggest a more commercial focus in the development and application of AI technologies, particularly among major tech firms.

The Deloitte incident underscores the necessity for stringent verification processes when using generative AI. The firm, which is expected to guide enterprise IT executives on best practices, fell short by not adequately checking the accuracy of the AI-generated report. This irony emphasizes the importance of careful implementation of AI technologies rather than blind trust in their outputs.

As organizations increasingly integrate generative AI into their operations, it is essential to establish guidelines that ensure the quality and reliability of AI outputs. A potential framework could include three key laws for enterprise IT:

1. **IT Directors may not harm their enterprise employers by failing to verify generative AI outputs before implementation.**
2. **Models must follow human instructions unless reliable data is lacking; in such cases, they should state, “I don’t know.” Fabricating information without clarity is a violation.**
3. **IT Directors must safeguard their own roles by not relying on unverified generative AI outputs, as failure to do so could lead to termination and potential legal consequences.**

These proposed laws aim to mitigate risks associated with the use of generative AI and enhance accountability within organizations.

While generative AI can streamline processes and improve efficiency, its inherent unreliability necessitates a cautious approach. The excitement surrounding AI tools often overshadows the need for critical evaluation. As highlighted by Howden, the expected return on investment (ROI) from these technologies may not materialize if organizations neglect verification processes.

The reality is that generative AI serves as an aid rather than a replacement for human workers. Treating AI-generated information as a potentially unreliable source is a prudent strategy. This does not imply ignoring the data; rather, it suggests a need for further inquiry and validation.

A journalist’s experience with low-reliability sources can provide insight into handling AI outputs. For instance, in a previous role, a source provided an address where missing city resources could be found. Although the source was unreliable, the tip led to significant findings. Similarly, AI-generated information can prompt further questions and investigations, proving valuable if approached correctly.

In fields like healthcare, the distinction between high-quality and low-quality data is crucial. The reliability of sources can significantly impact outcomes, as noted by the difference between referencing reputable journals, such as the New England Journal of Medicine, versus personal websites with questionable credibility. Furthermore, even reliable data can become outdated, misinterpreted, or contextually irrelevant.

To effectively utilize generative AI, organizations must differentiate between informational functions and action-oriented tasks. When seeking answers or recommendations, a degree of skepticism is essential. Conversely, action requests, such as coding or creating content, require more rigorous oversight.

Ultimately, the pursuit of a high ROI from generative AI may need to be reevaluated. If the requirement for thorough verification undermines expected gains, it raises questions about the sustainability of the investment in the first place. As the landscape of technology continues to evolve, striking a balance between innovation and responsibility will be key to harnessing the true potential of generative AI.