In the evolving world of artificial intelligence, corporate boards are facing the challenge of obtaining clear and actionable insights on AI risk. Recent evaluations of three leading AI tools—Claude, ChatGPT, and Gemini—mark a pivotal moment for organizations as they seek to handle this complex issue. Each tool created a single slide to present AI risk metrics for ACME Corporation, a fictional company with $500 million in revenue. The results not only showcase the capabilities of these AI systems but also highlight broader concerns regarding how organizations understand AI risk and governance.
Claude: A Focus on Accountability
Claude's output is noteworthy for its alignment with the practical needs of a Chief Information Security Officer (CISO). The slide created by Claude outlined five distinct risk categories, each with specific metrics, current values, target goals, and trend arrows. It highlighted metrics such as a 4.2% hallucination rate against a target of less than 2%, and three personally identifiable information (PII) exposure events year-to-date, along with actionable insights.
A significant aspect of Claude’s approach was its inclusion of a “Board Actions Required” section. This segment linked each identified risk to specific actions with deadlines, recommending that the board appoint an AI risk owner within 30 days, commission a full AI inventory within 30 to 90 days, and approve a risk budget ranging from $2.5 to $4 million within the same period. It also noted that only 34% of staff had completed AI risk training against an 80% target, highlighting a governance element often neglected by its counterparts.
Claude’s design philosophy emphasizes accountability and readiness for decision-making. However, the output reveals a challenge: while the metrics are strong, their usefulness relies on stable tooling and governance structures. Without these systems, teams may find it difficult to gather and maintain such metrics, potentially requiring significant manual coordination.
ChatGPT: Building a Governance Framework
Conversely, ChatGPT adopted a broader perspective by concentrating on the governance framework rather than just current metrics. Its output resembled a structured governance architecture that outlined what organizations should track and their progress along the way.
This governance-focused approach identified control coverage across six critical domains: Governance (72%), Data/Privacy (58%), Security (61%), Model Safety (63%), Monitoring (55%), and Vendor (49%). ChatGPT’s slide also included a bar chart displaying leading indicators and a 90-day action roadmap aimed at implementing data loss prevention (DLP) protocols, improving continuous integration/continuous deployment (CI/CD) pipelines with injection testing, and establishing vendor change notifications.
The strength of this output lies in its focus on establishing a baseline early in the governance process, which is vital for tracking the program's success over time. By prioritizing frameworks over immediate metrics, ChatGPT offers a strategic perspective that could be invaluable for organizations looking to create a comprehensive risk management strategy.
Gemini’s Approach
While Gemini’s response is less detailed in the provided context, it is essential to recognize that each tool's approach reflects different philosophies regarding AI governance and risk management. The variations in output suggest various strengths and weaknesses, mirroring the current state of AI tools and their application in real-world settings.
The Implications for Corporate Governance
As organizations increasingly depend on AI technologies, effectively communicating their risks to corporate boards becomes critical. The differing approaches of Claude and ChatGPT illustrate the ongoing effort to balance actionable insights with strategic governance frameworks. Claude focuses on actionable metrics aimed at immediate accountability, while ChatGPT offers a broader governance perspective that may assist organizations in developing a long-term strategy.
The insights derived from these AI tools emphasize that while technological advancements in AI are rapidly progressing, the frameworks that support them require careful consideration and improvement. For boards, having clear, actionable metrics integrated with a solid governance strategy may be essential for successfully navigating the uncertain waters of AI risk.
As the landscape continues to evolve, the demand for enhanced AI risk assessment tools and frameworks is expected to increase, prompting organizations to rethink their strategies and invest in stable governance models that can effectively support their risk management objectives.
