Appendix G: Executive AI Dashboard and Metrics

Executives do not need a technical telemetry screen. They do need a regular management view that shows where AI use is spreading, where exposure is rising, where controls are weak, and whether capability is becoming more durable.

The purpose of this appendix is simple: help leadership teams review AI as an operating portfolio rather than a collection of scattered pilots, vendor updates, and isolated incidents.

What An Executive Dashboard Should Answer

An executive AI dashboard should help leaders answer six practical questions:

  1. What material AI use exists now, and what changed this period?
  2. Where is value being created, and where is activity being mistaken for value?
  3. Where is risk, incident pressure, or policy breach rising?
  4. Which controls, approvals, or evidence requirements are weak or overdue?
  5. Which vendors, platforms, or jurisdictions are becoming strategic dependencies?
  6. Is the organisation becoming more governable as AI use expands, or less?

Dashboard Design Principles

A useful executive dashboard should be:

  • short enough to review in one meeting
  • comparative enough to show change over time
  • selective enough to focus on material systems, not every experiment
  • balanced enough to show value, exposure, and dependency together
  • actionable enough that each red signal has a named owner and next step

The common failure is to produce a dashboard that reports activity volume without helping leaders decide where to intervene.

Core Dashboard Sections

Section What leaders should see Why it matters
Inventory and visibility Number of material AI systems, high-impact workflows, and major vendors in current use; additions, retirements, or scope changes since last review Leaders cannot govern what they cannot see
Value and adoption Which use cases are producing measurable workflow, service, or strategic value; where adoption is shallow, forced, or workaround-heavy Usage growth alone is not evidence of value
Risk and incidents Incidents, complaints, overrides, near misses, policy breaches, and unresolved remediation items Incident visibility shows where AI use is outrunning control
Control and assurance Systems lacking current approval, testing, monitoring, documentation, owner confirmation, or audit evidence Weak control maturity is often invisible until something fails
Vendor and dependency Critical suppliers, concentrated model or cloud dependence, important contract or service changes, and weak fallback positions AI strength can hide strategic fragility
Workforce and trust Reliance, resistance, override behavior, training uptake, contestation, and recurring concerns about fairness, workload, or surveillance Legitimacy and adoption quality matter as much as technical availability

Suggested Monthly Or Quarterly Pack

Most executive teams should expect a short monthly or quarterly pack with five parts:

  1. Portfolio summary Material systems in use, new additions, retired systems, and major scope changes.

  2. Top green, yellow, and red signals A short list of the most important positive developments, emerging concerns, and urgent issues.

  3. Material incidents and control gaps High-priority incidents, complaints, policy breaches, unresolved remediation items, and overdue approvals or reviews.

  4. Strategic dependency view Vendor, platform, cloud, jurisdiction, or model changes that may require management action.

  5. Decisions required Items needing executive approval, escalation, pause, additional funding, or board visibility.

The purpose is not to flood leaders with operational detail. It is to make AI exposure, capability, and dependence visible enough for real oversight.

Suggested Executive Metrics

Not every organisation needs the same scorecard, but most should track some version of the following.

1. Inventory And Materiality

  • count of material AI systems in live use
  • count of high-impact or regulated AI use cases
  • number of new systems introduced since last review
  • number of paused, retired, or materially changed systems
  • percentage of material systems with a named accountable owner

2. Value Realization

  • percentage of material use cases with a defined outcome metric
  • number of use cases delivering verified workflow or service improvement
  • number of use cases with persistent adoption but weak measurable value
  • time from pilot approval to evidence of sustainable operational benefit
  • top three uses where investment is increasing or being withdrawn

3. Risk, Incidents, And Breach Signals

  • number of incidents, near misses, or significant overrides this period
  • number of complaints, contestations, or policy breaches linked to AI use
  • number of open remediation items past due
  • number of material systems with repeat incidents or recurring weak behavior
  • average time to detect, escalate, and contain important AI-related failures

4. Control And Assurance

  • percentage of material systems with current approval status
  • percentage with up-to-date testing, monitoring, and documentation
  • percentage with a current model or vendor review after major change
  • number of systems lacking adequate audit trail, records logic, or evidence pack
  • number of high-impact systems operating with temporary exceptions

5. Dependency And Sovereignty Exposure

  • number of material systems dependent on a single model provider or cloud path
  • count of critical uses with no credible fallback or degraded-mode plan
  • number of important vendor, contractual, or policy changes since last review
  • number of material systems involving cross-border data or inference dependence
  • count of uses where lock-in, residency, or portability concerns remain unresolved

6. Workforce, Use Quality, And Trust

  • training completion for managers and staff in approved AI use
  • rate of override or non-use in systems expected to be relied on
  • recurring workforce concerns about fairness, surveillance, workload, or deskilling
  • pattern of challenge, complaint, or escalation from affected users
  • leadership view of where trust is strengthening and where it is being spent down

Red Flag Signals

Executive dashboards should make certain red flags hard to miss:

  • management cannot produce a stable inventory of material AI use
  • adoption is increasing faster than approval, documentation, or monitoring
  • one vendor or model family is becoming critical without fallback
  • incidents are being closed administratively but the same failure pattern recurs
  • value claims are based on usage or enthusiasm rather than verified outcomes
  • staff are using unapproved tools because approved routes are too slow or unclear
  • major systems remain live under temporary exceptions for too long

A Simple RAG Review Model

Use a simple status model for material systems:

Status Meaning Expected leadership response
Green Value, control, and dependency are within expected limits Continue, monitor, and review on normal cycle
Amber One or more material concerns need management attention Assign owner, set timeline, and watch closely
Red Exposure, control weakness, or strategic dependence is outside tolerance Escalate, pause, narrow, or intervene immediately

The important point is not the color itself. It is whether the status forces a real management response rather than becoming presentation language.

What Should Reach The Board

Not everything belongs at board level. Boards should usually see:

  • major new high-impact AI deployments
  • significant incidents, public failures, or regulator-sensitive events
  • concentrated vendor or jurisdictional dependence in critical functions
  • unresolved control gaps in high-impact systems
  • material shifts in exposure, assurance, or strategic reliance

Routine operational details should stay with management unless they indicate a pattern that changes risk tolerance, trust, or strategy.

What Good Looks Like

A credible executive AI dashboard should allow a leadership team to say:

  • we know where material AI use exists
  • we can see whether value is real or overstated
  • we can identify where control is weakening before failure forces attention
  • we understand where vendor and cross-border dependence are becoming strategic
  • we can tell which issues need management action and which need board visibility

That is the difference between reporting on AI activity and governing AI as an institutional capability.

← Appendix F Next: Bibliography →
This site uses Just the Docs, a documentation theme for Jekyll.