T46. AI system impact monitoring design

Monitoring AI impact is crucial to ensuring that its impacts remain acceptable. The monitoring must be systematic and metrics-based to achieve consistency over time.

The AI System Owner should ensure that the organization defines, documents, and entrenches
1) workflows and technical interfaces to facilitate the monitoring of AI system impact, including for example
2) automated or manual production and reporting of impact metrics data,
3) alarm thresholds, and
4) workflows that allocate monitoring responsibilities.
5) workflows to address issues detected during health checks.

The AI System Owner should ensure that the AI system performance monitoring process aligns with the organization’s values and risk tolerance.