T25. Algorithm health checks design

Algorithms may be subject to performance deterioration over the medium and long term. In addition, the business, operational, IT, and regulatory environments and stakeholder pressures will change over time. These processes may jeopardize algorithm performance or lead to the emergence of unacceptable risks.

The Algorithm Owner should ensure that the organization conducts regular comprehensive reviews of the algorithm (algorithm health checks) to ensure that the algorithm aligns with the organization’s values and risk tolerance.

The Algorithm Owner should ensure that the organization defines, documents, and implements workflows and technical interfaces to review
1)    the AI system use case,
2)    the AI system users,
3)    the AI system operational environment,
4)    the AI system technical environment,
5)    the AI system deployment metrics,
6)    the AI system operational use metrics,
7)    the AI system version control practices,
8)    the AI system performance monitoring practices and
9)    the AI system health check practices.

The reviews should assess whether the algorithm aligns with the organization’s values and risk tolerance. If the review discloses misalignments, the Algorithm Owner initiates appropriate measures to regain alignment.