T41. Algorithm risk assessment

Algorithms constitute the backbones of AI systems. AI system performance driven by algorithm performance. Possible AI systems biases and unfair outcomes often emanate from algorithm design. If the organization has access to the algorithms in the AI systems, identifying the possible algorithm risks and assessing their gravity is key to sustainable AI system development and operation.

Algorithm risk assessment should cover, to the extent possible, a wide range of algorithm-related risk sources and causes.

The Algorithm Owner should at least ensure that the organization
1) explores and documents how the algorithm affects the operations of the entire AI system
2) explores and documents the possible risk of biased and, in particular, discriminatory outcomes, and
3) explores and documents the risk of unfair outcomes and harms the algorithm may generate.

As identifying biases and unfairness is often complex and contentious, the reviews should involve ethical and legal experts. Particularly if the organization intends to use the algorithm in a high-risk use case. In machine learning algorithms, testing algorithm outputs may be necessary for identifying biases and discriminatory outcomes.

In addition, if a machine learning algorithm incorporates inferences made from training data, the risk assessment should review and assess
4) the risk of detecting non-existing patterns and correlations in the data,
5) the level of algorithmic scrutability and explainability.