T40. AI system harms and impacts pre-assessment

Understanding what harms and societal impacts an AI system may create is a crucial precondition for sustainable AI system development.

The intensity of potential harms and impacts will vary significantly across different AI systems. Industrial AI systems with no direct effects on individuals or the environment will likely create limited harms and impacts. An AI system that makes irreversible decisions that affect the rights and obligations of individuals will have profound impacts and may generate significant harms.

The AI System Owner should ensure that the organization conducts a harms and impact pre-assessment at the outset of AI system development. The pre-assessment outcomes should be documented.

The pre-assessment should cover a wide range of potential harm and impact creation pathways. The pre-assessment should consider the harms the AI system may create and the impact it may have on its users, possible decision-making subjects, other affected parties, society at large, and the environment.

AI system risk assessments often focus on the direct harms the systems may create. The harms and impacts pre-assessment should, however, aim at also identifying the potential system-level harms and impacts. These include the social action affordances the system may create or modify, its potential wealth and power distribution implications and effects on equality.

The ethical advisory function should be involved in the pre-assessment of the harms and impacts if it is likely that the AI system will create a non-negligible risk of harm to individuals or the environment.

The AI System Owner should ensure that harms and impacts pre-assessment is conducted if the design parameters of the AI system undergo fundamental changes.