We continue our analysis of the second set of Specialised Technical Guides (Guides 3 to 15) issued by the Spanish Artificial Intelligence Supervisory Agency (AESIA) to support compliance with the European Artificial Intelligence Act (AI Act).

Today we look at key aspects of AESIA Guides 5 and 6: 


Guide 5: "Risk management"

Guide 5, titled "Risk management", supports compliance with article 9 of the AI Act from an operational perspective under the heading "Risk management system". This article establishes an obligation to implement an iterative and continuous process throughout a high-risk AI system's lifecycle to identify and mitigate potential risks. The onus for complying with this obligation is primarily on the provider, as the system's developer. However, Guide 5 clarifies that, if a deployer plays a role in system development, it must implement the measures established by the provider. The system should be geared towards protection of individuals' health, safety and fundamental rights.

With a view to devising a suitable risk management system, Guide 5 sets out a roadmap composed of eight interconnected phases.

Phases of the risk management system. Source: Guide 5, p. 15

A key aspect of the guide is determining appetite for risk, defined as the degree of risk that an organisation is prepared to tolerate to achieve its goals. This assessment is made on a scale of 1 to 15. If a system has a critical impact, such as the administration of insulin in the medical field, appetite for risk should be very low; conversely, in low-impact systems such as taste-based movie recommendations, tolerance may be much higher.

Finally, the guide warns of critical dangers such as algorithmic discrimination and the negative impact of biometric systems. Using case studies, it illustrates how data biases can unfairly penalise groups based on race or gender in access to health or employment.

 

Guide 6: "Human Oversight"

Guide 6, titled "Human Oversight", focuses on the operational implementation of the obligations defined in Article 14 of the AI Act. Its main objective is to ensure that high-risk AI systems are designed in such a way that they can be effectively monitored by natural persons during use.

Human oversight is not merely a technical requirement; it is a governance mechanism aimed at preventing AI from undermining human autonomy or causing harm to health, safety or fundamental rights. The guide stresses that human oversight enables accountability for the system's actions. Specifically, in order for humans to truly understand how a system works, the guide lists a number of measures that the design should incorporate, including:

  • "Counterfactual reasoning": the system should not simply describe why it made a decision but also explain what changes to the input data would have altered the outcome. This is essential for a supervisor or a data subject to understand, for example, under what conditions a rejected application would have been accepted.
  • Hierarchical breakdown: The interface should allow information to be navigated in a structured manner, from the most global (the model's general reasoning mechanism) to the most specific (the logic behind a specific individual decision).

The guide also explains the three fundamental approaches to gauge how much autonomy is given to the system:

  • Human-in-the-loop (HITL): There is human involvement in every decision cycle. This is the recommended level for high-risk cases where each action must be validated individually.
  • Human-on-the-loop (HOTL): Monitoring is performed after the fact or during operation, allowing intervention if an anomaly is detected.
  • Human-in-command (HIC): Humans retain overall command and ultimate responsibility for critical decisions and safe operation of the system.

One key element is emphasis on the need for appropriate interfaces. The provider must provide the deployer with tools that allow the AI results and reasoning to be interpreted simply, visually and in normal language.

Finally, the guide addresses a critical psychological risk: human tendency to blindly trust machine decisions. To combat this, it proposes measures such as implementing a 'forced error' mode, i.e. introducing deliberate failures in sandbox environments to test whether the supervisor maintains critical judgement, and ensuring awareness by training supervisors to understand the system's typical limitations and shortcomings.

Ver post @Linkedin

 

Related links

Related categories

Key contacts

Pablo García Mexía photo

Pablo García Mexía

Consultant - Head of Digital Law, Madrid, Madrid

Pablo García Mexía Iria Calviño Jaime Bofill