What changed between the plenary report and the adopted text

Article 10 – paragraph 2 – point b: (b) data collection processes;

Article 10 – paragraph 2 – point c: (c) data preparation processing operations, such as annotation, labelling, cleaning, updating enrichment and aggregation;

Article 10 – paragraph 2 – point d: (d) the formulation of assumptions, notably with respect to the information that the data are supposed to measure and represent;

Article 10 – paragraph 2 – point e: (e) an assessment of the availability, quantity and suitability of the data sets that are needed;

Article 10 – paragraph 2 – point f: (f) examination in view of possible biases that are likely to affect the health and safety of persons, negatively impact fundamental rights or lead to discrimination prohibited under Union law, especially where data outputs influence inputs for future operations (‘feedback loops’) and appropriate measures to detect, prevent and mitigate possible biases;

Article 10 – paragraph 2 – point f a (new): (f a) appropriate measures to detect, prevent and mitigate possible biases

Article 10 – paragraph 2 – point g: (g) the identification of relevant data gaps or shortcomings that prevent compliance with this Regulation, and how those gaps and shortcomings can be addressed;

Article 10 – paragraph 3: 3. Training datasets, and where they are used, validation and testing datasets, including the labels, shall be relevant, sufficiently representative, appropriately vetted for errors and be as complete as possible in view of the intended purpose. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons in relation to whom the high-risk AI system is intended to be used. These characteristics of the datasets shall be met at the level of individual datasets or a combination thereof.

Article 10 – paragraph 4: 4. Datasets shall take into account, to the extent required by the intended purpose or reasonably foreseeable misuses of the AI system, the characteristics or elements that are particular to the specific geographical, contextual behavioural or functional setting within which the high-risk AI system is intended to be used.

Article 10 – paragraph 5: 5. To the extent that it is strictly necessary for the purposes of ensuring negative bias detection and correction in relation to the high-risk AI systems, the providers of such systems may exceptionally process special categories of personal data referred to in Article 9(1) of Regulation (EU) 2016/679, Article 10 of Directive (EU) 2016/680 and Article 10(1) of Regulation (EU) 2018/1725, subject to appropriate safeguards for the fundamental rights and freedoms of natural persons, including technical limitations on the re-use and use of state-of-the-art security and privacy-preserving. In particular, all the following conditions shall apply in order for this processing to occur: (a) the bias detection and correction cannot be effectively fulfilled by processing synthetic or anonymised data; / (b) the data are pseudonymised; / (c) the provider takes appropriate technical and organisational measures to ensure that the data processed for the purpose of this paragraph are secured, protected, subject to suitable safeguards and only authorised persons have access to those data with appropriate confidentiality obligations; / (d) the data processed for the purpose of this paragraph are not to be transmitted, transferred or otherwise accessed by other parties; / (e) the data processed for the purpose of this paragraph are protected by means of appropriate technical and organisational measures and deleted once the bias has been corrected or the personal data has reached the end of its r…

Article 10 – paragraph 6 a (new): 6 a. Where the provider cannot comply with the obligations laid down in this Article because that provider does not have access to the data and the data is held exclusively by the deployer, the deployer may, on the basis of a contract, be made responsible for any infringement of this Article.

Article 11 – paragraph 1 – subparagraph 1: The technical documentation shall be drawn up in such a way to demonstrate that the high-risk AI system complies with the requirements set out in this Chapter and provide national supervisory authorities and notified bodies with the necessary information to assess the compliance of the AI system with those requirements. It shall contain, at a minimum, the elements set out in Annex IV or, in the case of SMEs and start-ups, any equivalent documentation meeting the same objectives, subject to approval of the competent national authority.

Article 11 – paragraph 2: 2. Where a high-risk AI system related to a product, to which the legal acts listed in Annex II, section A apply, is placed on the market or put into service one single technical documentation shall be drawn up containing all the information set out in paragraph 1 as well as the information required under those legal acts.

Article 11 – paragraph 3 a (new): 3 a. Providers that are credit institutions regulated by Directive 2013/36/EU shall maintain the technical documentation as part of the documentation concerning internal governance, arrangements, processes and mechanisms pursuant to Article 74 of that Directive.

Article 12 – paragraph 1: 1. High-risk AI systems shall be designed and developed with capabilities enabling the automatic recording of events (‘logs’) while the high-risk AI systems is operating. Those logging capabilities shall conform to the state of the art and recognised standards or common specifications.

Article 12 – paragraph 2: 2. In order to ensure a level of traceability of the AI system’s functioning throughout its entire lifetime that is appropriate to the intended purpose of the system, the logging capabilities shall facilitate the monitoring of operations as referred to in Article 29(4) as well as the post market monitoring referred to in Article 61. In particular, they shall enable the recording of events relevant for the identification of situations that may: / (a) result in the AI system presenting a risk within the meaning of Article65(1); or / (b) lead to a substantial modification of the AI system.

Article 12 – paragraph 2 a (new): 2 a. High-risk AI systems shall be designed and developed with, the logging capabilities enabling the recording of energy consumption, the measurement or calculation of resource use and environmental impact of the high-risk AI system during all phases of the system’s lifecycle.

Article 12 – paragraph 3: deleted

Article 13 – title: Transparency and provision of information

Article 13 – paragraph 1: 1. High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable providers and users to reasonably understand the system’s functioning. Appropriate transparency shall be ensured in accordance with the intended purpose of the AI system, with a view to achieving compliance with the relevant obligations of the provider and user set out in Chapter 3 of this Title. / Transparency shall thereby mean that, at the time the high-risk AI system is placed on the market, all technical means available in accordance with the generally acknowledged state of art are used to ensure that the AI system’s output is interpretable by the provider and the user. The user shall be enabled to understand and use the AI system appropriately by generally knowing how the AI system works and what data it processes, allowing the user to explain the decisions taken by the AI system to the affected person pursuant to Article 68(c).

Article 13 – paragraph 2: 2. High-risk AI systems shall be accompanied by intelligible instructions for use in an appropriate digital format or made otherwise available in a durable medium that include concise, correct, clear and to the extent possible complete information that helps operating and maintaining the AI system as well as supporting informed decision-making by users and is reasonably relevant, accessible and comprehensible to users .

Article 13 – paragraph 3 – introductory part: 3. To achieve the outcomes referred to in paragraph 1, information referred to in paragraph 2 shall specify:

Article 13 – paragraph 3 – point a: (a) the identity and the contact details of the provider and, where applicable, of its authorised representatives;

Article 13 – paragraph 3 – point a a (new): (aa) where it is not the same as the provider, the identity and the contact details of the entity that carried out the conformity assessment and, where applicable, of its authorised representative;

Article 13 – paragraph 3 – point b – introductory part: (b) the characteristics, capabilities and limitations of performance of the high-risk AI system, including, where appropriate:

Article 13 – paragraph 3 – point b – point ii: (ii) the level of accuracy, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any clearly known and foreseeable circumstances that may have an impact on that expected level of accuracy, robustness and cybersecurity;

Article 13 – paragraph 3 – point b – point iii: (iii) any clearly known or foreseeable circumstance, related to the use of the high-risk AI system in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, which may lead to risks to the health and safety, fundamental rights or the environment, including, where appropriate, illustrative examples of such limitations and of scenarios for which the system should not be used;

Article 13 – paragraph 3 – point b – point iii a (new): (iiia) the degree to which the AI system can provide an explanation for decisions it takes;

Article 13 – paragraph 3 – point b – point v: (v) relevant information about user actions that may influence system performance, including type or quality of input data, or any other relevant information in terms of the training, validation and testing data sets used, taking into account the intended purpose of the AI system.

Article 13 – paragraph 3 – point e: (e) any necessary maintenance and care measures to ensure the proper functioning of that AI system, including as regards software updates, through its expected lifetime.

Article 13 – paragraph 3 – point e a (new): (ea) a description of the mechanisms included within the AI system that allows users to properly collect, store and interpret the logs in accordance with Article 12(1).

Article 13 – paragraph 3 – point e b (new): (eb) The information shall be provided at least in the language of the country where the AI system is used.

Article 13 – paragraph 3 a (new): 3a. In order to comply with the obligations laid down in this Article, providers and users shall ensure a sufficient level of AI literacy in line with Article 4b.

Article 14 – paragraph 1: 1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they be effectively overseen by natural persons as proportionate to the risks associated with those systems. Natural persons in charge of ensuring human oversight shall have sufficient level of AI literacy in accordance with Article 4b and the necessary support and authority to exercise that function, during the period in which the AI system is in use and to allow for thorough investigation after an incident.

Article 14 – paragraph 2: 2. Human oversight shall aim at preventing or minimising the risks to health, safety, fundamental rights or environment that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter and where decisions based solely on automated processing by AI systems produce legal or otherwise significant effects on the persons or groups of persons on which the system is to be used.

Article 14 – paragraph 3 – introductory part: 3. Human oversight shall take into account the specific risks, the level of automation, and context of the AI system and shall be ensured through either one or all of the following types of measures:

Article 14 – paragraph 4 – introductory part: 4. For the purpose of implementing paragraphs 1 to 3, the high-risk AI system shall be provided to the user in such a way that natural persons to whom human oversight is assigned are enabled, as appropriate and proportionate to the circumstances:

Article 14 – paragraph 4 – point a: (a) be aware of and sufficiently understand the relevant capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

Article 14 – paragraph 4 – point e: (e) be able to intervene on the operation of the high-risk AI system or interrupt, the system through a “stop” button or a similar procedure that allows the system to come to a halt in a safe state, except if the human interference increases the risks or would negatively impact the performance in consideration of generally acknowledged state-of-the-art.

Article 14 – paragraph 5: 5. For high-risk AI systems referred to in point1(a) of Annex III, the measures referred to in paragraph 3 shall be such as to ensure that, in addition, no action or decision is taken by the user on the basis of the identification resulting from the system unless this has been verified and confirmed by at least two natural persons with the necessary competence, training and authority.

Article 15 – paragraph 1: 1. High-risk AI systems shall be designed and developed following the principle of security by design and by default. In the light of their intended purpose, they should achieve an appropriate level of accuracy, robustness, safety, and cybersecurity, and perform consistently in those respects throughout their lifecycle. Compliance with these requirements shall include implementation of state-of-the-art measures, according to the specific market segment or scope of application.

Article 15 – paragraph 1 a (new): 1 a. To address the technical aspects of how to measure the appropriate levels of accuracy and robustness set out in paragraph 1 of this Article, the AI Office shall bring together national and international metrology and benchmarking authorities and provide non-binding guidance on the matter as set out in Article 56, paragraph 2, point (a).

Article 15 – paragraph 1 b (new): 1b. To address any emerging issues across the internal market with regard to cybersecurity, the European Union Agency for Cybersecurity (ENISA) shall be involved alongside the European Artificial Intelligence Board as set out Article 56, paragraph 2, point (b).

Article 15 – paragraph 2: 2. The levels of accuracy and the relevant accuracy metrics of high-risk AI systems shall be declared in the accompanying instructions of use. The language used shall be clear, free of misunderstandings or misleading statements.

Article 15 – paragraph 3 – subparagraph 1: Technical and organisational measures shall be taken to ensure that high-risk AI systems shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

Article 15 – paragraph 3 – subparagraph 2: The robustness of high-risk AI systems may be achieved by the appropriate provider with input from the user, where necessary, through technical redundancy solutions, which may include backup or fail-safe plans.

Article 15 – paragraph 3 – subparagraph 3: High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs influencing input for future operations (‘feedback loops’) and malicious manipulation of inputs used in learning during operation are duly addressed with appropriate mitigation measures.

Article 15 – paragraph 4 – subparagraph 1: High-risk AI systems shall be resilient as regards to attempts by unauthorised third parties to alter their use, behaviour, outputs or performance by exploiting the system vulnerabilities.

Article 15 – paragraph 4 – subparagraph 3: The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training dataset (‘data poisoning’), or pre-trained components used in training (‘model poisoning’) , inputs designed to cause the model to make a mistake (‘adversarial examples’ or ‘model evasion’), confidentiality attacks or model flaws, which could lead to harmful decision-making.

Title III – Chapter 3 – title: OBLIGATIONS OF PROVIDERS AND DEPLOYERS OF HIGH-RISK AI SYSTEMS AND OTHER PARTIES

Article 16 – title: Obligations of providers and deployers of high-risk AI systems and other parties

Article 16 – paragraph 1 – point a: (a) ensure that their high-risk AI systems are compliant with the requirements set out in Chapter 2 of this Title before placing them on the market or putting them into service;

Article 16 – paragraph 1 – point a a (new): (a a) indicate their name, registered trade name or registered trade mark, and their address and contact information on the high-risk AI system or, where that is not possible, on its accompanying documentation, as appropriate;

Article 16 – paragraph 1 – point a b (new): (a b) ensure that natural persons to whom human oversight of high-risk AI systems is assigned are specifically made aware of the risk of automation or confirmation bias;

Article 16 – paragraph 1 – point a c (new): (a c) provide specifications for the input data, or any other relevant information in terms of the datasets used, including their limitation and assumptions, taking into account the intended purpose and the foreseeable and reasonably foreseeable misuses of the AI system;

Article 16 – paragraph 1 – point c: (c) draw-up and keep the technical documentation of the high-risk AI system referred to in Article 11;

Article 16 – paragraph 1 – point d: (d) when under their control, keep the logs automatically generated by their high-risk AI systems that are required for ensuring and demonstrating compliance with this Regulation, in accordance with Article 20;

Article 16 – paragraph 1 – point e: (e) ensure that the high-risk AI system undergoes the relevant conformity assessment procedure, prior to its placing on the market or putting into service, in accordance with Article 43;

Article 16 – paragraph 1 – point e a (new): (e a) draw up an EU declaration of conformity in accordance with Article 48;

Article 16 – paragraph 1 – point e b (new): (e b) affix the CE marking to the high-risk AI system to indicate conformity with this Regulation, in accordance with Article 49;