Artificial Intelligence (AI)

CPME Rapporteur: Prof. Dr Christian LOVIS (CH)
CPME Secretariat: Ms Sara RODA

CPME monitors the developments and the implementation of the Artificial Intelligence Act (AIA), and advocated in the Digital Omnibus on AI to maintain AI used for medical purposes in the scope of application of the AIA’s requirements for high-risk devices.

The uptake of AI in healthcare is still low due to several factors, including the complex environment of the sector, the wide range of products available on the market, the majority of which are not certified by a third-party, and a lack of confidence in using AI systems based on data from unknown data sources or data collection processes. 

AI products should be seamlessly integrated into the healthcare information system and reflect the real needs of healthcare professionals, patients and their carers. Doctors should be free to decide whether to use an AI system, bearing in mind the best interests of the patient, and to retain the right to disagree with an AI system,without repercussions.

CPME supports applying the strict liability regime for AI systems (as there is no need for the victim to prove fault) and mandatory insurance for high-risk AI systems, which should include “tail” cover.

CPME welcomes the AIA risk-based approach, the development of the EU database for high-risk AI and the risk management system. The list of stand-alone high-risk AI of Annex III should include the use of AI for assessing medical treatments and for health research. Certain systems cannot be deployed without clear validation as there can be misuse leading to discrimination and harm.

Providers of AI systems should designate an AI officer to ensure internal compliance of companies with the AI Act, in particular when processing special categories of data for bias detection and correction, or for producing AI outputs, or for disclosing or making personal data available to third parties.

Human oversight must be of ‘high quality’ and the provider is appropriately resourced for the effective performance of the task. The CE marking should only be given to AI systems that comply with EU law, including the General Data Protection Regulation. The possibility of exercising data subject rights must be provided in the AI system from the very beginning.