With your expertise in both life sciences and artificial intelligence, how do you see generative AI reshaping the regulatory landscape for medical devices and healthcare technologies in Europe?
Generative AI is frequently described as the most significant disruptor for the EU regulatory framework for healthcare over the next decade, due to its intrinsic capacity to challenge the assumptions on which such a framework was built. I believe three main regulatory challenges can be identified. First of all, the current Regulation 2017/745 (Medical Device Regulation – MDR), under which generative AI systems may need to be assessed if they have a medical intended purpose, is conceptualised for and has always been applied to regulate deterministic technologies, i.e. products with relatively stable, predictable functionalities and risk profiles, while the inherent capacity of these systems to evolve may require adaptation, interpretation or amendments of the regulation in place. Secondly, complying with the normative framework for the protection of personal data as set forth by the GDPR can be challenging as well, especially for compliance with the principle of data minimisation (among other challenges) both at the moment of training the AI system and during its deployment. This principle may need to be reinterpreted in the context of generative AI, or at least adapted for it. Finally, generative AI, as well as AI systems in general, raises difficult ethical and normative questions regarding liability allocation, especially in the healthcare field, which might require direct legislative intervention rather than the application of existing standards and norms.
In your work on data governance and data stewardship, what are the key challenges in ensuring that health data is used responsibly while still enabling innovation in AI-driven healthcare solutions?
Two of the main challenges in this regard, I believe, are ensuring that health data can be processed and reused to train and deploy AI solutions, thus fostering innovation while at the same time adequately protecting patients’ fundamental rights, and providing clear guidance on whether, and if so how, this data can be anonymised for these purposes. The two challenges may be addressed as part of a unified policy framework, in the sense that helping stakeholders navigate the complex set of norms addressing the concepts of personal data and anonymisation may increase their confidence in using such data (which I believe can be considered an instrument for the protection of patients’ rights), thus at the same time increasing the amount of data available for AI-driven solutions.
Drawing from your experience as both a researcher and consultant, what practical steps should policymakers and industry leaders take to build trustworthy and compliant AI systems in the healthcare sector?
Adopting a life-cycle and proactive approach I believe is fundamental for managing projects that involve AI systems in the healthcare sector. On the one hand, a life-cycle approach, although being required by the applicable norms, makes it possible to design projects involving AI in a compliant-by-design manner from the outset, anticipating future necessities or developments and laying the foundations from the very beginning to meet them effectively. On the other hand, proactiveness is a necessity, due to the need to update the governance plan according to the amendments and developments in the applicable regulations, but can also become a strategic advantage for stakeholders to remain competitive and foster innovation.