European Parliament Advances Stringent Requirements for AI Regulation, Including Safeguards for Foundational Models

In a significant development, the European Parliament has approved a series of amendments to the draft AI legislation in a recent vote. Among the changes, MEPs have agreed upon a set of requirements for foundational models that support generative AI technologies like OpenAI’s ChatGPT. These amendments oblige providers of foundational models to conduct safety checks, implement data governance measures, and mitigate risks before bringing their models to the market. Moreover, providers must consider potential risks to health, safety, fundamental rights, the environment, democracy, and the rule of law.

In addition to safety measures, foundational model makers are expected to reduce energy consumption and resource usage, while also registering their systems in an EU database established under the AI Act. Generative AI technology providers, such as ChatGPT, must comply with transparency obligations, ensure user awareness of machine-generated content, apply safeguards to the generated content, and provide a summary of copyrighted materials used in their AI training.

Notably, MEPs have focused on preventing general-purpose AI from evading regulatory requirements. The European Parliament has been engaged in intense debates concerning biometric surveillance and has agreed to amendments aimed at strengthening fundamental rights protections.

MEPs from the Internal Market Committee and the Civil Liberties Committee voted on thousands of amendments, ultimately adopting a draft mandate for the AI Act with strong support. The Parliament’s aim is to ensure that AI systems are overseen by humans and adhere to principles of safety, transparency, traceability, non-discrimination, and environmental friendliness.

The adopted amendments also include an expanded list of prohibited practices, banning intrusive and discriminatory uses of AI systems. This includes real-time and post remote biometric identification systems, biometric categorization systems using sensitive characteristics, predictive policing systems, emotion recognition systems, and indiscriminate scraping of biometric data for facial recognition databases.

While the vote in the committees signifies a major step toward setting the Parliament’s negotiating mandate, further discussions are expected during trilogue talks with the Council. Member State governments may push back on certain provisions, particularly those related to national security considerations.

The amendments proposed by MEPs align closely with the demands of civil society groups. However, concerns remain, particularly regarding AI’s use in migration control and the self-assessment of AI developers to determine the level of risk their systems pose. Digital rights organizations stress the need for stronger prohibitions on certain AI practices and improved enforcement mechanisms.

Despite these concerns, the amendments represent significant progress in establishing a comprehensive regulatory framework for AI in the European Union. The forthcoming plenary vote in the Parliament will be crucial in finalizing the mandate and initiating trilogue negotiations. With the AI Act, the EU aims to lead the way in creating human-centric, trustworthy, and safe AI while protecting fundamental rights, ensuring democratic oversight, and establishing an effective system of AI governance and enforcement.