EU Frontier AI Initiative: Developing frontier AI solutions that are safe and computationally efficient within Apply AI (RIA)

Apie kvietimą

Expected Outcome: The Apply AI Strategy[1] also seeks to bolster EU capabilities and achieve excellence in AI to support the development of European frontier models. As part of the Frontier AI Initiative, which brings together Europe’s leading actors in the field, this topic will support the development of sovereign frontier AI ensuring safety by design. This topic directly contributes to the Apply AI Strategy. Project results are expected to contribute to all of the following expected outcomes: Strengthened European capabilities in the development of frontier AI models. Improved computational efficiency of frontier AI models, resulting in reduced computational costs. Enhanced safety of advanced AI systems based on frontier AI models through the development and implementation of safe-by-design principles and/or AI agents acting as safety evaluators. Scope: To advance developments of frontier AI models towards highest-level performance, while ensuring energy efficiency, addressing computational constraints, and strengthening safety. The approach of this topic is twofold. First, it aims to advance the AI field through the development and training of a frontier AI model. The AI model should demonstrate state-of-the-art performance, have multimodal capabilities, and be optimized for agentic AI capabilities such as tool use, reasoning, and autonomous problem-solving. Second, this topic supports research on comprehensive methods to reduce the computational demands of frontier AI models and to ensure their safety, including technical methodologies such as automated testing and interpretability. The primary drivers behind computational efficient AI systems are the urgent challenges posed by the growing energy footprint of AI and current computational limitations. Modern AI models, especially frontier AI models, require substantial computational resources, with a significant impact in the environment. Additionally, they create barriers to entry to those interested in advancing the AI field. Key research areas include compression and distillation techniques aimed at reducing the complexity of large AI models. Innovations in AI architectures are also relevant, with a focus on innovative models that significantly lower computational demands for training and inference. Further, algorithmic approaches aimed at minimizing computational load during pre-training, post-training, and inference can also be considered. Ensuring the safety of AI systems is essential, especially as AI models become increasingly sophisticated and pervasive. Potential research areas to be considered include addressing misalignment, particularly the unintentional misalignment of large AI models. Work in this area could explore methods to detect and mitigate sophisticated misbehaviour, such as alignment faking, reward hacking of human oversight, and encoded reasoning in chain-of-thought (CoT). Additionally, research could focus on enhancing robustness against adversarial attacks…