EU Frontier AI Initiative: Advanced Scaling Frameworks for High-Performance AI Models (RIA) - Apply AI

Apie kvietimą

Expected Outcome: This topic aims to advance the scientific and technical foundations for the scaling of AI models, enhancing the performance of training and inference processes and providing European AI labs with the methodologies required to compete at the global frontier. Project results are expected to contribute to some of the following outcomes: Establishment of robust scaling laws and predictive frameworks that allow for the reliable estimation of model performance before large-scale compute investment. Enhanced competitiveness of European advanced AI models through optimised dataset mixtures, refined architectures, and superior hyperparameter configurations. Optimised development for frontier AI models, achieved via automated and scientifically grounded scaling methodologies. Scope: To maintain a leading position in AI, Europe must master the methodologies required to scale AI models’ performance, training and inference processes. Establishing a predictive methodology for AI model development ensures that limited computational resources are deployed with maximum efficiency, thereby bolstering the EU and AC’s technological sovereignty. While previous actions have focused on training High-Performance AI models, this topic addresses the research gap in the underlying theoretical basis, methodologies and tools for predictable scaling of AI models and inference. By providing the technical foundations for high-performance systems without the costs of unguided experimentation, this action complements training-centric efforts. In this light, proposals should deliver, as an outcome of the research, a toolset designed to support the scaling of AI models. Potential research areas include, but are not limited to: Automated Scaling Law Derivation: Research into scaling laws to predict performance across diverse modalities and tasks, moving beyond simple power laws. Exploratory Architecture Search: Developing automated methods to discover and refine model architectures that maintain performance while scaling both in training and inference, including innovations in sparsity, memory efficiency, and long-context handling. Dataset Mixture Optimisation: Research into the systematic selection, weighting, and sequencing of datasets. This includes developing tools for automated data quality assessment. Advanced Hyperparameter Tuning and Stability: Developing methodologies for stable training at large scales, including automated tuning of optimisers and learning rate schedules that remain effective as models grow. Proposals should ensure that the developed frameworks and scaling methodologies are compatible with diverse hardware architectures, demonstrating that scaling is studied across different systems to avoid optimisation for specific proprietary hardware. The selected project should actively support Advanced AI labs and High-performance Computing Centres already engaged in the development of frontier AI, ensuring that methodological advances are…