The ELLIOT project (short for European Large Open Multi-Modal Foundation Models For Robust Generalization On Arbitrary Data Streams) marks a new chapter in European artificial intelligence (AI) research. This four-year initiative is funded by the Horizon Europe programme with a budget of €25 million and brings together 30 leading institutions from 12 European countries. Its goal is to develop a new generation of trustworthy and general-purpose AI models capable of processing diverse real-world data streams.
From the Czech Republic, the Czech Technical University in Prague (CTU), specifically the Czech Institute of Informatics, Robotics and Cybernetics (CIIRC), and the innovation-driven deep-tech startup RoboTwin are participating in the project. CIIRC CTU will serve as the main national point of contact for disseminating the ELLIOT project’s results to both the research community and industry stakeholders.
At the core of the ELLIOT project (European Large Open Multi-Modal Foundation Models For Robust Generalization On Arbitrary Data Streams) is the development of a new generation of open, multimodal foundation models. These are AI systems capable of learning from vast amounts of diverse data—ranging from text, images, and videos to satellite imagery, sensor signals, and industrial manufacturing data. The aim is for these models to leverage acquired general knowledge across a broad spectrum of tasks, from robotics and autonomous driving to natural phenomena prediction.
Unlike current models that struggle with generalization and multimodal integration, the models developed under ELLIOT will support robust generalization and simultaneous processing of multiple data types. Crucially, they will be able to adapt to complex, dynamic, and noisy real-world conditions—such as those encountered in autonomous driving, drone-based environmental monitoring, or robotic manipulation.
The Czech Institute of Informatics, Robotics and Cybernetics at the Czech Technical University in Prague (CIIRC CTU) is contributing to a wide range of core research tasks within ELLIOT, including leading roles in several areas. The institute focuses primarily on adapting (fine-tuning) large multimodal models to real-world scenarios and evaluating their capabilities. A key research area is robotic perception, where models interpret human instructions in the form of language, gestures, or images in combination with sensor and camera data, enabling them to plan and execute robot actions.
Another major focus is the safety and robustness of these models—their ability to withstand attacks, such as training data manipulation. CIIRC will help develop evaluation tools to ensure compliance with both technical and ethical requirements, aligned with the new European AI Act.
“We already have advanced generative AI models—well-known examples like ChatGPT or DALL·E can process text and images,” explains Karla Štěpánová, head of the Robotic Perception Group at CIIRC CTU. “However, most current models cannot yet efficiently handle other critical modalities for human-like perception and decision-making, such as gestures or tactile input. They also lack temporal understanding—essential for modeling motion in dynamic environments, tracking manufacturing processes, or detecting landscape changes in satellite data. ELLIOT will fill these gaps with multimodal spatiotemporal models.”
CIIRC CTU researchers are also involved in numerous supporting tasks, such as testing, fine-tuning, and optimizing models for computational efficiency and energy consumption. The team brings extensive experience in leveraging national and European supercomputing infrastructure. In the past year alone, researchers used over 400,000 GPU hours on the Karolina supercomputer in Ostrava and the LUMI system in Finland, one of the most powerful supercomputers in the world.
“Our teams will also focus on developing tools to enhance the resilience of these new AI systems against cyberattacks and improve their overall safety,” adds Vladimír Petrík, machine perception expert for robotics in Josef Šivic’s IMPACT team at CIIRC CTU. “Equally important is our institute’s role in transferring and adapting foundation models to concrete applications in manipulation robotics, where CIIRC will collaborate closely with the company RoboTwin.”
RoboTwin will primarily contribute to the part of the project focused on collecting and generating domain-specific data essential for adapting foundation models to real-world scenarios such as robotic assembly, manipulation, or automation of manufacturing operations. The RoboTwin team will also validate the adapted models in real robotic applications within the manufacturing sector. With its strong focus on robotics and AI innovation, RoboTwin plays a key role in translating ELLIOT’s research into practical industrial use cases.
ELLIOT is closely aligned with European and international open-source and open-science communities, such as LAION and open-sci, and will build on their extensive experience in developing open foundation models and datasets. This collaboration ensures that the entire development pipeline—from data creation to model training, fine-tuning, and evaluation—remains open, reproducible, and verifiable, allowing easy adaptation for both industrial and public-sector needs.
“ELLIOT will significantly strengthen Europe’s sovereignty and independence in AI, particularly in the fast-evolving field of multimodal foundation models. By leveraging European scientific excellence, computing infrastructure, and a strong commitment to open and trustworthy innovation, the project marks a major step toward shaping Europe’s own future in AI,” says Dr. Yiannis Kompatsiaris, project coordinator from the Centre for Research and Technology Hellas (CERTH‑ITI), Greece.
The project will utilize cutting-edge European supercomputing infrastructure—including EuroHPC systems like JUPITER, Leonardo, MareNostrum, LUMI, and Alps at the Swiss National Supercomputing Centre (CSCS). It will combine real and synthetic data from trusted sources to train and develop a suite of open models, datasets, and methodologies. This will empower the AI community in Europe and globally to create, explore, deploy, scale, and evaluate open models in a fully reproducible way—at a scale previously unavailable to academia. ELLIOT will act as a catalyst for open research and open-source approaches in AI, reinforcing the vision of European AI sovereignty.
Project outcomes will also enable breakthrough applications and give Europe a competitive edge in areas such as media, Earth modeling, robotic perception, autonomous systems, computational engineering, and workflow automation. Key research activities will include community building and training the next generation of European machine learning and AI researchers, building on the internationally recognized ELLIS Society (European Laboratory for Learning and Intelligent Systems), whose Czech node—ELLIS Unit Prague—is led and hosted by CIIRC CTU.
The consortium is coordinated by the Information Technologies Institute of the Centre for Research and Technology Hellas (CERTH-ITI) in Greece and includes leading academic and research institutions, innovative SMEs, public bodies, and nonprofit organizations with a broad range of expertise.
