Enabling robots to evolve from specialized, task-specific systems to versatile, adaptive generalist agents is an open and challenging problem. The rapid advancements in generative models, such as diffusion models and multimodal foundation models, have shown great potential in the development of generalist robots capable of performing a wide range of tasks across diverse environments. One key aspect of a generalist robot is the embodied multimodal intelligence, which emphasizes the comprehension of language and multisensory inputs, the grounding across different modalities, and the generation of action in environments based on these inputs. Frontier techniques in pre-training, post-training, reinforcement learning (RL), chain-of-thought reasoning, and simulation for multimodal robotic foundation models will be covered in this workshop, providing more insights on training generalizable and adaptive generative policies to the community.

Tool use isn't just for humans anymore — we've long known that animals like crows can manipulate objects with remarkable skill. Yet getting robots to handle tools with similar dexterity remains a major challenge. In this talk, I'll present our current efforts in combining Large Language Models with visual feedback to enable robots to understand and execute tool-based tasks. Our method translates natural language instructions into concrete motion sequences, guided by a new tool affordance model that helps the robot navigate even tight spaces. I'll demonstrate how this hybrid approach bridges the gap between human commands and robotic actions, bringing us closer to more adaptable and capable robotic systems. Through experimental results, I'll show how our methodology performs across different manipulation scenarios, highlighting both its current capabilities and future potential.

To power the next generation mobile robots and drones, the field of spatial perception has made much progress from robust multi-sensor SLAM to dense, semantic, and object-level maps, with the aim of understanding open-ended environments as a basis for mobile robot navigation and environment interaction. I will show recent progress in reliable and real-time state estimation and 3D scene understanding using vision, LiDAR, IMUs, and more. Scenes to be reconstructed may contain dynamic objects and even people, whose poses, postures, and motions we can estimate in a tightly-coupled manner. In our works, we fully embrace the power of machine learning-based approaches, but typically integrated in modular, complex robotic systems that may include model-based methods as well. Our approaches are demonstrated as crucial enablers of a range of robot applications, from mobile manipulation on construction sites to dronesexploring obstructed indoor spaces or flying through the forest.

Nature systems that have bodies with many degrees of freedom are often considered the ultimate models for machines. To confer the motion performance advantage of animal systems on robotic machines, we conducted in-depth studies on the motion characteristics of biological systems at the biomechanical level. We then used the insights that we obtained to develop intelligent biomimetic robots to achieve "intelligence," "environment adaptation," "flexibility," and "energy-saving." In this talk, I will introduce the bioinspired snake robots we have developed and discuss the evolution of their control methods, from shape-based to neural oscillator-based and then to embodied intelligence, to endow snake robots with motion intelligence.

Researchers have been working on the design of mobile robot body configurations and control algorithms for decades. This report describes Tencent Robotics X's research journey on robot body design, control algorithms, application background, and ecosystem construction over the past eight years. The content includes the wheeled balancing robot Robicycle, the quadruped robot Max, the wheel-legged robot Ollie, the industrial intelligent inspection robot, and the elderly care robot The Five. Recent works also include the data-driven approaches and applications based on the multi-robot platforms.

Mobile manipulators are increasingly deployed in complex environments, where effective perception and control are crucial for interaction. However, equipping every robot with a full suite of sensors is often impractical due to cost and design constraints. This challenge is particularly evident in non-prehensile manipulation tasks—such as pushing, sliding, or rolling objects—where high-fidelity sensory feedback can significantly impact performance. In this talk, I will discuss how AI-driven approaches can enable robots to adapt to varying sensor configurations and improve non-prehensile manipulation. A key challenge arises when a robot trained with rich sensory inputs, such as tactile skin, needs to be replaced or augmented by a system with a more limited sensor set, like LiDAR or RGB-D cameras. To address this, we propose a machine learning framework that allows robots to substitute missing sensory inputs by learning a mapping between available perception data and the information provided by absent sensors. Beyond sensor substitution, AI models can enhance non-prehensile manipulation by learning robust policies that generalize across different sensing modalities and task variations. I will present experimental results demonstrating how mobile manipulators can leverage AI to perform complex pushing tasks with limited sensing, achieving performance comparable to or even exceeding that of robots using direct tactile feedback. This approach paves the way for more adaptable and cost-effective robotic systems capable of learning and optimizing their interactions in diverse environments.

From collaborative industrial robots to personal AI assistants, the integration of AI into our daily lives highlights the critical need for effective and reliable coordination among agents, as well as between agents and humans. This challenge centers on creating agents that not only align with user intentions but also possess the flexibility to adapt to evolving circumstances, such as the introduction of novel agents. The pursuit of multi-agent cooperation extends beyond individual interactions to encompass broader societal considerations. In this talk, I will discuss the challenges of cooperative AI, and our contributions on multi-agent cooperation, human-ai coordination and cooperative alignments.

Recent developments in the fields of AI and Generative AI have enabled a new level of interaction between user and robot systems. For the first time the users can explain the task for the robot in fully unstructured natural language and robot motion can be generated accordingly. In this talk I will present activities conducted in ABB Corporate Research Center in Sweden that highlight different mechanisms in which AI and Generative AI can be introduced into robotic applications and industrial automation.