Embodied AI focuses on creating robots that can perceive, reason, and act in the physical world. It brings together dexterous hand manipulation, humanoid whole-body control, and foundation models that link perception, language, and action. By combining these elements, embodied AI aims to enable robots that can generalize and perform complex real-world tasks with human-like adaptability. Dexhand Manipulation Dexterous hand manipulation focuses on enabling robots to interact with objects with the precision, adaptability, and coordination of human hands. [Read More…]

Recently developed foundation models, such as large language models and multi-modal models, open great opportunities to build generally capable agents, combined with reinforcement learning. This project focuses on learning skills and foundation models and connecting them to build generalist agents. In the following, we introduce some of our studies. For more details, please refer to the papers. Plan4MC We study building a multi-task agent in Minecraft. Without human demonstrations, solving long-horizon tasks in this open-ended environment with reinforcement learning (RL) is extremely sample inefficient. [Read More…]