Vision-Based Robotic Manipulation through SE(3)-Equivariant Diffusion Models

The MLCS research team from the Department of Mechanical Engineering, led by Professor Jongeun Choi (first author: Master's student Hyunwoo Yoo), presented the world's first vision-based SE(3)-Equivariant Diffusion Models for robotic manipulation at the 2024 “IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),” the top international computer vision conference in the AI field (h5-index: 422). This study is a follow-up to the “Equivariant Descriptor Fields (EDFs)” research presented at the 2023 “International Conference on Learning Representations (ICLR),” the top international machine learning conference (h5-index: 303). To advance the diffusion model version of the previous research, new theories were established. By utilizing SE(3)-Equivariance, the model robustly handles previously unseen orientations of target objects, making it a data-efficient robotic learning model that does not require learning for every possible scenarios. While typical visual robot learning models require hundreds or thousands of hard-to-obtain expert training data points, this study achieves effective learning with only 5-10 data points. Additionally, unlike existing studies, this approach enables end-to-end learning that anyone can easily perform. Recognized for its excellence, this research was selected as a Highlight paper (top 2.8% of all submitted papers, top 11.9% of published papers) at the 2024 “IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).”

The link: https://openaccess.thecvf.com/content/CVPR2024/html/Ryu_Diffusion-EDFs_Bi-equivariant_Denoising_Generative_Modeling_on_SE3_for_Visual_Robotic_CVPR_2024_paper.html