Posts by Collection

portfolio

publications

See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion

Published in Science Robotics, 2019

This work proposes a methodology for a robot to learn complex manipulation skills, exemplified by playing Jenga, by emulating hierarchical reasoning and multisensory fusion. The approach involves formulating game mechanics using a temporal hierarchical Bayesian model, capturing latent structures in force and visual domains. The robot then uses these learned representations to infer block behavior patterns and states, adjusting its actions and strategy similar to human gameplay. The method demonstrates fidelity in a real-world implementation compared to standard baselines.

Recommended citation: Fazeli, N., Oller, M. , Wu, J., Wu, Z., Tenenbaum, J. B., & Rodriguez, A. (2019). "See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion." Science Robotics. https://www.science.org/doi/full/10.1126/scirobotics.aav3123

Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control

Published in 6th Conference on Robotic Learning (CoRL 2022), 2022

Our method learns soft tactile sensor membrane deformation dynamics to control a grasped object’s pose and force transmitted to the environment during contact-rich manipulation tasks such as drawing and in-hand pivoting.

Recommended citation: Oller, M. , i Lisbona, M. P., Berenson, D., & Fazeli, N. . (2022). " "Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control". " CoRL 2022 1. https://arxiv.org/pdf/2209.13432.pdf

TactileVAD: Geometric Aliasing-Aware Dynamics for High-Resolution Tactile Control

Published in 7th Conference on Robotic Learning (CoRL 2023), 2023

Our paper introduces TactileVAD, a decoder-only control method that resolves tactile geometric aliasing, improving performance and reliability in touch-based manipulation across various tactile sensors.

Recommended citation: Oller, M. , Berenson, D., & Fazeli, N. (2023). " "TactileVAD: Geometric Aliasing-Aware Dynamics for High-Resolution Tactile Control". " CoRL 2023 1. https://openreview.net/pdf?id=FefFLN5FvIM

talks

teaching

Introduction to Manipulation

Graduate Course, University of Michigan, Robotics, Fall 2020 & Fall 2021

This course is an introduction to the field of manipulation. The course covers the fundamentals of manipulation, including kinematics, dynamics, control, and planning. The course also covers the fundamentals of grasping and manipulation, including grasp planning, grasp stability, and manipulation planning.

Robot Learning for Planning and Control

Graduate course, University of Michigan, Robotics, Winter 2023

An introduction to modern machine learning methods for control and planning in robotics. Topics include function approximation, learning dynamics, using learned dynamics in control and planning, handling uncertainty in learned models, learning from demonstration, and model-based and model-free reinforcement learning. Students implement the above learning algorithms on robots in simulation.