See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion
Published in Science Robotics, 2019
Recommended citation: Fazeli, N., Oller, M. , Wu, J., Wu, Z., Tenenbaum, J. B., & Rodriguez, A. (2019). "See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion." Science Robotics. https://www.science.org/doi/full/10.1126/scirobotics.aav3123
This work proposes a methodology for a robot to learn complex manipulation skills, exemplified by playing Jenga, by emulating hierarchical reasoning and multisensory fusion. The approach involves formulating game mechanics using a temporal hierarchical Bayesian model, capturing latent structures in force and visual domains. The robot then uses these learned representations to infer block behavior patterns and states, adjusting its actions and strategy similar to human gameplay. The method demonstrates fidelity in a real-world implementation compared to standard baselines.
[//]: # (Recommended citation: Your Name, You. (2009). “Paper Title Number 1.” Journal 1. 1(1).) Recommended citation: Fazeli, N., Oller, M., Wu, J., Wu, Z., Tenenbaum, J. B., & Rodriguez, A. (2019). See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion. Science Robotics, 4(26), eaav3123