Carnegie Mellon University Personal Robotics Lab

RI 16-843: Manipulation Algorithms

Instructors: Katharina Muelling (kmuelling -- and Henny Admoni (henny --

Lectures: T/Th, 9:00am--10:20am, NSH 3002


This is an advanced graduate-level class on the theory and algorithms that enable robots to physically manipulate their world, on their own or in collaboration with people. The class will first focus on functional aspects of manipulation, such as synthesizing robust and stable grasps for dexterous hands and motion planning in these spaces, as well as learning for manipulation, such as how to predict stable grasps from demonstration and experience. Moving forward, we will discuss additional requirements that arise from performing manipulation tasks collaboratively with people: moving from purely functional aspects of motion to incorporating the human into the loop, and coordinating human and robot actions via understanding and expressing intent.

Course Syllabus

Important Dates

October 11
Project proposals due
November 1
Project check-in due
December 8
Final presentations
December 15
Final report due
December 21
Final grades due


# Day Date Topic Reading Notes
1TAug 30 Introduction Creation Myths: The Beginnings of Robotics Research, Mason, IEEE RAM 19(2), 2012. URL
The Summer Vision Project, Papert, AIM, 1966. URL
2ThSep 1 Robotic Foundations Planning Algorithms: Chapter 3, LaValle, 2006. PDF Lecture
3TSep 6 Configuration Spaces Spatial Planning: A Configuration Space Approach, Lozano-Perez, 1980. PDF Lecture
4ThSep 8 Experiment Design Challenges for Robot Manipulation in Human Environments, Kemp et al., 2007 PDF Lecture
5TSep 13 Experiment Design Toward a Science of Robotics: Goals and Standards for Experimental Research, Takayama, 2009. PDF Lecture
6ThSep 15 Student Presentations Paul Calhoun: Developing performance metrics for the supervisory control of multiple robots, Crandall & Cummings, 2007. PDF
Xiang Zhi Tan: The YCB Object and Model Set: Towards Common Benchmarks for Manipulation Research, Calli et al., 2015. PDF
7TSep 20 Student Presentations Siwei Zhu: Real-Time Configuration Space Transforms for Obstacle Avoidance, Newman & Branicky, 1991. PDF
TBD: A simple motion planning algorithm for general robot manipulation, Lozano-Perez, 1986. PDF
8ThSep 22 Motion Planning Lecture
9TSep 27 Student Presentations Rushat Gupta Chadha: Manipulation Planning among moving obstacles, Stilman et al., 2007. PDF
Puneet Puri: Randomized Kinodynamic Planning, LaValle and Kuffner, 2001. PDF
10ThSep 29 Motion Planning Lecture
11TOct 4 Project Overview Lecture
12ThOct 6 Motion Planning Lecture
13TOct 11 Student Presentations Prasanth Palli: Data-Driven Grasp Synthesis - A Survey, Bohg et al., 2014. PDF
Shivam Gautam: Chomp - Gradient Optimization Techniques for Efficient Motion Planning, Ratliff et al., 2009. PDF
Mohak Bhardwaj: Planning Motions with Intentions, Koga et al., 1994. PDF
14ThOct 13 Grasping Chapter 5, A Mathematical Introduction to Robotic Manipulation, Murray et al., 1994. PDF Lecture
15TOct 18 Student Presentations Zhe Zhong: Grasp Synthesis in Cluttered Environments for Dexterous Hands, Berenson et al., 2008. PDF
Gauri Gandhi: Automatic Grasp Planning Using Shape Primitives, Miller et al., 2003. PDF
Ningyan Zhu: Passive Force Closure and its Computation in Compliant Rigid Grasps, Shapiro et al., 2001. PDF
16ThOct 20 Grasping Lecture
17TOct 25 Student Presentations Richa Varma: Anytime RRTs, Ferguson and Stentz, 2006. PDF
Kazu Otani: Visibility-based probabilistic Roadmaps for Motion Planning, Simeon, 2000. PDF
Smruti Amarjyoti: Learning object-specific grasp affordance densities, Detry et al., 2009. PDF
Focus Questions
18ThOct 27 Learning from Demonstration Lecture
19TNov 1 Student Presentations Luis Valle: E-Graphs: Bootstrapping Planning with Experience Graphs, Phillips et al., 2012. PDF
Dorothy Kirlew: Apprenticeship Learning via Inverse Reinforcement Learning, Abbeel & Ng, 2004. PDF
Focus Questions
20ThNov 3 Learning from Demonstration Lecture
21TNov 8 Student Presentations Puru Rastogi: Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields, Park et al., 2008. PDF
Lekha Walajapet Mohan: Policy Search for Motor Primitives in Robotics, Kober & Peters, 2009. PDF
Siwei Zhu: Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection, Levine et al., 2016. PDF
Focus Questions
22ThNov 10 Task Planning Lecture
23TNov 15 Student Presentations Paul Calhoun: Goal based action priors, Abel et al., 2015. PDF
Yujie Fang: Constructing skill trees by demonstration, Konidaris et al., 2011. PDF
Zhichao Jin: Hierarchical Planning in the Now, Kaelbling and Lozano-Perez, 2011. PDF
Focus Questions
24ThNov 17 Guest Lecture Learning and Modeling
25TNov 22 Student Presentations and Make-Up Quiz Mohak Bhardwaj: Legibility and Predictability of Robot Motion, Dragan et al., 2013. PDF
Lekha Walajapet Mohan: A Human Aware Mobile Robot Motion Planner, Sisbot et al., 2007. PDF
Focus Questions
26ThNov 24 No class - CMU holiday
27TNov 29 Student Presentation and Human-Robot Collaboration Zhichao Jin: Robots for Humanity: Using Assistive Robots to Empower People with Disabilities, Chen et al., 2013. PDF Focus Questions
28ThDec 1 Guest lecture Intent Recognition
29TDec 6 Student Presentation and Task Planning Puru Rastogi: FF: The Fast-Forward Planning System, Hoffmann, 2001. PDF Focus Questions
30ThDec 8 Final Presentations Presentation Schedule