This is an old revision of the document!


Tested with Cram v0.7.0, ROS version: Kinetic, Ubuntu 16.04

Simple Mobile Manipulation Plan

This tutorial demonstrates how to write a simple mobile manipulation plan and error handling and recovery behaviors for it. In this tutorial we will be using the Bullet world as a lightweight simulator. But as plans in CRAM are independent of the hardware platform, the same plan that can be executed in simulation can just as well be executed on the real robot, if we stick to motion and action designators defined in CRAM.

This tutorial assumes that you have a basic understanding of plans, and have undergone the Bullet World tutorial, and are now familiar with manipulating various elements in the Bullet world simulation. You can find the code of this tutorial on Github.

Installing the projection project

As the system we will be working with will get quite complex things might not go as expected. If you get into trouble while following this tutorial feel free to consult the Support page or contact the person who last edited this tutorial, you can find the username in the footer.

If you followed the installation instructions for the full CRAM on the Installation page, you should be ready to go.

Environment Setup

For this tutorial we need to set up the environment similar to how it is done for the real system: we will need an occupancy map for navigation, an IK solver for manipulation, a map of the static parts of the environment with the furniture etc.

First, let's set up the environment in our terminal by calling the launch file. It already brings up a roscore from within:

$ roslaunch cram_bullet_world_tutorial world.launch

REPL Setup

Now, let's load the package in the REPL (start the REPL with $ roslisp_repl):

CL-USER> (ros-load:load-system "cram_bullet_world_tutorial" :cram-bullet-world-tutorial)
CL-USER> (in-package :cram-bullet-world-tutorial)
BTW-TUT> (roslisp-utilities:startup-ros)

You'll have your bullet world launched with the PR2 in the kitchen. Now you're good to go.

Some Useful Designators

Before we proceed further let's look at some of the designators that are supported by the robot in the bullet world (and this works in the real world robot as well). To refresh your memory on designators, visit the tutorial Creating motion designators for the TurtleSim. Here are some of the supported motion and action designators:

  • Motion Designators - They describe low-level atomic motions that the robot can execute.
    •  (desig:a motion (type moving-torso) (joint-angle ?angle-val))

      is a motion for moving the torso of the robot and expects a key joint-angle with an value of the required joint angle.

    •  (desig:a motion (type going) (target ?loc-desig)) 

      is a motion that moves the body of the robot and expects a key target, whose value should be of type location-designator.

    •  (desig:a motion (type looking) (target ?loc-desig))

      is a motion that moves the head of the robot to look at the specfied location. It expects a key target with value of type location designator.

    •  (desig:a motion (type detecting) (object ?obj-desig)) 

      is a a motion that perceives and detects the specified object. It expects a key object with a value of type object-designator

  • Action Designators - These describe the high-level actions which may consist of multiple calls to different low-level motion to carry out a small plan.
    •  (desig:an action (type picking-up) (arm ?grasp-arm) (grasp ?grasp-pose) (object ?obj-desig)) 

      is an action that picks an object with the specified arm and grasp pose. It expects an object key with value of type object-designators, an arm with the value specifying the keyword for the arm to choose, and a grasp key whose value specifies the keyword of the grasp pose to pick the object up from.

    •  (desig:an action (type placing) (arm ?grasp-arm) (object ?obj-desig) (target ?loc-desig)) 

      is an action that places the object specified to the target. It expects a key arm with the keyword of the arm to use, an object specifying value of type object-designator and a target key with value of type location-designator.

We will use these designators throughout this tutorial.

Constructing plans

The goal of this chapter is to write a plan for a simple task - pick and place an object from one position to another position in the world - something you'll already be familiar with from the previous tutorial. Let's go ahead and dive into the code:

Any time you want a clean environment, you can run:

BTW-TUT> (init-projection)

Now, Let's define a method to spawn a bottle into the world and apply the physics using the Bullet physics engine and simulate the world for 10 seconds, and run it:

BTW-TUT>
(defun spawn-bottle ()
  (unless (assoc :bottle btr::*mesh-files*)
    (add-objects-to-mesh-list))
  (prolog:prolog
   `(and (btr:bullet-world ?world)
         (assert (btr:object ?world :mesh bottle-1
                             ((-1.6 -0.9 0.860) (0 0 0 1))
                             :mass 0.2 :color (1 0 0) :mesh :bottle))
         (btr:simulate ?world 100))))
BTW-TUT> (spawn-bottle)

You should now see a red bottle on the table. To make our lives easier, let's define some parameters. The *final-object-destination* is the place where the pr2 has to place the red bottle in the end. The *base-pose-near-table* is the pose, where pr2 should stand near the table, when he approaches the bottle and *base-pose-near-counter* is the pose, to access the the counter to place the bottle.

BTW-TUT>
(defparameter *final-object-destination*
  (cl-transforms-stamped:make-pose-stamped
   "map" 0.0
   (cl-transforms:make-3d-vector -0.8 2 0.9)
   (cl-transforms:make-identity-rotation)))
 
(defparameter *base-pose-near-table*
  (cl-transforms-stamped:make-pose-stamped
   "map" 0.0
   (cl-transforms:make-3d-vector -1.447d0 -0.150d0 0.0d0)
   (cl-transforms:axis-angle->quaternion (cl-transforms:make-3d-vector 0 0 1) (/ pi -2))))
 
(defparameter *downward-look-coordinate*
  (cl-transforms-stamped:make-pose-stamped
   "base_footprint" 0.0
   (cl-transforms:make-3d-vector 0.65335d0 0.076d0 0.758d0)
   (cl-transforms:make-identity-rotation)))
 
(defparameter *base-pose-near-counter*
  (cl-transforms-stamped:make-pose-stamped
   "base_footprint" 0.0
   (cl-transforms:make-3d-vector -0.150d0 2.0d0 0.0d0)
   (cl-transforms:make-quaternion 0.0d0 0.0d0 -1.0d0 0.0d0)))

Let's note down the steps that would involve picking the bottle from the table and placing it on the counter.

  • Move the robot near the table.
  • Move the arms out of the perception range, so that the robot can see the bottle without obstruction.
  • Look towards the object area.
  • Detect the object that has to be picked.
  • Pick up the object and once again, keep the arms away from the front.
  • Move the robot near the counter.
  • Place the bottle on the destination.

Implementing all the above steps in code, will look something like as shown below:

(defun move-bottle ()
  (spawn-bottle)
  (pr2-proj:with-simulated-robot
    (let ((?navigation-goal *base-pose-near-table*))
      (cpl:par
        (exe:perform (desig:a motion 
                              (type moving-torso)
                              (joint-angle 0.3)))
        (pp-plans::park-arms)
        ;; Moving the robot near the table.
        (exe:perform (desig:a motion
                              (type going)
                              (target (desig:a location 
                                               (pose ?navigation-goal)))))))
    ;; Looking towards the bottle before perceiving.
    (let ((?looking-direction *downward-look-coordinate*))
      (exe:perform (desig:a motion 
                            (type looking)
                            (target (desig:a location 
                                             (pose ?looking-direction))))))
    ;; Detect the bottle on the table.
    (let ((?grasping-arm :right)
          (?perceived-bottle (exe:perform (desig:a motion
                                                   (type detecting)
                                                   (object (desig:an object 
                                                                     (type :bottle)))))))
      ;; Pick up the bottle
      (exe:perform (desig:an action
                             (type picking-up)
                             (arm ?grasping-arm)
                             (grasp left-side)
                             (object ?perceived-bottle)))
      (pp-plans::park-arms :arm ?grasping-arm)
      ;; Moving the robot near the counter.
      (let ((?nav-goal *base-pose-near-counter*))
        (exe:perform (desig:a motion
                              (type going)
                              (target (desig:a location 
                                               (pose ?nav-goal))))))
      (coe:on-event (make-instance 'cpoe:robot-state-changed))
      ;; Setting the bottle down on the counter
      (let ((?drop-pose *final-object-destination*))
        (exe:perform (desig:an action
                               (type placing)
                               (arm ?grasping-arm)
                               (object ?perceived-bottle)
                               (target (desig:a location 
                                                (pose ?drop-pose))))))
      (pp-plans::park-arms :arm ?grasping-arm))))

Note that the plan is nested under pr2-proj:with-simulated-robot indicating that all the methods resolved by the designators is being called for the projection environment (You can see these low level methods being called under the cram_pr2_projections package under low-level.lisp). If pr2-pms:with-real-robot is used to replace this, the functions from the real robot ROS interfaces will be called. Also note that it is possible to execute actions which are independent in parallel (In this case, moving the robot torso up and moving the robot base are independent actions done in parallel)

Now run (move-bottle) and you will see that the robot successfully picks the bottle and places it on the counter.

Note:- Every time you run move-bottle you need to reset the world with (init-projection). You can include the call at the top of the test function, but maybe you want to comment some parts out to see how the plan behaves.

Increasing the Effectiveness by Improving the plan

The previous example worked perfectly because, we knew the exact coordinates to look for the bottle. This is hardly true for the real life scenario, especially since we are dealing with a kitchen environment, far from being precise compared to a factory floor. What would happen if the bottle was moved a little bit to the right of its previous spawn position? For this, let's redefine our spawn-bottle

BTW-TUT>
(defun spawn-bottle ()
  (unless (assoc :bottle btr::*mesh-files*)
    (add-objects-to-mesh-list))
  (prolog:prolog
   `(and (btr:bullet-world ?world)
         (assert (btr:object ?world :mesh bottle-1
                             ((-2 -0.9 0.860) (0 0 0 1))
                             :mass 0.2 :color (1 0 0) :mesh :bottle))
         (btr:simulate ?world 100))))

Now try running (move-bottle) again. Don't forget to clean up the projection before running. And the output will look like this.

BTW-TUT> (move-bottle)
; Evaluation aborted on #<CRAM-COMMON-FAILURES:PERCEPTION-OBJECT-NOT-FOUND {1013AC5B93}>.

Clearly the robot cannot find the object anymore because even though the robot is near the bottle, it is out of the field of vision of the robot due to the predefined base and object pose in our code. Let's try fixing this issue.

Recovering from Failures

To understand the syntax and get a refresher on failure handling, you can refer to Failure Handling. We're going to add some code into fixing perception in our case, so that the robot would still be able to find the bottle. Let's list down a plan of action for this.

  1. Tilt the head of the robot downwards
  2. Try to detect the bottle
  3. If,
    • successful in finding the bottle - continue with the rest of the code.
    • failed to find the bottle - turn the head to a different configuration (eg., left/right) and try detecting again.
  4. When all possible directions fail, error out.

Let's define some additional parameters to aid us:

BTW-TUT>
(defparameter *left-downward-look-coordinate*
  (cl-transforms-stamped:make-pose-stamped
   "base_footprint" 0.0
   (cl-transforms:make-3d-vector 0.65335d0 0.76d0 0.758d0)
   (cl-transforms:make-identity-rotation)))
 
(defparameter *right-downward-look-coordinate*
  (cl-transforms-stamped:make-pose-stamped
   "base_footprint" 0.0
   (cl-transforms:make-3d-vector 0.65335d0 -0.76d0 0.758d0)
   (cl-transforms:make-identity-rotation)))
 

We defined two coordinates *left-downward-look-coordinate* and *right-downward-look-coordinate* with respective to our robot base footprint as alternative directions to look for when downward look fails.

Now we define a method find-object which encapsulates our plan with failure handling :

(defun get-preferred-arm-for-direction (direction-looked)
  (let ((preferred-arm :RIGHT))
    (when (eq direction-looked *left-downward-look-coordinate*)
      (setf preferred-arm :LEFT))
    preferred-arm))
 
(defun find-object (?object-type)
  (let* ((possible-look-directions `(,*downward-look-coordinate*
                                     ,*left-downward-look-coordinate*
                                     ,*right-downward-look-coordinate*))
         (?looking-direction (first possible-look-directions)))
    (setf possible-look-directions (cdr possible-look-directions))
    (exe:perform (desig:a motion 
                          (type looking)
                          (target (desig:a location 
                                           (pose ?looking-direction)))))
 
    (cpl:with-failure-handling
        ((cram-common-failures:perception-object-not-found (e)
           ;; Try different look directions until there is none left.
           (when possible-look-directions
             (roslisp:ros-warn (perception-failure) "~a~%Turning head." e)
             (exe:perform (desig:a motion 
                                   (type looking) 
                                   (direction forward)))
             (setf ?looking-direction (first possible-look-directions))
             (setf possible-look-directions (cdr possible-look-directions))
             (exe:perform (desig:a motion 
                                   (type looking)
                                   (target (desig:a location
                                                    (pose ?looking-direction)))))
             (cpl:retry))
           (cpl:fail 'common-fail:looking-high-level-failure)))
 
      (let ((?perceived-bottle
              (exe:perform (desig:a motion
                                    (type detecting)
                                    (object (desig:an object 
                                                      (type ?object-type)))))))
        (values ?perceived-bottle (get-preferred-arm-for-direction ?looking-direction))))))
 

Let's see what this method does. The with-failure-handling clause here deals with cram-common-failures:perception-object-not-found which if you remember, was the error raised when the robot couldn't find the bottle. So instead of only looking downwards, the code will now iterate between downwards, left and right, and only upon a failure in all these, will an error be bubbled up - thus increasing our effective field of view. Also note that the find-object suggests an arm to use, depending on the direction it looked. Let us also update our move-bottle to use this method.

(defun move-bottle ()
  (spawn-bottle)
  (pr2-proj:with-simulated-robot
    (let ((?navigation-goal *base-pose-near-table*))
      (cpl:par
        (exe:perform (desig:a motion 
                              (type moving-torso)
                              (joint-angle 0.3)))
        (pp-plans::park-arms)
        ;; Moving the robot near the table.
        (exe:perform (desig:a motion
                              (type going)
                              (target (desig:a location 
                                               (pose ?navigation-goal)))))))
    ;; Find and detect the bottle on the table.
    (multiple-value-bind (?perceived-bottle ?grasping-arm) 
        (find-object :bottle)
      (exe:perform (desig:an action
                             (type picking-up)
                             (arm ?grasping-arm)
                             (grasp left-side)
                             (object ?perceived-bottle)))
      (pp-plans::park-arms :arm ?grasping-arm)
      ;; Moving the robot near the counter.
      (let ((?nav-goal *base-pose-near-counter*))
        (exe:perform (desig:a motion
                              (type going)
                              (target (desig:a location 
                                               (pose ?nav-goal))))))
 
      (coe:on-event (make-instance 'cpoe:robot-state-changed))
      ;; Setting the object down on the counter
      (let ((?drop-pose *final-object-destination*))
        (exe:perform (desig:an action
                               (type placing)
                               (arm ?grasping-arm)
                               (object ?perceived-bottle)
                               (target (desig:a location 
                                                (pose ?drop-pose))))))
      (pp-plans::park-arms :arm ?grasping-arm))))

Clean up and run (move-bottle) again, and this time, you'll find that the robot succeeds in transporting the bottle.

Expanding Failure Management Capabilities

Everything is good so far, but let's call this a lucky coincidence. For the robot, knowing which arm to use to pick up the bottle is not always enough. There are many positions with which we can grasp objects - from the object's front, back, left, right, etc. One might think that since the bottle is a rotationally symmetric object, it doesn't matter which side you approach from. But consider the bottle as an object model, which has a specific front, back and sides according to its orientation with respect to the bullet world. In which case, the side with which the arm approaches the bottle greatly matters, accounting for the configuration of the joints the robot arm will make while trying to grasp - some poses may be unachievable, while others may result in the collision of the arm with the table.

Let's try to visualize this issue, by spawning the bottle in yet another position:

(defun spawn-bottle ()
  (unless (assoc :bottle btr::*mesh-files*)
    (add-objects-to-mesh-list))
  (prolog:prolog
   `(and (btr:bullet-world ?world)
         (assert (btr:object ?world :mesh bottle-1
                             ((-1.1 -0.75 0.860) (0 0 0 1))
                             :mass 0.2 :color (1 0 0) :mesh :bottle))
         (btr:simulate ?world 100))))

Now run move-bottle once more, and the output should be something similar to as below:

BTW-TUT> (move-bottle)
[(PICK-PLACE PICK-UP) INFO] 1550502686.470: Opening gripper
[(PICK-PLACE PICK-UP) INFO] 1550502686.470: Reaching
[(PICK-PLACE MOVE-ARMS-IN-SEQUENCE) WARN] 1550502686.797: #<POSE-STAMPED 
   FRAME-ID: "torso_lift_link", STAMP: 0.0
   #<3D-VECTOR (0.6819813024319948d0 0.4206671881870251d0 -0.11278482277792945d0)>
   #<QUATERNION (-0.005172943672216379d0 0.0055962335340426494d0 -0.7845776913102387d0 0.6199836767360266d0)>> is unreachable for EE.
Ignoring.
[(PICK-PLACE MOVE-ARMS-IN-SEQUENCE) ERROR] 1550502687.092: #<POSE-STAMPED 
   FRAME-ID: "torso_lift_link", STAMP: 0.0
   #<3D-VECTOR (0.6797228574484719d0 0.4210222500057509d0 -0.2627674055849005d0)>
   #<QUATERNION (-0.005172943672216379d0 0.0055962335340426494d0 -0.7845776913102387d0 0.6199836767360266d0)>> is unreachable for EE.
Failing.
[(PP-PLANS PICK-UP) WARN] 1550502687.092: Manipulation messed up: #<POSE-STAMPED 
   FRAME-ID: "torso_lift_link", STAMP: 0.0
   #<3D-VECTOR (0.6797228574484719d0 0.4210222500057509d0 -0.2627674055849005d0)>
   #<QUATERNION (-0.005172943672216379d0 0.0055962335340426494d0 -0.7845776913102387d0 0.6199836767360266d0)>> is unreachable for EE.
Ignoring.
; Evaluation aborted on #<CRAM-COMMON-FAILURES:MANIPULATION-POSE-UNREACHABLE {10100E64C3}>.

The robot has failed to grasp again, even though the bottle is well within perception and grasping range.

So what went wrong? If you go back to definition of get-preferred-arm-for-direction:

(defun get-preferred-arm-for-direction (direction-looked)
  (let ((preferred-arm :RIGHT))
    ;; Always prefers right arm unless looking at the left side
    (when (eq direction-looked *left-downward-look-coordinate*)
      (setf preferred-arm :LEFT))
    preferred-arm))

And the part where we pick up the object:

        (exe:perform (desig:an action
                               (type picking-up)
                               (arm ?grasping-arm)
                               (grasp left-side)
                               (object ?perceived-bottle)))

We see that the robot defaults the right arm when the object is in the center and will always try to grasp the left side of the bottle, even when the object is slightly favoring the left side under it (Refer the figure below).

Once again, let's formulate a strategy like the previous case here. The plan we are going for will look something like this:

  1. Choose the favored arm.
  2. Get all possible grasp poses for the given type of the object and the arm.
  3. Choose one grasp from the possible grasp list.
  4. Try to grasp the object.
  5. If,
    • Grasp succeeds - Continue with the rest of the code
    • Grasp Fails - Choose a different grasp pose from the list.
  6. When no more possible grasp poses, try changing the arm used to grasp Once and try resuming from Step (2)
  7. When attempted with all arms and grasps, error out.

Let's encapsulate all this in a method called pick-up-object:

(defun pick-up-object (?perceived-object ?object-type ?grasping-arm)
  (let ((?possible-arms '(:right :left)))
    ;;Retry by changing the arm
    (cpl:with-retry-counters ((arm-change-retry 1))
        (cpl:with-failure-handling
            ((common-fail:object-unreachable (e)
               (roslisp:ros-warn (arm-failure) "Manipulation failed: ~a~%" e)
               (cpl:do-retry arm-change-retry
                 (setf ?grasping-arm (car (remove ?grasping-arm ?possible-arms)))
                 (cpl:retry))
               (roslisp:ros-warn (arm-failures) "No more retries left")))
 
          ;; Retry by changing the grasp
          (let* ((?possible-grasp
                   (cram-object-interfaces:get-object-type-grasps ?object-type nil nil nil ?grasping-arm))
                 (?grasp (cut:lazy-car ?possible-grasp)))
            (cpl:with-retry-counters ((grasp-retries 3))
              (cpl:with-failure-handling
                  (((or cram-common-failures:manipulation-pose-unreachable
                        cram-common-failures:gripper-closed-completely) (e)
                     (roslisp:ros-warn (grasp-failure)
                                       "~a~%Failed to grasp from ~a using ~a arm "
                                       e ?grasp ?grasping-arm)
                     (cpl:do-retry grasp-retries
                       (when (cut:lazy-car ?possible-grasp)
                         (roslisp:ros-info (trying-new-grasp)
                                           "Trying to grasp from ~a using ~a arm"
                                         ?grasp ?grasping-arm)
                         (setf ?possible-grasp (cut:lazy-cdr ?possible-grasp))
                         (pp-plans::park-arms)
                         (setf ?grasp (cut:lazy-car ?possible-grasp))
                         (cpl:retry)))
                     (roslisp:ros-warn (grasp-failures) "No more retries left")
                     (cpl:fail 'common-fail:object-unreachable)))
                ;; Perform the grasp
                (exe:perform (desig:an action
                                       (type picking-up)
                                       (arm ?grasping-arm)
                                       (grasp ?grasp)
                                       (object ?perceived-object)))))))))
  ?grasping-arm)

With this, the pick-up-object can now iterate through all possible grasp configurations stored in ?possible-grasp. The cram-object-interaces:get-object-type-grasps is a useful method which will give these values as a lazy list, provided the grasping arm and the object type.

There are two nested failure-handling clauses here. The inner failure-handling part will take care of cram-common-failures:manipulation-pose-unreachable and cram-common-failures:gripper-closed-completely - both of which are errors which can occur during a failed grasp. The recovery method will iterate through all possible grasp configurations for a given arm. Upon failure of this, the outer failure handling clause will take care of the common-fail:object-unreachable error thrown by the inner part and change the arm it's trying to grasp the object with and then retry the entire procedure again.

And only upon the failure of all these will the error be bubbled up. The method also returns the grasping arm it used to pick the object up so that the rest of the code is aware of the arm in which the bottle rests. Also, note that the arm we decided on earlier in find-object now acts as a bias to reduce the number of searches we do to find a successful configuration. We have also written appropriate warning statements to be informed about the actions the robot is taking.

Also let's redefine move-bottle again to include these:

(defun move-bottle ()
  (spawn-bottle)
  (pr2-proj:with-simulated-robot
    (let ((?navigation-goal *base-pose-near-table*))
      (cpl:par
        (exe:perform (desig:a motion 
                              (type moving-torso) 
                              (joint-angle 0.3)))
        (pp-plans::park-arms)
        ;; Moving the robot near the table.
        (exe:perform (desig:a motion
                              (type going)
                              (target (desig:a location 
                                               (pose ?navigation-goal)))))))
 
    (multiple-value-bind (?perceived-bottle ?grasping-arm) 
        (find-object :bottle)
      (setf ?grasping-arm (pick-up-object ?perceived-bottle :bottle ?grasping-arm))
      (pp-plans::park-arms :arm ?grasping-arm)
      ;; Moving the robot near the counter.
      (let ((?nav-goal *base-pose-near-counter*))
        (exe:perform (desig:a motion
                              (type going)
                              (target (desig:a location 
                                               (pose ?nav-goal))))))
 
      (coe:on-event (make-instance 'cpoe:robot-state-changed))
      ;; Setting the object down on the counter
      (let ((?drop-pose *final-object-destination*))
        (exe:perform (desig:an action
                               (type placing)
                               (arm ?grasping-arm)
                               (object ?perceived-bottle)
                               (target (desig:a location 
                                                (pose ?drop-pose))))))
      (pp-plans::park-arms :arm ?grasping-arm))))

You should see a result that looks like the one below. [Some messages that would come up are suppressed here for readability.]

BTW-TUT> (init-projection)
BTW-TUT> (move-bottle)
[(PICK-PLACE PICK-UP) INFO] 1550504321.279: Opening gripper
[(PICK-PLACE PICK-UP) INFO] 1550504321.279: Reaching
[(GRASP-FAILURE) WARN] Failed to grasp from LEFT-SIDE using RIGHT arm 
[(TRYING-NEW-GRASP) INFO] 1550504800.749: Trying to grasp from RIGHT-SIDE using RIGHT arm
[(PICK-PLACE PICK-UP) INFO] 1550504800.789: Opening gripper
[(PICK-PLACE PICK-UP) INFO] 1550504800.789: Reaching
[(GRASP-FAILURE) WARN] Failed to grasp from RIGHT-SIDE using RIGHT arm 
[(TRYING-NEW-GRASP) INFO] 1550504801.577: Trying to grasp from BACK using RIGHT arm
[(PICK-PLACE PICK-UP) INFO] 1550504801.601: Opening gripper
[(PICK-PLACE PICK-UP) INFO] 1550504801.602: Reaching
[(PICK-PLACE PICK-UP) INFO] 1550504801.939: Gripping
[(PICK-PLACE PICK-UP) INFO] 1550504801.973: Assert grasp into knowledge base
[(PICK-PLACE PICK-UP) INFO] 1550504801.974: Lifting
[(PICK-PLACE PLACE) INFO] 1550504802.356: Reaching
[(PICK-PLACE PLACE) INFO] 1550504802.508: Putting
[(PICK-PLACE PLACE) INFO] 1550504802.619: Opening gripper
[(PICK-PLACE PLACE) INFO] 1550504802.655: Retract grasp in knowledge base
[(PICK-PLACE PLACE) INFO] 1550504802.660: Retracting

The robot has once again succeeded in grasping the object.

You can now try spawning the bottle in different points on the table and observing how the robot resolves the arm and grasp for it. Subsequently, you will notice that our plans will work as long as the bottle is within reach of the robot. You could try implementing the adjustment of the robot's position to facilitate the fetch.

Since this is a simple tutorial in formulating and understanding mobile plans using CRAM, developing advanced plans and recovery behaviors, is left up to you.

Useful Macros

Many a time, you will find yourselves writing very similar failure handling strategies, because the solution in most cases is to iterate safely between the solutions of a designator or the elements of a list. CRAM provides a couple of useful macros to deal with this to avoid the cumbersome process of rewriting the same logic again and again. You will find these macros under cram_common_failures. Some examples are:

  • common-fail:retry-with-designator-solutions - Will retry by iterating over different solutions of the specified designator upon failure.
  • common-fail:retry-with-loc-designator-solutions - Same as the previous case but specialized to location designators. This macro also ensures that the location used is unique (outside a specified threshold) in successive iterations.
  • common-fail:retry-with-list-solutions - Will retry by iterating over elements of a list upon failure. This also supports lazy lists.

To demonstrate this, let's take a look at our existing pick-up-object:

defun pick-up-object (?perceived-object ?object-type ?grasping-arm)
  (let ((?possible-arms '(:right :left)))
    ;;Retry by changing the arm
    (cpl:with-retry-counters ((arm-change-retry 1))
        (cpl:with-failure-handling
            ((common-fail:object-unreachable (e)
               (roslisp:ros-warn (arm-failure) "Manipulation failed: ~a~%" e)
               (cpl:do-retry arm-change-retry
                 (setf ?grasping-arm (car (remove ?grasping-arm ?possible-arms)))
                 (cpl:retry))
               (roslisp:ros-warn (arm-failures) "No more retries left")))
 
          ;; Retry by changing the grasp
          (let* ((?possible-grasp
                   (cram-object-interfaces:get-object-type-grasps ?object-type nil nil nil ?grasping-arm))
                 (?grasp (cut:lazy-car ?possible-grasp)))
            (cpl:with-retry-counters ((grasp-retries 3))
              (cpl:with-failure-handling
                  (((or cram-common-failures:manipulation-pose-unreachable
                        cram-common-failures:gripper-closed-completely) (e)
                     (roslisp:ros-warn (grasp-failure)
                                       "~a~%Failed to grasp from ~a using ~a arm "
                                       e ?grasp ?grasping-arm)
                     (cpl:do-retry grasp-retries
                       (when (cut:lazy-car ?possible-grasp)
                         (roslisp:ros-info (trying-new-grasp)
                                           "Trying to grasp from ~a using ~a arm"
                                         ?grasp ?grasping-arm)
                         (setf ?possible-grasp (cut:lazy-cdr ?possible-grasp))
                         (pp-plans::park-arms)
                         (setf ?grasp (cut:lazy-car ?possible-grasp))
                         (cpl:retry)))
                     (roslisp:ros-warn (grasp-failures) "No more retries left")
                     (cpl:fail 'common-fail:object-unreachable)))
                ;; Perform the grasp
                (exe:perform (desig:an action
                                       (type picking-up)
                                       (arm ?grasping-arm)
                                       (grasp ?grasp)
                                       (object ?perceived-object)))))))))
  ?grasping-arm)

We can see that we do iterate through different grasp poses in a list in the inner failure handling clause. Let us rewrite this using the macro:

(defun pick-up-object (?perceived-object ?object-type ?grasping-arm)
  (let ((?possible-arms '(:right :left)))
    ;;Retry by changing the arm
    (cpl:with-retry-counters ((arm-change-retry 1))
        (cpl:with-failure-handling
            ((common-fail:object-unreachable (e)
               (roslisp:ros-warn (arm-failure) "Manipulation failed: ~a~%" e)
               (cpl:do-retry arm-change-retry
                 (setf ?grasping-arm (car (remove ?grasping-arm ?possible-arms)))
                 (cpl:retry))
               (roslisp:ros-warn (arm-failures) "No more retries left")))
 
          ;; Retry by changing the grasp
          (let* ((?possible-grasp
                   (cram-object-interfaces:get-object-type-grasps ?object-type nil nil nil ?grasping-arm))
                 (?grasp (cut:lazy-car ?possible-grasp)))
            (cpl:with-retry-counters ((grasp-retries 3))
              (cpl:with-failure-handling
                  (((or cram-common-failures:manipulation-pose-unreachable
                        cram-common-failures:gripper-closed-completely) (e)
                     (common-fail:retry-with-list-solutions
                         ?possible-grasp
                         grasp-retries
                         (:error-object-or-string (format nil 
                                                          "~a~%Failed to grasp from ~a using ~a arm"
                                                          e ?grasp ?grasping-arm)
                          :warning-namespace (grasp-failure)
                          :rethrow-failure 'common-fail:object-unreachable)
                        (setf ?grasp (cut:lazy-car ?possible-grasp)))))
 
                (exe:perform (desig:an action
                                       (type picking-up)
                                       (arm ?grasping-arm)
                                       (grasp ?grasp)
                                       (object ?perceived-object)))))))))
  ?grasping-arm)

This code will behave exactly like before but all the repetitive logic has been moved to this macro. The first two arguments are always the iterating list (in this case, the list containing possible grasp pose) and the number of retries. The keyword error-object-or-string can accept a warning string or just the plain error object and it will be used for logging. warning-namespace will accept the name in which the logging information is classified into. Since our method rethrows common-fail:object-unreachable we pass that along into rethrow-failure, and if none is passed, it will rethrow the same error it received.