Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
tutorials:advanced:unreal [2020/01/13 14:18]
hawkin [Prerequisites] adding new Chapter which explains the core Idea better
tutorials:advanced:unreal [2020/01/13 14:37]
hawkin [Usage and Code] added loading of cram_ROBOT_description as a step
Line 6: Line 6:
  
 ==== Idea ==== ==== Idea ====
 +We want to essentially teach a robot how to perform every day activities without having to dive deep into code, but rather simply show the robot what we want him to do using Virtual Reality. This would allow robots to learn from humans easily, since this way, the robot can acquire information about where the human was looking for things and where things were placed. Of course, this could also be hard coded, but that would take a lot more time and be prone to failure, since we as humans, often forget to describe very minor things which we automatically take for granted, but which can play a huge role in the success of a task for the robot. E.g. if the cooking pot is not in its designated area, we would automatically check the dishwasher or the sink area. This is something the robot would have to learn first.
 +
 +The advantage of using Virtual Reality for this is also, that we can train the robot on all kinds of different kitchen setups, which can be build within a few minutes, instead of having to move around physical furniture. This would also allow for generalization of the acquired data and would add to the robustness of the pick and place tasks. ​
 ==== Prerequisites ==== ==== Prerequisites ====
 This tutorial assumes that you've completed the [[tutorials:​intermediate:​json_prolog|Using JSON Prolog to communicate with KnowRob]] tutorial and therefore have ROS, CRAM, KnowRob and MongoDB installed. In order to be able to use the kvr package, a few specific changes have to be made. Within the ''​knowrob_addons''​ the ''​knowrob_robcog''​ package has to be replaced by this one [[https://​github.com/​robcog-iai/​knowrob_robcog.git This tutorial assumes that you've completed the [[tutorials:​intermediate:​json_prolog|Using JSON Prolog to communicate with KnowRob]] tutorial and therefore have ROS, CRAM, KnowRob and MongoDB installed. In order to be able to use the kvr package, a few specific changes have to be made. Within the ''​knowrob_addons''​ the ''​knowrob_robcog''​ package has to be replaced by this one [[https://​github.com/​robcog-iai/​knowrob_robcog.git
Line 35: Line 38:
 </​code>​ </​code>​
  
-To launch all the necessary ​components, simply execute: ​+If this is used in simulation and depending on if the PR2 or Boxy robot is supposed to be used, the respective description needs to be loaded first. 
 + 
 +For PR2: 
 +<code lisp> 
 +CL-USER> ​ (swank:​operate-on-system-for-emacs "​cram-pr2-description"​ (quote load-op)) 
 +</​code>​ 
 + 
 +For Boxy: 
 +<code lisp> 
 +CL-USER> ​ (swank:​operate-on-system-for-emacs "​cram-boxy-description"​ (quote load-op)) 
 +</​code>​ 
 + 
 + 
 +To launch all the necessary ​initializations, simply execute: ​
 <code lisp> <code lisp>
 CL-USER> (kvr::​init-full-simulation :namedir '​("​ep1"​) :​urdf-new-kitchen?​ nil) CL-USER> (kvr::​init-full-simulation :namedir '​("​ep1"​) :​urdf-new-kitchen?​ nil)
 </​code>​ </​code>​
  
-This will create a lisp ros node, clean up the belief-state,​ load the episodes that get passed to the init function as a list of strings in the ''​namedir''​ key parameter, e.g. in our case "​ep1",​ spawn the semantic map of the episode and the items and initialize the location costmap. This process may take a while, so please have some patience. When the function has finished running through your bullet world should look like this: +This will create a lisp ros node, clean up the belief-state,​ load the episodes that get passed to the ''​init'' ​function as a list of strings in the ''​namedir''​ key parameter, e.g. in our case "​ep1",​ spawn the semantic map of the episode and the items and initialize the location costmap. This process may take a while, so please have some patience. ​(or go grab a coffee meanwhile. It can really take several minutes on Ubuntu 16.04.) ​When the function has finished running through your bullet world should look like this: 
  
 Now, let's execute the pick and place plan: Now, let's execute the pick and place plan: