Differences

This shows you the differences between two versions of the page.

Link to this comparison view

tutorials:advanced:unreal [2019/04/15 15:59]
hawkin created. Moved MongoDB instructions from json-prolog tutorial
tutorials:advanced:unreal [2019/06/07 16:55] (current)
hawkin
Line 1: Line 1:
-==== Importing new episode data into MongoDB (Additional information) ====+====== ​ Using the Cram-KnowRob-VR package ====== 
 + 
 +Tested under CRAM version v 0.7.0  
 + 
 +This tutorial will introduce you to the ''​cram_knowrob_vr (short: kvr)''​ package, which uses the data recorded in the Virtual Reality environment using [[http://​robcog.org/​|RobCog]],​ extracts information from them using [[http://​www.knowrob.org/​|KnowRob]] and executes the CRAM high-level-plans based on this data either on the real robot or in the CRAM [[tutorials:​advanced:​bullet_world|bullet world]]. 
 + 
 +==== Prerequisites ==== 
 +This tutorial assumes that you've completed the [[tutorials:​intermediate:​json_prolog|Using JSON Prolog to communicate with KnowRob]] tutorial and therefore have ROS, CRAM, KnowRob and MongoDB installed. In order to be able to use the kvr package, a few specific changes have to be made. Within the ''​knowrob_addons''​ the ''​knowrob_robcog''​ package has to be replaced by this one [[https://​github.com/​robcog-iai/​knowrob_robcog.git 
 +|knowrob_robcog github]]. If it doesn'​t build because dependencies are missing, please install them. If it still doesn'​t build, you can try to pull this fork of the knowrob_addons [[https://​github.com/​hawkina/​knowrob_addons|knowrob_addons github]] instead. 
 +The ''​knowrob_robcog''​ package contains the VR-specific queries, which we need in order to be able to extract the information we need for the plans. 
 + 
 +=== Roslaunch === 
 +Launch a ''​roscore''​ first. Then, in a new terminal for each launch file, launch the bullet_world,​ json_prolog and roslisp_repl 
 +<code bash> 
 +    $ roslaunch cram_pr2_pick_place_demo sandbox.launch  
 +    $ roslaunch json_prolog json_prolog.launch  
 +    $ roslisp_repl 
 +</​code>​ 
 +The bullet world is needed for visualization. The json_prolog node allows us to access information in KnowRob from CRAM.  
 + 
 +==== Usage and Code ==== 
 +The following will describe what the different files and their functions do, when and how to use them and why they are needed. The explanation will follow the order files in the .asd file. It is separated into a usage and a files section. The usage section will focus on how to get everything ​ to run and how to execute a demo while the files section will look a bit more in depth into the code, and explain what is going on there.  
 + 
 +=== Usage === 
 +Here we will first explain on what needs to be done to get the robot to execute and perform a pick and place plan in the simulated bullet world. In the next paragraph, we will take a closer look at the individual source files and explain their function. 
 + 
 +Before you load the package, navigate to the ''​init.lisp''​ file and set the ''​*episode-path*''​ parameter to the path of your episode data. This is important. Otherwise it won't be possible to load the episode data properly. 
 + 
 +Now you can load the ''​cram_knowrob_vr''​ package with: 
 +<code lisp> 
 +CL-USER> ​ (ros-load:​load-system "​cram_knowrob_vr"​ :​cram-knowrob-vr) 
 +</​code>​ 
 + 
 +To launch all the necessary components, simply execute:  
 +<code lisp> 
 +CL-USER> (kvr::​init-full-simulation '​("​ep1"​)) 
 +</​code>​ 
 + 
 +This will create a lisp ros node, clean up the belief-state,​ load the episodes that get passed to the init function as a list of strings, e.g. in our case "​ep1",​ spawn the semantic map of the episode and the items and initialize the location costmap. In the code section below it will be explained in more detail, what is loaded and when. This process may take a while, so please have some patience. When the function has finished running through your bullet world should look like this:  
 + 
 +Now, let's execute the pick and place plan: 
 +<code lisp> 
 +CL-USER> (cram-pr2-projection:​with-simulated-robot (kvr::​demo)) 
 +</​code>​ 
 +With this call we first say that we want to use the simulated bullet-world PR2 robot instead of the real one, and then we simply call the demo. The demo will read out the VR episode data and extract the positions of the objects that have been manipulated,​ which hand was used and the positions of the human head and hand.  
 + 
 + 
 +=== Code === 
 +== mesh-list.lisp ==  
 +Contains a list of all the meshes which we want to spawn based on their locations in the semantic map. Some of them are commented out, e.g. walls, lamps and the objects we interact with, in order to keep the bullet world neat and clean. In unreal however, the walls and lamps are being spawned. We simply currently don't need them in bullet. 
 + 
 +== mapping-urdf-semantic.lisp == 
 +Mapps the urdf kitchen to the semantic map, since they differ in how some furniture is organized and called.  
 + 
 +== init.lisp == 
 +TODO 
 + 
 +== queries.lisp == 
 +TODO 
 + 
 +== query-based-calculations.lisp == 
 +TODO 
 + 
 +== designator-integration.lisp == 
 +TODO 
 + 
 +== fetch-and-deliver-based-demo.lisp == 
 +TODO 
 +==== Importing new episode data into MongoDB ​and KnowRob(Additional information) ==== 
 +In order for us to be able to query data for information,​ we first need to import that data into KnowRob and MongoDB. 
 +The MongoDB contains all the poses of the objects, camera, hand and furniture of the episode. So the pose is read out from there. KnowRob takes care of everything else. It knows of all the object classes and instances used in the episodes. So in order to be able to use the data, we need to import it into MongoDB and KnowRob. 
 + 
 +=== MongoDB ​===
 If you record data in the Virtual Reality using [[http://​robcog.org/​ | RobCog]], you essentially get the following files: If you record data in the Virtual Reality using [[http://​robcog.org/​ | RobCog]], you essentially get the following files:
 ''​RawData_ID.json''​ <- this contains all the recorded events that happened, where everything was, who moved what where when...etc. ''​RawData_ID.json''​ <- this contains all the recorded events that happened, where everything was, who moved what where when...etc.
Line 49: Line 121:
 $ mongorestore -d Own-Episodes_set-clean-table -c RawData_qtzg RawData_qtzg.bson $ mongorestore -d Own-Episodes_set-clean-table -c RawData_qtzg RawData_qtzg.bson
 </​code>​ </​code>​
 +
 +=== KnowRob ===
 +KnowRob needs to be able to find the .owl and .bson files. It's important to have a directory for all the Episodes. You can either have a look or download these [[ftp://​open-ease-stor.informatik.uni-bremen.de/​|episodes]] for reference. ​
 +
 +=== Performance ===
 +Depending on how many collections your database has, it can get slow when quering for information. One way to make it faster, is to include an index over timestamp for all collections. One way to add this, is to install [[https://​www.mongodb.com/​products/​compass|compass]] for mongodb. Launch it, and connect it to your database. The defautl settings should be fine so just click ''​ok''​ when it launches. Then go to your collection -> indexes -> create index. Call the new index ''​timestamp'',​ select the field ''​timestamp''​ and set the type to ''​1 (asc)'',​ click create. Repeat for all the collections. It will improve the query speed greatly.
 +