Differences

This shows you the differences between two versions of the page.

--- tutorials:advanced:unreal-engine [2022/03/08 11:15] – created vanessa
+++ tutorials:advanced:unreal-engine [2022/03/08 14:15] (current) – [Spawn objects in Unreal from Cram] vanessa
@@ Line 1: / Line 1: @@
-Test
+====== Using Cram in Unreal Engine======
+**Description:** In this tutorial you will learn..
+**Requirements:**
+  * Unreal Engine version 2.25.4/2.26/2.27.
+    * [[https://docs.unrealengine.com/4.26/en-US/SharingAndReleasing/Linux/BeginnerLinuxDeveloper/SettingUpAnUnrealWorkflow/|Install Guide]]
+    * Unreal requires a lot of disc space, at least 60 GB for the engine itself, and with several projects even more.
+    * Making the Unreal engine takes a lot of computation time. Schedule it adequately.
+    * Either:
+           - Check the installation of Unreal by launching the engine under <unreal-path>/Engine/Binaries/Linux/UE4Editor. Since it requires OpenGL or Vulkan, running it from a remote machine can get finicky.
+           - Or use [[https://www.jetbrains.com/de-de/rider/nextversion/|Rider for Unreal Engine]]
+    * Clone the Demo Project: <code bash> git clone --recurse-submodules git@github.com:urobosim/DemoProject.git</code>
+    * For further information how to Setup the Demo Project please take a look into this [[https://github.com/urobosim/DemoProject|README]].
+===== Setup =====
+To communicate with the robot in the Unreal Engine from host-machine additional components are required, including the host-side rosbridge, shared ROS messages and descriptions. First, install the following packages for communication via WebSockets, the TF2 buffer, kinematic models and a state publisher.
+<code bash>
+sudo apt install ros-melodic-rosbridge-server ros-melodic-robot-state-publisher ros-melodic-joint-state-publisher-gui ros-melodic-tf ros-melodic-tf2 ros-melodic-tf2-ros ros-melodic-pr2-arm-kinematics
+</code>
+Further software stacks are needed to control the robot in Unreal via CRAM plans and Giskards motion planner, as well as perception and knowledge inference and logging.
+Follow the installation instructions for each of the following software stacks, picking the ones required for your specific purpose. Using separate, overlayed workspaces is recommended.
+  * [[https://github.com/SemRoCo/giskardpy|Install Giskard]], the motion planner.
+  * [[https://github.com/knowrob/knowrob|Install Knowrob]], the long-term memory, especially for NEEMs
+  * [[https://github.com/cram2/cram/blob/noetic/README.md|Install CRAM]], the noetic branch, please follow the Github-ReadMe.
+Now create a catkin workspace for deps from repositories. Use wstool for git imports and rosdep for apt sources.
+<code bash>
+mkdir -p ~/unreal_project_ws/src
+cd ~/unreal_project_ws/src
+wstool init
+wstool merge https://raw.githubusercontent.com/urobosim/DemoProject/master/rosinstall/unreal_demo_project.rosinstall
+wstool up
+rosdep update
+rosdep install --from-paths --ignore-src . -r
+</code>
+Build the workspace either with the ros-native catkin or with python-catkin-tools, which can be added via apt.
+<code bash>
+cd ~/unreal_project_ws
+catkin_make                                  # or 'catkin build'
+source ~/unreal_project_ws/devel/setup.bash  # rather put the top-level workspace in your .bashrc
+</code>
+If the build process wasn't successful, install the missing packages via apt, find missing repositories in the code-iai GitHub group or ask the contributors for help.
+====== Running the System ======
+Finally, the connection to the ROS network can be established by launching the following nodes. Make sure the workspace is sourced.
+<code bash>
+#Launch rosbridge for communication between unreal and ROS
+roslaunch rosbridge_server rosbridge_websocket.launch
+# Launch urobosim world
+roslaunch urobosim_ros_config world.launch
+</code>
+Hit the 'Play' button in Unreal. The rosbridge terminal should indicate, that the connection has been established. Also use rostopic list to check, if the controller topics for the robot are published.
+===== GISKARD =====
+Giskard requires the whole_body_controller topics of the robot. To achieve that, make sure the rosbridge to Unreal is running properly.
+<code bash>
+roslaunch giskardpy giskardpy_pr2_unreal.launch
+</code>
+This launches Giskards trajectory planner. There may occur one timeout error for the state topic, which is fine for now.
+Launch RViz by executing rviz. Add the Robot Model to the scene. Add InteractiveMarkers and choose the topic; if /eef_control/update is not available, check the giskard setup again. Use the markers to move the robot and see the robot in Unreal move.
+===== CRAM =====
+cram-unreal provides
+  * A :detecting PM and plan to detect objects from Unreal to put them in Bullet
+  * The service clients to spawn and move objects in a running Unreal world (and stubs to extend to further clients)
+  * Some unused plans for pouring (feeding-plans)
+cram-pr2-unreal-process-modules (aka pr2-unreal-pms) provides
+  * An extension to pr2-pms as
+  * The with-unreal-robot macro uses unreal-detection while using the same manipulation pms as pr2-pms:with-real-robot
+  * Also a few changes to Giskard's constraints, which seemed to be outdated. Giskard won't work without them.
+Note: Remove CATKIN_IGNORE in the cram_pr2 packages and cram_external_interfaces/cram_urobosim to build them.
+Play the map in Unreal. The PR2 should twitch and wiggle its arms a bit if it's good. If not, check the DemoProject repo ReadMe how to troubleshoot.
+====== Start the Demo ======
+Startup roslisp_repl. First load cram_pr2_pick_place_demo, then cram_pr2_unreal_process_modules. Start a rosnode with (roslisp-utilities:startup-ros). To start the setting demo, run
+<code bash>
+(pr2-unreal-pms:with-unreal-robot
+   (demo::setting-demo '(:spoon)))
+</code>
+====== Spawn objects in Unreal from Cram ======
+Run the same launch files as above, but without giskard. Open Emacs, load cram_urobosim and start a node. Try to spawn an object with
+<code bash>
+(unreal:spawn-object :spoon (cl-tf:make-identity-pose))
+</code>