Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
tutorials:advanced:unreal-engine [2022/03/08 12:11] – [Using Cram in Unreal Engine] vanessa | tutorials:advanced:unreal-engine [2022/03/08 14:13] – vanessa | ||
---|---|---|---|
Line 13: | Line 13: | ||
- Or use [[https:// | - Or use [[https:// | ||
* Clone the Demo Project: <code bash> git clone --recurse-submodules git@github.com: | * Clone the Demo Project: <code bash> git clone --recurse-submodules git@github.com: | ||
- | * | + | * For further information how to Setup the Demo Project please take a look into this [[https:// |
+ | ===== Setup ===== | ||
+ | |||
+ | To communicate with the robot in the Unreal Engine from host-machine additional components are required, including the host-side rosbridge, shared ROS messages and descriptions. First, install the following packages for communication via WebSockets, the TF2 buffer, kinematic models and a state publisher. | ||
<code bash> | <code bash> | ||
- | $ roscore | + | sudo apt install ros-melodic-rosbridge-server ros-melodic-robot-state-publisher ros-melodic-joint-state-publisher-gui ros-melodic-tf ros-melodic-tf2 ros-melodic-tf2-ros ros-melodic-pr2-arm-kinematics |
</ | </ | ||
+ | |||
+ | |||
+ | |||
+ | Further software stacks are needed to control the robot in Unreal via CRAM plans and Giskards motion planner, as well as perception and knowledge inference and logging. | ||
+ | |||
+ | Follow the installation instructions for each of the following software stacks, picking the ones required for your specific purpose. Using separate, overlayed workspaces is recommended. | ||
+ | |||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | |||
+ | Now create a catkin workspace for deps from repositories. Use wstool for git imports and rosdep for apt sources. | ||
<code bash> | <code bash> | ||
- | $ rosrun turtlesim turtlesim_node | + | mkdir -p ~/ |
+ | cd ~/ | ||
+ | wstool init | ||
+ | wstool merge https:// | ||
+ | wstool up | ||
+ | rosdep update | ||
+ | rosdep install --from-paths --ignore-src . -r | ||
</ | </ | ||
+ | Build the workspace either with the ros-native catkin or with python-catkin-tools, | ||
+ | <code bash> | ||
+ | cd ~/ | ||
+ | catkin_make | ||
+ | source ~/ | ||
+ | </ | ||
+ | If the build process wasn't successful, install the missing packages via apt, find missing repositories in the code-iai GitHub group or ask the contributors for help. | ||
- | ===== Moving ... ===== | + | ====== |
+ | Finally, the connection to the ROS network can be established by launching the following nodes. Make sure the workspace is sourced. | ||
+ | <code bash> | ||
+ | #Launch rosbridge for communication between unreal and ROS | ||
+ | roslaunch rosbridge_server rosbridge_websocket.launch | ||
- | Go to the '' | + | # Launch urobosim world |
+ | roslaunch urobosim_ros_config world.launch | ||
+ | </code> | ||
- | < | + | Hit the ' |
- | test example | + | |
+ | ===== GISKARD ===== | ||
+ | |||
+ | Giskard requires the whole_body_controller topics of the robot. To achieve that, make sure the rosbridge to Unreal is running properly. | ||
+ | |||
+ | < | ||
+ | roslaunch giskardpy giskardpy_pr2_unreal.launch | ||
</ | </ | ||
- | You will also need to update your '' | + | This launches Giskards trajectory planner. There may occur one timeout error for the state topic, which is fine for now. |
+ | Launch RViz by executing rviz. Add the Robot Model to the scene. Add InteractiveMarkers and choose the topic; if / | ||
+ | |||
+ | ===== CRAM ===== | ||
+ | |||
+ | cram-unreal provides | ||
+ | |||
+ | * A :detecting PM and plan to detect objects from Unreal to put them in Bullet | ||
+ | * The service clients to spawn and move objects in a running Unreal world (and stubs to extend to further clients) | ||
+ | * Some unused plans for pouring (feeding-plans) | ||
+ | |||
+ | cram-pr2-unreal-process-modules (aka pr2-unreal-pms) provides | ||
+ | |||
+ | * An extension to pr2-pms as | ||
+ | * The with-unreal-robot macro uses unreal-detection while using the same manipulation pms as pr2-pms: | ||
+ | * Also a few changes to Giskard' | ||
+ | |||
+ | Note: Remove CATKIN_IGNORE in the cram_pr2 packages and cram_external_interfaces/ | ||
+ | |||
+ | Play the map in Unreal. The PR2 should twitch and wiggle its arms a bit if it's good. If not, check the DemoProject repo ReadMe how to troubleshoot. | ||
+ | |||
+ | ====== Start the Demo ====== | ||
+ | |||
+ | Startup roslisp_repl. First load cram_pr2_pick_place_demo, | ||
+ | |||
+ | |||
+ | |||
+ | <code bash> | ||
+ | (pr2-unreal-pms: | ||
+ | | ||
+ | </ | ||
+ | |||
+ | ====== Spawn objects in Unreal from Cram ====== | ||
+ | |||
+ | <code bash> | ||
+ | (unreal: | ||
+ | </ |