====== Writing tests ======

**Description:** In this tutorial you will learn how to write tests for your CRAM functions and features.

**Previous Tutorial:** [[tutorials:beginner:failure_handling|Implementing failure handling for the TurtleSim]]


===== lisp-unit =====

The library we are going to use to write our unit tests is called ''lisp-unit'', it is one of the simplest ones out there for Common Lisp. There are more elaborate ones that can do more things but in most cases ''lisp-unit'' is going to be more than enough for you.

For more information on the library check it's [[https://github.com/OdonataResearchLLC/lisp-unit/wiki|documentation page]] and the [[https://github.com/OdonataResearchLLC/lisp-unit|GitHub repo]].


===== Setting up the infrastructure and our first test =====

Let's create a new ASDF system for our test suite, it should be in the root of our ROS package, right next to ''cram-my-beginner-tutorial.asd''. Call it ''cram-my-beginner-tutorial-tests.asd'' and put the following barebones code inside:

<code lisp>
(defsystem cram-my-beginner-tutorial-tests
  :depends-on (cram-my-beginner-tutorial
               lisp-unit))
</code>

In addition to creating a new ASDF system, we are going to create a new directory within our ROS package, called ''tests'', where the test files will go.
We need a ''package.lisp'' file for our new ASDF system, so let's create it and put it in the ''tests'' directory. Put the following code into it:

<code lisp>
(defpackage :cram-my-beginner-tutorial-tests
  (:nicknames :tut-tests)
  (:use :common-lisp :lisp-unit))
</code>

In this tutorial we are going to test the function ''relative-angle-to'' from ''simple-plans.lisp''.
Therefore, we are going to create a file in our test suite called ''simple-plans-tests.lisp''.
**Name the files with test functions the same as the original files, just with ''-tests'' as a suffix.**
So, let's create ''simple-plans-tests.lisp'' inside of ''tests'' directory and put the following inside:

<code lisp>
(in-package :tut-tests)

;;; Turtle world is like this:
;;;  Y
;;;  ^
;;;  |
;;;  |
;;;  o-----> X
;;; Identity orientation is looking right in the direction of X axis.

(define-test relative-angle-to-dont-turn
  ;; turtle is in origin with identity orientation, wants to look at (3 0 0)
  (let* ((goal (cl-transforms:make-3d-vector 3 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle 0.0)))
</code>

So, we are testing the ''relative-angle-to'' function, which returns an angle, with which the turtle has to turn,
in order to look towards a goal point.

What does the test checks the following situation:
if the turtle is standing in the origin of the coordinate frame of the Turtlesim world,
looking in the identity orientation, i.e., to the right in the direction of the X axis,
what would be the angle with which it would have to turn to look at coordinate ''(3 0)''.
As ''(3 0)'' lies on the X axis, and the turtle is already looking in that direction, it would have to move with the angle of ''0.0''. And that is what the test asserts.

Let us add just another test case: if the turtle is standing at the origin with identity orientation
and wants to look at ''(-1 0)'', what would be the angle?
Let us create a new test function for this, such that your ''simple-plans-tests.lisp'' would look as following:

<code lisp>
(in-package :tut-tests)

;;; Turtle world is like this:
;;;  Y
;;;  ^
;;;  |
;;;  |
;;;  o-----> X
;;; Identity orientation is looking right in the direction of X axis.

(define-test relative-angle-to-dont-turn
  ;; turtle is in origin with identity orientation, wants to look at (3 0 0)
  (let* ((goal (cl-transforms:make-3d-vector 3 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle 0.0)))

(define-test relative-angle-to-look-back
  ;; turtle is in origin, wants to look at (-1 0 0)
  (let* ((goal (cl-transforms:make-3d-vector -1 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle pi)))
</code>

You can see that we needed to use the ''cl-transforms'' library for the test, as well as ''roslisp'' and ''turtlesim-msg''.
So let us add these to the ''.asd'' file of our test suite. We should also add the new files that we have created, ''package.lisp'' and ''simple-plans-tests.lisp'' to the ASDF system. It should now look something like this:

<code lisp>
(defsystem cram-my-beginner-tutorial-tests
  :depends-on (cram-my-beginner-tutorial
               lisp-unit
               cl-transforms
               roslisp
               turtlesim-msg)
  :components ((:module "tests"
                :components
                ((:file "package")
                 (:file "simple-plans-tests" :depends-on ("package"))))))
</code>


===== Running the test suite =====

Now let us load the test package in REPL and switch into its namespace.
You might need to restart your Emacs at this point, because the new ASDF system might not be visible to Emacs without restarting.

<code lisp>
CL-USER> (ros-load:load-system "cram_my_beginner_tutorial" :cram-my-beginner-tutorial-tests)
CL-USER> (in-package :tut-tests)
</code>

To run the tests written with the ''lisp-unit'' library, we call the ''run-tests'' function:

<code lisp>
TUT-TESTS> (lisp-unit:run-tests)

Unit Test Summary
 | 2 assertions total
 | 2 passed
 | 0 failed
 | 0 execution errors
 | 0 missing tests

#<TEST-RESULTS-DB Total(2) Passed(2) Failed(0) Errors(0)>
</code>

We can also specify which test specifically we want to run:

<code lisp>
TUT-TESTS> (lisp-unit:run-tests '(relative-angle-to-dont-turn))
Unit Test Summary
 | 1 assertions total
 | 1 passed
 | 0 failed
 | 0 execution errors
 | 0 missing tests

#<TEST-RESULTS-DB Total(1) Passed(1) Failed(0) Errors(0)>
</code>

If we are in a different package in the REPL than the test suite package, we can also specify the package to test explicitly:

<code lisp>
TUT-TESTS> (in-package :cl-user)
#<PACKAGE "COMMON-LISP-USER">
CL-USER> (lisp-unit:run-tests :all :cram-my-beginner-tutorial-tests)
Unit Test Summary
...
</code>

==== Integrating into the ASDF testing operation ====

As there exist multiple testing libraries in Lisp, the ASDF facility has a unified way of calling the test suites, independent of the library.
So let us integrate our test suite with the ASDF interface.
For that, we are going to add a '':perform'' tag to our ''cram-my-beginner-tutorial-tests.asd'' file, such that it now should look as following:

<code lisp>
(defsystem cram-my-beginner-tutorial-tests
  :depends-on (cram-my-beginner-tutorial 
               lisp-unit
               cl-transforms
               roslisp
               turtlesim-msg)
  :components ((:module "tests"
                :components
                ((:file "package")
                 (:file "simple-plans-tests" :depends-on ("package")))))
  :perform (test-op (operation component)
                    (symbol-call :lisp-unit '#:run-tests :all :cram-my-beginner-tutorial-tests)))
</code>

And to use the ASDF interface for testing, we simply need to call the ''asdf:test-system'' function on our package.
We can do that at any point in the REPL. So let us start a completely fresh REPL with nothing loaded in it and try the testing function:

<code lisp>
CL-USER> (asdf:test-system :cram-my-beginner-tutorial-tests)
T
</code>

You will get some compilation prompts in between and in the end it should return T.
Now let us destroy one of our tests and see what happens: change the angle in the first test in ''simple-plans-tests.lisp'' from 0.0 to pi:

<code lisp>
(define-test relative-angle-to-dont-turn
  ;; turtle is in origin with identity orientation, wants to look at (3 0 0)
  (let* ((goal (cl-transforms:make-3d-vector 3 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle pi)))
</code>

Now let us rerun the tests:

<code lisp>
CL-USER> (asdf:test-system :cram-my-beginner-tutorial-tests)

Unit Test Summary
 | 2 assertions total
 | 1 passed
 | 1 failed
 | 0 execution errors
 | 0 missing tests

T
</code>

To get more detailed information, set the ''*print-failures*'' parameter of ''lisp-unit'' to T:

<code lisp>
CL-USER> (setf lisp-unit:*print-failures* t)
T
CL-USER> (asdf:test-system :cram-my-beginner-tutorial-tests)
 | Failed Form: PI
 | Expected 0.0d0 but saw 3.141592653589793d0
 |
RELATIVE-ANGLE-TO-DONT-TURN: 0 assertions passed, 1 failed.

RELATIVE-ANGLE-TO-LOOK-BACK: 1 assertions passed, 0 failed.

Unit Test Summary
 | 2 assertions total
 | 1 passed
 | 1 failed
 | 0 execution errors
 | 0 missing tests

#<TEST-RESULTS-DB Total(2) Passed(1) Failed(1) Errors(0)>
</code>

Now go back and fix the angle to the correct value and see if the tests pass again.


===== Adding some more assertions =====

Now let us add more tests for the ''simple-plans.lisp'' file into our suite.
Let's add some more tests for the ''relative-angle-to'' to check if the number are correct with ''assert-number-equal'' function.
In addition to that assertion, there are also many different types of assertions, so we are going to try the ''assert-error'' one.
Put the following into your ''simple-plans-tests.lisp'':

<code lisp>
(in-package :tut-tests)

;;; Turtle world is like this:
;;;  Y
;;;  ^
;;;  |
;;;  |
;;;  o-----> X
;;; Identity orientation is looking right in the direction of X axis.

(define-test relative-angle-to-dont-turn
  ;; turtle is in origin with identity orientation, wants to look at (3 0 0)
  (let* ((goal (cl-transforms:make-3d-vector 3 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle 0.0))

  ;; turtle is in (3 0 0) with identity orientation, wants to look at (4 0 0)
  (let* ((goal (cl-transforms:make-3d-vector 4 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose
                                         :x 3.0 :y 0.0 :theta 0.0))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle 0.0))

  ;; turtle is in (0 0 0) looking up in direction of Y, wants to look at (0 1 0)
  (let* ((goal (cl-transforms:make-3d-vector 0 1 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose
                                         :x 0.0 :y 0.0 :theta (/ pi 2)))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle 0.0))

  ;; turtle is in (1 1 0) looking towards the origin, wants to look at (0 0 0)
  (let* ((goal (cl-transforms:make-3d-vector 0 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose
                                         :x 1.0 :y 1.0 :theta (+ pi (/ pi 4))))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle 0.0)))

(define-test relative-angle-to-look-back
  ;; turtle is in origin, wants to look at (-1 0 0)
  (let* ((goal (cl-transforms:make-3d-vector -1 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle pi))

  ;; turtle is in (1 0 0), wants to look at origin
  (let* ((goal (cl-transforms:make-3d-vector 0 0 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose
                                         :x 1.0 :y 0.0 :theta 0.0))
         (angle (tut::relative-angle-to goal pose-msg)))
    (assert-number-equal angle pi))

  ;; turtle is in (-1 -1 0) looking towards origin, wants to look away from origin
  (let* ((goal (cl-transforms:make-3d-vector -2 -2 0))
         (pose-msg (roslisp:make-message 'turtlesim-msg:pose
                                         :x -1.0 :y -1.0 :theta (/ pi 4)))
         (angle (tut::relative-angle-to goal pose-msg)))
      (assert-number-equal angle pi)))

(define-test relative-angle-to-type-error
  (assert-error 'type-error 
                (tut::relative-angle-to 
                 (cl-transforms:make-identity-pose)
                 (roslisp:make-message 'turtlesim-msg:pose))))
</code>

At this point, all the tests should be successful.

Please beware that if you define a test with a specific name, then delete it from the file but not restart the REPL, the test will still be in the current REPL session. So if you delete a test from a file, also think about restarting your REPL.