Using JSON Prolog to communicate with KnowRob

Tested under CRAM version v0.7.0

The following tutorial will show how to query the knowledge processing system, KnowRob, for information using Prolog queries within LISP and CRAM, using the json-prolog package, and also how to process and extract the data from the gained results.

This tutorial requires CRAM, roslisp_repl and ROS to have already been installed. Go through the previous tutorials and the installation page if you don't have CRAM installed yet. Make sure you have Java 8 and Gradle 5.1 already installed. Otherwise you won't be able to build KnowRob. Other versions could work, but have not been tested yet, so use them at your own risk!

We need to install KnowRob. For that, please follow the tutorial on the knowrob website The basic KnowRob installation is enough.
Note: make sure to select the right branch for your current ROS version.

If it complains about missing swi-prolog, you can install it via:

    $ sudo apt-get install swi-prolog

Make sure however, that it is version 7. There are major problems with locations of libraries with version 8, which is the version suggested by the swi-prolog website. apt-get should install version 7 by default on Ubuntu 16.04. To make sure, you can check it with:

    $ dpkg -s swi-prolog

In addition to a roscore and the roslisp_repl you need to launch json_prolog in a Terminal with the following command:

    $ roslaunch rosprolog rosprolog.launch

or if you're using an older version of KnowRob, then the following command:

    $ roslaunch json_prolog json_prolog.launch 

In order for us to be able to communicate with KnowRob from LISP, we need to load the cram_json_prolog package in our repl, since it implements the necessary interface for the CRAM to KnowRob communication. We also need the cram_utilities package in order to process lazy lists. (Note: Lazy lists will be explained later in the tutorial. If you are curious and want to jump right there, you can do so by clicking here.)
In order to load the packages, do the following in your repl:

CL-USER> (ros-load:load-system "cram_json_prolog" :cram-json-prolog)
CL-USER> (ros-load:load-system "cram_utilities" :cram-utilities)

If you want to add these dependencies directly into your package already, you can add them to your .asd file, as cram-json-prolog and cram-utilities.

To be able to communicated with KnowRob from CRAM over ROS, we need to start a ROS node in the REPL:

  CL-USER> (roslisp:start-ros-node "my_node")

There are multiple ways to call a query, depending on if one needs only one result or all possible results. We will go through them in the following, explaining the differences between them. In general, it helps if one is familiar with Prolog, since essentially everything that can be done in Prolog, can be done via cram-json-prolog as well. Some Prolog tutorials are provided here: Prolog tutorial.

The simplest way is using the prolog-simple-1 function, which gets the query as a string parameter. Any query you could use within Prolog, you can use within this function. It can be used in the following way:

    CL-USER> (json-prolog:prolog-simple-1 "member(X, [1,2,3])")

The result in this case is:

    CL-USER> (((?X . 1)))

X being the variable we asked Prolog to fill with a value. Whenever Prolog returns a value for a variable, the variable name gets prefixed with a question mark. “1” is the first value in the list we passed on to the member predicate of Prolog. The member predicate is true if the element is a member of the given list, or in this case since it is a variable, it will return the first element of the list. In a later section we will look in more detail into the bindings we get as a result from Prolog and explain a bit more in depth why we get these results.

Why do we get ?X sitting in 3 brackets? The inner bracket connects ?X with it's value 1. What does the second bracket do? Let's add another variable to the query – the variable ?Y.

CL-USER> (json-prolog:prolog-simple-1 "member(X, [1,2,3]), Y = X.")

This binds ?X to member of the [1,2,3] list and ?Y gets the same value as ?X. The result of the query is

(((?X . 1) (?Y . 1)))

Therefore, the second bracket makes a list of bindings for each variable in the query:

((?X . 1) (?Y . 1))

is a list of bindings for variable ?X and ?Y.

The last outer bracket is explained next.

Prolog simple works the same way as prolog-simple-1 does, but instead of getting just one result, we get multiple in a lazy-list (lazy-lists will be explained below).

    CL-USER> (json-prolog:prolog-simple "member(X, [1,2,3])")
      (((?X . 1))
         . #S(CRAM-UTILITIES::LAZY-CONS-ELEM
             :GENERATOR #<CLOSURE (LAMBDA () :IN JSON-PROLOG::PROLOG-RESULT->BDGS)
                {1005F52FAB}>))

The result is basically still the same, but it looks a little bit more complex on the first glance. What we get is a list which contains two elements. The first one is a list in a list with two elements which we already know from the previous call with prolog-simple-1 and which describes the value Prolog found for ?X . The second element of the list is #S(CRAM-UTILITIES::LAZY-CONS-ELEM :GENERATOR #<CLOSURE (LAMBDA () :IN JSON-PROLOG::PROLOG-RESULT→BDGS) {1005F52FAB}>)) which is a lazy list element generator.

The results we obtain from the queries are stored in lazy lists, meaning the information is loaded on demand, resulting in us seeing only one solution of the query. If we want to see all the results, we can expand the list and force LISP to show all of them to us, for example, by using:

    CL-USER> (defparameter *our-list* nil)
    CL-USER> (setq *our-list* (json-prolog:prolog-simple "member(X, [1,2,3])"))
      (((?X . 1))
         . #S(CRAM-UTILITIES::LAZY-CONS-ELEM
             :GENERATOR #<CLOSURE (LAMBDA () :IN JSON-PROLOG::PROLOG-RESULT->BDGS)
                {1005F52FAB}>))
 
    CL-USER> (cut:force-ll *our-list*)
         (((?X . 1)) ((?X . 2)) ((?X . 3)))

We save our list in a variable for convenience purposes. This allows us also to expand the so stored list, and we can see all the possible values Prolog found for the variable ?X, which are 1,2,3.

Now we see why we need the outer bracket: the result of the query is a (lazy) list of all possible bindings for the query variables. Let us add another variable to the query again:

CL-USER> (cut:force-ll (json-prolog:prolog-simple "member(X, [1,2,3]), Y = X."))
(((?X . 1) (?Y . 1)) ((?X . 2) (?Y . 2)) ((?X . 3) (?Y . 3)))

We get a list of all possible bindings of the query variables, where the bindings is itself a list with one element for each variable.

If we evaluate our original member query with prolog-simple-1 we can see the difference between prolog-simple-1 and prolog-simple better:

CL-USER> (cut:force-ll (json-prolog:prolog-simple "member(X, [1,2,3])"))
(((?X . 1)) ((?X . 2)) ((?X . 3)))
CL-USER> (cut:force-ll (json-prolog:prolog-simple-1 "member(X, [1,2,3])"))
(((?X . 1)))

In the first case, meaning prolog-simple, we see that all possible bindings for X are contained within the lazy list, all of them being lists themselves, mapping the variable to a possible value. While for prolog-simple-1 even after expanding the list, the result is still the same, since it returns only one possible value for ?X to begin with.

One peculiarity of Prolog is that it can return an infinite number of bindings for a variable. In that case, using cut:force-ll on the lazy list would have to return a list of infinite length, so the program will compute forever. This is why lazy lists are useful when working with Prolog – if the number of solutions is infinite or very big, using lazy lists one can work with one value at a time by iterating through the lazy list and getting new solutions on demand.

Now that we have a lazy list with interesting information, how can we access the elements contained within?

We can use (cut:lazy-car *our-list*) to access the elements of the list. It is used the same way the normal car is:

CL-USER> (cut:lazy-car (json-prolog:prolog-simple "member(X, [1,2,3])"))
((?X . 1))

Same goes for cdr:

CL-USER> (cut:lazy-cdr (json-prolog:prolog-simple "member(X, [1,2,3])"))
(((?X . 2))
 . #S(CRAM-UTILITIES::LAZY-CONS-ELEM
      :GENERATOR #<CLOSURE (LAMBDA () :IN JSON-PROLOG::PROLOG-RESULT->BDGS)
                   {10084ADDCB}>))

When we have one element of the lazy list, we can use (cut:var-value VAR LIST) to access the value of a specific variable. In our case we asked for possible values of the variable X. Therefore:

CL-USER> (cut:var-value '?X (cut:lazy-car (json-prolog:prolog-simple "member(X, [1,2,3])")))
1

Gives us the result 1, since this is the value of X in the first element of the lazy list. That way, one can extract the necessary information, adjust it if needed, and pass it on to the plans which need it.

You can also see the pattern here probably. Basically, any operation one can perform on a list in LISP, has an equivalent in the cut package with the word lazy as a prefix. e.g. mapcar → cut:lazy-mapcar, dolist → cut:lazy-dolist, car → cut:lazy-car etc.

It also good practice to add the package the prolog-simple call is used in as a parameter. So instead of the above, use:

    CL-USER> (json-prolog:prolog-simple "member(X, [1,2,3])" :package :NICKNAME-OF-YOUR-PACKAGE)

There are many packages within KnowRob which define different kinds of queries, depending on what kind of data one is working with and what one wants to achieve. Those packages are very useful, and can be imported, so that their queries can be used as well. A package can be imported by doing the following:

    CL-USER> (json-prolog:prolog-simple-1 "register_ros_package(knowrob_common)")
       (NIL)

In JSON Prolog (NIL) means success, and NIL means failure. This is explained in detail in the next section.

This allows us to use the knowrob_common package, which allows us to access many utility functions and also the most commonly used queries of KnowRob.

With this imported package we could now do the following:

    CL-USER (json-prolog:prolog-simple "lowercase('HELLOWORLD',RESULT)")
       (((?RESULT . |''helloworld''|))
        . #S(CRAM-UTILITIES::LAZY-CONS-ELEM
              :GENERATOR #<CLOSURE (LAMBDA () :IN JSON-PROLOG::PROLOG-RESULT->BDGS)
                           {100659E4AB}>))

This query converts the first elements upper case letters into lower case, and stores the obtained result in the RESULT variable.

An overview of the currently available packages and their queries can be found here. If when loading a package Emacs only returns NIL then it might be that the package does not exist or is not found on your system. In such cases, you can check the terminal in which json-prolog was launched. If it cannot be found, you will see the following error:

   [rospack] Error: package 'knowrob_maps_tools' not found
   [librospack]: error while executing command

If you call a query for which the package is not loaded yet, Emacs will throw the following error:

    Prolog query failed: PrologException: error(existence_error(procedure, '/'(u_load_episodes, 1)), context(':'(system, '/'(call, 1)), _1))
    [Condition of type SIMPLE-ERROR]

and if you check your terminal, in which the json_prolog node is running, there will be a huge java print as well, basically explaining the same thing.

In the former query we got a weird result from Prolog which looked like this:

|''helloworld''|

What do the pipes stand for in this case?

Lisp is case-insensitive, that means that variable of function names, or more correctly said, any symbol, can be written in any capitalization and will refer to the same symbol:

CL-USER> (eq 'hello 'HELLO)
T
CL-USER> (eq 'hello 'Hello)
T

Prolog, on the other hand, is case-sensitive. And in some cases, one would like to have a case-sensitive variable in Lisp as well. This is where the pipes come into play:

CL-USER> (intern "HELLO")
HELLO
:INTERNAL
CL-USER> (intern "Hello")
|Hello|
NIL

The function intern creates a symbol from a string. If the string is all capital, we get a normal usual symbol, in the other case we get the pipes.

The same is with Prolog, if your variable or any other symbol is not all caps, you will get a symbol in Lisp which has pipes around it:

CL-USER> (json-prolog:prolog-simple-1 "member(X, [1,2,3])")
(((?X . 1)))
CL-USER> (json-prolog:prolog-simple-1 "member(MYVAR, [1,2,3])")
(((?MYVAR . 1)))
CL-USER> (json-prolog:prolog-simple-1 "member(MY_VAR, [1,2,3])")
(((?MY_VAR . 1)))
CL-USER> (json-prolog:prolog-simple-1 "member(My_var, [1,2,3])")
(((|?My_var| . 1)))

In that case, to get the value of the variable you should also use pipes:

CL-USER> (cut:var-value '|?My_var|
                        (car (json-prolog:prolog-simple-1 "member(My_var, [1,2,3])")))
1

Please note that in Prolog, all variables have to start with a capital letter and cannot contain dashes:

CL-USER> (json-prolog:prolog-simple-1 "member(x, [1,2,3])")
NIL
CL-USER> (json-prolog:prolog-simple-1 "member(MY-VAR, [1,2,3])")
NIL

If we want to check, if a certain number is member of the list, we must declare the variable accordingly. e.g.

    CL-USER> (json-prolog:prolog-simple-1 "member(2, [1,2,3])")

and while one would expect Prolog to return “true”, in json-prolog what we get is

    (NIL)

Meaning a list with only one element, the element being NIL. Since in LISP everything that is not NIL, is true, the answer we get here means true, since it's a list of NIL and not NIL. If one were to test this out within the Prolog console itself, the result would be “true”. We can actually test this with json-prolog as well, to prove our assumption.

    CL-USER> (json-prolog:prolog-simple-1 "true")
    (NIL)
 
    CL-USER> (json-prolog:prolog-simple-1 "false")
    NIL

We can note that if the result is true we get returned a list, even if NIL is the only member of this list. If the result is false, we just get the element NIL as a result, which is LISPs way of saying “false”.

The question is, why is that so?

The json-prolog returns a list of all possible bindings that can be true for the Prolog query. For example,

    CL-USER> (cut:force-ll (json-prolog:prolog-simple "member(X, [1,2]), Y = 3."))

returns a list of bindings for X and Y (Note: force-ll expands the lazy list we get returned from Prolog. cut is the nickname of the cram-utilities package that one might need to import if this was not done prior):

    (((?X . 1) (?Y . 3)) ((?X . 2) (?Y . 3)))

whereas

    CL-USER> (json-prolog:prolog-simple "member(1, [1,2]), 3 = 3.")

doesn't have any variables that it needs to bind so it returns a list of bindings – bindings do not exist – so therefore we get a list of NIL.