Common Lisp Jupyter Setup

This tutorials is a step-by-step guide to setup JupyterLab with Common Lisp and ROS support.

All the terminal commands assume a set up ROS environment with the ROS workspace sourced.

You need Python3 to install the newest version of JupyterLab.

This installation was tested on Ubuntu 16.04 and 20.04 (python 3.8).

First, if you don't have it, install pip or rather pip3:

$ sudo apt install python3-pip python3-dev

Then install JupyterLab:

$ sudo -H pip3 install jupyterlab

You can test if it worked like so:

$ jupyter-lab

In a terminal:

$ roscd && cd ../src
$ catkin_create_pkg jupyter
$ cd jupyter
$ curl -O https://raw.githubusercontent.com/cram2/cram/boxy-melodic/jupyter/common-lisp-jupyter.rosinstall
$ wstool init && wstool merge common-lisp-jupyter.rosinstall && wstool update
$ roscd && cd ..
$ catkin_make # or catkin build if this workspace has been build like that before 

Currently, the CFFI library in CRAM master and other branches is out of date and the Jupyter Common Lisp kernel needs a newer version. So, until the newest version is pushed to the CRAM master, you'll need to cherry-pick the commit:

$ roscd cffi
$ git fetch origin
$ git cherry-pick 5448d11dc6d0750d92c117350f9a2cf39d5907b7

In a terminal:

$ sudo apt install libzmq3-dev
$ roslisp_repl

In the Lisp REPL:

CL-USER> (asdf:load-system :common-lisp-jupyter)
CL-USER> (defmethod jupyter:command-line :around ((instance jupyter/common-lisp::user-installer))
           (let* ((original-command-line (call-next-method))
                  (additional-entries `("--dynamic-space-size" 
                                        "4096"
                                        ,jupyter/common-lisp::+eval-flag+
                                        "(load (parse-namestring (concatenate 'string (sb-ext:posix-getenv \"ROS_ROOT\") \"lisp/scripts/roslisp-sbcl-init\")))")))
             (concatenate 'list
                           (subseq original-command-line 0 1)
                           additional-entries
                           (subseq original-command-line 1))))
CL-USER> (cl-jupyter:install)

This should write the config files of the Common Lisp Kernel into a directory that Jupyter will search through.

Make sure you kill any old versions of JupyterLab that are running. Then:

$ jupyter-lab

Inside Jupyter, go to File → New → Notebook and in the list of Kernels you should see Common Lisp now.

Have fun.

Sometimes the PIP version on the Debian repos for your Ubuntu might be outdated. In that case you will see an error when calling the JupyterLab installation command, it will tell you exactly that, that your PIP version is too old. In that case, you can install pip from a different repo, like so:

$ curl -O https://bootstrap.pypa.io/pip/3.5/get-pip.py    # be careful with the version! I had Python 3.5, you might have newer.
$ sudo -E python3 get-pip.py

But maybe you should just update your Ubuntu. Before 20.04 python 2.7 is the default, after it's 3.6+. Check your version with 'python –version', and also your 'pip –version', because whether you install a package with pip or pip3 will only install it for the respective version.

If there are multiple people working on the same PC and you want to install the stuff only for yourself, you can specify that with 'pip install –user <package>'.

When the jupyter installation tells there are version requirements for certain packages you can install the specific version like this. Check the required version from your command prompt, it may be different from this.

# use pip3 instead if pip is for python2
pip install --user traitlets==5.2.2.post1
pip install --user ipython==7.23.1

That specific version of traitlets also fixes a bug with jupyter-lab. ipython can be outdated if previously installed via aptitude.

If ASDF doesn't seem to find the common-lisp-jupyter package when you're loading its ASDF system, something must be wrong with your workspace or you downloaded the packages outside of the ROS workspace. Maybe you need to restart your REPL or Emacs completely. You can try the command below to help ASDF find the packages, but, really, just make sure your ROS setup is clean, then you won't need any hacks like this:

(let* ((ros-package-path (sb-ext:posix-getenv "ROS_PACKAGE_PATH"))
       (first-colon-index (position #\: ros-package-path))
       (ros-workspace-path (subseq ros-package-path 0 first-colon-index))
       ;; (packages '("common-lisp-jupyter" "bordeaux-threads" "cl-base64" "cl-indentify"
       ;;             "closer-mop" "dissect" "eclector" "ironclad" "multilang-documentation"
       ;;             "pzmq" "puri" "static-vectors" "trivial-do" "trivial-mimes" "shasht"))
       (jupyter-dir-path (concatenate 'string ros-workspace-path "/jupyter/")))
  (pushnew (parse-namestring jupyter-dir-path) asdf:*central-registry* :test #'equal))

If you get compilation errors when loading the ASDF system of common-lisp-jupyter, it means you have version incompatibilities. On Ubuntu 16.04 the SBCL installed from the Debian repos is really old. The common-lisp-jupyter package uses a bit newer version where some packages got renamed etc. If that is the case for you, try to check out an older version of common-lisp-jupyter, the newest version that still supports Ubuntu 16.04 is as following. Execute the command inside the common-lisp-jupyter Git repo:

$ git checkout 689f47667ab6a4a9865776fdd159d8bbdc222f20

If that still doesn't help, probably some of the dependencies of the package introduced breaking changes. Unfortunately, the person who made the rosinstall was too lazy to write down all the commit numbers that worked for the person, so you might have to debug and see, which specifically package is causing you problems.