Hacking on SLip

SLip is entirely self-contained; there are no dependencies and no server-side needed. You only need a local HTTP server like Nginx, and you should configure a SSL-enabled virtual host, and enable WebDAV. SSL is needed for using various browser API-s which are otherwise restricted, and WebDAV is needed so that Ymacs is able to save files.

Clone the Git repository somewhere. Define a name in /etc/hosts, for example:

slip.local 127.0.0.1

Define a virtual host for your HTTP server. Mine (using Nginx) looks like this:

server {
    listen 443 ssl;
    server_name slip.local;
    charset utf-8;

    ssl_certificate         /home/nginx/ssl/test.crt;
    ssl_certificate_key     /home/nginx/ssl/test.key;
    ssl_ciphers             HIGH:!aNULL:!MD5;

    location / {
        root /path/to/SLip/repo;
        autoindex on;
        dav_methods PUT DELETE MKCOL COPY MOVE;
        create_full_put_path on;
        dav_access group:rw all:r;

        ## the following two are required for more precise timings
        ## in performance.now(). feel free to skip them if you'd like
        add_header Cross-Origin-Opener-Policy same-origin;
        add_header Cross-Origin-Embedder-Policy credentialless;
    }
}

(you probably need to make sure that the nginx process can write to that directory; in my case it simply runs as my user, though I guess that's not quite a good idea.)

You'll have to use Google or ChatGPT for figuring out how to generate the SSL certificate, or to configure SSL if something is wrong; I could never remember this stuff. Any case, after restarting Nginx you should be able to open https://slip.local/ in your browser (Chrome or Firefox) and it should look pretty much the same as the REPL on this website. The info.md file that opens by default contains some information about available key bindings; if you're familiar with Emacs you should feel pretty much like home. 1

I wrote all the Lisp code from SLip in this very environment. It's convenient because it can talk to the running Lisp, so I can quickly test stuff. It would be nice if, at some point, I could just load SWANK and then use Emacs and SLIME, but that won't happen anytime soon.

So, let's say you'd like to add vector support for sequence functions, because currently they can only operate on lists. You'd probably first want to take a look at one of these functions, for example FIND. You type FIND in the REPL, and press M-. on it to jump to definition, and Ymacs conveniently opens lisp/seq.lisp for you. You browse through the code, get amazed (and perhaps horrified) at my magnificent with-list-frobnicator macro, which is used in almost all sequence functions, and figure you could “easily” add a few more cases to it so that it can take a vector as well. As you work on it, press C-M-x (or C-c C-c) to compile it, and do the same on one of the functions using it, then test it in the REPL. Or press C-c ENTER on a form using it to macro-expand it one level, or C-c M-m for complete macro-expansion. The output goes to the REPL, so make sure to have that open (press C-c s r at any time to go to the REPL.)

When you save files, they get saved directly on disk, in the Git repo, thanks to WebDAV. When you're happy with your work and want to recompile the FASL-s, type M-x sl-recompile-everything (completion is available, you can type just slre, or whatever minimal prefix works for you). “Recompile everything” will reload the page, appending ?recompile to the URL, and some JS code in ide/boot.js will figure that out and start rebuilding all FASL-s. If everything is fine that'll end with “DONE — press ENTER to reload”, and then you'll get a REPL with your upgraded sequence functions.

Next, you'd like to run the tests. Type (load "test/all.lisp") in the REPL (this takes a few seconds), and then (run-tests :all t), or (run-tests :log nil :all t) to skip logging (that'll be a little faster). They must all pass. If they do, go to Emacs, open Magit, commit and spawn a new branch, and then please send over a pull request. 🥹 (of course, it would be nice to add new tests for the stuff you've implemented, but since I hate it when my PR-s are rejected for not containing tests, I won't enforce that on my contributors. I'd rather not have tests, or write them myself, than to reject an otherwise good PR.)

When something goes wrong

When sl-recompile-everything fails for whatever reason, you might end up with a non-working medium (because Ymacs integration itself requires a working Lisp system). This is one of the challenges of working on a self-hosting environment. You can check if there are any errors in the browser's developer console, but those rarely help. Next, you can try to remove ?recompile from the URL manually and see if the IDE loads. If not, some FASL-s might be broken, in which case you want to run this in the repo:

  find . -iname "*.fasl" | xargs git checkout

That'll reset the FASL-s, so loading the environment now should work. Your Lisp files will still contain your changes, so you can reopen them, figure out what's wrong and then retry to recompile everything.

Another thing to be aware of is that Ymacs saves files both via WebDAV but also to the browser's local storage. If you need to edit files from Emacs as well, make sure to first remove it from browser's local store, as otherwise Ymacs will still load the version from there. From the REPL you can clear the local storage with (%:%ls-clear-store); if the REPL is momentarily broken, you must open the browser's developer tools and clear local storage, e.g. evaluate this in the console: localStorage.removeItem(".slip").

I kinda got used to the workflow, but I am aware that this part could be improved. I might work on polishing it someday. If you get stuck and need any help, feel free to file an issue on Github (if you think it's a bug in SLip) or otherwise email me directly.