SLip news - more Common Lisp!
SLip is an aspiring Common Lisp environment in the browser.
Since my earlier post in April, I wanted to replace the object
system with Closette, as it's more complete than TinyCLOS. Rather
than hammering down Closette to fit it into my limited Lisp, I thought I would add sufficient Common Lisp support so
that we're able to compile Closette as is, or at least almost. This is the glorious commit where I
added Closette with just two tiny changes — I rewrote this function because (at the time) I
didn't have typecase, and I removed the :test #'equal from a couple hash tables (I don't support that
yet, but it's easy and I'll add it soon). It took some more work from there to actually replace TinyCLOS, but
eventually I got it done.
Long story short, in order to reach this point I had to add support for a fair amount of Common Lisp. Here is what's new since my last post:
- multiple-value returns and the associated functions
- proper setters with the underlying machinery (define-setf-expansion etc.)
- progv and local declarations (specifically,
(declare (special ...))) - symbol-macrolet
- compiler macros
- looping constructs (do, do*, dolist, dotimes)
- many list and sequence functions (though they only work on lists so far)
- structures (defstruct)
- type system (deftype, typep, typecase)
- test suite (a very small subset of ansi-test)
These were not implemented in isolation, but are designed to work closely together. For example, SETF applied on a symbol macro, or a local macro defined via macrolet, will do the right thing. Or, macroexpand-all will properly mimic the compiler environment so that it's able to expand local macros or symbol macros correctly. Or you can define local setter functions using flet or labels. I didn't even know you can do that, and I didn't know that I don't know… but ansi-test did. ansi-test is anal 😅 I mean this in a positive way, it really stresses edge cases that you wouldn't even think of.
Some of these features were not really required for loading Closette, but I got them done anyway as a byproduct of delving into the immensity of this language. We should consider ourselves lucky to have not one, but several, high quality implementations, both free and commercial. I hope one day I won't be embarrassed about changing the main package names in SLip to COMMON-LISP and CL-USER, but that day is still far.
Performance
I used to think it's slow, but I've revised this opinion. SLip is actually faster than any other in-browser Lisp REPL I tried, except JSCL. Now, JSCL compiles to JavaScript (so we could say it's “native”) — that's in a different league. Of course SLip is slower, since it compiles to a custom bytecode which is then interpreted by a VM written in JavaScript.
But (1) performance is still okay (and I frequently find new opportunities to improve it) and (2) the VM approach has one neat advantage: the VM is interruptible. This allows us to provide green threads (which we do). It also enables us to do this (you can try it in the REPL):
(let ((*standard-input* (sl-stream:open-url "ide/info.md")))
(loop repeat 10 do (write-line (read-line))))
Internally, open-url uses an asynchronous JS fetch, and returns a stream based on JS ReadableStream — which, again, is asynchronous. In theory this should be real streaming, although I think the reality is that the browser will wait for the full body to come in before resolving those promises… but they are promises, therefore asynchronous. And our Lisp code doesn't have to care. When an async primitive is called, the VM will just pause until the promise is resolved. I could completely automate the workflow, but it's better to have some manual control over it, so async primitives will need to take this into account; but it's not hard. And the Lisp code is nicely sequential, the way humans are supposed to write programs.
Hacking
I've put together a hacking page that shows how to setup a local Web server for development. It's not hard, you only need Nginx (or Apache.. any Web server, really). There are no dependencies. It would be nice for this project to gain some contributors. I'm not making any illusions, though — the Lisp community is quite small, and the Common Lisp community even smaller. I can't reasonably expect people to put their time into something that might, some day, be useful. But, there's that. If you'd like to hack on it and get stuck on anything, please feel free to email me directly.