Compiler

Sytes is not a compiler, it's rather a “fast interpreter”. I'm still refer to the “compiler” because that's the phase that happens after the templates are parsed (in SICP that's called “analysis” which is probably more appropriate). In the future I might implement compilation to Common Lisp.

After a template is parsed, the resulted syntax tree is compiled into a Lisp closure. Compilation is pretty straight-forward—a bunch of primitive operators are supported, like quote, lambda, if etc.

The compiler supports classic macros via the defmacro form. When a macro form is encountered, it is evaluated at compile-time and the resulted expression is compiled instead. For example the “foreach” form is a macro:

{(macroexpand-1 '(foreach i '(foo bar baz) {<li>{i}</li>}))}


==>

(strcat (mapcar (lambda (i)
                  (strcat "<li>" i "</li>"))
                (quote (foo bar baz))))

The compiler supports a single namespace for functions, macros and variables.

Every S-expression (including of course lambdas) is compiled into a Common Lisp function, which is pretty neat because this means you can freely call CL functions from a template (you just need to make it available via def-primitive), or call a template from Common Lisp. For example the “mapcar” function is not implemented in our little language, instead we're using the CL function, which is better because it's a lot more efficient.

For more information about the language and library, see the language page.

Contexts

Bindings of names to values are stored in an execution context. A context has the following properties:

• name — the name of the context (it's commonly the template name)

• symbols — a hash table with all symbols defined in this context; originally I thought each template would intern symbols in its own context, but this complicated things too much, so all symbols are now interned in a single top-level context.

• parent — the parent context; accessing a name which is not found in the current context will result in a lookup in its parent context, and return that value instead if defined. However, note that variables in upper contexts are not writable from a template running in the lower context — in other words, you cannot modify the meaning of cons for example. More exactly, you can alter it for the current context, but not for other contexts that inherit from the toplevel context. This safety belt ensures that even if you want to screw up one website, you won't affect others running in the same Lisp image.

• global — the global environment. This is an association list mapping symbol to current value.

• env — the current lexical environment. This is modified each time we enter a function to include its arguments.

All templates are compiled and executed in their own context, but in order to share things between them (such as defining a common library of functions) they all inherit from other contexts. There is one top-level context, and in turn each Syte present in the image will create its own context which inherits from *toplevel-context*.

Performance

I applied the fundamental rule of performance optimization, quoted below:

“DO NOT!”

Sytes is interpreted, although the lexical analysis is done only once (I call that phase “compile-time” but don't confuse it with a real compiler). Execution is pretty slow if you need anything fancy; for example the following takes about 4 seconds:

(defun sum (n ret)
  (if (zerop n)
      ret
      (sum (1- n) (+ ret n))))

(sum 1000000 0)

That's iterating a million times to compute 1 + 2 + ... + 1000000. Four seconds is extremely slow, running the same in Common Lisp takes like 10 milliseconds.

Incidentally, note that even if dog slow, the interpreter is tail-recursive; it doesn't blow the stack, but for this to happen it needs that the host Lisp system do tail call elimination. SBCL does it if the debug level is not set above 2.

We could speed up big here and there (for example the interpreter resolves variables at run-time, but lexical vars could be figured out at compile-time; each variable access involves an alist lookup, and that's really slow) — however it seems to me that it doesn't make much sense to optimize an interpreter. On the other hand, templates are relatively simple stuff; normally you'd do the heavy lifting in Common Lisp and pass any data to be displayed to the template using a Syte handler, so in practice the slowness is negligible (well, at least it seems so to me right now). For the future, I might try to rewrite it and compile templates to Common Lisp.

Final advice: if you need something really fast you might want to look into Ruby on Rails, or Drupal. (bwwaahahaha!!)