Why Lisp?

  • Published: 2012-08-11
  • Modified: 2012-08-15 23:20
  • By: Mishoo
  • Tags: lisp, evanghelism
  • Comments: 12 (add)
Aug
11
2012

Why Lisp?

As this new website is written in Lisp, I thought a first post to praise Lisp would be in order. If you're a Lisper, you know all this already ;-) and if you're not, I hope it inspires you.

Early 2010 I joined the IT team of a small research center in Italy. Our manager at the time was a smart guy, a programmer himself—Emacs user and Haskell lover. We started a big project and, although their initial plan was to develop it in PHP (having already a few PHP devs there), the manager agreed I can use whatever language I want to bring us faster to the objective. He'd prefer Haskell, but I don't know it. I was having some 10 years experience with JavaScript, but not on server-side (NodeJS was in early stages those days), 8 years with Perl and only a few months with Common Lisp. I picked Lisp and I still congratulate myself for that choice.

No dependency hell

Earlier this year I changed my old server for a better one. To do that I could have simply copied the full HDD contents from the old server to the new one, but my old server was a messy combination of Debian stable + testing + unstable and I thought the only way to clean the mess is by starting all over. Having a freshly reconfigured server, I had to reinstall all my Perl stuff, such as my old website. That was somewhat painful and I realized just how easy it is to deploy Common Lisp applications versus Perl (or anything else, probably).

With Lisp, you just build your application and get one binary; you copy that to your server and you're done. With Perl, you have to install a load of modules on the server; some of them are available in Debian via apt-get, others are only on CPAN, some require gcc/g++ and random C/C++ libraries you didn't even know you're using, and some simply fail to build anymore.

After 8 years of Perl, I can promise you that setting up a (web) development environment with Common Lisp is much much easier.

Not unreadable

In the years I've been using Common Lisp I noticed that I can look through code that I didn't touch in 2 years and immediately understand what it does.

More importantly, I can read code written by others and understand what they meant relatively easily. This is helped by a great feature of the environment (Emacs/SLIME) which provides cross-referencing tools, for example, I can place the cursor on the name of a function and ask “where is this defined”, and it jumps to the place of the definition, which could be in my own code or in third party code. But I can also ask “where is this function called” or “where is this variable referenced”. This is extremely useful for code refactoring and it beats every “modern” IDE I've seen.

Organic growth

In most other languages, the development cycle is, roughly, (1) edit, (2) compile (depending on the language), (3) run the program to test it. I can still remember the days when I had to run /etc/init.d/apache restart in order for Apache to pick up the changes I've done to my Perl modules.

In Lisp things are different. You're not just writing your program; you're growing it. Lisp is alive. Your program starts from nothing—you really just start the REPL—and then you keep adding to it and changing it as it runs. You talk to it all the time; you can inspect and modify data and functions as the program runs; even when problems arise, such as a hard error, you can drop into a debugger and inspect local variables at various stack frames, or even evaluate arbitrary expressions in a certain stack frame.

Many of these ideas, which Lisp pioneered, were implemented later in other languages. For example you can attach the GDB debugger to a running process and inspect live data; however, you cannot change the running process, like for example you can't redefine functions.

The only way to understand how great this is is to try it; you won't get it from this short blog post, nor from a thousand pages book.

Stable as a rock

When I was younger, I had the feeling that if some software library was not updated in 5 years it's unmaintained and it's probably broken. That, sadly, is usually the case with many programming languages—because they're moving targets. They change frequently and they depend on lots of external factors—operating system, compilers, shared libraries etc.

That doesn't seem to apply to Common Lisp. The language is extremely stable. You often find code written a decade ago that still functions properly. You might think that this happens because Common Lisp did not “evolve”, and that's one face of the truth.

Programmable programming language

The fact is, Common Lisp is powerful enough that you can modify the language from itself, without the need for a new spec (and by that I don't simply mean that you can define functions and variables). I don't know of a single programming paradigm that wasn't brought to CL; in fact most paradigms, and even whole programming languages, were prototyped in Lisp long before they had a life on their own.

Support for macros, of course, is most of what makes this possible. But the fun part is that even if Lisp didn't have macros, you could implement them in Lisp itself with a bit of effort. That can't be said about other languages because it's only the Lisp syntax that makes macros possible. There were attempts to add macros to other languages (most notably C's #define) but they're not even close to Lisp macros. But of course, Lispers know this already, and non-Lispers still wonder what macros are good for.

Going further than “ordinary” macros, you can define reader macros. Experienced Lispers avoid them unless they're really necessary. The extent to which you can change the language is only limited by your imagination, and that's probably why the standard was not updated in almost twenty years; it's very good as it is; like the United States Constitution, it gives power to the people. I guess nobody wants to change that (well, not the people, anyway).

Lisp is simple!

The Lisp beginner might read this in disbelief. I know for myself, before using Lisp I thought that it's a weird language and that it must be extremely complicated. Nothing is further from the truth, but the truth is hard to see when your brain is wired to think in terms of for and while statements, in blocks delimited by { curly brackets }, and especially when you expect the language to read your mind about operator precedence.

A Lisp interpreter can be written in just a few lines of code, as this great paper by Paul Graham shows. My first Lisp interpreter was written in Perl, based only on ideas in this document and with a quick and dirty parser.

Like mathematics, Lisp is based on just a few axioms. Common Lisp did get big and complex—like mathematics—but it's written in a simpler Lisp, which is in turn written in an even simpler Lisp. The analogy goes on—complex mathematics is defined in terms of simpler mathematics, which is itself defined in terms of even simpler maths. Eventually, it all goes down to the void set. It's based on absolute, incontestable truths.

If you bootstrap a very simple Lisp compiler, you can then define and redefine the language in terms of itself. I did this and it was a fabulous experience. If you want to learn Lisp but you fear it's complex, my suggestion is to start looking at it from the bottom, from the simple things. Just as you learn maths.

My new blog

So this new website is powered by Common Lisp. I expect the code I wrote these days to function for decades, but this of course depends on other factors (there's a lot of CSS, HTML and JavaScript into it).

This is actually the first website I do in Common Lisp. I make a difference between websites and Web applications—in many ways, a website is harder to make. I've been working on a CL tool to make this easier—Sytes—using this tool and some other neat CL libraries like Quicklisp, buildapp and sb-daemon, I think it's as easy now to get started with writing websites in CL as you can do with PHP. I hope to prove that soon.

In the mean time, if you want to get in touch you can email me or leave a comment in these pages. Comments are moderated, because I'm fed up with comment SPAM; this means I'll have to approve your comment in order for it to appear on the website, but if you enter your email address you'll receive an email and will be able to validate your comment yourself. Your email address is not published.

12 comments. This is HOT!

Add your comment

# Nick Fitzgerald
2012-08-12 05:34
Now add feeds so I can keep up to date! :)
# Mishoo
2012-08-12 15:20
There's an Atom feed now at http://lisperator.net/atom
# Samar
2012-08-16 02:44
I'm a beginner in LISP. Where do you think I should I start to sink myself in it
# Simon
2012-08-16 06:23
Please start with SICP, 'The Structure and Interpretation of Computer Programs' until you are comfortable with Scheme. After that, you may want to go on to LISP, but I think you'll prefer to stick with Scheme--it can ruin you that way.
# Zmyrgel
2012-08-16 11:39
Samar, Pick something fun to work on and just use Lisp. That is in my opinion the only way to get started. If you meant what material to study to get started, check out Practical Common Lisp [http://www.gigamonkeys.com/book/]. Be sure to check out QuickLisp to help in installing libraries.
# Ashton Kemerling
2012-08-16 19:58
As a professional Lisp programmer, here's my take on your points, in order. No dependency hell: There are systems to deal with this. Lisp doesn't have many dependencies to count on, on the whole, which is why this isn't an issue. The flip side is that you'll probably need to write all of your libraries yourself. Not unreadable: That is not universally true. You'd be amazed how you can mangle Lisp up, given enough time. Organic growth: Which tends to lead to unplanned changes. Lisp has very few structures in place to protect private implementation details and provide structure. The result is that at its base most Lisp programs are an unstructured bag of functions. Read the Hunchentoot documentation to see what I mean. Stable as a rock: Not universally true. A lot of this is also because Lisp libraries tend to be of the "trivial" variety. They handle only the bare minimum functionality that the writer needed, rather than the entire set. As a result, they may be stable, but they're useless if your needs deviate an iota from what the writer needed. Programmable programming language: This means if you have multiple projects, each one is different. Lisp is simple: Lisp lacks the features to make life easier. It hasn't changed in decades, and a lot of the required features (posix interoperability, networking, threads) are implementation specific, so there are a large number of libraries that are merely shims to deal with functional differences. Lisp is an interesting language. But on the whole it has become a programming Ghetto. Very few people use it professionally, and most libraries are decades behind similar offers in Java, Ruby, and Python. In terms of utilities, Lisp is only in the same century as other languages because there's less than a century to be had. But fundamentally the worst part about Lisp is the community. At the heart of every Lisp believer, is the belief that the core unit of Lisp is a single hacker. It's a romantic, and vastly antiquated belief. We've past the point as a profession where much impact is had by single brilliant hackers. So in order for a language to be effective, it needs the tools, structures, and community behind it to support effective an effective team effort. Lisp lacks this. Samar: I'd recommend you futz around with Scheme and SICP a while to learn the basics. Maybe play with Haskell if you want. Lisp and its cousins are most helpful as learning opportunities, not as serious skills. The best you can hope from Lisp is for you to have a deeper understanding of some more functional design principles, and an appreciation for where certain features in Ruby and Python came from (blocks, anyone?). Also, it'll make you realize how important the private and protected keywords in Java really are.
# Savvas
2012-08-16 21:09
There is SICP modified for LISP. That way you will avoid Scheme.
# Cake
2013-01-09 16:28
As a newcomer in the world of LISP, I'm definitely going to keep an eye on this blog. I can't wait to have a better understanding of the language
# John Hinnegan
2013-01-09 22:50
I recommend Practical Common Lisp. http://www.gigamonkeys.com/book/ I found it to be a good intro to the language, and you build 'real' projects towards the end of the book. Something I have wished to find in other programming books ever since I read this one.
# Tamas Herman
2013-01-12 15:05
Samar: have a look at http://picolisp.com/ It's an even simpler approach to Lisp, because it is interpreted, which gives amazing opportunities for simplifications (which makes it still fast compared to compiled lisp!) See tons of examples in http://software-lab.de/rosettacode.l To give a feel for its power: a ~300kbyte picoLisp executable is largely equivalent to this software stack for example: - ruby interpreter - thin http server - god/monit process watchdog - mysql with replication - ORM (like ActiveRecord or more like DataMapper)
# Mishoo
2013-01-13 12:21
@Tamas: PicoLisp is definitely an interesting project and a good proof that you can do great things with minimalistic design. However... no lexical scope? That can't be serious. I love dynamic scope, but it's a bit like nuclear fission. When used in a strictly controlled manner it can do good, but used sloppily it can blow off the planet.
# Piotrek
2013-10-28 02:20
"But the fun part is that even if Lisp didn't have macros, you could implement them in Lisp itself with a bit of effort" can you please tell something more about it? or at least give some links