I've been writing more and more Common Lips recently and while I reflected a
bunch on the experience in a recent post that I recently followed up
Why Ecosystems Matter
Most of my thinking and analysis of CL comes down to the ecosystem: the
language has some really compelling (and fun!) features, so the question
really comes down to the ecosystem. There are two main reasons to care about
ecosystems in programming languages:
a vibrant ecosystem cuts down the time that an individual developer or team
has to spend doing infrastructural work, to get started. Ecosystems provide
everything from libraries for common tasks as well as conventions and
established patterns for the big fundamental application choices, not to
mention things like easily discoverable answers to common problems.
The more time between "I have an idea" to "I have running (proof-of-concept
quality) code running," matters so much. Everything is possible to a
point, but the more friction between "idea" and "working prototype" can be a
a bigger and more vibrant ecosystem makes it more tenable for
companies/sponsors (of all sizes) to choose to use Common Lisp for various
projects, and there's a little bit of a chicken and egg problem here,
admittedly. Companies and sponsors want to be confidence that they'll be
able to efficiently replace engineers if needed, integrate or lisp
components into larger ecosystems, or be able to get support problems. These
are all kind of intangible (and reasonable!) and the larger and more vibrant
the ecosystem the less risk there is.
In many ways, recent developments in technology more broadly make lisp
slightly more viable, as a result of making it easier to build applications
that use multiple languages and tools. Things like microservices, better
generic deployment orchestration tools, greater adoption of IDLs (including
swagger, thrift and GRPC,) all make language choice less monolithic at the
I've really enjoyed working with a few projects and tools. I'll probably write
more about these individually in the near future, but in brief:
- chanl provides. As a current/recovering
Go programmer, this library is very familiar and great to have. In some
ways, the API provides a bit more introspection, and flexibility that I've
always wanted in Go.
- lake is a buildsystem tool, in the
tradition of make, but with a few additional great features, like target
namespacing, a clear definition between "file targets" and "task targets,"
as well as support for SSH operations, which makes it a reasonable
replacement for things like fabric, and other basic deployment tools.
- cl-docutils provides the basis
for a document processing system. I'm particularly partial because I've been
using the python (reference) implementation for years, but the
implementation is really quite good and quite easy to extend.
- roswell is really great for getting started with CL, and also for making it
possible to test library code against different implementations and versions
of the language. I'm a touch iffy on using it to install packages into it's
own directory, but it's pretty great.
- ASDF is the "buildsystem" component of CL, comparable to setuptools in
python, and it (particularly the latest versions,) is really great. I like
the ability to produce binaries directly from asdf, and the
"package-inferred" is a great addition (basically, giving python-style
automatic package discovery.)
- There's a full Apache Thrift implementation. While I'm not presently working
on anything that would require a legit RPC protocol, being able to integrate
CL components into larger ecosystem, having the option is useful.
- Hunchensocket adds
websockets! Web sockets are a weird little corner of any stack, but it's
nice to be able to have the option of being able to do this kind of
programming. Also CL seems like a really good platform to do
- make-hash makes constructing hash
tables easier, which is sort of needlessly gawky otherwise.
- ceramic provides bridges between CL
and Electron for delivering desktop applications based on web technologies
I kept thinking that there wouldn't be good examples of various things,
(there's a Kafka driver! there's support for various other Apache ecosystem
components,) but there are, and that's great. There's gaps, of course, but
fewer, I think, than you'd expect.
The Dark Underbelly
The biggest problem in CL is probably discoverability: lots of folks are
building great tools and it's hard to really know about the projects.
I thought about phrasing this as a kind of list of things that would be good
for supporting bounties or something of the like. Also if I've missed
something, please let me know! I've tried to look for a lot of things, but
discovery is hard.
- rove doesn't seem to work when multi-threaded results effectively. It's
listed in the readme, but I was able to write really trivial tests that
crashed the test harness.
- Chanl would be super lovely with some kind of concept of cancellation (like
contexts in Go,) and while it's nice to have a bit more thread
introspection, given that the threads are somewhat heavier weight, being
able to avoid resource leaks seems like a good plan.
- There doesn't seem to be any library capable of producing YAML formated
data. I don't have a specific need, but it'd be nice.
- it would be nice to have some way of configuring the quicklisp client to be
able to prefer quicklisp (stable) but also using ultralisp (or another
source) if that's available.
- Putting the capacity in asdf to produce binaries easily is great, and the
only thing missing from buildapp/cl-launch is multi-entry binaries. That'd
be swell. It might also be easier as an alternative to have support for some
git-style sub-commands in a commandline parser (which doesn't easily exist
at the moment'), but one-command-per-binary, seems difficult to manage.
- there are no available implementations of a multi-reader single-writer
mutex, which seems like an oversite, and yet, here we are.
- There are no encoders/decoders for data formats like Apache Parquet, and the
protocol buffers implementation don't support proto3. Neither of these are
particular deal breakers, but having good tools dealing with common
developments, lowers to cost and risk of using CL in more applications.
- No support for http/2 and therefore gRPC. Having the ability to write
software in CL with the knowledge that it'll be able to integrate with other
components, is good for the ecosystem.
- There is no great modern MongoDB driver. There were a couple of early
implementations, but there are important changes to the MongoDB protocol. A
clearer interface for producing BSON might be useful too.
- I've looked for libraries and tools to integrate and manage aspects of
things like systemd, docker, and k8s. k8s seems easiest to close, as things
like cube can be generated from updated
swagger definitions, but there's less for the others.
- Application delievery remains a bit of an open. I'm particularly interested
in being able to produce binaries that target other platforms/systems (cross
compilation,) but there are a class of problems related to being able to
ship tools once built.
- I'm eagerly waiting and concerned about the plight of the current
implementations around the move of ARM to Darwin, in the intermediate
term. My sense is that the transition won't be super difficult, but it seems
like a thing.