erlang-in-lisp manual |
Because Erlang-in-Lisp aims to copy the semantics of Erlang, much of this section highlights syntactic issues or places in which Erlang-in-Lisp differs from Erlang. It may be best to consult the Erlang documentation or an Erlang book if you are unfamiliar with its shared nothing approach to concurrency.
One of the advantages of Lisp and Erlang is the REPL. It allows for
an interactive style of development. To have the full power of
Erlang-in-Lisp at the REPL, too, it is necessary to call the
toplevel function. After this function has been called, it is
possible to send and receive messages at the REPL; it is a
full-fledged Erlang-in-Lisp process.
The toplevel function also takes the concurrency strategy as a
parameter. To call the toplevel function and use the unix
process based concurrency (it is the default):
(toplevel)
or to specify the concurrency method explicitly:
(toplevel :process-class 'unix-process)
Likewise, to specify thread based concurrency:
(toplevel :process-class 'thread-process)
A second function, toplevel-exit can be called when
Erlang-in-Lisp functionality is no longer desired at the REPL:
(toplevel-exit)
To spawn a process, simply pass the spawn function a name for
your new process, and a thunk that is to be executed inside this new
process. For example, to create a simple hello world process:
(spawn 'hello-world #'(lambda () (format t "hello world~%")))
spawn returns a process identifier, or pid, that is used for
sending messages to the process.
The send function is used to send a message to a process using
its process id. With the pid you simple:
(send pid 'this 'is 'my 'message)
Anything following the pid is sent as the message.
Receiving messages from another process is simply a matter of calling
the receive function with a block of code that contains the
message patterns to match, and the code to execute when such a match
is found. The receive pattern matching syntax follows that of
fare-matcher. receive
is a blocking call, and will not continue until a match has been
found.
As an example, say we’d like to receive a message starting with the symbol
hello followed by one variable (which we’ll bind to a),
and then print out that variable:
(receive
((list :hello a)
(format t "The value of a is: ~A~%" a)))
The receive-after function takes three parameters: a receive
block, an integer timeout, and an after block. The recive block is
the same as in the receive function, however, since
receive-after is implemented with a destructuring bind, it is
necessary to wrap the receive and after blocks in an extra set of
parenthteses. The receive-after function blocks for at least
the number of seconds specified as the timeout. If a message is still
not found, the code in the after block is executed.
Now we extend our receive example above to print a timeout message after 5 seconds if a message has not been received:
(receive-after
(((list :hello a)
(format t "The value of a is: ~A~%" a)))
5
((format t "No message received.~%")))
This syntax is certainly a bit tedious and is on the todo list of things to change/improve.
The Erlang-in-Lisp error system uses the condition system of Common Lisp directly. All errors that are not handled inside of a process (and thus interrupt the normal flow of that process) are converted to messages and sent to linked processes. At the moment, this means that all processes behave like Erlang system processes, and therefore there is no ’cascading failure’ behavior.
Below is the simple ping-pong example that is often to introduce the
the ideas of Erlang. It combines some of the functions discussed
above. The ping function takes a pid and an
integer, count as arguments. It then sends count
messages to the pid, waiting after each message for a response.
The pong function simple accepts ping messages and responds to each with a pong until the receipt of a done message, at which point the process exits.
(defun ping (pong count)
(loop for iter from 1 to count
do (send pong :ping iter (self))
do (receive
((list :pong) (format t "PING -- Got pong.~%"))))
(send pong :done))
(defun pong ()
(receive
((list :ping iter ping)
(format t "PONG -- Got ping.~%")
(send ping :pong)
(format t "PONG -- Sent pong.~%")
(pong)) ;;check that this tail call can be eliminated.
((list :done)
(format t "PONG -- Got done."))))
(defun ping-pong (count)
(let* ((pong (spawn 'pong #'pong)))
(ping (spawn 'ping #'ping pong count))))
This section is for those interested in in helping with the development of erlang-in-lisp.
erlang-in-lisp is kept in git repository. To get the code with git, create a new repository and pull in the lastest version:
mkdir erlang-in-lisp git init git pull http://common-lisp.net/project/erlang-in-lisp/git/ master
To get the sources compiled and running, you will need to the appropriate dependencies (see section 2.24Dependenciessubsection.2.2).
Obviously there are the regular git facilities to create branches, etc, but once changes are ready for the mainline you can push a patch to the master branch (the assumption is you are in the erlang-in-lisp group on common-lisp.net):
git push USER@shell.common-lisp.net/project/erlang-in-lisp/git
All the dependencies of erlang-in-lisp are either stored directly in
the repository or can be fetched with the makefile in the
deps directoy. Simple change to this directory and issue
the make get-deps command. The dependencies will download
(it can take a while). For this process to complete normally, your
system will need to have darcs, wget, tar, gzip, and git installed.
You can remove the downloaded dependencies with make clean.
Note that this makefile is not very robust and should probably be replaced with something more maintainable.
Though most of these are self-explanatory, a brief description of the directory structure as it currently stands follows:
src – erlang-in-lisp sources are stored here,
including the asdf system definition file.
test – Tests are stored here. This includes a
separate system defintion for testing purposes (as it may have
its own dependencies, etc). See section 2.24Dependenciessubsection.2.2.
deps – Dependencies for erlang-in-lisp are stored
here. Some are directly in the repository, and the other
dependencies can be fetched with the included makefile.
website – Files for the website on
common-lisp.net are stored here.
doc – Documentation, including this manual, is
stored here.
To get started, you must first load the erlang-in-lisp package.
In SBCL, this is as simple as (require 'erlang-in-lisp) when
you are in the same directory as erlang-in-lisp.asd. Once that
is done, it is simple to run some examples. Below, we run the
ping-pong example:
CL-USER> (in-package #:erlang-in-lisp)
#<PACKAGE "ERLANG-IN-LISP">
ERLANG-IN-LISP> (toplevel :process-class 'thread-process)
#<THREAD-PROCESS {A917FD1}>
ERLANG-IN-LISP> (ping-pong 3)
pong started
ping started
PONG -- Got ping.
PONG -- Sent pong.
pong started
PING -- Got pong.
PONG -- Got ping.
PONG -- Sent pong.
pong started
PING -- Got pong.
PONG -- Got ping.
PONG -- Sent pong.
pong started
PING -- Got pong.
PONG -- Got done.
1
ERLANG-IN-LISP> (toplevel-exit)
Because the concurrency strategies of erlang-in-lisp are interchangeable, the only change to run the above example with unix-process based concurrency is:
ERLANG-IN-LISP> (toplevel :process-class 'unix-process)
Likewise if any other concurrency strategies are defined in the future, the name of the class that holds the process metadata for that strategy need only be specified.
Erlang-in-Lisp uses the
fiveam testing
framework. For now the tests are embedded direcly in the files
alongside the code, but this may become untenable if extra
dependencies are introduced (though we could use the #-fiveam
read macro as well).
To run all the tests simple type:
ERLANG-IN-LISP> (run!)
To run on specific test (here, for example, the
simple-send-receive-test):
ERLANG-IN-LISP> (run 'simple-send-receive-test)
Tests are found at the end of the .lisp file whose code they test.
See eil.lisp as en example.
Here is a simple unit tests that sees whether or not five is equal to five:
(fiveam:in-suite erlang-in-lisp::eil-suite) ;;included once before unit tests (fiveam:test five-equality "Test whether five is equal to five." (is (eql 5 5)))
Erlang-in-Lisp has a design inspired by the metaobject protocol.
There is a thin, user-facing macro layer that in turn calls an
extensible generic function based layer. The macro layer can be found
in core-eil.lisp; higher level abstractions built on top of
this layer can be found in eil.lisp. The heart of the generic function
layer is in process.lisp. Two extensions of the generic
function layer that provide the concurrency strategies discussed below
can be found in process-unix.lisp and process-threads.lisp.
The original plan for Erlang-in-Lisp was to start with fork as the primary method for spawning language-level processes using a “one OS process per Erlang process” model indeed. The reasons follow.
So no, fork-based concurrency not the be-all end-all of Erlang-in-Lisp, but I really think it’s the best way to start.
Some simple thread based concurrency has been implemented on top of the bordeaux-threads library. Because the threads share the same namespace, there is plenty of room for programmer mischief. However, if no messages are mutated, all is well and the system should work properly.
It is implemented as a simple (and likely inefficient) lock and condition-variable based system. Currently its main goal is to show that pluggable concurrency strategies are possible and to organically evolve the mechanism by which the various concurrency strategies are implemented.
First off, it would be nice to eliminate receive-after and
instead have one function, receive with some loop-like
syntax where we could include a timeout and a block after an
after statement. Perhaps this is not very Lispy, but neither is
loop.
Also, timeouts are not currently not implemented in the thread based concurrency (we need to somehow parameterize our wait on the condition variable) and may be poorly defined in the unix based concurrency. In the unix based concurrency, the timeout is interpretted to mean how long the process blocks for a message after it has already searched the mailbox for matches. While the timeouts in Erlang are certainly meant to be soft, this particular implementatation could lead to problems if the mailbox is very full. A more sophisticated timing mechanism is needed.
Right now all unhandled Lisp conditions are convered to messages and sent to linked processes. More testing of this is needed.
Also, all processes are Erlang system processes. Thus, processes that are linked but not system processes should be made to fail when necessary. This could be tricky, but may not be a huge priority.
Currently the pids are serialized with read and print (this is necessary in order to send the pids as messages). It is a huge kludge. There must be a nicer way to do this.
One of the nicer ways is using
cl-store, and some
attempts have been made in the direction (the attempts remain in
process-unix.lisp and are commented out). A good discussion of
all of the options is available at:
http://www.pentaside.org/paper/persistence-lemmens.txt
Pids are not the only problem; to acheive true distribution, we must be able to serialize closures and send them over the wire. This has been accomplished in Termite with Gambit-C, but we would also like it to be portable across Lisp implementations. Some possible solutions may be found in the green threads discussion below.
It would be nice to implement a concurrency strategy that stay’s true to Erlang’s lightweight green threads. Several approaches:
It would be nice to have Erlang-in-Lisp programs that are able to interact with existing Eralng nodes and processes. One way to do this would be to create bindings to the erl_interface C library. This is probably most easily done with the cffi groveler. Lispier bindings could be created later.
An utlimate goal is to arse Erlang with something like ometa. Ideally we would not parse Erlang directly, but instead parse core erlang, which is specified in this document. Parsing Core Erlang should be much simpler and allows us to use the existing Erlang front end to deal with any language changes that are not reflected in the more stable Core Erlang spec.
Watching other processes is hard to do reliably in Unix. If the processes are linked by parent/child relationships, then the parent can easily watch if children die with SIGCHLD (and *must* do it to avoid zombies) - but this is not general enough mechanism for Erlang-in-Lisp because we are mapping a graph of Erlang processes onto the unix process tree.
Another option is to share a pipe with the other process before you even fork, and add the watching of it in your epoll loop, so you can say the process is dead when you get an EPOLLHUP. This depends on all the considered processes following the proper convention of sending around the fd to the pipe, and so is not applicable to watching arbitrary other processes, but may work.
It might be possible to open an fd on some file in /proc/$pid and receive an event when the process dies.
These solutions require experimentation, but have the advantage that they are not subject to the same race conditions as having to register with a central server after you’ve forked.
This document was translated from LATEX by HEVEA.