SLURP

I have written Common Lisp, Erlang and Javascript versions of a library I call SLURP. It’s based on the ideas used in the Smalltalk SRP library.

Right now it’s pretty rough, but useable enough to transfer simple objects with named slots (eg CLOS objects, Javascript objects). On Erlang I’m using some very ugly code to get it to work with Erlang records (which are a compile-time phenomena).

It currently supports some basic types such as bytes, unsigned integers, strings, lisp-style cons cells and arrays of the basic types.

In the future I might look at performance optimization and writing a formal definition (machine checkable) for the protocol.

The Good

Encoding native lisp objects is as simple as calling slurp:encode

(defmethod send-state-change (object-id state-change)
  (let* ((encodedState (slurp:encode state-change))

The Bad

In Javascript, we have to add additional properties to a prototype to tell SLURP how to encode it:

function FuraxSubscriptionRequest(objectId, clientId) {
        this['OBJECT-ID'] = objectId;
        this['CLIENT-ID'] = clientId;
}

FuraxSubscriptionRequest.prototype['SLURP_TYPE'] = "furax-subscription-request";
FuraxSubscriptionRequest.prototype['SLURP_PACKAGE'] = "FURAX";

The Ugly

In Erlang we have to grab the record descriptions at compile time and save them in a dictionary for later reflection

The source is available by getting:

hg clone http://subvert-the-dominant-paradigm.net/repos/hgwebdir.cgi/slurp

SRP (State Replication Protocol)

While investigating squeak Smalltalk I discovered SRP, a serialization library.

What I really like about SRP:

  • Compact encoding – Unlike XML markup there are no redundant closing tags and meta information (like the tag name) only has to be recorded once
  • Preservation of the object graph – It can deal with cycles in the object graph and will make sure an object is only written to the output stream once
  • Tolerance of schema/class changes – In languages with powerful reflection mechanisms (like Smalltalk) it’s possible to reconstruct the stored object even when the original class structure has changed or been removed. Since field access in Smalltalk is just a message send, the library can build an object that responds to these messages even if it is not an instance of the original class used when storing the object.

As an exercise I did a quick port of SRP to the slate environment.

I need a mechanism similar to SRP or JSON to transfer objects between Common Lisp, Erlang and Javascript for my Guli project.

Persistence

The Java Virtual Machine that I adopted provided a feature called orthogonal persistence.

Ever since being introduced to the idea I have been convinced that developers are unnecessarily burdened by the need to manually manage storage and retrieval of persistent data.

Although several systems have implemented various ways of providing persistence to developers and end-users, it has never become mainstream.

See persistent operating systems on DMOZ for a description of the persistent Java project at Sun and the L3 system.

Beyond Java

Python logo
Squeak logo
Lisp logo

Over time I have become disillusioned with Java and the effort to produce an Open Source JVM and clean-room Java libraries.

Despite the massive popularity of Java and Microsoft technologies, there are many alternative high-level languages available to programmers today:

Python is very popular and a powerful tool for rapid development of small projects. Squeak Smalltalk is a versatile graphical development environment which also allows rapid prototyping. The best thing about Squeak is the ability to modify code and have the system updated immediately.

I also looked at slate, a modern incarnation of Smalltalk that has a more powerful dispatch system (allowing better organization of code) and uses prototyping instead of a class hierarchy.

I was drawn to Lisp after watching Rainier Joswig’s Slime demo and reading a couple of chapters of Practical Common Lisp.