Introduction

Elephant is a persistent object database for Common Lisp with full transaction semantics. A persistent metaprotocol overrides class creation and slot accesses, making slot values automatically persistent. Objects can be indexed by class and slot value and retrieved via unique-id, map and list operations over slot values and constraint-based queries.

Elephant is split between a common API and several data stores. Berkeley DB, CLSQL and postmodern interfaces are provided. Elephant inherits the ACID properties of these stores and all database operations are thread-safe by design.

Elephant is used predominantly in website applications, such as with the weblocks framework. Elephant runs on SBCL, Allegro, Lispworks, and ClozureCL.

The elephant code base is available under the LLGPL license. Data stores each come with their own, separate license.

Development work on Elephant been limited since 2009 and support not complete for the latest versions of SBCL and ClozureCL. It hasn't been tested at all with recent Allegro and Lispworks releases.

Key Features

Persistent Objects and Values: A persistent metaclass enables CLOS instances to store slot values in the database
Stored values can be any lisp object other than functions, closures, and class objects.
Persistent class redefinition is supported with both greedy and lazy update policies
Rich slot semantics

Persistent classes support slot types with different behaviors:
Persistent - Slot writes are automatically persist in the database
Indexed - the database maintains an index that maps slot values to instances
Derived index - names an index mapping a function of other slots back to instances.
Transient - reverts to standard CLOS slot behavior
Cached - a persistent slot that can be converted to and from a transient slot for fast access.
Set-valued - an unordered set of values; insert and delete operations are persistent
Associations - A protocol for maintaining N:1 and N:N relations between two classes

Indexing API

Objects can be retrieved by class, indexed slot values or a range of slot values.
Efficient minimally-consing mapping operators are provided.

Lightweight Persistent Aggregates

The database exports four persistence collection types:
Persistent set - a persistent collection of unique objects with insert, delete, map and find operations.
A Btree - an efficient O(log n) key-value data structure with both map and cursor APIs
Duplicate btree - A btree that supports duplicate keys.
Indexed Btree - An indexed BTree is an extension to the btree that allows additional index trees to be defined that automatically index from some function of a btree value back to that key-value entry.

Transactional Architecture

Elephant is a transactional database supporting complete ACID properties for database operations. Operations on database objects are thread-safe. The system provides a convenience macro, with-transaction, that bundles multiple database operations into atomic transactions and automatically handles transaction restarts due to contention or deadlock.

Multiple Data Stores

There are currently three different data stores that support the Elephant API: Berkeley DB, Postgresql via the postmodern library, and any database supported by the CLSQL library including SQLite3.
Transparent migration between store instances.

Automatic deadlock detection

An online mark and sweep garbage collector

Berkeley DB transparently supports multiple lisp images on the same machine.

CLSQL and postmodern stores support Elephant persistence across multiple machines and lisp images.

An all-lisp store is planned as an eventual feature at which time the CL-SQL store will be deprecated.

Object Query Language (Prototype)

Use a simple constraint language to define sets of objects to retrieve or map. The query planner exploits slot indices when available and performs efficient in-memory set intersections.
Rich slot semantics: Persistent classes support slot types with different behaviors:
Persistent - Slot writes are automatically persist in the database
Indexed - the database maintains an index that maps slot values to instances
Derived index - names an index mapping a function of other slots back to instances.
Transient - reverts to standard CLOS slot behavior
Cached - a persistent slot that can be converted to and from a transient slot for fast access.
Set-valued - an unordered set of values; insert and delete operations are persistent
Associations - A protocol for maintaining N:1 and N:N relations between two classes
Indexing API: Objects can be retrieved by class, indexed slot values or a range of slot values.
Efficient minimally-consing mapping operators are provided.
Lightweight Persistent Aggregates: The database exports four persistence collection types:
Persistent set - a persistent collection of unique objects with insert, delete, map and find operations.
A Btree - an efficient O(log n) key-value data structure with both map and cursor APIs
Duplicate btree - A btree that supports duplicate keys.
Indexed Btree - An indexed BTree is an extension to the btree that allows additional index trees to be defined that automatically index from some function of a btree value back to that key-value entry.
Transactional Architecture: Elephant is a transactional database supporting complete ACID properties for database operations. Operations on database objects are thread-safe. The system provides a convenience macro, with-transaction, that bundles multiple database operations into atomic transactions and automatically handles transaction restarts due to contention or deadlock.
Multiple Data Stores: There are currently three different data stores that support the Elephant API: Berkeley DB, Postgresql via the postmodern library, and any database supported by the CLSQL library including SQLite3.
Transparent migration between store instances.

Automatic deadlock detection

An online mark and sweep garbage collector

Berkeley DB transparently supports multiple lisp images on the same machine.

CLSQL and postmodern stores support Elephant persistence across multiple machines and lisp images.

An all-lisp store is planned as an eventual feature at which time the CL-SQL store will be deprecated.
Object Query Language (Prototype): Use a simple constraint language to define sets of objects to retrieve or map. The query planner exploits slot indices when available and performs efficient in-memory set intersections.

Limitations

Elephant is subject to some limitations that you may want to consider. Functions, closures, and class objects cannot be stored (although their list forms can be).

Perhaps the most frustrating limitation, and the one requiring the most effort to work around is that built-in aggregate objects (arrays, lists, hash tables) do not support persistent operations. Any list that is loaded from a slot or index will result in a freshly consed list. Elephant addresses many common cases via the persistent aggregate data structures described above.