2.5 Serialization

What can you put into the store? An ever-growing list of things: numbers (except for complexes, which will be easy to support), symbols, strings, nil, characters, pathnames, conses, hash-tables, arrays, CLOS objects. Nested and circular things are allowed. You can store basically anything except lambdas, closures, structures, packages and streams. (These may eventually get supported too.)

Unfortunately Berekely DB doesn't understand Lisp, so Lisp data needs to be serialized to enter the database (converted to byte arrays), and deserialized to be read. This introduces some caveats (not unique to Elephant!):

1. Lisp identity can't be preserved. Since this is a store which persists across invocations of Lisp, this probably doesn't even make sense.

             * (setq foo (cons nil nil))
             => (NIL)
             * (add-to-root "my key" foo)
             => NIL
             * (add-to-root "my other key" foo)
             => NIL
             * (eq (get-from-root "my key")
                   (get-from-root "my other key"))
             => NIL
        

2. Mutated substructure does not persist

             * (setf (car foo) T)
             => T
             * (get-from-root "my key")
             => (NIL)
        

   This will affect all aggregate types: objects, conses, hash-tables, et cetera. (You can of course manually re-store the cons.) In this sense elephant does not automatically provide persistent collections. If you want to persist a collection on every access see See Using BTrees.

3. Serialization and deserialization are pretty fast, but it is still expensive to store large aggregate objects wholesale. Also, since object identity is impossible to maintain, deserialization must re-cons the entire object every time. This eager allocation is contrary to how most people want to use a database: one of the reasons to use a database is if your objects can't fit into main memory all at once.
4. Merge-conflicts in heavily multi-process/threaded situations. More on this later.

But don't despair, we'll solve most of these problems in the next section.....