how does one _model_ data from relational databases in clojure?

Question

I have asked this question on twitter as well the #clojure IRC channel, yet got no responses.

There have been several articles about Clojure-for-Ruby-programmers, Clojur

Answer 1

This question hasn't been answered in a while, but Korma http://sqlkorma.com is another project that also helps reduce the dissonance between SQL and Clojure. It's been worked on more recently and should work with more recent Clojure versions.

Answer 2

A new micro SQL DSL built upon clojure.contrib.sql has been started. It's on Github weighing in at a lightweight 76 loc.

There's a lot of examples and theory behind the DSL on the author's blog, linked on his Github profile (new user SO hyperlink limit bah). Its design allows SQL to be abstracted into much more idiomatic Clojure, I feel.

Answer 3

There are a couple ORM-like libraries in the works nowadays.

• clj-record
• Carte
• Oyako (Disclaimer, I wrote this one.)
• ClojureQL is more of an SQL-generating lib from what I can see, but it deserves mention.

On the mailing list, some (smart) people recently described some other models for how this might work. Each of these libraries takes a fairly different approach to the problem, so be sure to take a look at them all.

Here's an extended example using Oyako, for example. This library isn't production ready and still under heavy development, so the example may be invalid in a week, but it's getting there. It's a proof-of-concept in any case. Give it some time and someone will come up with a good library.

Note that clojure.contrib.sql already lets you fetch records from a DB (via JDBC) and end up with immutable hash-maps representing records. Because the data ends up in normal maps, all of the myriad Clojure core functions that work on maps already work on this data.

What else does ActiveRecord give you? I can think of a couple of things.

A concise SQL-query DSL

The way I model this mentally: First you define the relationship between the tables. This doesn't require mutation or objects. It's a static description. AR spreads this information out in a bunch of classes, but I see it as a separate (static) entity.

Using the defined relationships, you can then write queries in a very concise manner. With Oyako for example:

(def my-data (make-datamap db [:foo [has-one :bar]]
                              [:bar [belongs-to :foo]]))

(with-datamap my-data (fetch-all :foo includes :bar))

Then you'll have some foo objects, each with a :bar key that lists your bars.

In Oyako, the "data map" is just a map. The query itself is a map. The returned data is a vector of maps (of vectors of maps). So you end up with a standard, easy way to construct and manipulate and iterate over all of these things, which is nice. Add some sugar (macros and normal functions), to let you concisely create and manipulate these maps more easily, and it ends up being quite powerful. This just just one way, there are a lot of approaches.

If you look at a library like Sequel for another example, you have things like:

Artist.order(:name).last

But why do these functions have to be methods that live inside of objects? An equivalent in Oyako might be:

(last (-> (query :artist) 
          (order :name)))

Save/update/delete records

Again, why do you need OO-style objects or mutation or implementation inheritance for this? First fetch the record (as an immutable map), then thread it through a bunch of functions, associng new values onto it as needed, then stuff it back into the database or delete it by calling a function on it.

A clever library could make use of metadata to keep track of which fields have been altered, to reduce the amount of querying needed to do updates. Or to flag the record so the DB functions know which table to stick it back into. Carte even does cascading updates (updating sub-records when a parent record is altered), I think.

Validations, hooks

Much of this I see as belonging in the database rather than in the ORM libary. For example, cascading deletes (deleting child records when parent records are deleted): AR has a way to do this, but you can just throw a clause onto the table in the DB and then let your DB handle it, and never worry again. Same with many kinds of constraints and validations.

But if you want hooks, they can be implemented in a very lightweight way using plain old functions or multimethods. At some point in the past I had a database library that called hooks at different times in the CRUD cycle, for example after-save or before-delete. They were simple multimethods dispatching on table names. This lets you extend them to your own tables as you like.

(defmulti before-delete (fn [x] (table-for x)))
(defmethod before-delete :default [& _]) ;; do nothing
(defn delete [x] (when (before-delete x) (db-delete! x) (after-delete x)))

Then later as an end user I could write:

(defmethod before-delete ::my_table [x] 
  (if (= (:id x) 1)
    (throw (Exception. "OH NO! ABORT!"))
    x))

Easy and extensible, and took a couple seconds to write. No OO in sight. Not as sophisticated as AR maybe, but sometimes simple is good enough.

Look at this library for another example of defining hooks.

Migrations

Carte has these. I haven't thought much about them, but versioning a database and slurping data into it doesn't seem beyond the realm of possibility for Clojure.

Polish

A lot of the good of AR comes from all the conventions for naming tables and naming columns, and all the convenience functions for capitalizing words and formatting dates and such. This has nothing to do with OO vs. non-OO; AR just has a lot of polish because a lot of time has gone into it. Maybe Clojure doesn't have an AR-class library for working with DB data yet, but give it some time.

So...

Instead of having an object that knows how to destroy itself, mutate itself, save itself, relate itself to other data, fetch itself, etc., instead you have data that's just data, and then you define functions that work on that data: saves it, destroys it, updates it in the DB, fetches it, relates it to other data. This is how Clojure operates on data in general, and data from a database is no different.

Foo.find(1).update_attributes(:bar => "quux").save!

=> (with-db (-> (fetch-one :foo :where {:id 1})
                (assoc :bar "quux")
                (save!)))

Foo.create!(:id => 1)

=> (with-db (save (in-table :foo {:id 1})))

Something like that. It's inside-out from the way objects work, but it provides the same functionality. But in Clojure you also get all the benefits of writing code in an FP kind of way.