Clojure and DynamoDB with Faraday, Part 1
DynamoDB and Clojure
There are two main options of accessing DynamoDB in Clojure right now - Amazonica, which provides a Clojure client through reflection that’s comprehensive but a direct translation of Amazon’s; and Faraday, which does not take the reflection approach and provides a simpler, more succinct access than one would otherwise get.
Both have a paucity of examples. Amazonica can probably better get away with it since it gets to piggy-back on the AWS examples out there, but Faraday’s tests have been growing and can double as examples.
I started with Faraday, since its more concise API was more appealing and I liked its reasonable defaults. In the process, I’ve added some missing functionality, expanded tests, and noticed that the examples from Amazon’s Getting Started tutorial weren’t fully covered.
Let’s fix that.
Methodology
I’ll be referring to the Javascript examples on basic DynamoDB operations, since they are the ones where the data set-up will more closely match Clojure, instead of Java’s nested method calls and fluent property setters.
You may want to follow along Amazon’s original explanation as well. That’ll help with additional details on DynamoDB which I won’t duplicate here, as well as showcase how much more compact the Faraday version is.
Step 0: Before we get started
First off, you’ll need a Clojure project with Faraday referenced. We’ll use Faraday 1.9.0alpha 3, since it has support for features not yet included on the stable 1.8 release.
1 | [com.taoensso/faraday "1.9.0"] |
You’ll also need a DynamoDB instance. You can run these tests against a live instance if you wish, but I’ll assume a local instance for simplicity.
See the notes on the Faraday README for more details.
Step 1: Create a table
Step 1 shows you how to create a table via the JavaScript console. We’ll do this through the REPL.
We’ll start by requiring Faraday, and defining our connection options.
1 | (require '[taoensso.faraday :as far]) |
We then create a table with :artist as a hash key, and a range key called :song-title.
1 | (far/create-table client-opts :music |
The :block? parameter indicates the create-table call should not return until the table is actually active.
If you’re following along the JavaScript example, you’ll see that Faraday’s call is more compact - we don’t need to reference :song-title and :artist on both the key schema and attribute definitions, since Faraday takes care of that behind the scenes.
This results in our table’s information:
1 | {:lsindexes nil, |
Notice that we used an attribute with a dash in the name, to stick to more Clojuric conventions. While that is valid, DynamoDB does not like having dashes on attribute names on some expressions (which we’ll see later), so you may want to keep it in mind when selecting your key names.
Step 2: Get Information About Tables
On step 2 we’ll query DynamoDB to return information about the tables we have.
Retrieve a Table Description
Retrieving a table description is also straightforward:
1 | (far/describe-table client-opts :music) |
This results on a description like the one returned by the create-table call.
1 | {:lsindexes nil, |
The first two nil values are the local secondary and global secondary indexes - we don’t have any yet, but we’ll create some later.
Retrieving a List of Your Tables
Getting a list of tables is also trivial.
1 | (far/list-tables client-opts) |
Which will return only one table name, unless you’re testing against a live database.
1 | (:music) |
Step 3: Write Items to the Table
So far so good, but we need to get data in. DynamoDB supports both putting individual items or making batch requests.
Write a Single Item
To write an item, we call put-item with a table name and a hashmap of item attributes.
1 | (far/put-item client-opts |
We can of course use nested maps for the attributes, such as in the case of :tags. This operation will return nil.
It’s also possible to do conditional writes, where we send a expression that needs to be true for the write to succeed. To only put the item on if neither the artist nor the song title exist, we’d do it this way:
1 | (far/put-item client-opts |
If you execute that command after having already created the item, you’ll get a ConditionalCheckFailedException telling you that the operation could not complete because of your conditions.
Notice we need to use an alias on :expr-attr-names because the attribute name has a dash in it. This is something I’ll expand on during Part 2.
Write Multiple Items
We can use batch-write-item to put or delete multiple items at once, from multiple tables. On the following example, we’ll define a batch request and then execute them all at once:
1 | (def batch-request |
As you can see, the request is a hash map where the keys are the tables to write to and the values as hash map of operations. For the operation hash-map, the key is an operation type (either :put or :delete), and the values a vector of items to be put, or a vector of keys to be deleted.
The batch-write-item call will result on a map with the unprocessed requests, as well as the consumed capacity units (we won’t get the latter on a local DynamoDB).
1 | {:unprocessed {}, :cc-units {}} |
And we can of course retrieve any of these items by using the :artist and :song-title, which we had defined as the key.
1 | (far/get-item client-opts :music {:artist "No One You Know" |
Gets us…
1 | {:artist "No One You Know", |
Next steps
That’s it for now, since this is getting long. On part 2 I’ll look into more options for retrieving items, including adding a projection for getting a subset of attributes, as well as getting multiple items at once with a batch request.
Published: 2016-01-04
