AWS Lambda, Clojure and ClojureScript

Introduction

In case you’re not familiar with them, AWS Lambdas are a fascinating idea - they are server functions you can create and run without first provisioning servers: you write your code, upload it, and pay only for the time that is executed.

There’s only one (somewhat minor) catch: given that there is no provisioned server capacity at all, the first time that you execute a lambda it has to be warmed up. The code stays live for a period of time after the first execution, but given Clojure’s start up time cost, I was wondering what effect that would have on a Clojure lambda.

Why am I doing this?

I’m looking to convert an application that gets irregular burst use to lambda. The usual approach would be that each function gets its own lambda, so I wanted to test the start up and execution time to see at which point it became an issue.

There was a second goal as well. Lambda supports multiple environments, including Java and Node.js. This means that I could choose to write the lambda functions in Clojure or ClojureScript. My gut feeling was that Clojure execution time would be faster, once it spun up, but ClojureScript instances running on Node.js would spin up much faster - I wanted confirmation, though.

Test case

The test case is simple enough. Since I wanted to focus on testing the time to warm up lambda from cold state, I came up with a small “glue code” test that depended on external APIs.

The test case:

• Receives a URL as a parameter
• Gets the page and converts the HTML to text
• Calls an external Algorithmia API for text summarization

The initial Clojure example obtained the page by slurping it and then using enlive to obtain all paragraph text in the body. While I got a different implementation working on ClojureScript, it seemed like it was starting to be an unfair comparison, since the parsing code would be different.

In the end, I decided to use a different Algorithmia API call from both the Clojure and ClojureScript versions. That allowed me to focus on testing the spin up time, and get an example on somewhat equal footing of how both would perform when acting as glue between API calls.

But first, a word on libraries

I wrote the Clojure implementation using uswitch’s lambada, and the ClojureScript version using cljs-lambda.

Both provide an easy wrapper for the lambda API, but cljs-lambda has a slightly better workflow. Mainly:

• You can easily define multiple lambdas in a project,
• It has straightforward lein tasks for building, deploying and invoking your lambdas (which wouldn’t be too hard to add to lambada, but are not there yet).

On the other hand, lambada has the advantage that you’re going to be able to benefit from all the existing Clojure libraries, as well as Clojuric wrappers for Java libraries.

While in theory you’d get the same advantage with ClojureScript, I found that several libraries I attempted to use assume that you’re running on a browser, and try to access the page or reference classes Node doesn’t provide. If you do decide on the ClojureScript route, I expect you’ll end up having to take a closer look at how libraries you’re used to taking for granted are implemented, or accessing the Node.js alternatives directly yourself.

Toooling on both Clojure and ClojureScript lags behind what you’d get for a purely Javascript approach with Serverless, though. There is no way to locally test lambdas, no support for multiple stages or regions, or any other management tools. Both options I tested are very much only API wrappers at this stage.

The results

The initial test bore my expectations: ClojureScript lambdas were warmed up much faster than Clojure lambdas. I also had to assign twice the RAM to the Clojure version for it to run at all - with 128MB we ran out of RAM before returning.

Both cases are calling out the same APIs, which on other tests take together between 800ms-1200ms to complete. Times presented are the billed duration, when invoking the lambda directly - a round trip through API Gateway will be slower.

Given that lambda scales allotted CPU power proportionally with RAM (although I have yet to find any documentation on what the proportion is), this begat another question: how far do we need to raise the RAM on the Clojure version to get it to spin up as fast as the version hosted on Node.js?

Clojure and CPU increases

I got the following results on repeated tests against the same Clojure 1.7 lambda, with different RAM assignments:

As we can see, we need to crank up the RAM all the way up to 1024MB for the assigned CPU to spin up the lambda as fast as the ClojureScript version. Execution performance once warm also increases, of course.

This got me wondering… do we get a proportional increase on ClojureScript and Node.js?

ClojureScript and CPU increases

Let’s try an equivalent test on for the ClojureScript instance:

Interesting. For the Node.js/ClojureScript lambda, not only there was barely any difference in warm up time (I guess there’s only so far you can push it), but execution time did not change at all. The Clojure version, on the other hand, was not only already running faster than the ClojureScript version on our first test, but did get faster as we increased the RAM and CPU.

Why worry about the warm up time?

Of course, you might wonder why worry about the warm up time at all. This might not be a concern if you’re invoking your lambda because of an event, since it has little effect beyond your paying a bit more for the first execution. It becomes an issue however if you expect to make your lambda available via API Gateway, which has a hard, non-configurable 10 second timeout on the code being invoked.

On our test case, a very basic Clojure lambda with only 256MBs assigned is cutting it pretty close with its average 8900ms, and on several of my tests it failed to finish under 10s when cold. This means that an even slightly more complex lambda, or one with a larger codebase or requiring more external calls, would need a larger RAM assignment (and higher cost) to minimize the chances of the first invocation timing out.

The tests

If you want to run some timing tests yourself you can invoke the APIs via the URLs below, by passing a URL-encoded page address as the parameter after summarize.

For ClojureScript and Node.js, with 128MBs:

http https://zle9inihva.execute-api.us-west-2.amazonaws.com/prod/summarize/http%3A%2F%2Fblog.jetbrains.com%2Fdotnet%2F2016%2F01%2F13%2Fproject-rider-a-csharp-ide%2F

And for Clojure and Java, with 512MBs:

http https://0l2ulqu44m.execute-api.us-west-2.amazonaws.com/prod/summarize/http%3A%2F%2Fblog.jetbrains.com%2Fdotnet%2F2016%2F01%2F13%2Fproject-rider-a-csharp-ide%2F

Conclusion

While the experiments bore my initial expectations, as far as start up time vs. performance went, I have to say I was surprised at the huge difference between both. I probably shouldn’t have been, considering other tests, but having to increase the RAM and CPU 8x to get the same warm-up performance was a bit of a shock.

Having said that, performance will scale better on Clojure with more RAM, you won’t have to wonder if the library you’re using depends on the browser, and frankly, lambda pricing is low enough that even at 1024MB it likely wouldn’t even register. Given those factors, unless what you’re doing is rather basic glue code, I’d still go with a Clojure lambda over a ClojureScript at this point.

Published: 2016-01-18