Let’s talk about some things that we as open source creators need to get better at, including supporting other developers ourselves.
(A talk I delivered at at the first CodeDAO meet-up during Berlin Blockchain Week in August 2019, about sustainable open source development).
I’ve been advising a startup on the data transformation space. As part of this, we re-wrote the core engine in Clojure. The new version is, at the worst case, 16 times as fast in the same hardware, and in some cases over 200 times faster. And it does it with a fraction of the lines of code.
We did this in under 3 months of part-time work. We couldn’t focus our entire attention on it, as we had other concerns as well - I was involved with general team and management tasks, and the second developer was helping on other internal projects as well. To further raise the bar: we had to keep it functionality-compatible with the current version, so I had to get acquainted with the existing feature set, and it was the other developer’s first Clojure project.
Clojure made our lives so much easier. But this is not a post about why Clojure is cool.
Note: I wrote this in 2018, while I was helping a young startup and convincing them to open source their core application. If I were to write it nowadays, in 2024, I’d elaborate a bit more, including James Governor’s evergreen remark about how “You can make money with open source, but it’s extremely hard to make money from open source.” (source). Leaving it as-is, however.*
I’ve been advising a startup on the data transformation space. As part of this, we re-wrote the core engine in Clojure. The new version is, at the worst case, 16 times as fast in the same hardware, and in some cases over 200 times faster. And it does it in a fraction of the lines of code.
We did this in under 3 months of part-time work. We couldn’t focus our entire attention on it, as we had other concerns as well - I was involved with general team and management tasks, and the second developer was helping on other internal projects as well. To further raise the bar: we had to keep it functionality-compatible with the current version, so I had to get acquainted with the existing feature set, and it was the other developer’s first Clojure project.
Clojure made our lives so much easier. But this is not a post about why Clojure is cool.
I’ve been arguing about why they should open source anything that is not enterprise-specific, including this layer. There’s many advantages, which I won’t go over right now, but there’s also a looming threat.
Layers are getting commoditized faster and faster. More and more, there is demand for people who are good at wiring things together (beyond the gem install hairball approach), or tools that help with that wiring.
I suspect that’s a big part of what’s driving how many companies like Seldon are going open source-first, or how Unreal opened their code as a way to compete with Unity.
“what’s driving how many… are going” is a doozy
Remember: This rewrite took about 3 man-months, with our attention pulled in multiple directions, while we strove to remain feature-compatible with the old engine. You have to assume anyone else who has the technical chops but doesn’t have that baggage can do it as well. Do you want to be disrupted by some motivated, random person who thought what you were doing was cool, but neither needed your entire feature set nor wanted to be shackled to your cloud version?
Better to commoditize yourself before someone else does it to you. You get to have a say in how it happens, use it as fuel to propel you somewhere new. And you get to tell your customers “if you think this thing we give away is cool, you should see the part we charge for”.
I’ve released Relevance, my smart tab organizer for Chrome, under the MIT license. You can find the ClojureScript source code on GitLab, and the extension on the Chrome web store.
UnitySteer 3.1 is out on GitHub.
The main update on this version is the addition of 2D support as a separate set of behaviors, thanks to the pull requests sent by @GandaG and extra changes by @PJOHalloran.
Perhaps the trickiest thing when developing an open source library is figuring out the balance between changes you feel are necessary, and respecting early adopters.
Take the case of khroma, the ClojureScript library for providing idiomatic access to the Chrome extensions API. It’s very much a work in progress, and I’m developing it to use on an extension of my own. The way I go about it is exposing functions and event handlers mostly as I need them, to avoid getting sidetracked into surfacing functionality for which I don’t have a practical use case right away.
However, when developing something organically, the question then becomes: where do you draw the line between unifying the way the APIs behave after adding functionality, and not breaking things for those very early users?
Up until now whenever someone asked about our open source projects I referred them directly to my Github page. That’s the repository, sure, but might be a bit noisy as it contains a mix of the projects I’ve created and maintain, with repositories I’ve forked (some of which I contribute to, but not all), with older projects.
I decided it was time to clear things up a bit, so I’ve done a brief write up on those that I’m actively maintaining. It has a better introduction to each, including both their application and licensing details.
Head over to the open source page to read more, and let me know if you have any questions.