The first time I coded in something called a "framework" was a really long time ago in CakePHP, a framework that apparently still exists to this day and taught me a lot about working within the constraints of a library to achieve what you want, and about the limitations of such an approach. This was in my teenage years, before I even started university, so I didn't really last long with it or even with PHP, and soon started to dabble in other languages. For a while I didn't really do any web programming, and when I started doing that again, it was Node.js that grabbed my attention.

Node.js, however, was then characterized by a pretty apparent lack of big-name frameworks. It was the time of Spring, Symfony, Django, and many many others, but in Node we had to make by with microframeworks. But that was fine, Node was essentially the new kid in the block and starting a project with a microframework is incredibly easy so I don't think anybody really cared at first. It wasn't until more mature projects started realizing that the DIY nature of microframeworks doesn't solve many problems that we started getting something a bit more serious.

And yet, what are our options right now? If you look at any lists of "Node.js frameworks", you'll probably find Express, Koa, Fastify and some other libraries that we wouldn't typically classify as fully-fledged frameworks. The most common addition to these lists is NestJS, which is a bit more of a serious contender, but my experience with it has been pretty terrible, so I'd rather not comment.

I've been working in backend web development with Node.js since fall of 2015. Almost ten years, and about 8 of those have been working with Express or a short while with Restify, which is pretty much the same but designed for purely API servers. The other two have been two years working with Hapi that were just ok. In two of those jobs I got to design and implement a layer on top of Express to hide some of the details and make working with it easier, and those were the happiest years of my professional life by far.

So you've read the title: after all those years, thinking a lot about what I would love to see in a framework, and not getting to do anything fun to make my life happier and easier at work for a year an a half, I'm about to start coding my own framework.

Another JavaScript framework?

If that's what you immediately thought, I don't blame you: There is a recurring meme about the oversaturation of frameworks in javascript, but that's pretty much a frontend problem. The Node.js space is also not empty, but I think there's still a space that nothing has been able to fill.

Now, my idea is not just to do some alternative to Express, or even NestJS. These are both HTTP frameworks, and they solve that, and only that. I think we're lacking something slightly more ambitious. I'm thinking of what I'm calling an application framework.

And whether that's a thing that already exists, I don't know, probably, I'm not inventing anything new, but what I mean is that this thing shouldn't just solve the HTTP layer, but the whole backend application set of needs.

What's an application framework?

What I've felt missing in most projects I've been on is, precisely, everything that a simple framework doesn't solve: Managing background jobs, dispatching and consuming events, doing errors and logging right, or even basic stuff like managing the startup and shutdown of the application. I don't think any of the projects I've laid my hands on have ever been able to do a graceful shutdown before I arrived.

So the first thing I consider that an application framework should do is managing the application. Startup, shutdown, and general lifecycle of components. Essentially, an in-language init system. This system is usually in charge of managing the order of startup and shutdown based on declared dependencies, and can double as a dependency injection system in its own right. This is one of the few things that bigger framweroks tend to get right, though, at least to an extent.

Another thing that I really want my application framework to do is to completely decouple itself from HTTP. While most server side applications are HTTP APIs of some sort, not every interaction with them is an HTTP call. There are times that you need to launch management one-time commands, scripts that interact you the application, etc. So instead, the framework should define its endpoints completely decoupled from the idea of HTTP, and then some other layer maps those to HTTP calls.

Another thing that I miss in most frameworks are distributed task queues and event systems. Many frameworks include some way of running asyncronous jobs of some kind, but doing so asynchronously and in a distributed manner (i.e., the workers don't necessarily share a process with the task initiators). Similarly, event systems are typically either in-process, providing no solution for highly scaled systems or fault tolerance, or way too heavyweight, intended for eternal storage and some form of "event sourcing" instead of just an asynchronous notification that should survive crashes.

Isn't that too ambitious?

Maybe. Probably. More than likely. But I'm just thinking about it too much lately, and I think it's time for me to start coding the thing I wish I already had. I did exactly that when I created Polyethylene and even though I'm probably its only user, I'm really glad I made it every time I use it.

It's also the way I've decided I need to address my growing problem of coding making me miserable. Because I know too well that it's not coding that's making me miserable, it's whatever project I'm working on, it's having to work with suboptimal frameworks while my head insists I have a solution and I'm not making it. So it's time to make it.

It was also a good way to finally write something here, and to calm my mind down from some recent developments at work. I hope I can come back to this topic later and expand a bit on what I'm really thinking about for this framework, because I believe is way more doable than it sounds.