Monolith vs Microservices

A decade ago, I was responsible for a monolith SaaS application, and microservices were the new hotness. Every little thing gets its own service. I was skeptical, but I did start deploying a few services outside of the monolith. I found that to be a helpful architecture that allowed most of the primary application code to live in one place while experiments, bespoke processes, and other tangentially-related things could live in their own repositories, their own hosting.

Up until about a year ago, this is the way that I continued to operate with the various products that I was responsible for. It worked well enough.

The worst of both worlds

Then I joined a company with 37 microservices running in a Kubernetes cluster. I thought that this would be my chance to find out what microservices are all about, finally. What I saw was not exactly microservices.

No service could be deployed without all of the other services also being deployed. They were all so interdependent that they were effectively one application split across multiple builds. A distributed monolith. Worse yet, each one of these had its own repository in a git submodule of the main repo, and they shared a database and a data model. Nearly any change to any single service meant a need to change to all of the other services.

I was brought in to get control of the architecture. The first thing I did was combine all of those repositories into a single monorepo. That quick change meant we could at least review updates across services in a single pull request rather than having to open 37 separate pull requests that needed to be merged at the same time. 😳

Since then, on top of regular application development responsibilities, I have been working toward combining the set of services that are actually a monolith in disguise. I settled on combining those that share a database and a data model while allowing those that are truly separate to remain separate.

But why?

During this process, I discovered the original intent for the “microservices”: to mitigate traffic spikes in any certain area of the application from affecting the other areas. There are certain critical API endpoints that must respond in double-digit milliseconds or less, while others have a lot more tolerance. The old guard had scaled up the number of replicas on the services with critical endpoints and kept them separate from services with endpoints that demanded a lot more processing time or memory.

They had the right idea: separate parts of your application from one another so that they don’t adversely affect each other unnecessarily. The thing is you don’t need to build them as entirely separate services to achieve this. You don’t need to trade simplicity for reliability.

The middle ground

Of the original 37 microservices, I consolidated 28 of them into a single monolith build. I achieved the reliability goal with the monolith architecture by hosting the same monolith build in three different API service tiers. Critical API endpoints have their traffic routed to a critical tier whose SLA dictates that responses must be very fast and the memory footprint must be relatively low. We can scale up this tier horizontally as needed and we can keep the reporting traffic out of it.

Much heavier endpoints like reporting are served by a heavy tier. This tier has an SLA that allows responses to be several seconds long and has much more memory allocated to it.

Everything else in the monolith goes to a middle “interactive” tier, and the few remaining actually-separate services still get their own builds and deployments.

Each of these tiers is the same application build - the monolith. The only difference is in the number of replicas for each tier, the amount of CPU and memory resources given to each replica, and how the ingress rules route traffic among them.

And as an added bonus, the codebase and the architecture are much easier to understand as a whole while still retaining the reliability promised by the old microservices.