Scaling Up the Prime Video Audio/Video Monitoring
Marcin Kolny (via Hacker News):
We designed our initial solution as a distributed system using serverless components (for example, AWS Step Functions or AWS Lambda), which was a good choice for building the service quickly. In theory, this would allow us to scale each service component independently. However, the way we used some components caused us to hit a hard scaling limit at around 5% of the expected load. Also, the overall cost of all the building blocks was too high to accept the solution at a large scale.
[…]
To address the bottlenecks, we initially considered fixing problems separately to reduce cost and increase scaling capabilities. We experimented and took a bold decision: we decided to rearchitect our infrastructure.
We realized that distributed approach wasn’t bringing a lot of benefits in our specific use case, so we packed all of the components into a single process. This eliminated the need for the S3 bucket as the intermediate storage for video frames because our data transfer now happened in the memory.
[…]
Moving our service to a monolith reduced our infrastructure cost by over 90%. It also increased our scaling capabilities.
That really sums up so much of the microservices craze that was tearing through the tech industry for a while: IN THEORY. Now the real-world results of all this theory are finally in, and it’s clear that in practice, microservices pose perhaps the biggest siren song for needlessly complicating your system. And serverless only makes it worse.
I won’t deny there may well be cases where a microservices-first architecture makes sense, but I think they’re few and far in between. The vast majority of systems are much better served by starting and staying with a majestic monolith.
Update (2023-05-08): Adrian Cockcroft (via Hacker News):
They state in the blog that this was quick to build, which is the point. When you are exploring how to construct something, building a prototype in a few days or weeks is a good approach. Then they tried to scale it to cope with high traffic and discovered that some of the state transitions in their step functions were too frequent, and they had some overly chatty calls between AWS lambda functions and S3. They were able to re-use most of their working code by combining it into a single long running microservice that is horizontally scaled using ECS, and which is invoked via a lambda function. This is only one of many microservices that make up the Prime Video application. The problem is that they called this refactoring a microservice to monolith transition, when it’s clearly a microservice refactoring step, and is exactly what I recommend people do in my talks about Serverless First.
[…]
In contrast to commentary along the lines that Amazon got it wrong, the team followed what I consider to be the best practice. The result isn’t a monolith, but there seems to be a popular trigger meme nowadays about microservices being over-sold, and a return to monoliths. There is some truth to that, as I do think microservices were over sold as the answer to everything, and I think this may have arisen from vendors who wanted to sell Kubernetes with a simple marketing message that enterprises needed to modernize by using Kubernetes to do cloud native microservices for everything. What we are seeing is a backlash to that messaging, and a realization that the complexity of Kubernetes has a cost, which you don’t need unless you are running at scale with a large team.
Less pointless abstraction makes a pipeline more efficient? Shocker!
I suppose we should be grateful that this is essentially an Amazon introspective. That in itself is news IMO.