Scaling without the rewrite

What is a scalable architecture?

A scalable architecture is a system design that absorbs growth in users, data or transaction volume by adding resources, without a fundamental redesign. That is the entire test. If the answer to more load is a larger bill, the design scales. If the answer is a rewrite, it does not, whatever the diagram claims.

Two levers sit under every scaling decision. Horizontal scaling, or scale-out, adds more machines or instances and distributes load across them. Vertical scaling, or scale-up, raises the capacity of a single machine: more CPU, more memory, faster storage. Most architecture debates are arguments about which lever to pull, in which order, and how much runway each one buys.

A third variable is rarely stated and usually decisive: how much engineering time an option costs before it returns anything. A team of nine does not have the same menu as a team of ninety. Sound architecture decisions are priced in engineer-weeks as well as in instance hours.

Is a scalable architecture the same as a reliable one?

No. Scalability asks whether the system can handle more load. Reliability asks whether it survives a component failing. A platform can pass a load test at several times peak traffic and still go dark because one payment provider, one region or one shared connection pool stopped answering. Treating these as a single question is how teams end up with an elegantly horizontal application tier sitting on a single point of failure.

Run them as two separate reviews. The load question is answered with numbers: requests per second, p99 latency, queue depth, connection saturation. The failure question is answered with a list: every dependency, what happens when it is slow rather than down, and who notices first. Most post-incident reviews turn up a failure of the second review, not the first.

By the hour, the figures track company size. More than 90 percent of mid-size and large enterprises lose over $300,000 for every hour of downtime, and 41 percent put their own exposure between $1 million and $5 million or higher.² At Fortune 500 scale the average runs from $500,000 to $1 million per hour, and finance and healthcare can exceed $5 million.³

Those are the invoices. The durable cost is the retention damage afterwards, which is why architecture options should be scored against customer-facing commitments rather than infrastructure cost per request alone. A design that trims the compute bill and adds an hour to recovery time is not a saving.

Horizontal versus vertical scaling: which lever comes first?

Vertical, in most mid-stage cases. A larger instance, a corrected index and a cache in front of the ten heaviest queries usually buy more runway per engineering hour than a horizontal redesign, and the work fits in a sprint rather than a quarter. Scale-up has a hard ceiling and a single-machine failure mode, both real, but the ceiling normally sits further away than the team assumes.

The pragmatic sequence is measure, optimise, scale up, scale out, and only then split the system. Teams that invert that order pay the complexity tax of distributed systems before proving they need the capacity.

Scale-up is not obsolete. For most platforms it is still the cheapest engineering hour available, and skipping past it is how a six-week problem becomes a two-quarter programme.

Which scalability patterns survive growth?

Six hold up across most platforms, in roughly the order they earn their keep.

1. A stateless application tier. Sessions in a shared store, never in process memory. Nothing else on this list works until any request can be served by any instance.
2. Caching at three distinct levels. Edge and CDN for static and semi-static responses, an application cache for computed results, and a query cache for expensive reads. Caching is the highest-leverage pattern available because it removes work rather than redistributing it. The discipline it demands is invalidation: every cached item needs an owner, a time to live, and a written rule for what clears it.
3. Asynchronous work behind a queue. Anything the user does not wait for (email, exports, thumbnails, webhooks, reconciliation) belongs off the request path. This is also the cheapest resilience win on offer, because a slow downstream service becomes a longer queue instead of a timeout.
4. Read replicas before partitioning. Most workloads are read-heavy. Splitting reads across replicas is a smaller change than sharding and reversible if it does not help.
5. Orchestrated, autoscaled compute. Containers plus an orchestrator turn horizontal capacity into a policy rather than a project.
6. Distributed deployment across zones or providers. Multi-cloud and distributed deployments record 17 percent fewer total outages than single-vendor deployments.⁵

The orchestration argument has settled. Kubernetes production use reached 82 percent in the 2025 CNCF annual survey, up from 80 percent in 2024 and 66 percent in 2023.⁶ A year earlier, production plus pilot use already covered 93 percent of the organisations surveyed.⁷ The open question is no longer whether to use it, but whether a given team should be operating it themselves.

Cost pressure sits behind all of it. Gartner expects worldwide public cloud end-user spending to grow 21.3 percent to $850 billion in 2026, inside total IT spending growth of 10.8 percent to $6.15 trillion, with AI workloads as the main driver.⁸ Scaling by adding resources works until the resource line becomes the problem, which is why the patterns that remove work outrank the ones that spread it.

Where do growing platforms actually hit their limits?

Almost always in state. Compute scales horizontally with a configuration change. Databases, caches with affinity, file stores and stateful services do not. Autoscaling the application tier without a plan for the data layer relocates the bottleneck into the one component with the longest lead time to fix.

The recurring offenders, in the order they tend to surface:

• A single shared primary database serving every read and write, with connection pools exhausted long before CPU is.
• N+1 query patterns that were invisible on a small table and fatal on a large one.
• Synchronous calls to third parties on the request path, with no timeout, retry budget or circuit breaker.
• Background jobs competing with live user traffic for the same database.
• Session affinity that quietly makes a stateless tier stateful again.
• Missing or wrong indexes, still the most common reason a database looks overloaded when it is not.

Sharding, which splits data horizontally across independent partitions, is the answer once a single primary can no longer hold the write volume or the working set. It is also the least reversible decision here. The shard key determines which queries stay cheap and which become cross-partition scans, and changing it later means a migration under load. Plan it before you need it, and exhaust read replicas, partitioning within one engine, and archiving cold data first.

Does a scalable architecture require microservices?

No. Microservices are one way to scale an organisation and one way to scale workloads that differ from each other. They are not a prerequisite for handling traffic. Splitting too early is a common and costly failure mode: the team inherits network latency, partial failures, distributed transactions and a far harder debugging story, without having solved the original bottleneck, which was usually sitting in the database.

A modular monolith with clean internal boundaries, deployed horizontally behind a load balancer, scales further than most teams expect. The honest trigger for decomposition is asymmetry: two parts of the system have genuinely different scaling profiles, release cadences or failure tolerances, and the coordination cost of one deployable has grown larger than the coordination cost of several. If you cannot name the service you are extracting, the reason, and where its data boundary falls, the split is premature. That call belongs in the design phase of the software development life cycle, not in the middle of an incident.

When should you re-architect instead of adding resources?

Keep adding resources while the cost curve stays flat and the fix is a purchase. Re-architect when adding resources stops working, or when the cost of the next increment exceeds the cost of the change. Five signals that the second condition has arrived:

1. Latency rises with data volume rather than with traffic. That is a data model problem, and no instance size fixes it.
2. The largest available instance is within one growth cycle of being fully consumed.
3. Every incident traces back to the same component, and the mitigation is always the same manual step.
4. Deployment risk has become the constraint, so teams batch releases because a single deployable makes small changes expensive to ship.
5. Infrastructure spend grows faster than usage, which means you are buying headroom to work around a design rather than capacity to serve customers.

The gap between those two numbers is the work. Preparedness is not a bigger cluster. It is a named owner for every dependency, a rehearsed failover, a load test shaped like real traffic, and a data layer plan written before write volume demands one. None of that requires a rebuild. It requires deciding which of the two questions, load or failure, the platform is weakest on, and putting the next quarter of engineering there.

For an outside read on that, our engineering practice runs the diagnostic across the application tier, the data layer and the deployment path, and the same architecture work sits behind our web platform builds. You can start that conversation here.

Written by the group's editorial team with the practice leads who run these builds. Reviewed before publish. Spotted an error? Tell us and we will fix it.