This post is more about changing perspective than going deep into these two concepts. Firstly, we should have some context to understand how these strategies appear and why we need to acknowledge them.
Context
At Mestreclique, we have some legacy pieces of our solution that still live far away from recent technologies. Even though we have great backup for those specific legacy pieces, they tend to have much worse performance than the more recent ones, as expected for this kind of service.
These legacy pieces have a direct influence on our front office by being the backbone for specific, isolated user actions. What we didn't know was that the channel established to a specific legacy service was shared. Among legacy services, there was a centralized communication object that tried to manage consecutive, rather than concurrent, requests.
Obviously, this was a recipe for disaster. But what impressed us the most was what happened when a timeout was obtained from a specific service. A single lock was acquired for the connection and, despite the operation being interrupted by the timeout, the lock was never properly released.
This meant that a single failed operation could prevent other operations from progressing, effectively turning a local problem into a much bigger one.
All this story only serves to justify what we did next, mainly supported by the pressure of being unable to refactor or rewrite an entire service in a reasonable time to prevent a generalized partial outage.
As we had to write something at a "lower" level than what typical recent frameworks provide, our solution was not particularly fantastic. Despite being way more reliable, it still had performance issues, and sometimes we had a lot of request losses due to the suffocation of services that simply could not be replicated at that moment.
Response Mechanisms
This led us to explore further in other components. One thing we had for sure: there was no way we would write something much better in a reasonable amount of time, so we tried to look at other components in our architecture.
The most obvious one was the front office being served.
Now comes the technical part.
We could not improve serving capacity at that moment, and creating a safeguard by permanently overprovisioning servers would waste money without guaranteeing a safe response. Instead, servers should adjust how they respond to load based upon current conditions.
You can use two specific strategies here:
- Load shedding is done by returning errors rather than serving requests.
- Throttling of clients is done by delaying requests or responses to control the rate at which work reaches the system.
So the idea here is to play with three variables: which requests I have to serve first, which ones I can delay, and which ones I can simply ignore in an overload situation.
This is an important shift in perspective. Instead of asking "How can we process everything?", we start asking "What is the most valuable work we can still process?"
The figure 1 illustrates a traffic spike that exceeds the capacity.
Figure 2 illustrates the effects of load shedding and throttling to manage the load spike. Note the following:
- The curve represents requests per second, and the area under it represents total requests.
- Whitespace represents traffic processed without failure.
- The backward-slashed area represents degraded traffic (some requests failed).
- The crosshatched areas represent rejected traffic (all requests failed).
- The forward-slashed area represents traffic subject to prioritization (important requests succeeded).
Figure 1 shows how the system might actually crash, leading to a greater impact in terms of both volume (number of requests lost) and time (duration of the outage extends past the traffic spike). Figure 1 also distinguishes the uncontrolled nature of degraded traffic (the backward-slashed area) prior to system crash.
Figure 2 shows that the system with load shedding rejects significantly less traffic than in Figure 1 (the crosshatched area), with the rest of the traffic either processed without failure (whitespace area) or rejected if lower priority (forward-slashed area).

Figure 1: Complete outage and a possible cascading failure from a load spike

Figure 2: Using load shedding and throttling to manage a load spike
Load shedding
Load shedding essentially helps prevent an overloaded system from failing completely. Instead of processing every request and risking a crash, the system rejects less important requests while preserving resources for critical tasks.
The important point is that rejecting a request can actually be the safer option. If a service is already operating at its limit, accepting more work does not necessarily increase throughput. It can instead increase queueing, memory consumption, contention and latency until the entire component becomes unstable.
A simple implementation could look like this:
csharp
public async Task<IResult> GetCustomerData(
string customerId,
CancellationToken cancellationToken)
{
if (_loadMonitor.IsOverloaded)
{
return Results.StatusCode(
StatusCodes.Status503ServiceUnavailable);
}
var customer = await _customerService.GetAsync(
customerId,
cancellationToken);
return Results.Ok(customer);
}
This may look counterintuitive. Returning an error seems worse than trying to serve the request.
But there is a fundamental difference between one request failing and the entire service failing.
We can also make this decision based on request priority:
csharp
if (_loadMonitor.IsOverloaded &&
request.Priority < RequestPriority.Critical)
{
return Results.StatusCode(
StatusCodes.Status503ServiceUnavailable);
}
Now the system is no longer treating every request equally.
A critical operation can continue while a low-priority operation is rejected. This becomes particularly useful when different operations have very different business values.
The goal is not to avoid errors entirely.
The goal is to control where those errors happen.
Throttling
On the other hand, throttling reduces the rate of incoming work by intentionally delaying requests.
Instead of immediately rejecting a request, the system can wait until resources become available:
csharp
await _throttler.WaitAsync(cancellationToken);
return await _legacyService.ExecuteAsync(
request,
cancellationToken);
This can be particularly useful when the underlying service is slow but still capable of processing requests if they arrive at a controlled rate.
A simple concurrency limiter can achieve something similar:
csharp
private readonly SemaphoreSlim _concurrencyLimiter = new(10);
public async Task<Result> ExecuteAsync(
Request request,
CancellationToken cancellationToken)
{
await _concurrencyLimiter.WaitAsync(cancellationToken);
try
{
return await _legacyService.ExecuteAsync(
request,
cancellationToken);
}
finally
{
_concurrencyLimiter.Release();
}
}
The finally is particularly important here.
Given our original problem with locks that were not released after timeouts, resource ownership must have a clearly defined lifecycle. If something acquires a resource, there must be a reliable path for releasing it regardless of whether the operation succeeds, fails or times out.
Throttling can also be applied selectively to clients that are generating excessive traffic:
csharp
if (_rateLimiter.IsExceeded(clientId))
{
return Results.StatusCode(
StatusCodes.Status429TooManyRequests);
}
This allows the system to protect itself without unnecessarily penalizing clients that are behaving normally.
Load shedding vs. throttling
Although both strategies are related, they solve slightly different problems.
Load shedding asks:
"What work should we stop doing?"
Throttling asks:
"How quickly should we allow work to happen?"
In reality, they can work together.
Imagine a service capable of handling 100 requests per second. A sudden traffic spike pushes the incoming rate to 300 requests per second.
Throttling can slow down the traffic reaching the service, while load shedding can reject requests that cannot reasonably be processed within the available capacity.
The combination gives the system more control over its failure mode.
And controlling the failure mode is arguably one of the most important parts of resilience.
Experience impact
Believe it or not, the impact of these strategies is notorious from a user experience perspective.
The important part is that the main areas of the page can remain fully functional. Instead of having the entire experience fail because one isolated operation depends on a slow or unavailable service, we can allow the rest of the page to continue working.
Imagine a page containing:
- Customer information
- Recent orders
- Recommendations
- Statistics provided by a legacy service
If the statistics service becomes overloaded, there is no reason for the entire page to fail.
The application can simply degrade that specific section:
csharp
var statistics = await TryGetStatisticsAsync(
customerId,
cancellationToken);
return new CustomerPageModel
{
Customer = customer,
Orders = orders,
Statistics = statistics
};
The UI can then treat missing statistics as a degraded state rather than as a catastrophic failure.
From the user's perspective, this is a huge difference.
They can still navigate, inspect their data and perform other operations while the problematic part is unavailable or slower than usual.
This is what makes these mechanisms particularly interesting from a UX perspective: resilience becomes visible as a better user experience, not simply as better infrastructure metrics.
The user might see a section taking longer to load, or a non-critical feature temporarily unavailable, but the application itself remains usable.
And that is often much better than showing a completely broken page part.
Conclusion
We were initially giving absolute attention to channel communication and trying to make the problematic legacy services more reliable.
That was necessary, but it was not enough.
The bigger lesson was that resilience cannot always come from making the problematic component better. Sometimes the component cannot be changed quickly enough, cannot scale horizontally, or simply has limitations that we have to live with.
In those situations, the surrounding architecture has to adapt.
Load shedding and throttling gave us a different way of thinking about the problem. Instead of assuming that every request must succeed immediately, we started considering the value of each request, the cost of processing it and the amount of capacity currently available.
A system operating under normal conditions can afford to treat requests almost equally. A system under pressure cannot.
When capacity is exceeded, doing less can actually allow the system to do more of what matters.
That was probably the biggest perspective change for us.
We were initially looking at resilience as a problem of communication reliability and service performance. We ended up looking at it as a problem of controlled degradation.
And this is probably the most important takeaway: a resilient system is not necessarily a system that never fails.
It is a system that, when something inevitably goes wrong, fails deliberately, selectively and in a way that protects the most important parts of the experience.
Sometimes the best way to keep a system alive is not to make it handle everything.
It is to teach it what it can afford to let go.
References
The main source of this information is Chapter 8 - Building Secure and Reliable Systems from Google, which describes these strategies in a much more formal way. I also took both images presented here from that source.