Caching Articles

Retries improve resilience when failures are temporary.

They can also increase load exactly when a dependency is already struggling.

Imagine a service receiving:

1,000 requests/sec

The dependency starts failing.

Your policy retries every request three times.

Now the dependency might see something closer to:

Original traffic:  1,000
Retry #1:          1,000
Retry #2:          1,000
Retry #3:          1,000
                    -----
Potential traffic: 4,000 requests/sec

And that's before considering retries at multiple layers.

Service A retries Service B.

Service B retries Service C.

The client retries Service A.

A reliability mechanism can become a traffic multiplier.

Good retry policies usually consider:

• Whether the failure is transient
• Exponential backoff
• Jitter
• Maximum attempts
• Retry budgets
• Request deadlines
• Retry-After guidance
• Whether another layer is already retrying

Retries don't create capacity.

If a dependency is overloaded, sending it more requests immediately might not be the recovery strategy you want.

Retry when another attempt has a reasonable chance of succeeding.

Then give the dependency some time before asking again.

Caching isn't only about deciding what data to store.

You also need to decide what makes that data unique.

Suppose you cache product information:

product:42

Looks reasonable.

Then you discover that the product 42 has different prices by country:

product:42:PL
product:42:DE

Then prices differ by currency:

product:42:PL:PLN
product:42:DE:EUR

Then the application becomes multi-tenant:

tenant:17:product:42:PL:PLN

Your cache key quietly contains assumptions about your data model.

It determines:

• What data is considered equivalent
• Which requests can share cached data
• Tenant and user isolation
• How invalidation works
• Cache cardinality
• Whether callers receive the correct result

A missing dimension can cause incorrect cache hits.

Too many dimensions can destroy your hit rate and create excessive cardinality.

Before writing:

cache.Set("products", value)

Ask what uniquely identifies the value you're caching.

Your cache doesn't understand your domain.

Your cache key has to explain it.