Chapter 20: Performance Optimization

20.1 Before Any Number Means Anything

20.1.1 The Three Shapes of a Deleted Benchmark

The cold open showed all three. They differ in how obvious they are.

The result is discarded. mix(i, 3) with nothing on the left. The compiler proves the call has no effect and removes it. This is the loudest of the three, because the number that comes back is absurd — sub-nanosecond for real work — but only if you know what the real work should cost.

The result is stored somewhere the compiler can see is dead. A local variable that is never read afterwards is the same case with a longer proof, and the number is identical.

The input is constant. mix(1, 3) can be computed once and hoisted out of the loop, leaving a loop that assigns the same constant N times. Measured, that form reports 0.4282 ns against the honest 1.0710 — a plausible number, measuring the wrong thing, and no control catches it unless you vary the input.

you wrote      for i := range b.N { mix(i, 3) }
                 │
it proved      the result is never read
                 │
it emitted     for i := range b.N { }
                 │
you measured   0.2141 ns  ── the loop, not the call ──►

the control    for i := range b.N { }  = 0.2146 ns
               the two agree to four figures

This chapter hit that trap three separate times while being written, on three different techniques, and each time the first number was better than the honest one. That is not a coincidence. The deleted version is always faster, so dead-code elimination systematically produces results that look like successful optimisations. It is the one measurement error that never disappoints you, which is why it survives review.

20.1.2 The Elimination Ladder

The obvious fix is to use the result. It is worth watching how many times that fails before it works.

Five benchmarks, all allocating the same four kilobytes, differing only in what they do with it:

// Illustrative snippet — not a complete program
var byteSink byte
var sliceSink []byte

func BenchmarkR1Discard(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = make([]byte, 4096)
    }
}

func BenchmarkR2WriteRead(b *testing.B) {
    for i := 0; i < b.N; i++ {
        buf := make([]byte, 4096)
        buf[0] = 1
        byteSink = buf[0]
    }
}

func BenchmarkR3LoopVar(b *testing.B) {
    for i := 0; i < b.N; i++ {
        buf := make([]byte, 4096)
        buf[i%len(buf)] = byte(i)
        byteSink = buf[i%len(buf)]
    }
}

func BenchmarkR4KeepAlive(b *testing.B) {
    for i := 0; i < b.N; i++ {
        buf := make([]byte, 4096)
        runtime.KeepAlive(buf)
    }
}

func BenchmarkR5Escape(b *testing.B) {
    for i := 0; i < b.N; i++ {
        sliceSink = make([]byte, 4096)
    }
}

Rung 2 is the fix most people write: touch the buffer, read the value back, store it somewhere package-level. Rung 3 is the fix people write when rung 2 does not work: make the index depend on the loop counter so nothing can be precomputed.

Rungs 1 through 3 are all measuring an empty loop, and they agree with the control to three decimal places. Rung 2's write and read-back did not help: the compiler folds buf[0] = 1 followed by reading buf[0] into the constant 1, and a constant needs no buffer. Rung 3's dependence on i did not help either, because the buffer still dies inside the iteration that made it, and a buffer nobody outside the loop can observe does not need to exist.

Rung 4 is the reason this is a ladder rather than a pair. runtime.KeepAlive did what it promises — the allocation is no longer deleted, and 31.56 ns is real work. But B/op is still zero. The buffer exists, on the stack. It never escapes the function, so the compiler put it in the stack frame, where creating it costs a pointer bump and zeroing four kilobytes, and costs the collector nothing at all.

Only rung 5 measures a heap allocation, and it is fifteen times rung 4 and two thousand times rung 1.

Three of the five rungs report a number a reader would happily paste into a commit message. Only one of them measures what the code does in production, and nothing in the output distinguishes them.

20.1.3 The One Line the Benchmark Cannot Tell You

Nothing in the benchmark output distinguishes rung 1 from rung 3, and 0 allocs/op appears on four of the five. The compiler will tell you directly, though, if you ask:

Only the last says escapes to heap. That output answers “is this benchmark measuring what I think”, and it is available before the benchmark runs.

This is worth building a habit around, because it generalises past benchmarks. §20.6 returns to it as an optimisation technique in its own right: the allocation that does not escape needs no pool, no arena and no tuning, because it is not an allocation.

20.1.4 What b.Loop Protects, and What Inlining Takes Back

Go 1.24 added b.Loop, and it is widely described as the fix for this whole problem. It is a real improvement and it is not that.

Read what it promises:

Every word is load-bearing, and most readers stop at the end of the first sentence. Function calls. Assigned variables. A discarded make is neither: make is a builtin rather than a call, and _ = is a blank assignment rather than an assigned variable. A discarded alloc4k() is one of them — it is a function call, and the promise covers the result of a function call whether or not anybody keeps it.

The second row is the documented boundary. Under b.Loop, the discarded make costs 1.129 ns against an empty b.Loop's 1.134 — which is to say it is still gone. b.Loop did not save it, because there was no call and no assignment to keep alive.

The third row is the one the documentation is technically right about and nobody reads that way. Wrapping the allocation in a function should make b.Loop's promise apply, because the result of a function call is exactly what it covers. It reports 1.134 ns: the floor. The call was inlined — which b.Loop has allowed inside its body since Go 1.26; before that it suppressed inlining there, and the third row would have read like the fourth — and after inlining there is no function call, so there is no function-call result to keep alive. The promise held; the thing it was promised about stopped existing.

Line 81 is the function, where the make escapes to heap. Line 90 is the call site inside b.Loop, where after inlining the same make does not escape. The fourth row adds //go:noinline and the protection arrives: 510.7 ns and a real 4096 B/op.

Two more clauses of that documentation are worth reading before you trust a b.Loop benchmark, and both are one line each. The protection applies only between the braces, so a helper called from the loop protects nothing inside itself. And the condition must be written exactly b.Loop() — for b.Loop() && !done compiles, runs, reports a number, and silently has no protection at all, which is a quieter failure than the inlining one because there is no -gcflags=-m line to find.

One clause is useful rather than dangerous: after Loop returns false, b.N holds the total number of iterations that ran. That is the mechanism §20.1.8's custom metrics need, and it is the one legitimate reading of b.N in a b.Loop benchmark.

The two forms do differ in one measured way. An empty b.N loop costs 0.2146 ns per iteration; an empty b.Loop costs 1.134. b.Loop is a function call and a bounds check per turn rather than a compare-and-increment, so it carries a floor about 0.92 ns higher. For anything costing tens of nanoseconds that is irrelevant. For the operations this chapter spends most of its time on — measured, an uncontended atomic add at 5.50 ns, an uncontended mutex at 10.79 ns, a sync.Pool round trip at 11.32 ns — it is between eight and eighteen per cent of the measurement.

20.1.5 benchstat, and the Run That Looked Like a Win

Chapter 16 stopped at “one run is not a number” and handed the tool here. This is the tool doing the only job that matters.

Two implementations of the same sum, one with a four-way unrolled loop, ten runs each:

The first of those compares the two implementations. The second compares two samples of identical code — nothing was changed between the runs at all.

        │   old.txt   │               new.txt               │
        │   sec/op    │   sec/op     vs base                │
Sum-16    1.813µ ± 1%   1.000µ ± 1%  -44.82% (p=0.000 n=10)

        │    a1.txt    │              a2.txt                │
        │    sec/op    │   sec/op     vs base               │
Sum-16   1.859µ ± 14%   1.828µ ± 1%   -1.67% (p=0.010 n=10)

── the second pair is the same binary twice ──►

The first is a result. A 45% improvement, a 1% spread on both sides, and a p-value that leaves no room to argue.

Read the second carefully, because there are two traps in four numbers.

The first is that p=0.010 looks significant. It is, statistically — and the code is byte-for-byte the same. With ten samples and a noisy first run, benchstat will report a significant difference between a thing and itself.

The second is the ± 14%, and it is the one that saves you. A sample with a 14% spread cannot support a claim about a 1.67% difference. The ± column is the result; the percentage is a story about it.

And here is what a single run of each would have shown. Measured: the first run in each file was 2112 ns and 1829 ns — a 13% improvement that does not exist. Taking the best of the new sample against the worst of the old, which is what happens when somebody runs a benchmark until they like it, reads as 22.3% faster.

Producing those two files correctly is most of the work, and it is where the errors that survive benchstat come from. The shape that holds up:

Four details in that, each of which has produced a wrong answer for somebody.

-run '^$' stops the tests running before the benchmarks. Tests warm caches, allocate, and start goroutines that may still be running; none of that belongs in the first benchmark’s numbers, and it explains a surprising number of first-run outliers.

-count=10 in one invocation, not ten invocations appended. A single process runs all ten samples against the same code, the same heap and the same CPU state.

-benchmem. B/op and allocs/op are deterministic where ns/op is not — the same code allocates the same amount every time — so they carry a claim that timing cannot. A change that removes allocations has proved something even on a laptop running a video call.

The two runs must be close together in time. Thermal state, other processes and the page cache all drift. Building both binaries first and running them alternately is the paranoid version, and it is what you do when the difference you are claiming is under 10%.

One thing that shape deliberately does not do is compare across machines. There is no correction factor for a different CPU, and benchstat will compute a percentage between two machines as happily as between two commits.

20.1.6 Variance Is a Result, and the Machine Is Part of the Measurement

An optimisation has two outputs and most write-ups report one.

A service at p50 20 ms and p99 400 ms has a spread problem, and a change that takes the mean from 20 ms to 18 ms while leaving p99 alone has improved nothing anyone notices. benchstat's ± column is the benchmark-scale version of the same fact.

The intuition is that contended code is noisier, and it is worth checking rather than assuming. Measured: §20.4.1's own two counters — one mutex against 64 padded shards, under b.RunParallel at -cpu 16 — twelve runs each on a quiet machine, and §20.5.2's narrow false-sharing shape, ten runs each. Two separate benchstat invocations, with their result rows collected here:

The intuition does not survive, and it fails the same way twice. In both comparisons the worse version is the tight one at ± 1%, and the improvement is four or five times noisier.

Both results are clean — p=0.000, and neither ± is anywhere near the gap; the counter improved 6.4× and the padding 1.47×. But in both, the improved arm is the wider one. On a benchmark that is a win; on a service it is a question, because a p99 is built out of the spread and both changes made the distribution wider while making the middle better. §20.4.5 measures how far that can go.

That 1.47× is the same physical fix the cold open reports at 8.2×, on the same machine, in the same week — and §20.5.2 reports the same benchmark at 1.37× in a different run, which is the ± column doing its job. Neither the 1.47 nor the 8.2 is an error. §20.5.2 is the section that explains the gap between them, and the explanation is the chapter’s thesis: the ratio belongs to the benchmark’s shape, not to the optimisation.

Three things move a benchmark result without any code changing, and all three have been mistaken for optimisations.

Thermal state. A machine that has been running benchmarks for ten minutes is slower than one that has not. pmset -g therm on macOS reports CPU_Speed_Limit; while this chapter was being drafted it read 56, which is to say the processor was running at a little over half its nominal speed, and every absolute number taken in that window was wrong by an unknown factor. Interleaved -count samples, which is what benchstat compares, survive this. Ten runs of the old version followed by ten of the new do not.

Other processes. The ± 14% above was a browser. A system indexing daemon at 127% CPU produced a four-fold spread on a contention benchmark in the same session.

The garbage collector. A benchmark that allocates runs collections at unpredictable points, so the allocation and the collection land in different samples. -benchmem is how you see past it.

20.1.7 ns/op Means Something Else Under RunParallel

Everything above applies to any Go benchmark. This one is specific to concurrent ones, it is not documented anywhere you will trip over it, and it silently invalidates the comparison people most want to make.

Here is a function containing a fixed microsecond and a half of pure CPU work — no locks, no sharing, no allocation — measured twice. The body is byte-for-byte identical.

The operation did not become eleven times faster. Each call still burns its microsecond and a half; that is what the function does. What changed is the denominator.

b.RunParallel starts GOMAXPROCS goroutines and hands b.N iterations out among all of them, then divides the wall-clock time by the total. So ns/op from a parallel benchmark is inverse throughput — how often the system completes an operation — and ns/op from a sequential one is latency, how long one operation takes. Same units, same column heading, different questions.

sequential   ops ──────────────────►  1,547 ns each
             one goroutine, wall time = sum

parallel     ops ══════════════════►    134 ns each
             16 goroutines, wall time = max
             latency is still ~1,547 ns

── 134 x 16 = 2,144: the goroutine count is the ratio ──►

Multiplying back gives the check: 134 x 16 = 2,144 against a true latency of 1,547 — in the right place, and over rather than exact because sixteen GOMAXPROCS on eight physical cores does not scale linearly.

The division is worth doing, once you have said out loud what it is. Derived, 1,547 / 134 = 11.5× is a throughput ratio: the system completes 11.5 times as many operations per second. It is not a latency ratio, and no single call got faster. That is the number worth having and neither benchmark printed it.

Three practical consequences.

Never read a RunParallel figure as a latency, and never divide it into a sequential one without naming which quantity you have. The same two numbers support “11.5 times the throughput” (true) and “11.5 times faster per call” (false), and only the second one ever ends up in a commit message. The comparison people actually want is §20.3.1's crossover question, and answering that needs both benchmarks written the same way — either both sequential over the same total work, or both parallel.

A RunParallel regression can be a latency improvement, and the reverse. A change that adds queueing raises latency and can raise throughput. If the product cares about p99, §20.1.8's custom metric is the only way to see it.

b.SetParallelism(n) multiplies GOMAXPROCS, it does not set it. SetParallelism(2) on a sixteen-thread machine runs 32 goroutines, not 2. To vary the core count, use -cpu — which is §20.4's whole method.

20.1.8 Measuring Something Other Than Time

ns/op is the default and often the wrong metric. b.ReportMetric lets a benchmark report what it is actually about:

// Illustrative snippet — not a complete program
func BenchmarkPipeline(b *testing.B) {
    var handled int64
    for b.Loop() {
        handled += processBatch()
    }
    b.ReportMetric(float64(handled)/b.Elapsed().Seconds(), "items/s")
}

Three cases where this matters more than the default, and the first is not hypothetical.

Throughput under a fixed cost. A batching change makes each call slower and the system faster, because each call now carries more items. Measured: the same journal commit under a mutex, four batch sizes, five runs, medians.

ns/op gets 10.5× worse and throughput gets 48.6× better, in the same runs, from the same change. A reviewer reading the default column rejects a change that made the system five times faster in the metric the product is measured on. §20.6.1 is where that change is made on purpose.

Tail latency. A change that lowers the mean and raises p99 is usually a regression, and ns/op cannot express it. Collecting per-iteration durations and reporting p99 as a custom metric turns §20.1.6's argument about variance into a number benchstat will compare for you.

Latency under RunParallel. §20.1.7's ambiguity has a one-line fix: multiply the elapsed nanoseconds per iteration by runtime.GOMAXPROCS(0) and report it as ns/op-latency. The throughput figure and the approximate per-operation latency then appear side by side, which is what a reviewer needs to tell an improvement from a redistribution. This is also the one place b.N is legitimate inside a b.Loop benchmark: after the loop it holds the iteration count, which is the denominator every custom metric needs.

The general rule: benchmark the thing you would put on a dashboard. If nobody would alert on ns/op for this code path, it is not the metric to optimise.

20.1.9 The Apparatus Is Also Code

One reminder that the tools are software, found while writing this chapter.

// Illustrative snippet — not a complete program
var p = sync.Pool{New: func() any {
    b := make([]byte, 1024)
    return &b
}}

func BenchmarkICE(b *testing.B) {
    for b.Loop() {
        v := p.Get().(*[]byte)
        (*v)[0] = 1
        p.Put(v)
    }
}

This one no longer does, as of Go 1.27 — the same benchmark compiles and runs there — and that does not much matter; compiler bugs get fixed. What matters is the reflex. When a benchmark will not build, or produces a number you cannot explain, the possibilities include a mistake in your code, a misunderstanding of the API, and a defect in the toolchain — and the third is not so rare that you should spend a day assuming it is impossible. A build failure containing the words “internal compiler error” is the toolchain telling you which one it is.

20.1.10 Common Mistakes

Summary: Before Any Number Means Anything

A concurrent optimisation is a claim about a difference between two programs, and the difference is usually small enough that the measuring apparatus decides the answer. The compiler deletes work nothing observes, which inside a benchmark is the work you meant to time — and it does so silently, accurately, and always in the direction that flatters you. Four of the five obvious ways to stop it do not work, b.Loop protects a narrower set of things than its reputation and loses even that to inlining, and one line of -gcflags=-m settles the question before the benchmark runs. A single run is a sample rather than a number; benchstat is what says so out loud, and its ± column is the half of the result that tells the next person whether to trust the other half.

Self-Check Questions: Before Any Number Means Anything

20.2 A Profile Is Not a Decision

Chapter 19 hands you a profile. It is accurate, it was cheap to take, and reading it correctly is a skill that chapter spent seven sections on. This section is about the step nobody teaches: a profile tells you where the time went, and that is not the same as telling you what to change.

20.2.1 The Profile That Names Nothing You Can Edit

Here is a counter behind a mutex, hammered by sixteen goroutines. It is the simplest contended thing in the book.

// Illustrative snippet — not a complete program
func (c *oneMu) inc() { c.mu.Lock(); c.n++; c.mu.Unlock() }

Duration: 3.92s, Total samples = 14.86s (379.44%)
  flat  flat%   sum%      cum   cum%
 5.10s 34.32% 34.32%    5.11s 34.39%  runtime.pthread_cond_wait
 4.69s 31.56% 65.88%    4.69s 31.56%  runtime.usleep
 2.95s 19.85% 85.73%    2.95s 19.85%  runtime.pthread_cond_signal
 0.45s  3.03% 88.76%    3.05s 20.52%  sync.(*Mutex).lockSlow
 0.34s  2.29% 91.05%    0.34s  2.29%  runtime.procyieldAsm
 0.18s  1.21% 92.26%    3.23s 21.74%  sync.(*Mutex).Lock
 0.14s  0.94% 93.20%    4.15s 27.93%  sync.(*Mutex).Unlock
 0.10s  0.67% 93.88%    0.10s  0.67%  runtime.pMask.read

── 85.73% in three frames you cannot edit ──►

The sum% column does the arithmetic for you: 85.73% of the profile is three runtime functions you cannot edit. inc does not appear at all. Neither does the increment. The profile is entirely correct: that really is where the CPU time went, and the total exceeding the duration is sixteen threads' worth of samples over one wall-clock run.

It is also useless as a decision. Nothing in those eight lines tells you to shard the counter, shorten the critical section, or stop sharing — and those are the only moves available.

Read the listing once more before moving on. It is going to turn out that everything you needed was printed there.

pthread_cond_wait   34.3%  ──┐
usleep              31.6%  ──┼─ 85.7% of the profile
pthread_cond_signal 19.9%  ──┘  "the machine is waiting"

what you can change:
  the number of locks       ──► §20.4.4
  the size of the section   ──► §20.4.6
  whether it is shared      ──► §20.4.6
  which primitive it is     ──► §20.4.3

── none of them is named in the profile ──►

20.2.2 The Three Things a Hot Frame Can Mean

Every hot frame is one of three, and they take different actions.

It is the algorithm. The frame is doing work that is genuinely necessary given the approach, and the approach is wrong. No amount of making the frame faster helps, because the fix is to stop calling it that many times. This is the most common case and the one people miss most often, because “optimise the hot function” is such a natural reading. §20.2.3 is a worked example where the hot frame is runtime.memmove and the bug is a quadratic insert.

It is contention. The frame is fast in isolation and expensive here because goroutines are queued for it. The profile names the symptom; §19.3.3's block and mutex profiles name the waiter and the holder, which is more than the CPU profile gives you and still not a decision.

It is nothing you can move. Memory bandwidth, a syscall, the collector, a library you do not own, crypto/sha256.blockAVX2. It is real, it is necessary, and it is about as fast as it is going to get. The correct action is to do less of it (§20.6) rather than to make it faster.

Only the first two are actionable, and the action for the first is never “optimise this function”.

frame at the top of the profile
           │
   ┌───────┴────────┬────────────────┐
   │                │                │
the algorithm    contention      irreducible
   │                │                │
call it fewer    remove the       call it
times, or        serialisation    fewer times
differently      (§20.4)          (§20.6)
   │                │                │
   └────────────────┴────────────────┘
                    │
       every one of these is a hypothesis
       until §20.1 has tested it

20.2.3 Flat and Cumulative Are Different Questions

pprof sorts the same profile two ways and they disagree about what matters. Flat is time spent in a function; cumulative is time in it and everything it called.

Here is an index builder. It reads twenty thousand records, hashes each one, and keeps the hashes in sorted order so lookups are binary searches.

// Illustrative snippet — not a complete program
func (x *index) add(v uint64) {
    x.mu.Lock()
    i := sort.Search(len(x.keys), func(i int) bool {
        return x.keys[i] >= v
    })
    x.keys = append(x.keys, 0)
    copy(x.keys[i+1:], x.keys[i:])
    x.keys[i] = v
    x.mu.Unlock()
}

It takes 13.2 ms for twenty thousand records, which is slower than it looks like it should be. Chapter 19's instructions produce this:

  flat  flat%   sum%      cum   cum%
1890ms 54.31% 54.31%   1890ms 54.31%  runtime.memmove
 540ms 15.52% 69.83%    540ms 15.52%  sha256.blockAVX2
 270ms  7.76% 77.59%    270ms  7.76%  runtime.pthread_cond_signal
 140ms  4.02% 81.61%    140ms  4.02%  runtime.kevent
 110ms  3.16% 84.77%    170ms  4.89%  sort.Search
  60ms  1.72% 86.49%     60ms  1.72%  m4.(*index).add.func1
  50ms  1.44% 87.93%    600ms 17.24%  sha256.(*Digest).Write
  50ms  1.44% 89.37%     50ms  1.44%  sync.(*Mutex).Unlock

── and there is no way to optimise memmove ──►

runtime.memmove at 54.31% flat. It is the top frame by a wide margin, it is in the runtime, and there is no version of “optimise runtime.memmove” available to you. By §20.2.2's taxonomy it looks like case three: irreducible.

It is case one, and the column that says so is cum.

  flat  flat%       cum   cum%
     0     0%     2.93s 84.20%  m4.RunSerial
 0.04s  1.15%     2.20s 63.22%  m4.(*index).add     ◄──
 1.89s 54.31%     1.89s 54.31%  runtime.memmove
     0     0%     0.73s 20.98%  m4.digest
 0.02s  0.57%     0.71s 20.40%  crypto/sha256.Sum256

── 1.15% flat, 63.22% cumulative, and it is yours ──►

add is 63.22% cumulative and 1.15% flat. Nearly all of the memmove is add's copy, shifting the tail of the slice one element right on every insertion. Twenty thousand insertions into a slice that grows to twenty thousand elements is quadratic, and memmove is where the quadratic lives.

The same reading applies to §20.2.1's contended counter, where it is starker still. Sorted by cumulative time, inc — the function under test — is 50.07% of the profile with 0.2% flat time. It does no work at all; it waits, and everything it waits in is somebody else’s code.

The rule that survives: the target is the highest-cumulative frame that is your code. Everything above it is scaffolding — runtime.mcall, park_m, the benchmark closure, testing's wrapper — and everything below it is what it called. It is the top of the part you can edit, and its cumulative share is the number that goes into §20.2.4's arithmetic.

Now go back to §20.2.1's listing, which was the flat sort and which you have already read once. The cum column was in it the whole time. sync.(*Mutex).Unlock sits at 0.94% flat and 27.93% cumulative, and Lock at 1.21% flat and 21.74% cumulative. On a saturated mutex, releasing costs more than acquiring — the releasing goroutine is the one that has to wake a waiter, which is where the 19.85% of pthread_cond_signal comes from. Nothing was hidden. Both numbers were printed, in the cum column, on the sixth and seventh lines of a listing that had already been dismissed as unactionable.

That asymmetry is also why §20.4.6's “shorten the critical section” advice has a floor: a quarter of the cost is not inside the section at all.

20.2.4 Amdahl as Arithmetic, and Where It Has No Term

We now have a diagnosis: add is responsible for 63.22% of the time, and it holds a mutex while it does it. Two changes suggest themselves, and both are hypotheses.

The first is the reflex this whole book has been building toward and the one this chapter exists to question: the work is under a lock, records could be hashed concurrently, so parallelise it.

Before writing it, the arithmetic. If a fraction s of the runtime is serial and the rest parallelises perfectly across N workers, the best possible speedup is:

                     1
   speedup  =  ─────────────
               s + (1 - s)/N

s = the fraction that cannot be done in parallel
N = the number of workers

The serial fraction is not a guess here. add takes the mutex for its whole body, so its cumulative share is s: 0.6322.

Now the measurement. Measured: twenty thousand records, five runs, medians.

Parallelising made it 1.8 times slower, and adding four times as many goroutines changed almost nothing — the two parallel rows agree to within half a per cent, which is what a fully serialised program looks like from the outside. The hypothesis was wrong, and Amdahl did not predict how wrong: it predicted only that the upside was capped at 58%, which was already enough to decide against it.

That gap is worth naming, because it is the formula’s most misunderstood property. Amdahl’s law gives a ceiling, not a floor. Its worst case is no improvement: the serial fraction caps you, and nothing in the formula lets a change make the program slower. Real concurrent systems do not obey that. Measured in §20.4.1, one mutex under a rising core count is 5.6× slower at sixteen cores than at one, and measured here, sixteen goroutines are 0.55× a single one. Amdahl has no term for either curve.

The missing term is coordination. Every additional worker adds not just its share of the serial section but a cost paid by every other worker — the cache line moved, the waiter woken, the queue joined. That cost grows with the number of participants rather than staying fixed, so past some point each new worker subtracts more than it adds.

ideal      ▁▂▃▄▅▆▇█   linear in workers
Amdahl     ▁▂▃▄▅▅▅▅   flattens at 1/(1-s)
measured   ▁▂▃▄▅▄▃▂   flattens, then falls

── the third curve is §20.4.1's mutex, and only
   the third one is what your code does ──►

The practical use of that is not a better formula. It is a caveat on the estimate: Amdahl gives you a ceiling, and the floor can be below where you started. So the arithmetic is worth doing to decide whether an optimisation is worth attempting, and never as a substitute for §20.4's sweep, which is the only one of the two that can report a negative number.

20.2.5 The Change the Profile Was Actually Asking For

The second hypothesis follows from reading add as case one, the algorithm. It maintains sorted order on every insert, at a cost of one memmove per record. Nothing requires that: the same output — an ordered slice — comes from appending everything and sorting once at the end.

// Illustrative snippet — not a complete program
func RunSorted(in []string) int {
    keys := make([]uint64, 0, len(in))
    for _, s := range in {
        keys = append(keys, digest(s))
    }
    slices.Sort(keys)
    return len(keys)
}

No mutex, no goroutines, no concurrency of any kind. It replaces twenty thousand O(n) insertions with one O(n log n) sort.

slices.Sort rather than sort.Slice is not incidental. sort.Slice takes a func(i, j int) bool and reaches the elements through reflection; slices.Sort is generic and compiles to a direct comparison. Measured on twenty thousand uint64 in isolation: 1.840 ms against 0.957 ms, a 1.92× difference for a change of one identifier. In this workload, where SHA-256 is most of the remaining time, it is worth 1.17× — 5.03 ms down to 4.30 ms. A performance chapter has no business shipping the reflective one.

Removing the serial work — with no concurrency at all — bought 3.06×, which is twice Amdahl’s entire ceiling for the parallel version. And it did something more useful than that: it changed the serial fraction, so parallelising the remaining work now buys another 2.76× on top, for 8.43× overall.

That is the shape of the whole chapter in one example. Concurrency did not fix this program; it made it 1.8 times worse. The algorithm fixed it, and then concurrency was worth adding, because the thing standing in its way was gone. Note the order, because it is the reusable part: the change that made the parallel version worth writing was made in the sequential one.

20.2.6 Attribution Is Not Causation

The profile says 40% of the time is in json.Marshal. Removing it saves 40%, so the service gets 1.67 times faster.

That inference is wrong twice over. It is wrong because Amdahl bounds it — 1.67× is the ceiling for making the frame infinitely fast, which nobody does, and a realistic halving of a 40% frame buys 1.25×. And it is wrong because the 40% may not be removable at all: §20.2.2's third case is a frame doing necessary work, and §20.2.1's is a frame that is hot because fifteen goroutines are queued behind it.

Two readings. Removing the top frame of a profile almost never doubles anything — you need to delete half the program for that. And a 1.25× improvement is frequently not worth a week, which is worth knowing before the week rather than after.

The deeper error is treating the profile’s attribution as a causal claim. A profile is a record of where samples landed. “Removing this makes the program faster by that much” is a hypothesis about a counterfactual program that does not exist yet, and §20.1 is how you test it. §20.2.5's parallel version is what happens when the hypothesis is not tested: every step of the reasoning was sound and the result was a 1.8× regression.

20.2.7 What the Other Profiles License

§19.3.3 drew the distinction this section depends on: the block profile blames the waiter and the mutex profile blames the holder. Here is what each authorises.

A block profile with one dominant stack tells you where goroutines are parked. It does not tell you whether that is a problem — a worker parked on an empty channel is healthy and looks identical to one starved by a slow producer. The decision it licenses is narrow: find out what the other side of that channel is doing. It is a pointer, not a verdict.

A mutex profile with one dominant stack is the strongest signal in Chapter 19's set, because it names the code that held the lock while others waited. That maps directly onto §20.4.6's ladder: stop sharing, change the primitive, move work out, shrink the section, shard. It is also the instrument that would have found §20.2.1's counter, where the CPU profile named nothing.

A heap profile has two modes and confusing them wastes days. -inuse_space answers “what is resident right now”, which is the memory-leak question. -alloc_space answers “what has been allocated since the process started”, which is the garbage question and the one that matters for throughput. A service with a flat inuse_space and an enormous alloc_space has no leak and a great deal of collector work, and §20.6 is the section for it.

Measured in §20.6.2, the relevance: a four-kilobyte buffer that escapes costs 469.7 ns and 4096 B/op; the same buffer on the stack costs 56.99 ns and 0 B/op, and appears in no heap profile at all. alloc_space is where that difference shows up, and inuse_space is where it does not.

A flat CPU profile is itself a result. A service with a p99 of 800 ms whose CPU profile has no peak and sums to a small fraction of wall-clock time has not told you “there is no bottleneck”. A CPU profile samples goroutines that are running; time spent blocked on a channel, a mutex or a network read produces no samples, because the goroutine is not on a processor to be sampled. Flat CPU plus high latency means the time is spent waiting, which narrows the problem to exactly the class §20.4 is about — and points at the block and mutex profiles, both off by default, which §19.3.1 measured as reporting an empty profile rather than an error.

20.2.8 The Decision Procedure

Putting §20.2 together, in the order that costs least:

1  what fraction is it?     ──► Amdahl bounds the payoff
     │ worth it                  and s is the cum column
2  is it your code?         ──► no: read -cum and find
     │ yes                       the frame that is yours
3  which of the three?      ──► algorithm / contention /
     │                          irreducible
4  sweep -cpu               ──► worse with cores: §20.4
     │ scales
5  can you do less of it?   ──► §20.6, and it beats
     │ no                       making it faster
6  make it faster           ──► and prove it with §20.1

── six questions, and only the last is a code change ──►

Questions 1 through 5 cost minutes and no code. Question 6 is where everyone starts.

The worked handover, using this chapter’s own numbers. Chapter 19 gives you a CPU profile whose top three frames are 85.73% runtime waiting functions, and a mutex profile whose top stack is inc. §20.2.4 bounds it: inc is 50.07% cumulative, so eliminating the waiting entirely caps at 2.00×. Worth a day, so continue. §20.4's sweep answers the shape: the number rises with core count, so contention is confirmed and a bigger machine will make it worse. §20.4.6 supplies the moves in order: does this need to be shared at all? Is an atomic correct here? Can the work move out? Can the section shrink? Only then, shard. §20.1 checks the result: -count=10 on both, benchstat, and the improvement has to clear the spread.

Six questions, one code change, and the code change is last.

20.2.9 Common Mistakes

Summary: A Profile Is Not a Decision

A profile is an accurate account of where time went and a poor account of what to do. Measured on a contended counter, 85.73% of the CPU profile is three runtime functions nobody can edit and the function under test has 0.2% flat time — while carrying 50.07% of the cumulative. Every hot frame is the algorithm, contention, or something unmovable, and the three take different actions that the CPU profile cannot distinguish. The cumulative column is what names the frame you can change, and its share is the s that turns Amdahl’s law from a slogan into a bound — 1.53×, computed before any code was written, on a change that then measured 0.55×. Amdahl gives a ceiling and has no term for the floor, which is why the sweep in §20.4 is not optional. And the change that worked on that program was algorithmic, used no concurrency at all, bought 3.06×, and then made concurrency worth another 2.76×.

Self-Check Questions: A Profile Is Not a Decision

20.3 Is Concurrency the Answer?

Nineteen chapters have taught you to make concurrent code correct. This is the first one that asks whether the code should be concurrent at all.

One chapter has already asked it once. §2.5 measured a goroutine per item against a plain loop, on this machine and this toolchain, at a thousand items of trivial work: 269,521 ns against 1,868, which that section reports as 144× slower and correctly attributes to goroutine creation. It then stops, because Chapter 2 is about what a goroutine costs rather than about when to use one.

What no chapter has done is sweep the size until the answer changes sign. That is this section, and it turns §2.5's single point into a curve — one that crosses zero, keeps going, and then turns out to depend on something other than the number of items.

The omission produces a specific bug, and it is one of the most common shapes in Go code: a service where every operation is a goroutine, none of the goroutines has enough work to justify itself, and the profile shows time in the scheduler.

20.3.1 The Baseline Nobody Takes

Every optimisation claim in this chapter is a ratio, and a ratio needs a denominator. For concurrency the denominator is the sequential version — and almost nobody writes it, because the code is already concurrent by the time anyone benchmarks it. Almost every published concurrency benchmark compares one concurrent design against another concurrent design.

Writing the sequential version is usually twenty minutes and it settles three questions at once: whether concurrency helps at all, what the serial fraction actually is, and what the parallel overhead costs. §20.7's worst cases are all changes that were never compared against doing nothing.

The procedure:

1. Write the sequential version, however ugly. It does not ship.
2. Benchmark both at the sizes you actually see, not the size that makes the point.
3. If the parallel version does not win by a clear margin under §20.1's rules, keep the sequential one — it is simpler, it has no §18.4 protocol or lifetime bugs, and it cannot deadlock.

That last point is the one that gets forgotten. The sequential version’s advantage is not only speed at small n; it is that Chapters 2 through 18 do not apply to it.

20.3.2 The Crossover, Measured

The same arithmetic over n items, done sequentially and split across sixteen goroutines with a WaitGroup and a per-goroutine accumulator. No shared state, no locks, no channels — the friendliest possible parallel decomposition.

The concurrency did not become correct at a million items. It was always correct. It became worth it.

Three readings, in increasing order of usefulness.

The 1,000,000 row is the headline because it is the reproducible one, and 4.2× on a machine with eight physical cores is a good result. §20.3.5 is about why it is not sixteen.

It also lines up with §2.5 from the other end. That section’s 144× at a thousand items and this one’s 134× at a hundred are the same phenomenon at two sizes, measured a year and eighteen chapters apart, and neither of them is a statement about goroutines being slow. Both are statements about a thousand goroutines each doing a nanosecond of work.

The 100 row should be read as “two orders of magnitude”, not as 134. It is the least stable figure in this chapter: across runs the sequential baseline ranged from 38.77 to 144.1 ns, because 39 ns is close enough to the timer’s floor that the ratio is mostly measuring goroutine creation. The instability is the point rather than a flaw — at this size the concurrency costs so much more than the work that the exact multiple is uninteresting.

The 10,000 row is where people actually live, and it is still a loss. Ten thousand items feels like a lot. It is not enough to pay for sixteen goroutines, a WaitGroup, and the cache traffic of writing into a shared output slice.

20.3.3 It Is Work Per Goroutine, Not Item Count

The crossover above is not at “a million items”. It is at a total amount of work, and the same experiment with heavier items moves it by three orders of magnitude.

The summation costs well under a nanosecond per item, which is why it takes hundreds of thousands of them to cover the coordination. Replace the addition with a SHA-256 and everything else stays the same — same decomposition, same WaitGroup, same sixteen goroutines, same machine.

The crossover moved from somewhere past ten thousand items to somewhere between one hundred and four hundred — a factor of a thousand or more, for a change that touched neither the concurrency nor the machine. Derived, each hash costs about 183 ns, so at four hundred items each goroutine carries twenty-five of them: about 4.6 µs of work, which is where §2.5's band says to expect concurrency to start paying and where it does.

So the question to ask before parallelising is not “how many items?” but “will each goroutine do at least a few microseconds of work before it needs to coordinate again?” If the answer is no, the answer to the whole question is no — and the two tables put the same workload on opposite sides of §2.5's band: 4.6 µs of hashing per goroutine at the SHA crossover, against a few hundred nanoseconds of addition at a size where the sum still loses.

There is a number available for that, and it is §2.5's rather than this chapter’s. That section measured the goroutine round trip — creation through to completion — at 469 ns, and derived a band from it: under about a microsecond of work per goroutine, batch; past about fifty microseconds, spawn freely; in between, measure. This chapter’s own data sit inside that band exactly where they should. §20.3.2's 100-item row spends 5,244 ns on sixteen goroutines, which is 328 ns each — inside §2.5's measured 150–470 ns creation cost, and doing a hundred sub-nanosecond additions to earn it.

20.3.4 The Granularity Is the Knob

The previous three subsections ask whether to parallelise. This one is about the mistake people make once they have decided to, and it is the largest single factor in this chapter.

Here is one fixed job — apply a four-instruction hash to 1,048,576 values — split across the same sixteen threads six different ways. The only variable is how many items each goroutine handles.

The best and worst rows are both correct, both parallel, and both use all sixteen threads. They differ by 1,531×.

Divide the worst row out: 274 ms across 1,048,576 goroutines is roughly 261 nanoseconds of scheduling per item, against a hash that the bottom row prices at about 1.16 ns. The program spent two hundred and twenty-six times as long deciding who would do the work as doing it. That is not a scheduler defect — §2.5 measured a goroutine at 156 ns to issue and 469 ns to create and finish, and 261 ns is squarely inside that, multiplied by a million.

The curve is also the answer to “how many goroutines?”, which is the wrong question. The two parallel rows at 4,096 and 65,536 items per goroutine are within 20% of each other — 0.179 ms and 0.214 — so anywhere in that range is right, and the useful reading is that there is a floor, not an optimum. Give each goroutine enough work to clear the overhead — the ~1 µs of §20.3.3 — and the exact figure above that stops mattering. The last row is not part of that plateau: it is the one-goroutine control, and it is 6.8× the plateau because it uses one thread out of sixteen.

1          ████████████████████████████  274.00 ms
16         ██                             17.30 ms
256        ▏                               1.08 ms
4,096      ▏                               0.18 ms  ◄┐ within
65,536     ▏                               0.21 ms  ◄┘ 20%
1,048,576  ▏                               1.21 ms  1 gor.

── the knee is granularity, not core count ──►

20.3.5 Why Sixteen Threads Did Not Give Sixteen Times

The million-item row is 4.2× on sixteen hardware threads. The missing 11.8× has three causes and they are worth separating, because only one of them is yours.

The serial fraction. Setting up, splitting, and summing the sixteen partial results happens once, on one goroutine. §20.2.4's arithmetic bounds the rest.

Memory bandwidth. Sixteen threads reading one array saturate a shared resource that does not scale with core count. §20.5.4 measures it directly — the same loop scales 8.1× when each element is given arithmetic to do and 4.4× when it is not — and it is the ceiling most lock-free parallel code actually meets.

The machine. Sixteen GOMAXPROCS on an eight-core, sixteen-thread part is eight physical cores with hyper-threading — throughput scaling past eight is real but sublinear by construction.

None of the three is a bug, and all three are why the honest expectation for a parallel speedup is well under the thread count, and why claiming otherwise in a design document is a sign nobody measured.

20.3.6 Measuring Your Own Serial Fraction

§20.2.4's arithmetic needs s, and §20.2.3 read it off a profile because there was one function holding a lock for its whole body. Often there is not: the serial part is spread across a setup phase, a merge phase, and the coordination itself.

There is a direct way that needs no profile. Run the same workload at one thread and at N, and solve Amdahl backwards:

          T(N)/T(1) - 1/N
     s =  ───────────────
             1 - 1/N

two -cpu runs, one division, no profile required

Derived, from §20.3.2's numbers: sixteen goroutines produced 4.2× on the million-item case, so T(N)/T(1) is 0.238 and s = (0.238 - 0.0625) / 0.9375 = 0.19. Nineteen per cent of that workload is not parallelisable, which for a split-and-sum is the split, the sum, and the memory bandwidth together.

That number is worth having before the next optimisation, because it is the ceiling for every future parallel change to the same code. With s = 0.19, no amount of hardware takes it past 5.3×, and a proposal to move from sixteen threads to sixty-four is asking for a 1.2× improvement in exchange for four times the machine.

The estimate is rougher than the profile’s: it attributes all non-scaling to serial work, including contention and cache effects that are not strictly serial. That is arguably the more useful number, because those costs are real and they do not go away when you add cores either. And it converts an argument into a measurement — “this will scale if we give it more cores” is a claim about s, and s costs two benchmark runs.

20.3.7 Two Different Things Are Called Concurrency

§1.1 drew this distinction and the performance consequence is worth stating plainly, because the two have different crossovers — and one of them has none.

Parallelism for throughput. Split one job across cores to finish it sooner. This is §20.3.2's table. It needs the work to be divisible and large, the crossover sat near a million items there, and it is bounded by Amdahl, by memory bandwidth and by the split.

Concurrency for latency. Start a slow thing now so you can do something else while it runs. Two backend calls concurrently finish in max(a, b) instead of a + b, and this has no crossover at all — it wins whenever the operations are slow relative to a goroutine, which for anything involving a network is always.

throughput   split 1,000,000 items across 16 threads
             bounded by Amdahl, bandwidth, and the split
             payoff: 4.2x, and it needed a million items

latency      two 40 ms calls, concurrently
             bounded by the slower one
             payoff: 80 ms becomes 40 ms, at n = 2

── the second needs no crossover analysis ──►

The confusion between them produces both of this section’s bugs. A goroutine per item over a slice is parallelism applied where the work is too small — §20.3.4's top row. And a sequential loop over three independent HTTP calls is latency left on the table where one errgroup would have taken it, which is §14.4's whole subject arriving here as a performance point.

The question that separates them: is the work waiting, or is it computing? Waiting parallelises for free. Computing has a crossover, and §20.3.2 measured where.

This is also the clarification that keeps the section from being over-applied. A server handling ten thousand connections is concurrent because the connections are independent and each spends most of its life blocked; §1.3's I/O-bound case is where a goroutine’s few kilobytes buy a capability rather than a speedup, and none of this section’s numbers apply to it. And a change that splits a 100 ms request four ways to return in 30 ms is a win even though it burns more total CPU, because the thing being optimised is the wait. Decide which objective you are optimising before reading any table here.

20.3.8 Common Mistakes

Summary: Is Concurrency the Answer?

Concurrency has a fixed cost that is paid before any benefit arrives, and the benefit scales with the work while the cost does not. That makes it a crossover rather than a preference, and the crossover sits further out than almost anyone guesses: the same summation was 134 times slower across sixteen goroutines at a hundred items, still 2.2 times slower at ten thousand, and 4.2 times faster at a million. The unit that decides it is work per goroutine rather than item count: measured, swapping the addition for a SHA-256 moves the crossover from past ten thousand items to between one hundred and four hundred. And once you have decided to parallelise, granularity rather than core count is the knob — the same job split six ways across the same sixteen threads spans 1,531×, with a goroutine per item at the bottom of it, spending 261 nanoseconds of scheduling on 1.16 nanoseconds of work. None of which applies to I/O-bound concurrency, where a goroutine buys the ability to wait on many things at once and there is no crossover to find.

Self-Check Questions: Is Concurrency the Answer?

20.4 Contention as a Curve

§20.2 said a profile is a single point. Contention is not a point; it is a curve, and the shape of the curve is the finding.

The reason is structural. A mutex under no contention costs a compare-and-swap and nothing else. A mutex with two goroutines occasionally colliding costs a little more. A mutex with sixteen goroutines all wanting it costs the compare-and-swap, plus the handoff, plus the cache line moving between cores, plus the parked goroutines waking in an order the runtime chose. Those costs do not scale linearly with the number of contenders, and they do not scale the same way for different designs.

Which means a benchmark run at one core count is a single sample from a function nobody has looked at. go test -bench defaults to your GOMAXPROCS, so most people’s single sample is taken at whatever their laptop has.

20.4.1 The Same Code, Two Directions

One counter behind one mutex, and the same counter split across 64 shards, each padded to its own cache line. Identical work, identical machine, identical minute. The only thing that changes is GOMAXPROCS.

// Illustrative snippet — not a complete program
type oneMu struct {
    mu sync.Mutex
    n  uint64
}

func (c *oneMu) inc() { c.mu.Lock(); c.n++; c.mu.Unlock() }

type shardMu struct {
    s []struct {
        mu sync.Mutex
        n  uint64
        _  [48]byte // 8 + 8 + 48 = one cache line
    }
}

func (c *shardMu) inc(k uint64) {
    i := k % uint64(len(c.s))
    c.s[i].mu.Lock()
    c.s[i].n++
    c.s[i].mu.Unlock()
}

The padding is not decoration, and the size of it is not a guess: sync.Mutex is 8 bytes on this platform and the counter is 8, so 48 bytes of padding is what makes a shard exactly one 64-byte line. Get it wrong and the struct is 56 bytes, four shards straddle three lines, and the benchmark measures §20.5's problem instead of this one.

Measured, that is not a hypothetical. The same 64-shard counter, sixteen goroutines, seven runs, medians — padded 8.57 ns/op, unpadded 13.77 ns/op, a 1.61× penalty for eight bytes of arithmetic in a struct declaration. That figure is §20.5's subject leaking into a §20.4 benchmark, and it is an instance of §20.1's rule arriving from the other direction: the workload has to contain the thing you meant to measure, and only the thing you meant to measure.

Sixteen times the processors made the left column 5.6× slower. The same code, the same machine, the same afternoon, and the only thing that changed was -cpu.

Two findings, and the second is the one nobody quotes.

The single mutex degrades monotonically with processor count, 5.6× from one to sixteen. That is the convoy: one goroutine runs the critical section while the others queue, so adding processors adds queuers rather than throughput. What they add is cache-line traffic on the lock word, futex wakeups, and scheduler work, all of which scale with the number of contenders. Buying a bigger instance for a workload shaped like the left column does not merely fail to help. It costs you throughput, and the bill arrives as a latency regression that no code change explains — because the deploy was to a larger machine type.

And at one and two processors, sharding is the worse design. 13.50 against 10.79 at -cpu 1, 25% worse. The sharded version does more work per operation: an index computation, a bounds check, and a cache line that is probably cold because the previous operation touched a different shard. Sharding is not free and it is not always right.

That second finding has a specific and common production shape. A map is sharded on a sixteen-thread laptop, where it is genuinely faster. It is deployed to a container with a one-CPU quota, where it is measurably slower than the plain mutex it replaced — and because the change was benchmarked and the benchmark was honest, nobody re-checks it.

one mutex    1 │███                          10.8
             2 │████                         13.6
             4 │█████████                    31.0
             8 │███████████████              51.4
            16 │██████████████████           60.4  worse

64 shards    1 │████                         13.5
             2 │█████                        15.8
             4 │███                          10.0
             8 │███                          10.2
            16 │██                            7.6  better

── ns/op; more processors, opposite answers ──►

20.4.2 What Reproduces, and What Does Not

That table is one run, and this chapter’s own rule is that one run is not a number. Three independent sweeps of the sharded column, taken at different times on the same machine, disagree about a great deal:

Run A improves monotonically after two. Run B has a deep trough at four processors. Run C is flat and unremarkable. All three agree on the thing that matters — sharding removes the degradation — and none of them agrees on where the sharded curve’s best point is. A chapter that printed run B alone would have told you, with a straight face, that four processors beat sixteen.

The single-mutex column is the robust half: across runs it degrades by 5.6×, 4.8× and 3.8× from one processor to sixteen. Call it roughly four to five. The direction never changes.

There is a further trap underneath. Swap the counter payload for a map write and the sharded column reverses completely — 24.91 ns at one processor to 42.30 ns at sixteen — because that benchmark is measuring memory traffic across sixty-four maps rather than lock contention. Same idea, same sweep, opposite conclusion, and the only difference is what the critical section does.

20.4.3 The Primitive Is a Bigger Lever Than the Count

Before sharding a lock, it is worth asking whether it needed to be a lock.

Two things in that block are worth stopping on, and the first is a number this chapter has trained you to distrust.

0.124 ns is below the empty-loop floor. §20.1.1 says sub-nanosecond for anything with a memory access means the benchmark; §20.1.4 measured b.N's own floor at 0.2146 ns. A reader who took §20.1 seriously should stop here — and the resolution is §20.1.7's, one section later than most people remember it. This is a RunParallel figure, so it is inverse throughput and not latency: sixteen goroutines each performed a load, and the wall-clock time was divided by all sixteen. The per-operation latency is about 0.124 x 16 = 2.0 ns, which is an L1 hit and exactly what an atomic load on this architecture should cost. The benchmark is honest, the units are the trap, and the chapter’s own rule catches it.

The mutex costs about the same to read through as to write through — 62.25 against 53.11 — while the atomic differs by a factor of 140 between the two. Under sixteen-way contention you are not paying for the operation inside the critical section; you are paying for the lock, and §20.4.6 measures how little shrinking the section can do about that.

With those established, the read row is the one to sit with. An atomic load is around five hundred times cheaper than taking a mutex for the same read — 503× at the medians, and taking the worst atomic sample against the best mutex sample it is still 426×. There is no lock, no waiting, and nothing for sixteen cores to queue on. RWMutex sits in between at 2.35× better than Mutex, which is a real improvement and a rounding error next to the atomic.

Chapter 11 taught when an atomic is correct — a single word, no invariant spanning two variables — and §11.3 spent a section on where that stops being true. This is the performance side of the same decision, and the gap is not incremental. For writes it is smaller, 3.0×, but still larger than most sharding buys and it comes without a shard count to tune. §20.7.7 is the price of getting the correctness half wrong, and it is high.

one word, no invariant across fields
     └─► atomic          measured 0.124 ns read
reads dominate, sections are long
     └─► RWMutex         measured 26.4 ns read
anything else
     └─► Mutex           measured 62.3 ns read
still contended after all three
     └─► shards          and a parameter to tune

── going down: more general, more expensive,
   and more to maintain ──►

RWMutex deserves one caveat, because §9.3 measured where it stops paying and this is the same boundary from the performance side. An RWMutex does more bookkeeping than a Mutex — a reader count and a writer-pending flag — so it only wins when readers both dominate and hold the lock long enough to amortise it. The 2.35× above is an all-read workload doing nothing but a load; with a longer read the gap widens, and with a write-heavy mix RWMutex is slower than Mutex. §17.3.6 measured the case where it is decisive: swapping Mutex for RWMutex on a sharded limiter map took 128.5 ns to 49.6, which that section correctly called a bigger win than every sharding decision combined.

20.4.4 What Sharding Costs

Sharding is presented almost everywhere as free. It is not, and there are three bills. (Picking the shard needs a hash of the key; for any comparable key type maphash.Comparable supplies one, and Go 1.27's maphash.ComparableHasher packages hash and equality together as a maphash.Hasher.)

The first is diminishing returns, and §17.3.6 already measured it on a sharded limiter map:

Sharing costs 52.7 ns in that table — 180.3 down to 127.6 — and the first eight shards recover 43.2 of it. Sixty-four times the shards for the last 18%. Everything useful happened before the eighth shard.

That table also names the reason the curve flattens, which is more useful than the flattening itself. Every request still took the enclosing map’s own lock regardless of which shard it wanted, and a floor that does not shard cannot be broken by sharding. Before adding shards, find what every caller still touches; the distance between your current number and that floor is all the sharding has left to buy.

The second is memory and cache footprint. Each shard is a cache line you did not have, a hash you compute on every operation, and a map you cannot iterate consistently. §20.5 has the arithmetic.

The third goes unmeasured, because the benchmark that justifies sharding almost always measures the write. Reading a sharded counter means touching every shard.

The read cost is almost exactly linear — about 10 ns per shard, which is the uncontended lock from the first row, paid once per shard. That is the point: sharding does not remove the cost, it moves it from the write path to the read path, and the move is one-for-one.

Whether that matters is a question about your read rate, and it has two clean answers. If the counter is scraped every fifteen seconds, 5 µs some six thousand times a day is thirty milliseconds a day and the trade is free. If the counter is read on the request path — a quota check, a rate decision, an admission gate — you have made every request pay 5 µs to make every increment cheaper, and nobody benchmarked that direction.

So the shard count is a real parameter with a real cost on both sides, and the default reflex of “make it 256, why not” is measurably wrong.

20.4.5 Contention Is a Tail Phenomenon

Every number in this section so far has been a mean, and means are the reason contention ships to production unnoticed.

Here is the same mutex — a critical section of about thirty nanoseconds — with the distribution of the wait recorded rather than the average, at five worker counts.

Read the p50 column first. From one worker to sixteen it goes from 29 ns to 46 ns — it does not come close to doubling. Most acquisitions of a contended lock are uncontended: the goroutine arrives, the lock is free, it takes it, and nothing about that changed.

Now read the p99. It goes from 31 ns to 60.6 µs: 1,954 times worse, on the same lock, in the same run, for the same operation.

The mean is the number that would have been reported, and it describes neither end.

p50    ▏                                  46 ns
mean   ▏                               2,363 ns
p99    ████████████████████          60,578 ns
p99.9  ████████████████████████████ 135,623 ns

── the mean is 51x the median and 4% of the p99,
   and it is the only one of the four a dashboard
   shows by default ──►

The consequence is the one this chapter keeps arriving at in new places. A dashboard of medians cannot see contention at all. A service whose lock wait has a 46 ns median and a 60.6 µs 99th percentile looks perfectly healthy on every graph anyone draws by default, right up to the point where a p99 objective fires and there is nothing in the median to explain it. This is why §20.1.8 asks you to benchmark percentiles, and why §19's mutex profile — which accumulates total wait — finds this when a latency graph does not.

The mechanism is queueing, and §17.1.3's Little’s Law already gave it to you: as utilisation approaches one, the mean queue depth rises smoothly and the tail rises much faster. A mutex is a single-server queue with no admission control, so its waiting-time distribution has the shape every such queue has. The 430 µs maximum is one goroutine that lost fifteen coin flips in a row — with 320,000 acquisitions, some goroutine was always going to.

The one-worker row is the control and it is worth a glance. Its maximum is 146 ns, five times its own p99. An uncontended lock has no queue, so its worst case is a cache miss rather than a wait, and the entire distribution fits inside two orders of magnitude. Every row below it is the queue arriving.

20.4.6 Four Times the Answer Is Not “Shard”

Sharding is the most-reached-for fix and the fifth-best one. Here is the whole ladder first, so that the four rungs above it are visible before they are argued for:

─────────────────────────────────────────────────────
1  does this need to be shared at all?
     per-worker state, merged at the end      16.5x
2  is the primitive right?
     atomic where the invariant is one word    503x
3  can the work move out of the critical section?
     compute unlocked, publish locked         1.51x
4  can the critical section be shorter?
     §10.5.2, and measured                    2.70x
5  shard it
     and stop near GOMAXPROCS                  7.9x

── ordered by cost to you, not by payoff: the first
   four also help at -cpu 1, and sharding does not ──►

The ordering is deliberate and it is not the ordering by payoff — step 2 is the largest number on the list and it is second, because it is only available when the state is one word. Read the ladder as cheapest question first: each rung is a question you can answer by reading your own code, and the first four all improve the single-processor case as well as the sixteen-processor one. Sharding is last because it is the only rung that makes something worse, and §20.4.1 measured what: 25% worse at -cpu 1.

1. Remove the sharing. A per-goroutine accumulator combined at the end has no lock at all, which is what §20.3's parallel version did and why it scaled. §17.3.6's bottom row is the same finding from the other side: one limiter per goroutine at 10.9 ns/op against 180.3 shared, a factor of 16.5, available to no sharded design.

2. Change the primitive. §20.4.3's table, and it is the biggest single number in this section: 503× on the read path when the state is one word and no invariant spans two fields. §11.3 is where you check whether that is true, and §20.7.7 is what it costs when you decide wrongly.

3. Move the work out of the critical section. Compute under no lock, take the lock only to publish. §9.4's copy-release-call is the pattern, and §18.6.3 measured what happens when it is not done — a 50 ms deadline waiting 480 ms behind a dependency call held under a mutex.

Moving ninety per cent of the work out bought 1.51×. That is a real improvement and it is nothing like ten times, which is the number the intuition suggests.

4. Make the critical section smaller. The convoy’s severity scales with how long the lock is held, so shrinking the held time shortens the queue. §10.5.2 is the design rule. Here is what it buys when the lock is fully saturated — nothing at all outside the critical section — with the section itself shrinking rather than the work moving.

Ten times smaller buys 2.70×, not 10×. The remainder is the handoff — waking the next waiter and transferring the lock costs something no shrinking removes, and §20.2.3 measured where it lives: Unlock at 27.93% cumulative against Lock's 21.74%. A quarter of the cost is outside the section, so a quarter of the cost cannot be shortened out of it.

The two tables together give the rule: shrinking the critical section helps in proportion to how saturated the lock is, and even at full saturation the return is sublinear. Check the saturation before doing the work — a -cpu sweep that shows a rising curve is the evidence that it is worth anything at all.

5. Shard it. Last, and §20.4.1 and §20.4.4 are the two halves of why: it is 25% worse at one processor, and it moves cost onto the read path at about 10 ns per shard. When the four questions above have all been answered “no”, it works — 7.9× at sixteen processors in §20.4.1 — and it is worth doing. It is simply never the place to start, and it is where almost everybody does.

20.4.7 Why -cpu Belongs in Your Benchmark Suite

The convoy is invisible at any single processor count, and so is its inverse. The practical recommendation is narrow and cheap: for any benchmark that touches shared state, run -cpu 1,2,4,8,16 and look at the direction. It costs one flag and five times the benchmark duration.

The flag has one precondition that is easy to miss, and missing it produces a sweep that looks reassuring and means nothing. -cpu sets GOMAXPROCS; it does not make your benchmark concurrent. A benchmark body that calls the locked code in a plain loop runs on one goroutine no matter what the flag says.

The left column moves by 12% between its endpoints and never exceeds 17% of its own floor — drift, not a slope. The right column is the incident. Same lock, same machine, same afternoon — and the left column is what you get if you write the benchmark the way most benchmarks are written. A flat -cpu sweep is evidence of contention-free code only if the benchmark could have shown contention in the first place, which is §20.1's argument arriving in a new place: check the harness before you trust the number.

Three habits make the sweep useful rather than decorative.

Include -cpu 1. It is the cheapest detector of a change that only pays under contention, and it catches the design that is a pessimisation for every deployment smaller than your benchmark machine.

Sweep past your production core count. The question a sweep answers is not “how fast is this now” but “what happens when the machine gets bigger” — and that has to be asked before the machine gets bigger, because afterwards it arrives as an unexplained latency regression correlated with nothing in the deploy log.

Record the sweep, not the endpoint, so that a future change which bends the curve shows up as a change in shape rather than a 4% regression in one number nobody investigates. benchstat handles a sweep natively — each -cpu value is its own row — so §20.1's statistical discipline applies unchanged.

20.4.8 The Contention You Did Not Write

Every fix in §20.4.6 assumes you own the critical section. Often you do not, and the options change completely.

§17.3.6's table is the canonical example. A shared rate.Limiter costs 180.3 ns/op under sixteen-way parallel Allow(), against 10.9 ns/op for one limiter per goroutine — and every one of those nanoseconds is inside golang.org/x/time/rate, behind a mutex you cannot shrink, holding a critical section you cannot shorten. Three of §20.4.6's five rungs are unavailable.

Two of the five remain.

Shard the thing you do not own, by holding several of them. A keyed map of limiters is exactly this: 137.1 ns at eight keys against 180.3 shared. You have not changed the library; you have reduced how many goroutines meet at each instance of it.

Or stop sharing it, which is the same table’s bottom row at 10.9 ns and a 16.5× improvement — rung 1 of §20.4.6, and the better of the two that survive. Per-goroutine or per-worker instances, reconciled at the end if they need to be. This is available far more often than people check, because the instinct that a limiter must be shared to limit anything is correct for a rate and not for many other kinds of state.

The general shape: when the contention is inside a dependency, your only levers are how many instances exist and how many goroutines reach each one. Both are decisions you make outside the library, and both are visible in a -cpu sweep long before they are visible in production.

20.4.9 Common Mistakes

Summary: Contention as a Curve

A benchmark of shared state at one core count is one sample from a function nobody has plotted, and the function is not monotonic in the direction people assume. Measured, one mutex under b.RunParallel degrades 5.6× from one processor to sixteen while the same workload sharded 64 ways improves — and at one and two processors the sharded version is the worse design, by 25%. Which means a change that is honestly benchmarked as a win on a laptop can be a measured regression in a one-CPU container, and nobody re-checks it because the benchmark was real.

What reproduces from a sweep is the direction. The position of the optimum does not: three independent sweeps of the sharded column put its best point in three different places, and an earlier draft of this chapter published one of them as a finding.

Before sharding there are four cheaper questions, and all four also help the single-processor case. Does this need to be shared at all — measured at 16.5× in §17.3.6's limiter table. Is the primitive right — measured at 503× for a contended read that could be an atomic load. Can the work move out of the critical section — measured at 1.51× for moving ninety per cent of it. Can the section be shorter — measured at 2.70× for making it ten times smaller, which is sublinear because a quarter of the cost is in the handoff and not in the section.

And every mean in this section understates the problem. Measured, the same lock’s p50 moves 1.6× from one worker to sixteen while its p99 moves 1,954×. A dashboard of medians cannot see contention at all.

Self-Check Questions: Contention as a Curve

20.5 The Machine Underneath

Two of this chapter’s effects come from hardware Go hides almost completely — the cache line, and the memory bus. Neither appears in any profile Chapter 19 taught, both are measurable in ten minutes, and one of them is the best demonstration in the book of why §20.1 comes first.

20.5.1 What Chapter 11 Already Measured

§11.3.7 covers false sharing properly. It has the cache-line diagram, the padding fix, a Measured badge, and the sentence that matters most:

That is the mechanism and the blind spot, and this chapter re-derives neither. What §11.3.7 does not have — because no single measurement can have it — is the thing that makes false sharing a §20.1 problem rather than a §11 problem.

20.5.2 The Same Effect, Three Shapes, Nine Times Apart

Same cores, same cache, same 56 bytes of padding, same hour. The reported benefit of the identical optimisation ranges from 8% slower to 8.18× faster depending only on how the benchmark was shaped — and §11.3.7 publishes 2.3× for a fourth shape, which sits inside that range.

All three are correct measurements. They measure different things.

The first row is the cold open’s left column and the shortest of the three stories: with one goroutine there is no second core to invalidate anything, so there is no coherence traffic to remove, and all the padding does is make the struct eight times larger and slightly worse to touch. A benchmark of a fix for contention, run without contention, does not report zero. It reports a regression — which is worse than zero, because it is a number, and a reviewer will argue about it.

The second row’s narrow shape has sixteen goroutines fighting over two counters, so each goroutine spends most of its time in ordinary contention on a line it would be contending for anyway. Padding separates the two counters, but the sixteen writers still collide — eight per counter, on a line that is now exclusively theirs and still shared eight ways. The wide shape gives each of four goroutines its own counter, so once padded there is no sharing at all: the cores stop talking to each other entirely, and the increment becomes a register operation with a store.

alone     1 goroutine  ──► 2 counters
         padding splits one line into two
         1 writer per line before, 1 after
         payoff 0.93x — a regression

narrow   16 goroutines ──► 2 counters
         padding splits one line into two
         8 writers per line before, 8 after
         payoff 1.37x

wide      4 goroutines ──► 4 counters
         padding splits one line into four
         4 writers per line before, 1 after
         payoff 8.18x

── the question is writers per line after the
   fix, not before it ──►

20.5.3 Which Shape Is Yours

The question is not “am I falsely sharing” but “how many independent writers are there per cache line, and would separating them leave them independent”.

Three cases, in descending order of how much padding buys.

Independent per-writer state, packed together. A slice of per-worker counters, a struct of per-shard statistics, an array indexed by processor.

// Illustrative snippet — not a complete program
// ✗ eight workers, four to a cache line
stats := make([]struct{ hits, misses uint64 }, workers)

// ✓ one worker per line
type padded struct {
    hits, misses uint64
    _            [48]byte
}
stats := make([]padded, workers)

Each writer touches only its own element, so padding makes them genuinely independent and the gain is the wide shape’s — large. This is the case worth looking for, and §20.4.1's shard counter is an instance of it: measured at 1.61× there, on a struct whose padding was eight bytes short of a line.

Shared state that happens to be adjacent. Two counters both written by everybody. Padding separates the lines but the contention is real rather than false, so the gain is the narrow shape’s — modest, and you are now paying memory for a small win.

Read-mostly data. Cache lines shared by readers cost nothing; the coherence traffic is caused by writes. Padding a read-mostly struct is pure waste, and §20.7.2 has the arithmetic.

The diagnostic is a question about your data, not a profiler run — which is fortunate, because §11.3.7 already established that no profile will answer it.

20.5.4 The Other Shared Resource

Cache lines are the coherence problem. There is a capacity problem underneath them, it has nothing to do with sharing, and it puts a hard ceiling on parallel code that has no locks in it at all.

The second table is the shape everyone expects: near-linear to four threads, 7.4× at eight, 8.09× at sixteen, tailing off exactly where sixteen hardware threads stop being eight physical cores.

The first stops at 4.76× and then goes backwards. There is no lock in it. There is no shared writable state in it — every goroutine reads a disjoint slice and writes only its own accumulator. It stops scaling because the memory bus is a shared resource of fixed width, and past four threads the cores are queueing for it rather than for anything Go can see.

40 ops/element   1 │█                        1.00x
                 2 │██                       2.00x
                 4 │████                     3.99x
                 8 │███████                  7.40x
                16 │████████                 8.09x

read-only sum    1 │█                        1.00x
                 2 │██                       1.99x
                 4 │████                     3.80x
                 8 │█████                    4.76x
                16 │████                     4.35x  worse

── no locks in either; the bus is the lock ──►

This is the third of §20.3.5's three reasons sixteen threads did not give sixteen times, and it is the one people never look for, because the instinct that “no locks means it scales” is so strong. The diagnostic is arithmetic rather than instrumentation: count the bytes each goroutine touches and the operations it performs on them. A loop doing less than roughly one arithmetic operation per byte read is memory-bound, and adding cores to it buys progressively less and eventually nothing.

The fixes are not concurrency fixes at all. Touch less memory — a smaller element type, a filtered pass, a better layout. Do more per byte touched — fuse two passes into one so the data is read once. Or accept the ceiling, which is a real answer, and stop paying for cores that are queueing.

20.5.5 The Sharing You Did Not Write

§20.5.3 asks which shape your data has, which presumes you know where the adjacency came from. In practice false sharing arrives through three shapes nobody wrote on purpose.

A slice of small structs, indexed by worker. make([]stats, workers) puts every worker’s counters in one contiguous block, so eight workers with a 16-byte struct fit four to a cache line. This is the shape padding was invented for and the one where §20.5.2's wide measurement applies — the writers are genuinely independent, and separating them makes them independent in hardware too.

Adjacent fields in a long-lived struct. A server object with a requests counter next to an errors counter, incremented by different goroutines on different paths. They share a line because they were declared next to each other, which is a decision nobody made.

A slice that grew. append reallocates, and the new backing array’s alignment is not the old one’s. Code that was accidentally well-separated can become accidentally packed after a resize, which makes this the one shape that appears without any code change at all.

The first is worth padding. The second is worth reordering the struct — putting a large field between two hot counters costs nothing and is not padding. The third is worth knowing about mainly so that an unexplained regression after a capacity change has a candidate.

20.5.6 What Padding Costs

§11.3.7 gives the fix and this chapter prices it, because “add padding” is advice that gets applied by reflex.

That is the trade stated exactly: you have exchanged a cache-coherence problem for a cache-capacity one. §20.7.2 measures what that costs when the exchange is a bad one.

The rule is narrow: pad things there are few of and that are written concurrently. Shard counters, per-processor state, the hot fields of a long-lived server struct. Not elements of a large collection, and not anything read-mostly.

And check the size rather than assuming it. Measured with unsafe.Sizeof on go1.26.1/amd64: sync.Mutex is 8 bytes and sync.RWMutex is 24, so a shard of {sync.Mutex; uint64} needs [48]byte to reach 64 and a shard of {sync.RWMutex; uint64} needs [32]byte. A pad copied from one struct to another is a pad that is wrong, and §20.4.1 measured the penalty for being eight bytes short — 56 where 64 was needed — at 1.61×.

20.5.7 GOMAXPROCS, and the One Decision Left

§1.5 owns this topic and ends at the right conclusion: “on a current toolchain, do nothing.” Since Go 1.25 the runtime reads the CPU affinity mask and, on Linux, the cgroup quota, and it re-reads them periodically; setting GOMAXPROCS yourself disables that updating, and runtime.SetDefaultGOMAXPROCS restores it. Both behaviours are gated on the go directive in your go.mod, which is the same mechanism §18.2 measured for loopclosure. Read §1.5; none of it is repeated here.

What belongs in a chapter about decisions is the narrow set of cases where changing it still helps.

Lowering it to reduce contention. §20.4.1's left column is the argument: if your workload’s curve slopes the wrong way, fewer processors is a real if unsatisfying mitigation while you fix the lock. Measure it with a -cpu sweep first — you already have the number.

Lowering it because you are not the only tenant. A batch job sharing a machine with a latency-sensitive service is a scheduling decision, not a Go one, and GOMAXPROCS is one of several levers.

Raising it above the core count. Almost never. It does not create parallelism and it does add scheduler overhead; the exception is workloads dominated by blocking syscalls the runtime cannot see around, and those are better fixed at the syscall.

Everything else — and in particular reaching for automaxprocs on a current toolchain — is covered by §1.5's conclusion and by the fact that setting the value at all turns off the runtime’s own tracking.

20.5.8 Common Mistakes

Summary: The Machine Underneath

§11.3.7 already measures false sharing at 2.3× and already says no profile will name it. What one measurement cannot show is that the same effect, on the same machine, in the same hour, measures 1.37× and 8.18× depending only on the benchmark’s shape — sixteen goroutines over two counters against four goroutines over four. Both are honest, they measure different things, and neither transfers. The question that decides which you have is writers per cache line after the fix, and it is answered by reading your struct rather than by running an instrument.

Underneath coherence there is capacity. Measured, one loop over 256 MB scales 8.09× to sixteen threads when each element gets forty operations and 4.35× — peaking at eight and then falling — when it gets one. There is no lock in either version. Memory bandwidth is the ceiling most lock-free parallel code actually meets, and the diagnostic is arithmetic per byte, not a profile.

Padding costs 8 bytes turning into 64, which is free on a few hot structures and converts a coherence problem into a capacity one on a large collection. And GOMAXPROCS is §1.5's, which ends at “do nothing” — the only decisions left here are lowering it to mitigate a curve you have already measured, or lowering it because you are sharing the machine.

Self-Check Questions: The Machine Underneath

20.6 Doing Less Work

Every optimisation so far has made the same work faster or reduced the interference between the pieces of it. This section is about the class that beats all of them and has no Amdahl ceiling of its own: doing less work in the first place.

It comes last among the techniques because it is the one people reach for after concurrency, and it should be the one they reach for before. Removing an allocation helps at every core count. Batching helps at every core count. Neither has a left-hand end of the curve where it makes things worse — which, after §20.4, is worth something on its own.

20.6.1 Batching, Generalised

§20.4.6's ladder shrinks the critical section. Batching does something better: it reduces how often the critical section is entered at all. The lock stays exactly as it was, and n operations become one.

That is the largest ratio in this chapter, and the reason is arithmetic rather than cleverness: at batch size k the lock is taken 1/k as often, so the per-operation cost of the lock falls as 1/k until something else becomes the floor. Nothing was made faster. The number of times the expensive thing happens was divided.

The same shape applies to anything with a fixed per-call cost: a syscall, a network round trip, a database statement, a log write, a sync.Pool round trip. §5.4's buffered channel is batching in disguise, and §17.2's token bucket is a batch of permits.

unbatched   [lock][work][unlock] x 4096
            4,096 lock acquisitions

batched     [work x 4096][lock][merge][unlock]
            1 lock acquisition

── the work did not shrink; the lock count did ──►

20.6.2 What Batching Costs

There is a bill, it is always the same one, and it is not visible in the table above.

Latency. At batch size 4,096 the first item in a batch waits for 4,095 more to arrive before anything happens to it. If items arrive at 10,000 a second, that is 410 milliseconds of waiting for an operation that takes a fraction of a microsecond. Derived, and it is the only arithmetic that matters here: worst-case added latency is (batch size - 1) / arrival rate, and that number belongs in the design discussion beside the throughput one.

Loss on failure. A batch that is in flight when the process dies is n lost operations rather than one. For a metrics counter that is acceptable; for an audit log it is not, and the batch size is bounded by what you can afford to lose rather than by the throughput curve.

Memory. n items held is n items resident, and a batch that grows without a bound is an unbounded buffer, which §17.4 spends a section on.

The standard fix for the first is a timeout: flush when the batch is full or when it has been open for longer than a deadline, whichever comes first. That converts an unbounded latency risk into a bounded one and it is why almost every real batching layer has a FlushInterval next to its BatchSize.

20.6.3 The Allocation You Do Not Make

The cheapest allocation is the one the compiler turns into a stack slot. §20.1.3 introduced -gcflags=-m as a benchmark hygiene check; here it is the optimisation itself.

8.2×, and the second row is not a pool, a cache or an arena. It is the same make([]byte, 4096) with nothing keeping it alive past the iteration — the compiler put it in the stack frame, where it costs a pointer bump and zeroing four kilobytes and costs the collector nothing. The allocation did not get faster; it stopped being an allocation.

Two details in that table are worth pausing on, because both are §20.1 arriving inside a §20.6 measurement. allocs/op is deterministic where ns/op is not, so the 1-against-0 in the last column is the durable half of this result and will reproduce on your hardware when the 8.2× does not. And the second row only reports 56.99 ns because it is a b.Loop benchmark: measured, the identical body under for i := 0; i < b.N; i++ reports 0.2186 ns and zero bytes, because with nothing keeping buf alive the stack allocation is deleted too. b.Loop's assigned-variable protection is what makes the stack version measurable at all.

Four things make a value escape, and all four are visible in the -gcflags=-m output:

• Returning a pointer to it, or storing one in something that outlives the function
• Storing it in an interface. any, error, io.Writer — the value goes to the heap so the interface can hold a pointer to it
• Capturing it in a closure that outlives the call, which includes every go func() that references it
• The compiler not being able to prove otherwise, which is the case that surprises people: a slice whose size is not a constant, a value passed to a function the compiler cannot see into

The third is the one this book keeps producing. go func() { use(buf) }() moves buf to the heap even if the goroutine finishes immediately, because the compiler cannot prove it will.

20.6.4 The Allocations You Make Anyway

Some allocations are structural. Many are a formatting call nobody thought about.

Read the allocs/op column before the timing one, because §20.6.3 just argued it is the deterministic half — and on go1.27, which specialises the allocator for objects under 80 bytes, the timing column is also the one a newer toolchain will have moved: three allocations against one. Sprintf takes ...any, so each of the two string arguments is boxed into an interface — two allocations before the function is even entered — and the result is the third. On top of that the format string is parsed at run time and a reflective path decides what to do with each verb. 3.7× for replacing one call with an operator, and the resulting string is byte-identical. None of that work is visible at the call site, which is why it survives review.

The integer row is the same shape with one argument instead of two: two allocations against one, and 3.7× the time. strconv.Itoa boxes nothing, parses nothing, and writes its digits into a stack buffer before the single allocation that becomes the string.

strings.Builder is the interesting row, because it is slower than the operator. Grow plus three writes is the right tool for a loop that appends an unknown number of times; for a two-part join the compiler’s own concatenation already does one exactly-sized allocation, and the Builder adds bookkeeping to reach the same place.

Those three allocations are not a benchmark artefact. The exercise at the end of this chapter has a Record built exactly this way, and its gate reports the same number: “Record allocates 3 time(s) per call.”

This is not an argument against fmt. It is an argument about where: fmt is right in error paths, log messages that are usually disabled, and anything that runs once. It is wrong on a hot path that runs a million times a second, and the replacement is not clever — it is the operator the language already has.

20.6.5 sync.Pool as a Crossover, Not a Preference

§12.3 covers what sync.Pool is: a per-P private slot, a shared queue, and a victim cache that survives one collection and not two. This section is about when it is worth having, and the answer is a crossover rather than a rule.

The pool’s cost is flat — 11 to 14 ns whatever it holds, because a pooled Get is a bounds check and a pointer move regardless of what the pointer points at. Allocation’s cost is not: it rises with size, because the allocator has to find the span and the runtime has to zero the memory.

So the decision is a comparison between a constant and a line, and it has three regions.

Large objects, high churn: pool. Buffers of kilobytes and up, allocated and discarded per request. This is what sync.Pool was built for and what net/http and encoding/json use it for.

Small objects: usually not. 1.4× on a 64-byte slice is real on go1.26.1 — and narrower on go1.27, whose allocator specialises exactly this size class — and it is rarely worth the two ways a pool goes wrong — an object returned to the pool while still referenced, and an object taken from the pool with stale contents.

Objects that never escape: never. This is the trap, and §20.7.1 measures it: if the object was on the stack, pooling it adds the pool’s 12 ns to something that cost almost nothing.

20.6.6 The Collector Is a Knob Before It Is a Code Change

Every allocation you do not remove is work for the garbage collector, and the collector has a dial that costs nothing to turn.

GOGC sets how much the heap may grow between collections, as a percentage of the live set. The default of 100 means the heap doubles before the collector runs; 200 means it triples.

1.38× for an environment variable, on a service where a week of removing allocations might buy the same. That is worth knowing before the week starts, and it is worth knowing that it is a trade rather than a free lunch: at GOGC=800 this program uses roughly nine times its live set in heap, and the trade is only available if you have the memory.

The last row is the interesting one. GOGC=off is slower than the default — 57.21 ns/op against 37.76, half as much time again per operation, from switching the collector off entirely. Nothing is being collected, so nothing is being reused: every allocation gets fresh pages from the operating system, the working set grows past every cache level, and the program spends more on memory misses than the collector was costing. Turning the collector off is not the limit of the GOGC curve; it is off the end of it.

GOMEMLIMIT is the companion and the safer half. It sets a soft ceiling on total memory, and the collector runs harder as you approach it. The combination people actually want in a container is GOGC raised and GOMEMLIMIT set to something under the container limit: collect lazily while there is room, and collect aggressively rather than being killed when there is not.

20.6.7 Common Mistakes

Summary: Doing Less Work

The techniques in this section have no Amdahl ceiling and no left-hand end of the curve where they hurt, which makes them the ones to try first even though they are usually tried last.

Batching is the largest ratio in this chapter — measured at 393× for a batch of 4,096 — because it divides the number of times an expensive thing happens rather than making it cheaper. It costs latency, exactly (size - 1) / arrival rate, and the batch size should come from a latency budget rather than from the throughput curve, which never stops improving.

An allocation that does not escape is not an allocation: measured, 56.99 ns and zero bytes against 469.7 ns and 4,096, for the same make. -gcflags=-m tells you which one you have before you write anything. Of the allocations that remain, a surprising share are formatting calls — measured, fmt.Sprintf is 3.7× a concatenation and allocates three times against one, because ...any boxes every argument.

sync.Pool is a crossover, not a preference: its cost is a flat 12 ns and allocation’s is a rising line, so it wins hugely on kilobyte buffers, marginally on small ones, and never on objects that were on the stack. And before any of this, GOGC is four benchmark runs — measured at 1.38×, with GOGC=off at 0.66× the default’s throughput because nothing gets reused.

Self-Check Questions: Doing Less Work

20.7 The Changes That Made It Worse, and When to Stop

Every section so far has had a measured half nobody writes down: the version of the optimisation that was slower. This section is that half, collected, because it is the more useful one. Each of these was a reasonable idea, applied by an experienced person, benchmarked honestly, and wrong — and in every case the thing that would have caught it was cheaper than the change.

20.7.1 A Pool That Was Slower Than Doing Nothing

The pool is 7.0× slower than not pooling, and the allocs/op column is zero on both sides, which is the part that makes it survive review. A reviewer looking for the pool’s benefit finds a benchmark with no allocations, concludes the pool is working, and does not notice that the version without the pool also has no allocations — because the object was never on the heap to begin with.

The pool did exactly what it promises. It replaced an allocation with a Get and a Put. There was no allocation.

The check is §20.1.3's, it costs one compile, and it is the same check §20.6.5 asks for: -gcflags=-m on the original, before the pool exists. does not escape means there is nothing to pool.

20.7.2 Padding That Added Cache Lines

Derived, from §20.5.6: padding a uint64 counter to a cache line turns 8 bytes into 64. On sixteen shards that is 128 bytes becoming 1 KB, which nobody will ever notice. On a million-element collection it is 8 MB becoming 64 MB, and a last-level cache measured in tens of megabytes now holds an eighth as many elements.

The change removed cache-line coherence traffic between cores and added cache capacity misses on every access. Both are real; on a large collection the second is much larger, and it does not show up in the benchmark that motivated the padding because that benchmark had sixteen elements.

Two conditions have to hold together for padding to pay: few objects, and concurrent writers. §20.5.3's three cases are the taxonomy, and the reflex application skips the counting step entirely.

20.7.3 Pooling That Outlived the Object

sync.Pool retains what you put in it, and it retains the largest thing you put in it.

The naive pool holds a 4 MB buffer that one request in a hundred needed, and holds it across a collection because §12.3's victim cache is doing exactly what it was designed to do: keep pooled objects alive through one GC cycle so a burst does not have to re-allocate everything. The buffer is released on the second collection — but in a running service the pool is refilled between collections, so the second collection never comes for that slot and the retention is permanent.

The fix is one condition on the Put: return the buffer only if it is a size you want to keep.

// Illustrative snippet — not a complete program
func put(b *[]byte) {
    if cap(*b) > maxPooled {
        return // let it go; the next Get makes a new one
    }
    *b = (*b)[:0]
    pool.Put(b)
}

Measured, that takes the resident heap from 4.98 MB to 0.99 MB — the same 5× on a pool holding one oversized buffer, and the arithmetic scales with how many oversized buffers your traffic produces.

This is the failure mode that does not show up in a throughput benchmark at all. ns/op is unchanged. The cost is memory, it appears in inuse_space rather than alloc_space, and §20.2.7's distinction between the two heap profile modes is what finds it.

20.7.4 A Cache That Cost More Than the Computation

A cache replaces a computation with a lookup, which is a win exactly when the lookup is cheaper than the computation. That comparison is rarely made.

The top row is the failure. Caching a computation that costs four nanoseconds makes it 3.1× slower through a sync.Map and 19.6× slower through a mutex-guarded map, on a 100% hit rate — the best case a cache can possibly have. The mutex version is worse than the sync.Map version for §20.4's reason: sixteen goroutines are now queueing on a lock that the uncached version did not have.

The bottom row is the success, and it is the same code: a computation costing 44.73 ns becomes a 12.67 ns lookup, 3.53×.

The crossover is not subtle and it is arithmetic. A cache is worth having when the computation costs more than the lookup, which on this machine means more than about 13 ns for a sync.Map and more than about 83 ns for a mutex-guarded map under this much concurrency. Below that the cache is a pessimisation with a hit rate of one hundred per cent, and it gets worse from there — every real cache has misses, an eviction policy, and a memory footprint, none of which appear in the table above.

20.7.5 The Cost of a Bound That Never Fires

Chapter 17 established that every bound has a cost and left the arithmetic. Here it is.

Nothing was ever refused. Every Acquire succeeded immediately, every Allow returned true, every channel send found space. The cost is entirely the checking, and it is 21× to 45× the work being protected.

That ratio is the whole point, and it cuts both ways.

When the bounded work is small, the bound dominates. Three nanoseconds of computation behind a 140 ns semaphore is a semaphore benchmark. If the thing you are limiting is this cheap, you almost certainly do not need to limit it — and §17.4's argument for bounding is about resources, not about cheap CPU work.

When the bounded work is large, the bound is free. The same 140 ns in front of a 5 ms database call is 0.003% overhead, and §17.3's entire case for admission control applies unchanged. Bounds are for expensive, resource-holding operations, and they cost nothing there.

The mistake is not “using a bound”. It is using one on a hot path where the protected work costs less than the check — and then, because the bound is a shared object under contention, discovering that the limiter has become the bottleneck it was supposed to prevent. §17.3.6's shared rate.Limiter at 180.3 ns/op under sixteen-way Allow() is that outcome measured.

20.7.6 Sharding Past the Floor

§17.3.6's table again, because it is the cleanest measured example of an optimisation continuing after it stopped working:

The first eight shards recover 43.2 ns of the 52.7 ns that sharing was costing — 82% of the available gain. Sixty-four shards reach 98%. Five hundred and twelve reach 100%, for sixty-four times the shards and the last 18% of a gain that was already almost complete.

And then the section stops improving entirely, because the floor is not the shard count. Every request still takes the enclosing map’s own lock, whichever shard it wants. A floor that does not shard cannot be broken by sharding, and the distance between your current number and that floor is all the sharding has left to buy.

The habit worth forming: before adding shards, find what every caller still touches. Then the arithmetic is available in advance — if the floor is 127 and you are at 137, the remaining prize is 7%, and no shard count reaches it.

20.7.7 The One Where the Benchmark Was Right

This one is different from the others, and it is the reason this section exists.

A counter is incremented by sixteen goroutines under a mutex. §20.4.3's table says an atomic is far cheaper, atomic.Uint64 has Load and Store, and the replacement is obvious:

// Illustrative snippet — not a complete program
// before
mu.Lock(); n++; mu.Unlock()

// after
a.Store(a.Load() + 1)

go vet is quiet. go test -race is quiet — and it is correct to be quiet, because every individual operation genuinely is atomic and there is no data race in the memory-model sense.

The change lost 92% of its increments, three runs in a row, and under -race it lost 93.61% while reporting no race at all.

Load and Store are each atomic. The sequence of them is not. Between one goroutine’s Load and its Store, fifteen others load the same value and store the same result, and fifteen increments become one. This is the lost-update problem §8.4 names and §11.1 warns about, arriving through a door marked “performance optimisation”.

The correct version is faster still:

atomic.Add is a single read-modify-write instruction, it is 3.10× the mutex, and it does not lose anything. The performance instinct was right; the API choice was wrong, and no measurement in this chapter would have caught it.

20.7.8 Two That Are Not Optimisations At All

Both of these appear in performance PRs regularly and neither belongs in one.

Removing a defer. Since Go 1.14 an open-coded defer costs about a nanosecond, which is inside the noise of anything it would be guarding. Removing defer mu.Unlock() in favour of a manual Unlock on every return path trades a nanosecond for a class of bug — the early return added later that does not unlock — and §9.2 is the section that measures what that bug costs. This is a negative-value change.

Replacing a channel with a mutex “because channels are slow”. They are slower per operation, and §16.3.5 has the figures: an uncontended Lock/Unlock pair at 12.0–12.4 ns/op against a channel ping-pong at 222–277. But the two do different things: a channel transfers ownership and carries a happens-before edge, a mutex protects a region. Replacing one with the other is a redesign of the concurrency structure with a performance justification attached, and the redesign is what should be reviewed. If the channel is genuinely a hot path, §20.6.1's batching applies to it exactly as it applies to a lock.

20.7.9 When to Stop

Optimisation has no natural end, so it needs an imposed one. Four conditions, any of which is sufficient.

You are at the ceiling. §20.3.6 gives s from two -cpu runs. If you are at 90% of 1/s, the remaining parallel upside is 10% and it is not there.

The next change is worth less than its risk. §20.2.6's table: removing a 25% frame entirely buys 1.33×. If the removal means a cache with an invalidation protocol, the arithmetic is 1.33× against a class of bug that is hard to test for, and the answer is usually no.

The profile has gone flat. When the top frame is 8% and there are forty frames below it, there is no single change left worth making. That is a finding, and it means the next improvement is architectural — a different algorithm, a different data layout, or not doing the work — rather than another pass over the same code.

The number is inside the noise. §20.1.5's control run is the reference. On this machine a quiet benchmark’s spread is 1 to 9%; a 3% improvement cannot be distinguished from nothing, and pursuing it produces commits that are indistinguishable from reverts.

There is a fifth condition that is not about numbers. The code got harder to read and the win was small. Every optimisation in this chapter has a maintenance cost — a shard count to tune, a pool with lifetime rules, a batch with a flush timeout, a padded struct somebody will tidy up. A 5% improvement that adds one of those has a negative total return, and the person who pays it is not you.

20.7.10 Common Mistakes

Summary: The Changes That Made It Worse, and When to Stop

Every technique in this chapter has a measured failure, and in each case the check that would have caught it was cheaper than the change. Pooling an object that never escaped is 7.0× slower and reports 0 allocs/op on both sides. Padding a large collection trades coherence for capacity. A pool that keeps an oversized buffer holds five times the resident memory across a collection, and ns/op never notices. A cache in front of a four-nanosecond computation is 19.6× slower at a hundred per cent hit rate. A bound that never fires still costs +140 ns on three nanoseconds of work. And sharding past the floor buys the last 18% for sixty-four times the shards.

The one that matters most is the one where the benchmark was right. Replacing a mutex with Store(Load()+1) is 2.38× faster, passes go vet, passes -race, and loses 92% of its increments. No measurement discipline in §20.1 would have caught it; a test that asserts the total catches it immediately. Correctness gates come before performance claims, because a performance claim about incorrect code is not a claim about anything.

And stopping is a decision, not an absence of one. You are done when you are near Amdahl’s ceiling, when the next change is worth less than the bug class it introduces, when the profile has gone flat, when the improvement is inside the noise — or when the code got harder to read for a number nobody will notice.

Self-Check Questions: The Changes That Made It Worse, and When to Stop