Why ratios want geometric means

You may have heard that when taking the average of a set of ratios, it’s best to use the geometric mean instead of the normal arithmetic mean. For years, I took this as received wisdom, but recently I decided to think about it more carefully. It makes sense to me now—here’s my thought process.

Speed test

Suppose that you have two computer programs, Program A and Program B. You want to compare how fast the two programs run on a variety of test cases. You time them, and observe the following results:

• Trial 1: Program A is 2× faster.
• Trial 2: Program A is 4× faster.
• Trial 3: Program A is 0.2× as fast.

Call these ratios $a_1 = 2.0$, $a_2 = 4.0$, and $a_3 = 0.2$. Program A is faster than Program B on trial $i$ if $a_i > 1$. We can see that Program A is sometimes faster, but Program B is sometimes faster, too. Which program is faster “on average”?

With a simple arithmetic mean, we could compute:

So, “on average”, it looks like Program A is $2.1$ times faster.

But something’s not quite right here. Suppose that we had looked at the exact same timing data from the perspective of Program B instead. We would have observed these results:

• Trial 1: Program B is 0.5× as fast.
• Trial 2: Program B is 0.25× as fast.
• Trial 3: Program B is 5× faster.

That is, we have $b_1 = 0.5$, $b_2 = 0.25$, and $b_3 = 5.0$, where each $b_i = 1 / a_i$. Taking an arithmetic mean again, we find:

But then we are forced to conclude that, “on average”, Program A is $2.1$ times faster than Program B, and Program B is also $1.9$ times faster than Program A! Surely any framework that leads us to such a conclusion must be flawed. An arithmetic mean must not be the right tool: let’s return to the drawing board.

Composition

Consider for a moment what the arithmetic mean means. It says, “combine your three quantities by adding them together, then scale them down by a factor of three to counteract the change in magnitude”. It we have three quantities of distance, or of mass, or of time, this makes sense. If you travel $3$ meters and then travel $2$ meters, you’ve traveled $3 + 2 = 5$ meters.

But that’s not how ratios compose. If you make a program $1.2$ times faster, and then make it $1.2$ times faster again, you haven’t made it $1.2 + 1.2 = 2.4$ times faster—you’ve made it $1.2 \times 1.2 = 1.44$ times faster. Thus, to take an average in the same spirit as the arithmetic mean, we should combine our three ratios by multiplying them together, and then scale them down by a third root to counteract the change in magnitude. And this is precisely the geometric mean.

Returning to our speed comparison example, we can reëvaluate how much faster Program A is “on average”…

…and Program B:

Now our results are consistent: Program A is faster, and Program B is slower. Moreover, note that $\tilde a = 1 / \tilde b$, which makes sense. This is another property that carries over in spirit from arithmetic means. If you travel, on average, $3$ meters farther than me, then I travel $-3$ meters farther than you; and just as ratios compose with multiplication rather than addition, they invert with reciprocation rather than negation.

The logarithmic connection

Look more closely at these analogies that hold “in spirit” between ratios and the more general real numbers:

• Ratios live in $\mathbb{R}^+$, and reals live in $\mathbb{R}$.
• Ratios combine with multiplication, and reals with addition.
• Ratios scale down with $n$th roots, and reals with division.
• Ratios invert with reciprocation, and reals with negation.

The common structure underlying these mappings is the logarithm:

• The image of $\mathbb{R}^+$ under the logarithm is $\mathbb{R}$.
• For ratios $a$ and $b$, we have $\log(a \cdot b) = \log(a) + \log(b)$.
• For a ratio $a$ and positive scalar $c$, we have $\log(a^{1/c}) = \log(a) / c$.
• For a ratio $a$, we have $\log(1 / a) = -\mskip-1mu\log(a)$.

Because the logarithm preserves all the structure that we care about, we say that it is a homomorphism (“same shape”) from the ratios to the reals. Moreover, since the logarithm is invertible (via the exponential function), and its inverse likewise preserves this structure, it is an isomorphism, which means that these two spaces are really just different characterizations of the same thing. Thus, another way to think about the geometric mean is that it uses the logarithm to bring the ratios into “linear space”, takes their arithmetic mean, and then returns them to their natural habitat:

This also suggests that if we wanted to, say, summarize the distribution of a random variable representing ratios, we should probably not take its expectation. We would get a more meaningful result by taking the exponential of the expectation of its logarithm, $\exp(\mathbb{E}[\log Z])$. And this in turn brings to mind Jensen’s inequality, with which we immediately obtain a nice proof of the AM–GM inequality by taking $Z$ to be a uniform random variable over a set $z_1, \dotsc, z_n$:

with the second step due to Jensen’s inequality, the concavity of the logarithm, and the monotonicity of the exponential. All the pieces fit nicely together… one connection at a time.