Entropic Thoughts

Various Hill Plots

Various Hill Plots

I struggle to interpret Hill plots. Maybe the same is true for you? Here are a bunch of examples to help build intuition. Each example comes with five sets of generated data, to indicate how different they can be due to sampling error.

The Hill plot is sometimes used to estimate the tail exponent of a power law. Although estimating tail exponents accurately is very hard and often ends up being somewhat subjective in the end, I figured it might be useful to look at a few examples of Hill plots to get an intuition for what they can look like, and when one should start to expect odd tail behaviour.

Thin-tailed continuous distributions

We begin with the ever intrusive normal distribution. Its Hill plot resembles the first half of a cycloid or something.

hill-plots-01.svg

Increasing the variance of the distribution does not change anything about the Hill plot.

hill-plots-02.svg

Changing its mean does not change the shape of the plot, but it affects the magnitude of the alpha values.

hill-plots-03.svg

Next, we’ll move on to the exponential distribution. This ought to look a lot like the normal distribution since they have sort of the same type of tail.

hill-plots-04.svg

And it does! What about the logistic distribution, which we use as a poor man’s normal distribution, but which also looks like it has fatter tails?

hill-plots-05.svg

Well, that also looks like everything else we’ve seen so far. What about lognormal?

hill-plots-06.svg

It looks like maybe it doesn’t have as sharp of a point on the right end. And if we really crank up the variance on that one, it starts to look heavy-tailed on a histogram, but does the same thing happen on a Hill plot?

hill-plots-07.svg

No change, except the magnitude got a whole lot smaller! Let’s look at Student’s t distribution, which at low degrees of freedom should have fatter tails than the normal distribution. But first, high degrees of freedom:

hill-plots-08.svg

Basically normal. Checks out. Then low degrees of freedom.

hill-plots-09.svg

This certainly has a blunter right end than the normal distribution! What about with just one degree of freedom?

hill-plots-10.svg

Oh, yeah, look at that! Stabilises at a tail exponent of one for a while.

Let’s leave these relatively well-behaved continuous distributions aside for now, and check out some odd cases.

Bounded distributions

First up is the uniform distribution.

hill-plots-11.svg

Very aggressive slope down at the beginning, then slowly leveling out like the others. We can do the Beta distribution with small parameters – this is a kind of mixture of uniform and normal.

hill-plots-12.svg

Looks a lot like the uniform distribution on the Hill plot, which makes sense given the low parameters: it’s still noticeably bounded on the range 0–1. However, a Beta distribution representing many trials near 50 % ought to be more like a bell curve and not have such a hard cut-off. Does that change its Hill plot?

hill-plots-13.svg

Yes, this looks more like the normal distribution, albeit with a blunter right end, maybe – sort of like the lognormal distribution.

Discrete distributions

What about discrete distributions? Let’s try Poisson.

hill-plots-14.svg

Hahah, funny! Turns out this is what Hill plots look like for discrete distributions, like this one with binomial draws:

hill-plots-15.svg

I’m sure one could make art from this. It’s also reassuring the high-trial count binomial starts to look like a normal distribution, as one would expect.

Heavy-tailed distributions

Now, then, what does the Hill plot look like for actual heavy-tailed distributions? We’ll do a relatively well-known one first: Cauchy.

hill-plots-16.svg

Okay, stable at a tail exponent of one then drops off. What about a Lévy distribution?

hill-plots-17.svg

Look at that tail exponent! Let’s also see a real power law, the Pareto distribution.

hill-plots-18.svg

Oh yeah, fair enough!