I wrote logtrain.pl on my phone when I had some waiting time to kill. Even with the awkward keyboard, it probably took less than 20 minutes in total.4⁴ On the phone, I ended up using a lot of single-character variable names and didn’t write many comments. I have fixed some of that for the code in this article. While writing I also noticed some inconsistencies and small latent bugs that are fixed now. I remain convinced that self-reviewing code gets you at least 20 % of the value of having a trusted colleague review code.

I won’t dwell on this first version because it’s simple enough to be an introductory tutorial to a scripting language: generate a random number, ask the user for the logarithm of it, then print what the actual logarithm was so the user can check their result, then repeat until eof.5⁵ One thing of note, perhaps, is that this script generates a random value for \(\log{x}\) and then displays \(10^{\log{x}}\) to the user. The reason for doing that way is to give the user a uniform distribution of logarithms to guess from. Generating \(10^x\) directly (which would be the natural thing to do) would have biased the logarithms heavily upwards.

use v5.16;
use warnings;
use strict;

sub guess_round {
    my $n = 1/100 * int (10**(rand 6));
    # Format with appropriate number of decimals for number size.
    my $fmt = $n > 10 ? "%.0f" : ($n > 1 ? "%.1f" : "%.2f");
    printf "log $fmt ?\n", $n;

    # Read until EOF or valid guess.
    my $guess;
    while ($guess = <STDIN>) {
        last if $guess =~ /[0-9.]/;
        say "Invalid guess."
    }
    return unless defined $guess;

    # Print actual logarithm for self-checking.
    my $actual = log($n) / log(10);
    printf "actual: %.3f\n\n", $actual;

    # Indicate to outer loop that we want to go on.
    return 1;
}

while (guess_round) {}

At this point, this is the only component in the system and it doesn’t talk to anything else. It interacts with the user like this:

$ perl logtrain.pl
log 535 ?
2.7
actual: 2.728

log 2.8 ?
0.46
actual: 0.441

log 586 ?
2.8
actual: 2.768

log 1344 ?
3.15
actual: 3.128

log 2.0 ?
...

If looking for a place to stop programming and start practising logarithms, this is a great place to stop. The script has every feature it needs.

But! There should be something bothering you: we are trying to improve something (our mental logarithm skills) but we don’t have a quantitative measure of whether improvement is happening at all. Any improvement effort should involve trying to measure the improvement so we can know if we are doing something useful or wasting our time.

This means inserting some logging to logtrain.pl. One way to do it is to change guess_round to return the tuple of ($actual, $guess - $actual), and then change the main game loop to include the logging:

open(my $log, ">> :encoding(UTF-8)", "loglog.txt");

say $log "NEW SESSION";
while (my ($actual, $difference) = guess_round) {
    printf $log "%.5f %.5f\n", $actual, $difference;
    $log->flush;
}

This will generate a log file that looks like

$ cat loglog.txt
NEW SESSION
2.72818 -0.02818
0.44091 0.01909
2.76784 0.03216
0.29667 0.00333
NEW SESSION
-1.15490 -0.04510
2.01494 -0.01494

At this point, the system looks like

Printing the new session markers every time the script starts up is a crude way to track progress over time. By picking up those markers we can write an ugly oneliner to summarise performance over time:

$ cat loglog.txt | perl -lane 'if (/NEW SESSION/) { $n and print $ssq/$n; $ssq=0; $n=0; } else { $ssq+=$F[1]**2; $n++; }'
0.00053487022
0.82406979478
0.82982874455

This prints out the mean sum of squared errors for each session. If that number doesn’t start to go down with increased practise, we should be looking at some other way to get better.

Okay, great, now we can stop programming and start practising, right?

Yes, except the log file also contains a key to diagnosing our performance. We are measuring accuracy as the squared error, which means to reduce the error efficiently, we should focus on the biggest problems first.6⁶ Kahneman talks about this in Noise; any time you’re optimising a squared loss function, you get more from reducing a big error by a little than a small error by much. This makes sense since mound-shaped loss functions in general are synonyms for “don’t sweat the little things” thanks to their flatness around the optimum. I’m sure I’m better at some ranges of logarithms and worse at others. Where do I make the biggest errors? The log file knows, and we can extract that information by constructing a histogram for the various logarithm ranges of interest.7⁷ Since the non-fractional part of a base-ten logarithm is equal to the number of digits up to the units position, we ignore slips of the finger involving those, and focus on the fractions which are the difficult bit.

This is performed by loganal.pl.

use v5.16;
use warnings;
use strict;
use List::Util qw( max );

my $buckets = 10;

my @histogram;

open(my $log, "< :encoding(UTF-8)", "loglog.txt");
while (<$log>) {
    next if /NEW SESSION/;
    # This analyses only the fractional part, leaving errors of magnitude
    # estimation a problem for someone else.
    /^-?\d+\.(\d+) -?\d+\.(\d+)$/ or die "Unrecognised format $_";
    my $b = int($1/10**5 * $buckets);
    my $diff = abs ($2/10**5);
    $diff = 1-$diff if $diff > 0.5;
    # Measure squared error to penalise large mistakes more heavily.
    $histogram[$b]{ssq} += $diff**2;
    $histogram[$b]{n}++;
}

# Measure skill as the mean of the squared error for each bucket.
my @mean_error;
for my $b (0..$buckets-1) {
    # If this bucket has not been practised at all yet, give it a huge mean_error.
    defined($histogram[$b]) or $mean_error[$b] = 1, next;
    $mean_error[$b] = $histogram[$b]{ssq} / $histogram[$b]{n};
}
my $max = max @mean_error;

# Write histogram to file.
open(my $dist, "> :encoding(UTF-8)", "logdist.txt");
for my $b (0..$buckets-1) {
    my $lower_bound = $b/$buckets;
    my $upper_bound = $lower_bound + 1/$buckets;
    my $bar_length = 30/$max * $mean_error[$b];

    printf $dist "%.2f--%.2f %.10f %s\n",
        $lower_bound, $upper_bound, $mean_error[$b],
        '-' x $bar_length;
}

By the way, the histogram is a seriously undervalued data structure. It’s the way to store data approximately when order doesn’t matter, but it’s missing from virtually every standard library8⁸ Yes, even Python. The Counter class can only deal with categorical data, not floats.. Fortunately a crude version is easy to make on your own. If you want something for production use, I warmly recommend HdrHistogram.

This script will create a new file logdist.txt which contains something like

$ perl loganal.pl && cat logdist.txt
0.00--0.10 0.0001205906 --
0.10--0.20 0.0005869017 ---------
0.20--0.30 0.0003543788 -----
0.30--0.40 0.0003568321 -----
0.40--0.50 0.0017917269 ------------------------------
0.50--0.60 0.0005234944 --------
0.60--0.70 0.0003712724 ------
0.70--0.80 0.0005121528 --------
0.80--0.90 0.0001640814 --
0.90--1.00 0.0000484416

Great. We now know that I’m not very good at the 0.4–0.5 range, which corresponds to taking the log of numbers roughly between 2.5 and 3. This may be in part because I’ve memorised \(\log{3} \approx 0.5\) when it’s actually closer to \(0.47\). We learned what we wanted!

The system is now

Again, good place to stop programming and continue practising.

But look at what loganal.pl just produced. It made something that, if you squint, sort of looks like a probability density that predicts how bad I am at different logarithms. If we could generate logarithms with sort of that density we could have focused practise mainly on the numbers we’re bad at.

This is what loggen.pl does. First it reads the pretend density from logdist.txt and converts it to a cumulative probability distribution. Then it enters an infinite loop that draws random numbers from that cdf.

use v5.16;
use strict;
use warnings;

my @x = (0);
my @cdf = (0);

open(my $dist, "< :encoding(UTF-8)", "logdist.txt");
while (<$dist>) {
    /^\d\.\d+--(\d\.\d+) (\d.\d+)/ or die "Unrecognised format $_";
    # For each histogram bucket...
    push @x, $1;
    # ...create the corresponding cumulative probability.
    push @cdf, $cdf[$#cdf] + $2;
}

# Normalise distribution so it sums to 1.
$_ /= $cdf[$#cdf] for @cdf;

while (1) {
    my $r = rand();
    for my $i (1..$#x) {
        next if $cdf[$i] < $r;
        say (rand ($x[$i] - $x[$i-1]) + $x[$i-1]);
        last;
    }
}

The first few lines of its output might look like

$ perl loggen.pl | head -n 5
0.715224078842172
0.447936494098662
0.248107602352682
0.042294572367708
0.450150342244956

Now we have a system where the new part is neat, but it doesn’t really … do anything.

The last step is modifying logtrain.pl to pipe numbers from loggen.pl instead of generate its own. The changed parts are

...
sub guess_round ($) {
    # Get the fraction of log(n) from the random number source.
    my $fraction = shift;
    # Pick a random integer part for log(n).
    my $y = int(rand(6)) + $fraction;
    # Compute 10^log(n).
    my $n = 1/100 * int (10**$y);
    ...
}

# Pipe appropriately distributed random numbers to guess_round.
open(my $generator, "-| :encoding(UTF-8)", "perl", "loggen.pl");

...
while (my ($actual, $difference) = guess_round(<$generator>)) {
    ...
}

Producing the final system

This is the place I stopped programming. There are other ways we can still improve this9⁹ One thing that irks me is that loggen.pl depends on logdist.txt existing, which in turn depends on loglog.txt, meaning to cold start these scripts, we need to create an empty log file first and generate a uniform distribution from it. It would be nice if loggen.pl could generate uniformly distributed logarithms on its own when it doesn’t have a distribution to go by., but now is a good place to talk about architectural patterns instead.