Quarterly Cup 2024 Q1 Retrospective
Alternate title: the quarter I found out I suck at forecasting.
I have continued participating in the Metaculus Quarterly Cup, and as in the previous two quarters, I will summarise my experience and lessons.
Just the numbers
My average blind1 As in without seeing the community prediction. Brier score2 A lower Brier score is better, and the range people practically fall in is something like 0.07 (best) to 0.25 (random guessing.) was 0.25 – which is about the worst it could be!
Part of this is probably overconfidence from doing so well the previous tournament, and maybe this tournament was a little harder than the past two? I’m not sure.
What I did differently
I’m not going to share the rest of the results because it was just bad. It doesn’t matter exactly how bad. But! Aside from hubris, there were reasons. I did two things differently this quarter compared to the previous two.
- I did not look at the community prediction.
- I did not update very actively.
Basically, I wanted to participate in this tournament in hardcore mode, to see how much of my past result was actual forecasting skill, and how much of it was that I had learned to smooth out the noise of the community forecast. It turns out a large portion of it was about smoothing out the community forecast!
So now I have a new challenge ahead of me: learn forecasting again, this time without the crutches. I have a long way to go.