Entropic Thoughts

Readership maths skills

Readership maths skills

readership-math-skills.jpg

Many of you get notified of new articles via rss, and some of you stay tuned through the email newsletter. The email subscribers have, in the past three weeks, answered a survey on their understanding of maths topics. I asked three questions of increasing difficulty:

  1. How advanced maths have you formally studied?
  2. How advanced maths are you still comfortable using?
  3. How advanced maths do you know well enough to teach someone else?

The “advancedness” criterion for answers to this survey was interpreted as the syllabic order common in public education in countries around the northern Atlantic. This means the survey puts a heavy emphasis on what’s known as real analysis and doesn’t cover algebra, set theory, structures, combinatorics, and other discrete maths.1 I also notice now that some topics are suspiciously missing from the response alternatives even though they are typically part of the syllabus, such as those relating to statistics and linear algebra. Oh well.

Let’s briefly break down the results.

You have studied a lot of maths!

The following plot shows what fraction of respondents have formally studied up to a particular level of maths.2 Take this with a bucket of salt. I don’t think, for example, that there are actually readers who have not formally studied addition/subtraction. I suspect those are related to lizard men, rather than an actual lack of education.

maths-skills-01.svg

We can see, for example, that maybe 7 % of respondents have never formally studied maths beyond division, and thus might struggle following along an article using percentages – and I do see the irony in saying that.3 To be fair, the response alternative gave examples “3 % of 8, 154 increased by 14 %” which is slightly more advanced percentage use than just naming a fraction as a percentage.

What really surprised me was the other side of this distribution. Over 70 % of respondents have formally studied differential equations and beyond. That’s very advanced maths. Some of this can be explained by selection bias: those that choose to respond to a form asking about maths education will primarily be those that are proud of their level of maths education.4 Similar to that trope that in local Reddits, the median user appears to have a very high salary, because those commenting on “how much are you paid” questions are those that are proud of what they are paid. Some of it can be that it’s easier to have studied something than it is to know it today.

Estimating level of comfortable maths

The reason I asked these questions was to get some guidance on how many readers I lose when I use more advanced maths in articles. The level to which people have formally studied maths is not a useful guide, because what is studied can be forgotten. After all, I have also formally studied differential equations, but it was ages ago, and I don’t remember anything of that. Thus, the other two questions.

Most respondents say they are still comfortable using maths at the level they studied.5 Some are comfortable at a level much higher than they have studied – these are those that are self-taught. Since people tend to overestimate their abilities, I’m going to assume that the answer to “what can you teach” is really the best guide to what level of maths can be comfortably handled as part of explanations.

When we switch the question from “what have you studied” to “what can you teach” the response goes down by 1.6 levels on average. That’s a smaller drop than I would have expected, but the distribution of the drop has heavy tails: many respondents claim they could teach to the level they studied, while others have forgotten a lot of it.

This is the distribution corresponding to the maths people are still very comfortable with.

maths-skills-02.svg

Only a third of respondents are still comfortable with differential equations. That’s a more believable number! Here are some other important transitions:

  • When it comes to solving equations, 6 % of readers start having trouble understanding what goes on.
  • If the equations involve logarithms, another 7 % join in the woes.
  • When entering calculus territory, another 12 % of readers check out.
  • Going further into integration loses 13 % more readers.

I don’t know if any of this will affect what I write, or how I write it6 I am well aware that there are readers that wish for more maths, and those that wish for less of it. In the end, what determines what I write is where my interests take me. Sorry!, but it feels good to have a rough idea of these numbers. If I can explain something without involving differentiation, that means for every hundred readers, there’s twelve more that’ll be able to follow along. That’s valuable.

Further investigation

The survey data used in the analysis above are limited to the readers that choose to receive the email newsletter; it is possible the subpopulation using rss is different. The more severe limitation of the data comes from the survey design, though, which is limited by the relatively simplistic survey system offered by my newsletter provider, Buttondown. Their survey system is designed for quick vibe checks rather than deep insight, and that shapes what it makes possible.

It would be interesting to investigate further, and choose a better survey design that doesn’t rely on the order of common syllabi. If you would like to participate in such a survey, please email me a list of topics you think are roughly equally demanding in study time for a common education. List them in an approximate order of advancedness, in your subjective opinion. This order does not have to be very exact – I will design the survey so that it corrects for smaller mistakes in ordering. Also let me know what you would like to get out of such a survey, i.e. why it would be in your interest to respond to it. (You don’t have to speculate on why others might want to respond to it.)

I cannot guarantee that I will make the survey, but I’ll use your emails as an interest gauge. If the interest is high enough and there seems to be good enough reasons, I’ll do it.

Appendix A: full text of survey

This is the full text of the questions email newsletter readers responded to.

  1. How much maths have you formally studied? By “formally studied” I mean that your study has been under the oversight of someone whose job it is to teach other people that kind of maths.

    (Alternatives are ordered loosely by “advancedness”; choose the last you have formally studied.)

  2. How much maths are you comfortable with? (Choose the alternative where you start to get uncomfortable, even if you think you can do more advanced things.)
  3. How much maths do you know well enough to teach someone else, without looking anything up? (Calculators allowed!)

Survey design tip #1: ask as specific questions as possible. This reduces the risk that people interpret the questions differently, but it also helps jog people’s memory, and it engages their system 2, which guards against automatic but inaccurate answers. Even if you want to ask about something fuzzy and non-specific, it is better to design a handful of specific questions that all revolve around that fuzzy thing, instead of asking the fuzzy question outright.7 As a benefit, with a few separate questions, it is possible to statistically judge whether the fuzzy thing is captured by those questions at all. If the answers to the questions are not sufficiently correlated, the questions aren’t successful in targeting the fuzzy thing.

Then in this survey, the full text of the response alternatives were the same for all three questions, namely:

  1. Counting, recognising numbers (IIIIII, 6)
  2. Addition/subtraction (3+9, 9−3)
  3. Negative numbers (−3, 4−12)
  4. Multiplication (6×5)
  5. Fractions, division (3/4, 50/8)
  6. Percentages (3 % of 8, 154 increased by 14 %)
  7. Variables and functions (\(x\), \(f(x)\), \(y=3x\))
  8. Integer powers (\(3^8\), \(x^2\))
  9. Integer roots (square roots, cube roots)
  10. Fractional powers and roots (\(x^{1.2}\), \(x^{\frac{1}{12}}\))
  11. Finding solutions to linear equations (\(3x-4=5\))
  12. Finding solutions to quadratic equations (\(x^2 + 3x - 3 = 15\))
  13. Finding solutions that require logarithms (\(3^a = 5\))
  14. Basic trigonometry (sin 180°, cos \(\frac{\pi}{2}\), finding the size of the hypotenuse)
  15. Basic differentiation (\(\frac{d}{dx} 3x + x^2\), \(\frac{f'(x)}{f(x)}\) when \(f\) is known)
  16. Basic integration (\(\int t^3 \, dt\))
  17. Complex numbers (\(3+5i\), \(8e^{i \pi / 12}\))
  18. Differential equations (\(\frac{dy}{dx} = 5y\))

Survey design tip #2: examples serve not only to make the response item more specific, but it also helps against another failure mode when asking people about their proficiency. People – especially men – tend to overestimate their abilities when they are asked if they are comfortable with doing a thing. By including examples of that thing, I hope respondents treat that as an opportunity to self-check and verify if they’re actually as good with the thing as they think they are.