FlowRatio: Work Sampling Made Easy
If you’re doing time tracking, chances are you’re doing it wrong. In fact, if you’re counting anything, chances are you’re doing it wrong. The key insight is, as readers of that article know, that we don’t need an exact number; something in the right ballpark will do – where the size of the ballpark depends on the problem at hand.
Work sampling is a form of time tracking where we record what we are doing at random times of the day, and this can be used to paint a fairly accurate picture of how we spend our time. I have created FlowRatio Tracker as a one-page pure html and JavaScript implementation of that idea.
It’s not yet pretty, nor very featureful, but at this point it works well enough that I use it.
Even if you don’t want to track how you spend your time in detail, it can be useful for temporary time allocation studies, like figuring out how much of your work is unplanned, as opposed to planned. Run FlowRatio in the background for a few days and only record categories “planned” and “unplanned” – you’ll end up with a decent approximation for very little effort.1 The same principle applies for other distinctions – how much time do you spend on internal support? Fighting CI pipelines? Working around technical debt? Spend a few days running FlowRatio and then you’ll know. If you believe the proportion has changed since your last study, re-run a new study for a few days. It’s not hard but it can be very informational.
Why did I create FlowRatio? Let’s back up a little.
Conventional time tracking is intrusive
The two most popular forms of time tracking I know are:
- Any time we get a natural break, we try to remember what we have been doing for how long and write it down.
- Whenever we start working on something, we start a timer and then stop or restart it when we are done or move on to the next thing.
Note that both of these pretend to be exact counts, but they are not2 There is a possible third method which also pretends to be exact but is not: hire a person whose only job is to watch over our shoulder and record how we spend your time. A good zeroth-order approximation for the inaccuracy in such a measurement is 1 %, according to Deming.:
- Memory-based tracking is victim to the flawed human memory and attention span. It’s common to get sidetracked and then no longer remember exactly when one switched from doing one thing to another. Plenty of people I know just make shit up at the end of the day.
- Timer-based tracking requires great discipline with both starting and stopping timers3 Although combining this with something like Pomodoro helps.. When mistakes happen, we have to resort to manually adjusting the protocol and this reverts back to a memory-based tracking method.
My beef with both methods is that they are unnecessarily intrusive. Every time we do something, we need to remember to also do something else. They are intrusive because they pretend to yield exact counts. They pretend to account for everything – but they don’t!
What happens when we accept that a number in the right ballpark is sufficient?
Work-sampling is an old technique
In 1934, Tippett did exactly this, and invented the “snap-reading method of making time-studies”.4 A Snap-Reading Method of Making Time-Studies of Machines and Operatives in Factory Surveys; TIppett; The Journal of the Textile Institute; 1935. Tippett notes – and I have seen that claim replicated more recently5 A Comparison of Self-Reported and Observational Work Sampling Techniques for Measuring Time in Nursing Tasks; Ampt, Westbrook, Creswick; Journal of Health Services Research & Policy; 2007. that self-reporting and other attempts at a complete record results in biased data. Tippett’s snap-reading method is in part intended to trade bias for sample variance, which is easier to deal with.
The more popular term for snap-reading these days is work sampling, so that’s what I will call it too.
The basic idea is that instead of having a person stand next to a group of eight factory workers with a stopwatch, observing everything they do6 Why a group of eight? That’s about as many people one person can continuously observe, according to Tippett., we can have a person who occassionally (at random times!) walks across the factory floor and records whatever it is one worker (randomly chosen!) is doing. Repeat this enough times, and we get a good high-level picture of what our workers are doing. As long as we’re just interested in a rough time distribution, we get it for much cheaper with the latter method. If we want more detail, we can either run the study for longer, or send the observer walking more frequently – whichever is easier or cheaper in our situation.
Interrupting yourself might be fine
It’s easy to imagine software that implements work sampling time tracking: it interrupts us at random intervals and asks what we are doing. My major contention with this approach has been “interruptions bad”.7 At one point I even started working on a Slack bot that would message you and ask what you worked on last, on the idea that you don’t have to respond right away but can wait until your next natural break. However, that would bias results toward activities that lead to natural breaks.
However, I stumbled over Daniel Reeves’ article on his implementation of the work sampling concept, and he argues very elegantly for why interrupting the user to ask what they are doing is not a big deal:
we find it doesn’t interrupt our flow at all. We think the difference is that it’s not like, say, a pop-up email notifier that pulls your train of thought off in a new direction. It’s almost the opposite: it reinforces what you’re already focused on. It can even serve to get you back on track if your mind is starting to wander. “Oh yeah, this email is entirely unrelated to the work I intended to be doing. Whoops!”
Obviously, for this to work, the data entry has to be low-friction.
FlowRatio makes work sampling easy
Reeves’ implementation is open source and somewhat cross platform, but being based on X11, it’s a little tricky to get running under Windows. I figured, how hard can it be to create a pure JavaScript version that runs anywhere?8 Actually, my first attempt was a native gui application, but that turned out to be rather annoying to make cross-platform, mainly for a lack of documentation. Turns out the bare minimum is not that hard.
The result is FlowRatio Tracker, which is not yet pretty but now at a point where I can use it to understand where my time on the computer is going. (And to see how much time I spend on writing tests and documentation – an activity I want to do more of.)
The first time FlowRatio is opened, it will ask for notifications permissions, which are required for this type of work sampling to make sense. Once we allow those, the page will most likely say pending which means we should go back to work and do whatever we want to do. At some point in the future, we will get a notification from FlowRatio, and when we click that notification we will be taken to a text box where we fill in what we are doing. I recommend keeping it to short keywords, like “blog” or “work” or “email” or “news”.9 Undocumented feature: if you tag a sample with a colon, that counts as an upper-level category with a lower-level category in it. So “work:docs” will be in the “work” category in the report, and in the future you might be able to see how the sub-categories of “work” relate to each other.
If it takes more than 60 seconds to fill something in after the notification is sent, it will automatically record us as afk. If the tab is closed, the computer is off, or whatever, and FlowRatio later comes to life, it will retroactively impute off samples for that period – this means FlowRatio was off during this period, and there’s no way to know what happened then.
Once you have logged a few samples (that are not off or afk) you can click the report link and you’ll get a very rudimentary summary of how you spend your time.
If you’re as big a fan of this as I am and you want to contribute, the repo is on sourcehut.
Fewer samples are needed than you might think
A previous colleague challenged the efficiency of this method by claiming that any reasonable accuracy takes a lot of samples. At the time, I made the following crude observation: imagine we want to find out how much of our time we spend on unplanned work. What’s the error of conventional time tracking? Let’s say it’s 5 percentage points.
For work sampling to be good, it needs to get down to an error of 4 percentage points. In the worst case, this takes 150 samples. In other words, we can either obsessively track our time the conventional way, or we run FlowRatio for three weeks10 On average 45 minutes per sample, with 8 hour work days, means it takes just over 14 work days to get 150 samples. and we’re done – we’ll know with better than 4 percentage points accuracy how much time we spend on unplanned work.11 What if we want results faster than three work weeks from now? In principle, it’s just a matter of tuning FlowRatio up to interrupt us more often. However, it’s currently hard-coded to 45 minutes – turning those constants into settings is some ways into the future. In that case, you may want to run your own version. Just download the html file and edit it to your heart’s content.
Appendix A: Other implementations
There are other implementations of the same concept, with a neat list maintained by the folks at Beeminder. Many of those implementations share the problems of the original, and the only other web implementation requires signing up to use.