Single-File Stateful Haskell Web App
I recently announced Decision Drill. This is written in Haskell, and I’m not sure it would have happened any other way. Here’s why, and what I learned.
Avoiding stateful servers
As mentioned previously, I hate maintaining deployments of side projects. This hate is probably shared by anyone who has high standards for performance and availability, but very little time to fix things that break.1 Rant warning. I recently tried to quickly add a user account to one of my servers, which are configured with Ansible. My playbooks were broken on my new computer because it runs a newer version of Ansible. I don’t have time to debug Ansible playbooks that used to work just fine one computer ago. Nor do I want to figure out how to downgrade Ansible to get things going. It shouldn’t be like this! Considering switching to Nix to manage my servers. There are some tricks to reduce the number of moving parts in a side project:
- Keep it small. Work on it until it does exactly what it needs to but no more.
- If there is a cheap manual process that can replace automation that needs to be maintained, go with the cheap manual process.
- Make it entirely client side, ideally in something that runs just about anywhere with a good compatibility story; for me, that is Perl or JavaScript.
As a concrete example of these points, I use FlowRatio daily, and it seems like the sort of thing that would obviously need to talk to a backend. But it does not – it’s keeping everything in the browser’s local storage. That works because I’m only using it on one web browser. It’s also not very polished, but it does exactly what I need it to. There’s no automation ensuring the integrity of the data, but I download the csv export every so often and make sure it’s going with my computer’s backup.
Six years ago, I (very informally) vowed to keep side projects client side for this reason. Any server process that is not nginx makes things ten times as complicated to maintain. It gets even worse if the server process is supposed to be stateful, because then we shoulder responsibility not just for keeping a process healthy, but also the integrity and availability of user data!
I toyed with breaking this policy when making Fisher’s Fountain which does use a server for random generation, but that seemed fine because (a) it’s entirely stateless, and (b) if something about it breaks it can be reimplemented in JavaScript in the space of a weekend.
Making a small, stateful web service
Well, it happened. Decision Drill is a stateful server process. It was deliberately designed to hold data for at most a month, limiting the impact of failed data integrity. Way too many services start out by implicitly promising to keep data intact essentially forever. Sometimes it happens, and it’s very nice when it does, but it’s a silly assumption to start out with. The owners of these services often take them down a couple of years later, for lack of maintenance effort.
Deciding to keep state is one thing, how it’s kept is another. Decision Drill uses SQLite for its storage, because it is one fewer server process that needs to be running, and one fewer part that needs to be installed and configured correctly.2 Also SQLite is a really neat piece of software.
Avoiding registrations or identification of any kind beyond a little random value in a cookie is also intentional – I cannot trust myself to keep an OAuth implementation up-to-date – or worse, hold someone’s password!
At first I intended to make Decision Drill event sourced, in order to keep the database append-only. The main benefit of that would be that it’s less sensitive to bugs: if some view presents corrupted data, it’s guaranteed not to be an erroneous destructive update that caused it. However, I quickly realised that event sourcing would have gone against the idea of making it small and ensuring it does just what it needs to and no more. Event sourcing adds technical complexity that’s not necessary for solving the problem at hand.3 Although it is still annoying to have to be more careful around destructive updates of data. If someone wants to show me how Decision Drill can be easily designed with event sourcing, let me know!
The entire project, minus build scripts and project metadata, sits in one Haskell file of 650 lines. Stripping out embedded html and stylesheets, the actual Haskell code is about 400 lines, including blanks and comments. This is very manageable! Although some Haskell libraries suffer from deprecating their content when they are upgraded4 Though nowhere near the level of Python!, they can also be kept at older versions until one wants to spend the effort upgrading.
In regards to deployment, Decision Drill reuses what was learned on Fisher’s Fountain, so that turned out to be a complete non-event. If something in Haskell breaks and needs to be rebuilt, the main problem lies in standing up a Haskell development environment. Given the solution used for Fisher’s Fountain, that should be a non-problem going forward. Also Cabal works much better these days than it used to.
Possibly-weird design decisions
I promised “what I learned” so here are some things that happened which I’m okay with, but which might be questioned by someone else.
Routing scheme
The routing scheme of Decision Drill is along the lines of
/nomination/#DecisionId NominationR GET POST /nomination/#DecisionId/close NominationCloseR POST /nomination/#DecisionId/second/#ProposalId SecondR POST /voting/#DecisionId VotingR GET /results/#DecisionId ResultsR GET
rather than the perhaps more rest-like
/#DecisionId/nomination NominationR GET POST /#DecisionId/nomination/close NominationCloseR POST /#DecisionId/nomination/second/#ProposalId SecondR POST /#DecisionId/voting VotingR GET /#DecisionId/results ResultsR GET
This started out as a mistake, but it grew on me. The latter emphasises the decision as a resource, with various stages to it. The former emphasises the stages of the process, with the decision as a parameter. The former seems more decoupled, in that each stage of the process could be implemented by a separate application, with the url indicating which application should receive a request.
I don’t know what to think. Let me know if you have opinions!
Automatically updating session timeout
The Decision Drill session cookie is set to expire a month after it was created. This allows a person to create a decision, forget about it, come back some time later, and still have admin privileges on that question. One might think there’s a problem with that: if someone visits once, 27 days later opens a new decision, then comes back 9 days after that – their session will have expired. Oops.
Fortunately, Yesod updates the session timeout on each visit, so the session always expires a month after the latest visit. This guarantees one never loses admin privileges on decisions one has opened. (Unless we lose our cookie, of course.)
But wait don’t we need a cookie banner now? No! Stop it! Cookie banners are not needed for cookies essential to the functioning of the site. We only need to collect consent for creepy cookies.
Ill-behaved maximum number of alternatives
If a decision has over 50 alternatives submitted, trying to add one more will just throw an error. This is not good ux, but I hope that any sane decision process will not result in more than 50 nominations. Nobody can evaluate all that and remain sane! In other words, the assumption is that if it ever gets to 50, someone has made a mistake and is okay with restarting the decision process.
Things that ought to be done but are not
There are also some things I would have done differently had I been smarter.
Inefficient database interactions
The way Decision Drill interacts with the database is very primitive. To avoid having to learn Esqueleto at this point, it uses only Persistent, which means no joins, for example. I also didn’t learn anything beyond the most basic querying even using Persistent, so it ends up doing some aggregation in the application instead of the database.
I’m not expecting much traffic to the service, so this should all be fine.
Hard-coded scheduled close
The way the automatic closing of each stage works is by comparing the opening time plus a hard-coded constant to the current time. I realised after the fact that this is a bad way to do it, because if the hard-coded constant is ever reduced, it will instantly close a bunch of open stages.
The correct way to do it would have been to have a field in the record for the scheduled closing time, save that in the database, and then compare that to the current time. Then the closing time of ongoing decisions would not change when the constant in the code changes. Fortunately, it is not too late. If I ever feel like changing the hard-coded timeout, I will first have to update the schema and make sure all ongoing decisions have a scheduled closing time stored with them.
maybe (error “???”)
The voting handler code cannot be reached unless nominations has closed. This means the (optional!) closing time value always exists in the voting handler code, but the compiler does nothing to help us realise this. Thus there are some partial functions involved that get the closing times of things that we know have closed, but the compiler does not.
I wish I could figure out a better way to handle that, but in the brief moments I sketched it out, I did not.