Steady-State Actor–Motivation Forecasting
We looked before at how to improve our gut feeling forecast around the prospect of peacekeeping in Gaza on May 31. I teased then about the actor–motivation framework I’m trying to understand. Reader Ero Carrera pointed me in the right direction to learn more: the primary proponent of this framework is Bruce Bueno de Mesquita. I’m going to refer to Bueno de Mesquita lot.
A forecast in the spirit of Bueno de Mesquita is based primarily on information about the actors involved in controlling the outcome: what they want, how much they want it, and how much influence they wield over other actors.1 The Predictioneer’s Game; Bueno de Mesquita; Random House; 2009. This information is fed into a sort of argumentation simulation, that crunches a bunch of different scenarios regarding how these actors interact with each other, and then if we do this multiple times we get an ensemble of forecasts which lets us assign a probability to outcomes.
The great thing is that this apparently works well particularly in situations where base rates are hard to come by – which is exactly the situation we were in with the un peacekeeping tournament question.
It should be easy but it is not
When we are running a quantitative ensemble forecasting model, we are really looking at something like this:
|
| description of world
v
+-----------+
| random |
| outcome |
| generator |
+-----------+
|
| possible outcome
v
The random outcome generator generates a random outcome that is consistent with the description of the world (i.e. given what we know, the outcome is possible) and it does so roughly in proportion to how likely that outcome is. So we run the random outcome generator multiple times to get a distribution of possible outcomes, and that is our forecast.2 E.g. if we discover that in 209 out of 5000 runs, there are peacekeepers in Gaza on May 31, then our forecast for that outcome would be 4 %.
The critical part is that random outcome generator. In most cases, that is a fairly well-behaved function: any time we run it with the same inputs, we get outputs that are sort of in the same general area of reality. That is useful when we want to reverse engineer the outcome generator because we don’t know what it looks like on the inside. We can then start with a very rough approximation3 Think something along the lines of a linear transformation or a lookup table., and gradually refine it with more effort until the results are very close.
I don’t think this is possible for actor–motivation forecasting. The function that converts actor properties to outcomes is iterative and involves actors reacting to the actions of each other, and even actors trying to anticipate how other actors will react to their actions. This is a recipe for non-linearity. I suspect the same inputs can generate wildly different outcomes through the actor–motivation function.4 This doesn’t mean the model is useless for forecasting: the probability distribution over outputs is still stable. I think this is even a desirable property of this framework: it can account for very different possible outcomes and take a stab at quantifying how likely they are.
In other words: I don’t expect it to be possible to approximate the results – we have to run the full simulation.
There are some good news and some bad news.
- The bad news is that Bueno de Mesquita never sits down to explain all the details of the simulation. He has left breadcrumbs behind, dropping loose fragments of the framework every few years starting in the early 1980’s. If we go through the past 40 years of his writing, we might pick up enough context to reconstruct the actor–motivation function. I haven’t been able to do that – at least not yet.
- The good news is that the breadcrumbs that Bueno de Mesquita has left behind are often inconsistent with each other; it looks like Bueno de Mesquita is constantly tinkering with and improving the model. This means model performance might not be that sensitive to the exact details of it. We may be able to improvise something that looks like it has the right shape when we squint and it would be good enough for our purposes.
Unfortunately, to know which parts of the actor–motivation function are required for structural stability and which are just paintings on the wall, one needs a better understanding than I have of the theory Bueno de Mesquita draws on. I haven’t yet been able to reconstruct anything I even remotely believe in.
There is a simplified, steady-state model
In this article, we will not learn more about the full dynamic model. However, Bueno de Mesquita has also published a highly simplified model. The simplified model produces – if I understand correctly – the most likely steady-state outcome of the dynamic model, provided that the actors involved don’t significantly change throughout the negotiation process.
That is a very limited output – it doesn’t even come with a probability attached – and indeed the simplified model is useless for any practical forecasting. But! The things we learn when setting this up will be useful prerequisites for when we learn the fuller model later on.
We will pick up the same example as in the previous article, but as we discovered there, many factors go into a peacekeeping mission in Gaza. For this article, we’re going to focus on just the first:
Will there be a peace agreement in Gaza?
Note the absence of a timeline; the simplified model doesn’t deal with time.
The basic components: people, power, and preference
Here’s how we approach forecasting following in Bueno de Mesquita’s footsteps.
Model outcomes on a continuous scale
We start by putting the event of interest on a continuous scale ranging from 0 to 1. On this scale, zero might represent “intensified hostilities” and one represents “bilateral peace agreement”. In between, we have some middle grounds like “enforced peace agreement” where one side beats the other into peace, and “ceasefire” which is something of a temporary peace agreement.5 This method does not work for problems that don’t fit neatly onto one continuous scale. I don’t know what Bueno de Mesquita suggests, but I would assume a good first step is to try to subdivide such problems into smaller ones that each fit onto their own continuous scale.
Here’s how I would construct the scale:
0.0 Intensified hostilities 0.1 0.2 Status quo 0.3 0.4 0.5 Ceasefire 0.6 0.7 0.8 Enforced peace 0.9 1.0 Bilateral peace
The distance between the items on the scale is meaningful. This distance should reflect the opinions of people asked about the problem in a particular way. For example:
- Alice prefers an enforced peace. If she cannot get her preferred outcome (enforced peace), but has to choose between a bilateral peace and a ceasefire, I believe she would be more ready to accept the bilateral peace. Hence, bilateral peace and enforced peace are closer to each other than either of them are to the ceasefire.
- Bob prefers the status quo to be maintained. If Bob cannot get his preferred outcome, but instead has to choose between intensified hostilities or a ceasefire, I believe Bob would sooner choose intensified hostilities. Therefore, the first two are closer to each other on the scale than to the third.
- It might look weird that ceasefire is in the middle. Wouldn’t someone calling for a ceasefire prefer a bilateral peace to intensified hostilities? Maybe. But it’s very rare to find a person who truly prefers a ceasefire. Usually people have a preference either for continued hostilities or permanent peace, and ceasefires come out as compromises between these people, rather than something someone really prefers.
One thing needs to be said about the scale we just constructed: merely the act of constructing it has improved our thinking. When forecasting, a common fallacy is thinking in a black-and-white style: either there’s a bilateral peace, or hostilities continue. But reality has a way of inserting nuance into outcomes, and the end result is rarely clearly in one or another category. By constructing this scale, we have realised that the outcome space is larger than just two entries.
The result of our upcoming analysis is highly dependent on the shape of this scale. Especially the outcomes we assign to the endpoints of the scale matter a lot. I don’t know a scientific way to design the scale6 De Mesquita says that the scale should be constructed so that the extreme points represent the most extreme opinions available on the conflict. But he’s also said that the extreme ends should correspond to the most extreme outcomes that are possible. Either way, it’s not a particularly operational definition., and in some forecasts I have ended up revising the scale after the fact because I didn’t like the forecast that resulted from my initial attempt at creating the scale.7 This may sound similar to p-hacking, but it’s not. Using the coherence properties of probabilities to go back and forth and adjust inputs and correlations and outputs until everything looks just right is good forecasting technique.
Find out which actors are relevant
The next step in the actor–motivation model is to list the actors that are relevant for the issue, and estimate three properties for each:
- The salience of the issue for them, meaning how strongly they care and how much of their energy they will invest in getting their preferred outcome.
- Their influence, which indicates how easy it is for them to convince other actors to get with the program. This property, in particular, should be solicited as such: set the influence of the most influential actor to 1.0, then assign influences to the other in proportion to how much influence they wield compared to the most influential actor.
- Their preference, which is which outcome they prefer when they come into the situation, expressed as a number on the outcome scale we designed in the previous step.8 I would have been inclined to look at the preference as a distribution over the outcome scale, but there’s a good reason to make it a sigle number: together with monotonic utility, that results in a guarantee that a Condorcet winner exists. This will matter in the full model. In this simplified model, you can use an arbitrary distribution for the preference, if you want.
There are varying levels of detail at which we can create this list. The most detailed would be listing every living human, but that is unfeasible for any problem. If the stakes of the forecast are high, we might still want to fill the list with individuals, though: significant government heads around the world, relevant diplomats, leaders of intergovernmental organisations, and so on.
For our low-stakes analysis, we’ll limit ourselves to a few roles and rough groups of people. I know nothing about politics, so I have made some wild guesses as to the properties for each actor – but according to Bueno de Mesquita, the general framework should still produce a decent answer.
| Actor | Salience | Influence | Preference |
|---|---|---|---|
| Israeli government | 1.0 | 0.8 | 0.2 |
| Israeli people | 0.8 | 0.3 | 0.7 |
| Hamas | 1.0 | 0.7 | 0.2 |
| Gaza people | 1.0 | 0.1 | 1.0 |
| Palestinian Authority | 0.8 | 0.3 | 1.0 |
| us government | 0.5 | 1.0 | 0.4 |
| G7 nations excl. us | 0.4 | 0.3 | 0.9 |
| un Sec Council excl. G7 | 0.3 | 0.2 | 1.0 |
| Iranian government | 0.6 | 0.4 | 0.0 |
The executive summary of the above is that the three main players that control the outcome (Israeli government, us government, and Hamas) don’t seem to want a ceasefire.
The us is separate from the rest of the G7 economic bloc because the US has much larger influence by – from my understanding – being the primary supplier of arms for Israel. The reason I separate the G7 economic bloc from the rest of the un security council is I vaguely remember reading somewhere that the G7 bloc often votes together in the security council.
To reiterate, please note that I know next to nothing about international politics and all of these numbers could be completely crazy! But hopefully they are crazy in a way that somewhat cancels out.
Graphical view of the landscape of power
At this point, we can multiply the salience and influence of each actor, and add the resulting number as a sort of power mass concentrated at that actor’s preferred outcome. Given the numbers above, it would look like this:
0.0 ## Intensified hostilities 0.1 0.2 ########## Status quo 0.3 0.4 ## ---------------------------------------- 0.5 Ceasefire 0.6 0.7 ## 0.8 Enforced peace 0.9 # 1.0 ### Bilateral peace
If it wasn’t already, this makes it clear why we have ongoing hostilities: the major powers involved want it that way.
The dashed line represents the average of all the power mass9 The easiest procedure to get the average power mass is this: for all actors, multiply all three properties S×I×P and sum up the results. Divide by the sum of the products S×I., and this is what Bueno de Mesquita suggests should be the forecast of this simplified model: a reduction in intensity falling just shy of a ceasefire.
The steady-state solution is insufficient
As stated previously, this solution is useless for practical forecasting because (a) we aren’t given a probability, and (b) time does not affect the solution. The above represents the most likely outcome, given the actors and motivations we have suggested, if the actors get together and discuss for however long it takes them to come to an agreement. Then, according to Bueno de Mesquita, that agreement is probably going to be near the average of their power mass. This does not let us answer what the probability of peace is before May 31.
Another problem that might not be obvious is that the steady-state solution assumes there is a single, fixed set of actors that negotiate throughout the process. In reality, different actors may surface and become relevant, others may fade away during the negotiation process, and some will have their priorities change.
To handle all three of those problems, we need the dynamic model that evolves through time. Maybe, if we’re lucky, I can figure it out and that will be a future article. If not, I hope this was at least a little inspiring.