The Magical Widening Yellow Brick Road
Beeminder is a tool for getting yourself to do the things you want to do… when you’re thinking about what you want to do next week. For example, next week I want to eat healthy, work out at least 3 times, and work at least 40 hours on Beeminder. That’s in stark contrast to what I want to do right now. Namely, eat another brownie, sit on the couch a bit longer, not work out just yet, and watch just one more hilarious talking kitten video before getting down to work.
The discrepancy between what you want to do now vs later is called akrasia. As we explained in the inaugural article for this blog, akrasia is fundamentally a problem of overweighting immediate consequences. The immediate yumminess of this pie outweighs its far off damage to my goal of losing 20 pounds in 6 months. The immediate comfiness of this couch outweighs the far off damage of just one less workout. It makes sense to care more about the present than the distant future but humans take this to irrational extremes, making trade-offs we are fully aware are patently ridiculous. Such as dying years earlier for the pleasure of smoking.  Or spending 10 hours of the week on Facebook that could’ve been spent face to face with friends.
Stay on this road to your eventual goal.
The purpose of Beeminder’s Yellow Brick Road is to fix that problem by adding immediate negative consequences to redress the immediate positive consequences (the yumminess of the pie, the comfiness of the couch, the wrongness on the internet, etc). Stay on this road to your eventual goal; if you deviate from it even for a day, you lose.  Here’s how it actually works.
All you need to know
All you really need to know is: stay on this yellow brick road every day! But here’s just enough detail to know what that entails.
For non-weight-loss goals, where we assume there’s no randomness in what you’re measuring , the yellow brick road is very simple: Draw the dotted line from today to your goal — that’s the centerline of the road — and put a yellow-brick lane on either side that’s as wide as the targeted daily rate of change. All that really means is that if you’re in the right lane today and do nothing (stay flat) then you’ll be in the wrong lane tomorrow. The day after that you’ll be off the road. So the lanes simply serve to tell you how many days of safety buffer you have.
The road will not let you lose tomorrow if you’re in the right lane today.
With weight loss we have to account for random fluctuation. As with all Beeminder goals, we guarantee that you can’t lose tomorrow if you’re in the right lane today. But what if your weight is randomly up five pounds overnight? No problem, the road automatically widens to ensure you’re still on it. Once it widens though, it stays fixed. If you find yourself in the wrong lane then the top edge of the road is a fixed bright line that you’re not allowed to cross. Another random up-fluctuation will doom you so you need to scramble back below the centerline so that the auto-widening guarantee kicks in again. To play it totally safe, just make sure you stay below the centerline at all times. And remember that the “can’t lose tomorrow” guarantee only works if you report every day.
All you ever wanted to know
People who have actually lost weight by following a yellow brick road have not needed any more explanation than above in order to enter into a commitment contract to stay on their road — even people with hundreds of dollars at risk. So you can certainly stop reading now. But if you’re interested in the motivation behind why the yellow brick road works the way it does, read on.
If you’re still reading it’s likely because you’re wondering why it has to be so complicated, with the “magical” widening. Why not just draw a single fixed, bright line that’s safely above your starting point and gradually slopes down to your goal? Cross that line and you lose. That seems to give all the benefits of inexorable progress and daily accountability.
What’s missing, though, is that it doesn’t fully take into account the psychology of akrasia. We’ve learned this the hard way.
If your commitment contract is to stay below a fixed bright line then the right strategy is to always stay a safe distance from it, where “a safe distance” is something like “the most a random up-fluctuation could possibly be”. Of course you don’t know what that is yet, which is the first and simplest advantage of the automatic widening: You’re letting Beeminder worry about how to define “a safe distance” based on your actual data.
Maybe you behave for a few days, but that nice conservative line of safety has lost its bite.
But suppose you’ve got the fixed bright line version and you’ve worked out an absolute bare minimum safe distance such that on any given day there’s an extremely small chance of a random fluctuation pushing you above it. That way the overall chances of ever crossing the line are acceptably low.  That’s the strategy you’d like to follow, but you’re akratic. Today you’re right on that threshold of what you decided was a safe distance from the bright you-lose line. So you ought to be super careful today. But your short-term self, staring at today’s honey-glazed cheesecake special, knows that the chances of crossing the bright line are, by definition, extremely small. If you eat this cheesecake, the chances are still quite small. It’s an acceptable risk if it’s just this once. (It really would be an acceptably small risk if it really were just this once, which you tell yourself it will be.) So you eat the dessert. And sure enough, you’re only a little into that safety margin the next day. Maybe you behave for a few days, but that nice conservative line of safety has lost its bite. You gradually creep further and further past it until you’re really dangerously close to the bright line. This is motivating and you keep trying to get further below the bright line but unless you’re quite close to it, the fear can’t compete with the cake (and brownies, ice cream, etc). So you stay too close to the edge day after day and it’s only a matter of time before an unexpectedly large random fluctuation pushes you over. Cue the 8-bit rendition of Chopin’s Funeral March. 
Why would you be so silly? Akrasia! Same reason you came to us in the first place.
Note that this isn’t speculation. The yellow brick road used to be like that (worse actually; it would get wider but never narrower). We know.
What’s the solution? More immediacy! We want to take all that gradually building risk and concentrate it on single days. How? Roughly we want to provide a warning line that would be dangerously close to the bright you-lose line except we make it totally safe as long as you’re below it. (We move the bright you-lose line as necessary.) If/when you cross that warning line we throw all the risk at you all at once. Except, ironically, that’s so scary that it’s not so risky. You have no choice but to panic now; you’re right up against the bright line. The cheesecake can’t compete with that and you scramble back below the warning line to safety.
Notice the difference. In the simple fixed bright line version you would panic just as much if you were ever so close to the bright line. But you can’t ever get that close without taking on a pretty big risk of overshooting and crossing it. So you never get close enough to really panic or, more likely, you do and in doing so your risk of actually losing is way too big. (Remember that panic is our friend. It gets us off our fat butts.) Compare again to the magically moving bright line where you can and do get close enough to panic yet don’t take on the risk of overshooting and crossing it. You’re automatically safe until panic mode is triggered, and then you really have to panic. So you do, and you get safe again.
When you’re in the wrong lane, the road width does not change.
To map this back to how the yellow brick road actually works: what I’ve been calling the warning line here is really just the centerline of your road, the dotted straight-line path to your goal. It, naturally enough, separates the right lane from the wrong lane. And the bright you-lose line of course is the top edge of the road. The bright line magically moves because the road width changes to honor the guarantee that you can’t lose tomorrow if you’re in the right lane today.  When you’re in the wrong lane, the road width does not change — you’re in the case of the fixed bright line.
How does Beeminder decide how wide the yellow brick road should be?
Having laid out how and why the road magically widens to maintain the guarantee that you can’t fall off your road tomorrow if you’re in the right lane today, what determines the road width when you’re sitting pretty below the centerline? Beeminder computes the road width to reflect the variance in your actual data. Roughly, the width is such that if you retrofitted the road so that it followed your actual data as closely as possible, it would cover 90% of your datapoints (assuming no magical widening). That means that if you try to follow along the bottom edge of the road then it should happen infrequently (1 in 20 times) that a random deviation bumps you above the centerline into the wrong lane. If you thrive on panic, aim to be just barely below the centerline and you’ll be in the wrong lane and in serious danger of going off the road every other day, on average.
Another corollary of Beeminder’s road width algorithm: if you find yourself just slightly into the wrong lane then you have about a 5% chance of a random up-fluctuation pushing you off the road, if you don’t buckle down. The risk is correspondingly higher — and 5% should already be cause for alarm — the further into the wrong lane you are. But as long as you take the danger of being in the wrong lane seriously, you’ll actually keep the risk very low.
The bottom line on the top edge
For non-weight-loss goals (no random fluctuations in what you’re measuring) the lanes of the road just serve to give you a couple days warning before you lose. For weight loss it’s similar but the mechanism is trickier. But in terms of committing to the contract all you really need to know is that Beeminder has free rein to make the road any width it wants, except that it can’t let you lose tomorrow if you’re in the right lane today. If you’re in the wrong lane, just get back in the right lane as fast as humanly possible. No more magical widening when you’re in the wrong lane — the top edge of the road is now in fact the bright, fixed, you-lose line. Which you are suddenly way too close to.
Image by Krista Kennedy
 But this is not necessarily irrational. Some people, even when they step back and think about it, view this as a reasonable trade-off. That’s absolutely their choice to make. I’m just talking about smokers who say (and mean) that smoking is not in their overall best interest, yet keep smoking.
 For the present discussion we needn’t worry about the question of what “you lose” means. But typically it’s that you entered into a monetary commitment contract and losing means losing a painful amount of money. More generally, the idea is to approximate how bad it would be to fail at your long-term goal and then to arrange for a significant portion of that badness to hit you all at once if you deviate from the path to your goal on any given day. As opposed to what happens naturally: a negligible portion of that badness hits you on any given day that you’re off track. So no single day matters and so you stay off track day after day indefinitely!
Of course another key advantage of the yellow brick road — the flipside of its harsh mistress-ness — is that you can build up a safety buffer. You can get a comfortable distance below your road and then eat that pie guilt-free, because you’ve earned it and, as the yellow brick road shows, it will not throw you off track.
 On the back end this is actually all super general and there’s a boolean parameter, “noisy”, that determines whether the graph is measuring something with random fluctuations (aka noise). But since weight is the only popular example of a noisy graph, we keep things simple by not exposing generality like that in the interface.
 For the math here we can start with the useful fact that if something has a 1 in N chance of happening each time you try it and you try it N times, then there’s a 63% chance that it will happen on at least one of those N trials. If you’re going to do something risky over and over again and you want your overall risk to be, say, 1 in 100, then the risk each individual time better be much less — specifically 1 in 10,000 in this example if you’ll be repeating the risky thing 100 times.
 That’s the standard jingle for a video game character dying.
 Recall the first and simplest advantage of magic road widening: You don’t actually have to have any idea how to choose what safety buffer you need — how close to the you-lose line it’s safe to get.