Are you somehow stumbling upon this blog post without knowing anything about Beeminder? Hoo-boy are you in the wrong place. But here’s a frenzied attempt to catch you up in time: Beeminder graphs your progress toward goals by drawing a bright line, called the Yellow Brick Road, that you commit to having all your datapoints hew to. Like getting at least 10k steps per day on average as measured by your Fitbit or Garmin (we integrate with gadgets and webservices like that). If you don’t stay on track, we charge you money. It’s very motivating! And a key thing we keep you apprised of as you follow your yellow brick road is how much safety buffer you have — how many days until you derail. Derailing means you’ve gone off track and we charge you money. All clear?
So sometimes you accumulate too much safety buffer and what’s the fun in that? You want to skate the edge, with Beeminder nipping at your heels, pushing you to do more than you’d otherwise do (or less — some Beeminder goals are about doing less of something, like eating sugar or surfing Facebook). Or maybe you just want to set some daily bare minimum, no matter how much you’ve overachieved previously. We call that ratcheting because you’re making that yellow brick road jump closer to your current datapoint and, by the nature of the commitment contract, you can’t make it easier again once you’ve done that. Not without waiting a week, at least. Beeminder’s a little complicated, ok? But if you understood all that, we suppose you’re ready to read Bee’s post here!
Announcement! Ratcheting (reducing your safety buffer) and auto-ratcheting (capping your safety buffer) no longer change the dates of future changes (like breaks that you scheduled on purpose) in your yellow brick road. It also means the weekends-off feature plays nice with all those other features.
Background
Due to various historical circumstances, like lane width, and our own runaway “cleverness”, it used to be the case that when you ratcheted most goals (Do Less and Rationing goals not included), we would shift your entire future road over along the x-axis, until it was the appropriate number of days away from your current datapoint. That had the side-effect of moving the dates of all of your future road segments to be closer as well. In other words, if you ratcheted the road by one day, then your upcoming weekend, or vacation, or whatever, would move one day closer. Which isn’t usually how scheduled vacations work. The airlines don’t move your tickets one day earlier because you got extra work done before leaving for Hawaii.
That also meant that you couldn’t both (a) automatically cap your safety buffer, and (b) have automatic weekends-off. Quite the thorn in the side of you overachievers who wanted to do a minimum of 7 hours of work a day, while still setting aside weekends to pay attention to your family, or whatnot.
Anyhow, we’ve fixed it all up and everything is perfect now! …
… With the following caveats:
- Flat spots on roads are a little weird now.
- Do Less goals are a little different in how they do weekends-off, and nothing has changed with them + auto-ratcheting. (If they were working for you before, they still should. If they weren’t, uh, email support so I can hear about it!)
- For Do More goals, weekends-off doesn’t mean weekends are ignored when we count how many safe days you have, we just automatically schedule a two-day break every Saturday and Sunday.
The main motivation for this post is to explain the new weirdness with flat spots and help you figure out how to get what you want out of your yellow brick road. This is, in certain circumstances, a slight disimprovement for Do More ratcheters wanting to get rid of flat spots, and hence, by the Pareto Dominance Principle, we are blogging it.
The short version: when manually ratcheting, you may sometimes have to submit the form twice to get rid of all of your buffer when there’s a flat spot. Further, there may sometimes be amounts of buffer that are essentially off limits unless you make an adjustment to the flat spot elsewhere (other than by ratcheting). Taking those in reverse order, let’s dive in.
Some amounts of buffer are off limits
Think of it this way. If you have set the coming weekend “off”, then we don’t let you set an amount of safety buffer that will cause you to work on the weekend (without you going and changing the weekend so that it is “on”).
Example time! Let’s say it’s Monday, and you have weekends-off, so the coming Saturday and Sunday are flat:
You’ve currently got more than a week of buffer because you’ve done extra work, but you don’t want to be a slacker the rest of the week. You can shave off one day of buffer and put yourself at seven days, no problem:
You could also reduce the road down to just four days of buffer — that makes your next beemergency on Friday. That’s no problem:
Anywhere between zero and four days is fine, actually.
Weekend Weirdness
It’s at five and six days of buffer (in this case) that things are weird.
Notice the warning here, with that flat spot within our sights? “You’ve got an upcoming flat spot. You may not be able to remove all of your buffer.”
Because of that flat spot you cannot make it so that you have five or six days of buffer, because five days from now is a weekend. And you scheduled the weekend flat. So we’re going to take that seriously and the weekend days become off limits. If you try to ratchet to five or six, the road just returns seven days of buffer.
A nice side-effect of treating flat spots like you truly meant them is that capping your safety buffer (AKA auto-ratchet) now works fine with weekends-off and vacations without having to turn auto-ratchet off during your vacation and then restore the cap when you come back.
Mathematical Interlude
There’s a classic joke about a math professor where a student raises their hand in lecture to ask, “in step 17, how do we know such-and-such?” and the professor says “it’s obvious” and the student’s like “but what about blah blah blah?” and the professor says “oh. hmm… well…” and then stares at the board, then they start scribbling equations, fill the entire blackboard up, stare some more, then finally: “Aha! Yup, it is obvious.”
Working this out reminded us of that joke a few times. It can seem baffling that this can be as complicated as it is — “I have X days of buffer can you just make it be Y jeez” — and working out the underlying math can both cut away a lot of needless complexity and also make clear how much complexity is intrinsic. Anyway, here for your amusement — or to make the arithmophobes scatter like mice — is some of the reams of math we did, convincing ourselves that we implemented this right:
Wait, I see a flat spot ahead, but there’s no “you’ve got a flat spot” message
Check it out. When the road gets scooched up close enough to your current datapoint that the flat spot is above you, the warning about “upcoming flat spot” goes away. In our example case, that’s anywhere below five days of buffer.
In this case you do have a flat spot in the future, but it is not within your safety horizon, so it’s not relevant at this point. The flat spot is five days away, and your next beemergency is in four days, so the flat spot can’t currently affect any ratchets you make. Or, for a more visual explanation of what I mean by your “safety horizon”, see how your current datapoint is below the level of that flat spot in the road? So it’s just not relevant at this point, because you can only change the road a little bit right now.
Killing a future flat spot
Ok, that’s all well and good with a future flat spot. But what if you actually want to kill that flat spot that you scheduled?
Probably you got that break by using the break scheduler and if it’s still more than a week away, you can actually use it un-schedule a break too, by selecting the time period you want to unschedule, and changing the rate from 0 to … something. That is admittedly kind of a fussy interface for saying “uh no, I don’t want that break after all”, and if the change is less than a week away, you can’t use it anyway. (Yes, even though you’re not even making the road easier in this hypothetical world).
In that case you’ll have to get rid of it using the prototype road editor. The prototype road editor functionality is super slick and awesome and amazing, but the interface is not exactly newbee friendly. (Yet!) If you want to venture down this road, check out my screencast of removing a flat spot from my writing road using the road editor.
A couple things to remember:
- Switch to editor mode when you pick your graph
- Untick the “Fixed Slopes” before you delete the end of your flat spot
- Don’t panic! — there’s an undo button
- Nothing gets submitted to Beeminder until you click “Submit”
- (So click submit)
As long as the changes you make to the road don’t violate the pink zone (i.e. cause the graph to overlap that lightly pink-shaded zone), you’re good to go! The interface will let you know loud and clear if your changes do violate that constraint, though. And this is all happening in your browser; the changes aren’t sent to Beeminder unless you press the Submit button, so even if you make your graph go all wonky here in the road editor and everything turns red and sirens go off, it’s ok. You can just reload the page and we’ll never know!
Shortening flat spots
Another thing we were concerned about was the case of newbees setting up a goal with some initial safety buffer (the goal creation interface makes that easy) and then changing their minds about it and wanting to get started right away. We didn’t want the answer to that case to be “go use this prototype road editor thing” since that’s still wholly too deep-Beeminder for a newbee. You might also legitimately want to shorten a break for other reasons.
So here is the compromise we came up with. If you are actually on a flat spot, you can shorten the flat spot. But this only works with manual ratchets. It’s not a built-in / automatic part of the ratchet algorithm, but actually uses a different method to adjust your safety buffer in this case. We changed the form to alert you that something is different, and to be very explicit that what you are doing at this point is shortening a scheduled break.
Programming Philosophy Interlude
Couldn’t we have strung together these two different types of ratcheting using ✨Programming Magic✨ so that flat spots would get automatically removed without you having to think about it?
Yes… we could have but (1) we think it’s an improvement in 99% of ratchets to assume you really wanted that flat spot there, since aside from the newbee case, you don’t get a flat spot in a road without taking some kind of action. And, (2) haven’t you been paying attention around here? Trying to automatically adjust roads for you is fraught with potential pitfalls and weird corner cases depending on what values of the road are explicitly set by you, and which are inferred. It’s safer to make fewer assumptions and make you do more of the work.
To give you an example of this flat spot shortening business, I’ll be economical and also show you a case where you’d have to submit the form twice. You’re just a little above the road, with a flat spot starting today. The road has five days of buffer, and let us suppose that you would like to have two days remaining:
If you put in a “2” and ratchet, here is what you’ll get:
In addition to the super helpful banner telling you what’s up — “Ratcheted as much as we could, but you are now on a break. Ratchet again to reduce the length of your break.” — the ratchet section of the settings has changed to look like this:
Now if you change the buffer, we will slide the end of the flat spot over, reducing your break. This is a little bit like the old ratchet where we moved the whole road in the x-direction to reduce your buffer, but in this case we are only moving the single segment that describes the flat spot, not the entirety of the future road:
Usually this won’t really be noticeable. The case where you would be most likely to notice is if you have segments that specify road values. (We are getting way into the weeds now.) If the segment after the flat spot has a specific y-value specified then moving the beginning of the segment to be a day earlier will make that segment shallower. If the segment after the flat spot has a specific end date and rate specified — the more common case — then moving the beginning of that segment to be a day earlier will make the segment end at a higher y-value. In most cases you would never notice or care about that.
Since we’re this far into the weeds, let me show you visually a couple cases where the results of a flat ratchet might seem odd. (None of them violate the akrasia horizon constraint, so we decided that they are ok.)
Case 1: the shifted segment gets less steep
Here is a road that’s got a segment after the flat spot that says “reach a value of 7 by the 19th”:
If you flat-ratchet this to two days, here’s what you get; a segment that reaches a value of 7 by the 19th…
…but since you started three days earlier, the segment is less steep. It now looks like it’s got almost the same rate as the following segment, but if you look closely, you’ll see that there is still a kink on the 19th, where it reaches a value of 7 and then the slope changes.
Case 2: weird shenanigans
If you have really particular stuff set up, you can even cause non-monotonic roads by a flat ratchet. But this is pretty unlikely. Here’s an example though!
I’ve set up this road so that it has two segments after the flat one. It has a segment that increases at a rate of 1 until the 19th. That is followed by a segment that is effectively flat for three days, however it is specified as a Date + Value, saying “on the 22nd, be at a value of 4”:
Here’s the road matrix, for those of you following along at home:
[[2020-10-09, 0, null], (road starts on Oct 9 at a y-value of 0)
[2020-10-14, null, 0.57], (road slopes up at 0.57 per day until Oct 14)
[2020-10-14, 3, null], (road jumps to a y-value of 3 on Oct 14)
[2020-10-18, null, 0 ], (road is flat until Oct 18)
[2020-10-19, null, 1 ], (road slopes up at 1 per day until Oct 19)
[2020-10-22, 4, null]] (road goes to a y-value of 4 on Oct 22)
Here’s what the road looks like after ratcheting to two days of buffer:
Are you starting to believe me now, that automatically changing roads for you is more complicated than you might initially think? What happened here is perfectly logical, given how we represent and plot roads, and given how the road was specified. The section that originally appeared as a second flat section, was in fact a direction to “continue from your starting point until you reach a value of 4 on the 22nd of October”. In the original road it only happened to be flat because the original specification caused the previous segment to end at a value of 4. When we scooched the starting point of that segment, which was specified as “increase at a rate of 1 per day”, instead of increasing for one day, it is now increasing for four days. That causes the segment following it change from being a segment that goes from four to four in the course of three days, to one that goes from seven to four in three days. Phew.
(Side note: If there are any upcoming non-monotonicities in your road, the whole ratchet interface just says awhellno and links you to the road editor.)
Weekends-off doesn’t mean weekends are ignored
The two days that you are taking off for the weekend still count towards your safety buffer. So if you automatically cap your safety buffer, the auto-ratchet will eat some of your buffer when the weekend hits. That’s just Beeminder taking you literally when you say you want at most, e.g., two days of buffer and not treating weekends as a special case.
Let us explain. Say you have auto-ratchet set to two days, with weekends-off. You’re skating the edge, like a normal Beeminder user, but on Thursday you go nuts and dispatch all your work for Thursday, Friday, and Monday. So now your next beemergency is Tuesday. If you don’t count the weekend, you might think of that as three days of safety; you’re safe for Thursday (today), Friday, and Monday. But Beeminder still counts the weekend, so from our point of view you’re safe for … Friday, Saturday, Sunday, Monday, Tuesday … five days!
What then? Well, on Friday morning from your weekends-don’t-count point of view, you now have two days of buffer — but not really! Beeminder sees four days and that’s more than you said you wanted so Beeminder ratchets the graph. It won’t mess with your weekend so it can’t ratchet you to your specified two days — that would give you a Sunday beemergency — but it can ratchet you down to three, for a next beemergency day of Monday. So that’s what it does.
More generally speaking, because the two flat days of your weekend still count in your safety buffer calculation, the most extra buffer that you can carry through the weekend (if you have your safety buffer automatically capped that is) is Max Safety - 2.
Ok, we covered a lot of ground here. Regular ratchets, flat ratchets, prototype road editor, un-scheduling breaks with the break scheduler. Of course the bottom line is that the yellow brick road is an arbitrary piecewise-linear function that you can edit to your heart’s delight with the road editor but also if your heart doesn’t delight you can tell Beeminder how much safety buffer to ratchet away and count on us to just do that. “Just do that” is way easier said than done but we now do what we think is the best approximation to what you wanted, with what we think is a reasonable amount of transparency about the cases where we can’t.
Hopefully it’s all much simpler to use in practice than it was to understand this blog post! If it gives you any guff (frustration, confusion, wrongness, obtuseness, …) definitely let us know.
Image credit: Faire Soule-Reeves