# Announcement: The Yellow Brick Half-Plane Has Arrived

Tuesday, June 9, 2020
By dreeves

Until today Beeminder had a fundamental design flaw that was baked in from literally day one. The first line of code for what would become Beeminder was to draw a line on a graph in Mathematica from a target weight to a goal weight. But weight fluctuates, I thought to myself. Or maybe I said it out loud to Bee, to which she would’ve nodded sagely at the seemingly innocuous fact. So, we figured, let’s add some thickness to that line and say you just need to keep your weight measurements on that thicker line or below it. We can color it yellow and call it the Yellow Brick Road.

insert record-scratching sound here

After killing baby Hitler, I’d say the next most important use for a time machine is to burst in on us at that moment and explain that, sure, weight fluctuates but you can’t just add thickness to either side of the line like that! The line drawn on the graph has to be the critical edge, the bright line that you can’t cross.

Do you see the problem with how we did it? We defined the yellow brick road — the path from where you are today to your goal — in terms of the centerline but then allowed a fudge factor known as “the wrong lane”. In the so-called wrong lane, you’d be off from the goal you set for yourself but within a window that Beeminder still considered “on the road”. Besides violating a fundamental tenet of Beeminder it yielded a cascade of bugs and confusion and flat-out wrongness. Like how if you dialed in a goal to do X things by time T, Beeminder enforced something slightly but infuriatingly different.

## What does all this mean for actual users?

For one thing, let me make that last example more concrete. Until now, if you wanted to, say, write 50,000 words in November, you’d end up only having written 48,333 words by the night of the 30th if you were edge-skating. Now there’s no more falling behind into that spare lane-width of leeway. It means we don’t need three numbers telling us the distance to this lane vs. that lane, just the one clear number that tells us where we are from today’s requirements.

Besides quashing whole categories of bugs and greater consistency and general elegance, though, you should notice surprisingly little! We’ve worked our butts off to make the transition silky smooth. We’ve even lovingly crafted the aesthetics of the new graphs to mimic, at least for now, the old version with lanes on a road.

Before (do-more goal)

After (do-more goal)

So why are we telling you all this? Well, for one thing, there are so many moving parts that we want to warn you in case we break things (again [2]). And just because this is the blog of record for building Beeminder and this is a big deal for us, even if most of the work has involved minimizing the impact for you. But it should also mean paving the way for big new features in the future. Mostly, we’re really excited and wanted to share!

## Can we get back to the excruciating details?

#### “We have finally peeled off the layers of duct tape and solved the root of these problems”

More generally, you just couldn’t rely on the colors Beeminder showed you to correspond with your actual amount of safety buffer on your goal. Going into “the wrong lane” was perpetually messing that up on anything but the simplest graphs.

The only way out of the mess we’d built for ourselves was to redesign how the yellow brick road worked. We did this exceedingly carefully and painstakingly with the hopeful end result that casual users — in particular those with simple linear graphs — will hardly notice a difference.

It started with a more careful separation of concerns between what we call Beebrain and Beebody. We gradually got more and more logic — timezones, inferring the start of the yellow brick road, another misguided bit of magic involving weight loss leniency — out of Beebrain so it could focus just on drawing the graphs and computing statistics. The number of clusterfudgesicles — byproducts of forcing “lanes of the road” to work — that we had to gradually untangle was mildly staggering. [1]

Then there was killing off exponential roads, generalizing the “no-mercy” derails setting, fixing the infamous do-less loophole, killing off auto-widening roads, and killing off custom lane widths — all of which were prerequisites for the yellow brick half-plane. And of course a million backward-compatibility issues.

In parallel with all of that we were also keeping busy porting Beebrain from Python to Javascript. That’s about 16,000 new lines of code (including the forthcoming visual road editor), though a mercifully large chunk of it is about to be obliterated as soon as the last old-style graph is converted to yellow brick half-plane. Because, ultimately, behind the scenes, yellow brick half-plane is a dramatic simplification of how Beeminder works.

UPDATE: There’s another small change we need to mention, per the Pareto Dominance Principle: When you create a goal now, it defaults to just one day of initial safety buffer instead of two. (You can still pick any amount you like — this is just about the default if you don’t pick.) It’s a little more natural this way, to have the razor road start exactly where you are today, with your initial datapoint right on it.

UPDATE: On this day, June 18, in the year of our Lord 2020, the last old-style non-YBHP lane-ridden graph was converted to the Yellow Brick Half-Plane New World Order. Praise be! Also we finally thought to add before-and-after pictures to this post.

## Footnotes

[1] The story of the parameter known as “edgy” is just gobsmacking and epitomizes the elaborate ball of duct tape we backed ourselves inside of by not having a yellow brick half-plane from the start. It’s too far into the weeds to be worth proper blogging but I’m including our dev notes about it in this footnote for posterity and just to marvel at:

Suppose you start a pushups goal of 3/day but want the first 3 due on the first day. This means starting the centerline of the yellow brick road above the initial datapoint so the initial datapoint starts on the edge of the road. So an “edgy” goal is one where the initial point is on the bottom (or top, depending on “yaw”) edge of the road. That means moving the start of the road up (or down) by one lane width compared to where it would be if it started at the initial datapoint. But we have a chicken and egg problem: We need the road function in order to compute the lane width and we need the initial point, (tini,vini), to compute the road function. Which is to say, we need (tini,vini) to compute the lane width but we need the lane width to compute (tini,vini), to shift it.

Solution (sort of): The lane width is generally equal to the daily rate of the road. So we can have the road start at the initial datapoint but one day earlier. I.e., (tini,vini) -= (86400,0). Then when we know the road function we can move tini forward a day and vini by the appropriate amount, vini = rdf(tini+86400).

The only problem with this approach is that it only puts the initial datapoint at the edge if the rate of the first segment of the road is the same as the rate of the most recent datapoint, since it’s the rate there that determines the overall lane width. Trying to do this Really Right gets very messy or even impossible without introducing worse problems than the initial point not being on the actual edge.

I think the right solution to the ‘edgy’ confusion is to get rid of the edgy parameter — goals are always edgy. To get the edgy=false behavior, just use the road matrix to specify one initial flat day. And in fact goals should always have an initial road row that says rate 0 up till yesterday or today (because otherwise if you add a datapoint before the first datapoint then you’ll make the road change).

[later] Bethany has now convinced me that edginess should not be in Beebrain’s purview. The start of the road is always given explicitly by tini and vini. When you create a goal that should be edgy you know the initial rate (e.g., the generous estimate that the user provides for Set-A-Limit [now do-less] goals) so you can can bump vini up by that amount. So we’re going to do that! Phew!

[2] Here’s a story from last week as I related it in a daily beemail the next day:

Bee and I had a fun night last night. While working on the Yellow Brick Half-Plane transition (did you see the news that all new goals are YBHP now? key milestone! we couldn’t turn back now even if we wanted to!) we accidentally shrank the roads on 1200 graphs before they were ready which caused about 100 of them — the edge-skaters — to instantly derail. So we had to painstakingly fix them and email everyone to apologize and assure them that they could ignore the accidental legit check they got, etc etc. (We think there was only one person who was about to derail anyway and was able to opportunistically get out of it due to our screw-up. For everyone else we got things fixed up in time.) About 9 person-hours of work from one tiny bug. I guess we’ve seen a lot worse!

Image credit: Faire Soule-Reeves. Thanks to Kevin Lochner who, years ago, first said “Isn’t this yellow brick road more of a yellow brick half-plane?” to which we probably said “kind of but that sounds stupid” and thanks to @insti for first articulating the problem with lanes as we’d implemented them and suggesting the theoretically elegant solution of defining the road in terms of the critical edge. Biggest thanks to Uluç Saranlı — see his Beeminder blog post for even nerdier details on yellow brick half-plane. Last but not least, thanks to another Beeminder superfan, Kenn Hamm (@kenoubi on Beeminder), for a putting a bounty on this which we can now finally claim!

Tags: , , , , , , ,