Sometimes Beeminder goals have loopholes, like you could dehydrate yourself to get your datapoint below the bright line on your weight-loss graph (please don’t!). There are plenty of things like that and I probably shouldn’t think too hard about more examples. Sometimes loopholes like that can ruin a goal. But other times, ironically, such loopholes can be so blatant that they’re in no danger at all of ruining the goal. We call the latter ice cream truck loopholes. As in a loophole so big you could drive an ice cream truck through it. Could but won’t. You know that your Beeminder goal would be ruined the moment you exploited it, and so you’re not tempted to. It’s like a slope so slippery you’re not tempted to so much as set foot on it.
“Simply beemind touching the door of your gym’s building”
The original example of such a loophole was from the first incarnation of the Beeminder commitment contract back when we thought we needed an elaborate, literal commitment contract. We suggested that, to avoid fuzziness around the definition of “going to the gym”, you could simply beemind touching the door of your gym’s building. If you were legitimately injured or something and shouldn’t exercise, the loophole would be there for the taking. But otherwise you’d never be so lazy that you’d actually get yourself all the way to the door of the gym and then not go in. I mean, some people might. It depends on where exactly the akrasia is. Is the workout itself hard to make yourself do or is it just overcoming the inertia of getting out the door?
And that’s the point: thou knowest thyself. You can’t predict every loophole but for a given loophole you can predict whether it will ruin your goal.
Goodhart’s Law
One aspect of this, which we’ve talked about before, is that Beeminder is surprisingly robust to Goodhart’s law. Goodhart’s law states that any metric you try to optimize quickly becomes meaningless because people game the living crap out of it (that’s how I state it anyway). Our point with ice cream truck loopholes is that that doesn’t apply to a single person the way it applies in organizations. If you pay your employees based on how much time they spend at the office or how many lines of code they write, that may well be a Goodhartian disaster.
But for you personally, beeminding the amount of time you spend, say, at the gym, or outdoors, might well be highly aligned with your big picture goals — health, exercise, socializing. And you’re always ready to adjust if not. The feedback loop is tight and the incentives stay in line. Which makes sense: you’re the one constructing the incentives and also the one being incentivized. Goodhart’s law specifically depends on that not being the case.
Examples
We’ve talked about ice cream truck loopholes with users from day one and have collected a few examples. Here’s one, paraphrased, from someone writing a book:
I get an automatic +1 via RSS for each chapter. The (mostly) ice cream truck loophole is breaking it up into ever finer sub-chapters. I’ve abused that loophole just slightly, and rarely.
Another user said they’ve used the “check in at the gym without working out” loophole only once in two years and hasn’t been tempted otherwise.
Here’s one for household chores:
My metric for my to-do list goal is touching the item in question. For example, I could just move my laundry basket and get a +1 for doing the laundry. I’m not tempted to exploit it because I do actually want the laundry done, but if there’s a reason I can’t do the laundry (like not enough hot water) then I’m happy to use the loophole.
A couple months ago we had a kind of “this post is to keep Beeminder happy, the end” blog post. As David MacIver convinced us on Twitter, it’s not necessarily a ruinous loophole that you can mash your keyboard and hit publish to technically satisfy your blogging goal. If you define a blog post as “anything I’m not embarrassed to publish” and a small number of dummy posts meet that criterion then the system is working as designed. Presumably without Beeminder you’d let the blog go totally moribund, so it’s still a huge win. If you do start posting dummy posts one after another, you’ll restructure the goal (Beeminder lets you do that — you just need a week of lead time for any changes, including ending the goal altogether).
Finally, here’s an example from an old post of ours on what we dubbed “The Dirty Plate Club” in which we propose a way to break the habit of unthinkingly finishing all the food on your plate. Taking it to our characteristic extremes, we advocated a strict rule of leaving some food left over at every meal. A common reaction was to point out the loophole of overloading your plate to make it easy to follow the dirty plate club rule. But, you guessed it, that’s an ice cream truck loophole.
If you wanted to thwart the system, that would be a great way to thwart it. But you want the system to work. That’s why you’re adopting it in the first place.
We’re not saying you don’t have to worry about loopholes and other ways your nicely quantified, graphable Beeminder goals can be at odds with your true underlying goals. Just that the glaring, blatant loopholes tend not to be the ones you need to worry about.
Image credit: Faire Soule-Reeves