Should You Beemind The Moving Average?

Friday, March 20, 2015
By dreeves

Graph from the Hacker's Diet showing raw data with moving average overlaid

This was tied for the most popular topic for us to write about in a mini straw poll of the daily beemail readers. I decided to blog it instead because a surprising number of people have proposed this over the years (also it’s a blog beemergency day).

Beeminder users being, to put it mildly, rather sophisticated, you’ll know to apply Betteridge’s Law of Headlines to the title of this post. This comes up in the context of weight loss goals. It can be very frustrating to have a datapoint off the yellow brick road due to a random daily fluctuation. Fitness trainers often recommend weekly weigh-ins because daily random weight fluctuations freak people out. But of course the answer to that is not less data but data smoothing. The more data the better.

Beeminder of course has nice data-smoothing, a la The Hacker’s Diet: an exponentially weighted moving average. So, you ask, why not beemind that instead of the raw datapoints? Could that avoid the heartache of beeminding such a noisy metric as daily weight?

Ok, enough faux questions. Here’s why it doesn’t work, despite the moving average being the right thing to look at to see the trend.

Imagine your road is sloping down and you’re below it at first but you weigh in every day at the same weight, 100kg. You’re on a collision course with the road. After multiple days of 100kg each day, you find yourself on the wrong side of the road. It’s an emergency day and you need to lose a fraction of a kilogram to get back on. Except you’re beeminding the moving average so now you need to lose many times that to actually bring the average down enough. You’re actually requiring a bunch more foresight to keep from derailing — a much worse problem than the fluctuations!

Example Beeminder graph

In this example, you weigh in at 100kg every day until the eep day. Beeminding the raw datapoints you’d need to lose less than 0.1kg. Beeminding the moving average you’d need to lose a full kilogram.

“You need to lose many times that to actually bring the average down enough”

In general, being akratic and beeminding the moving average still means that you’re skating the edge and every day there’s some magic weight that you must weigh in at to keep from derailing. That’s fundamental to beeminding weight. No matter how you try to smooth the data or widen the road, if you eked by yesterday then there will necessarily be a magic weight that you absolutely must hit today to stay on track. It’s like a no-free-lunch theorem, figuratively speaking. Whatever we do ends up equivalent to having a single bright line that your (actual) weight is not allowed to cross every day.

But The Hacker’s Diet says…

Yes, and our moving average line is exactly as the good book recommends. The Hacker’s Diet then suggests that, when in intense weight loss mode, you make sure all your datapoints are below the moving average. That’s not too different from Beeminder’s insistence that all the datapoints be on the Yellow Brick Road.

More and more I’m convinced that the more fundamental solution to this kind of problem is to beemind actions over outcomes. That is, we recommend beeminding things under your direct control, like calories, steps, miles, minutes of exercise. Or at least beemind the outcomes conservatively, like cap the amount at risk at $5 and dial the yellow brick road to something super easy.


Image from the “Signal and Noise” chapter of The Hacker’s Diet

Tags: , , , , , , , , ,

  • chikitulfo

    I believe you have some problem with the the blog data collection, as you’ve posted on the blog, but the goal still derailed. You should check that asap.

  • MKK

    I’m sure most people would blench at this, but I treat my weight loss goal as a benchmark with unlimited tries to get under. I weigh myself whenever I feel like it, and if the weight is above the current data point, I don’t enter it. But if it’s below, I do. If I can’t get a data point under the current one by the time the goal derails, tough luck. I reset the road and keep trying. I know it’s somewhat against the spirit of Beeminder, but I’ve managed to lose 11 pounds this way (my total goal is 20 pounds), and it makes a weigh-in feel like “okay, let’s take another shot at this” rather than the make-or-break crisis it might feel like otherwise.

  • Dollar Flipper

    Just make it clear in your boilerplate and it doesn’t bother me…

  • Dollar Flipper

    I agree with most of this. It’s frustrating noticing the moving average not going down as sharp as I’d liked, but hey, I was the one who ate the burrito so it’s my own damn fault.

    The hardest part for me right now is that I’m using a Zone mentality. It lets me have 3 cheat meals (or one super big cheat day) and I worry that if I eat too much that day, that I’ll end up off of the road the next morning. It happened once before, but then again, instead of jumping right bag on the wagon, I just ate too much and didn’t work out enough! So that blows.

    To sum up my ramblings, the weight tracking for me is fine, you have a ton of different ways to look at the data, and pick the one that makes you feel the best. For me right now, it’s the polynomial fit.

  • Yonah Sienna

    Or beemind bites! :P

  • Daniel Reeves

    Thanks for pointing that out! Yeah, our wordpress plugin seems to have broken. We’re replacing it with an IFTTT recipe…