1000 Days of User-Visible Improvements

Thursday, November 21, 2013
By dreeves

An unfinished / unpolished Beeminder bee, being beeminded.

UPDATE: A revised and updated version of this article is now on Messy Matters.

It’s amazing where one trivial user-visible improvement per day will eventually get you to. We’ve made 1000 user-visible improvements (UVIs) to Beeminder in the last 1000 days. [1] We had to or we’d have owed one of our users $1000.

Our inspiration for this was a crazy-sounding claim by Paul Graham in his 2007 essay, How Not to Die. He points out that if one of his Y Combinator startups is regularly doing new deals and releases and, importantly, sending updates to Graham and colleagues, they’re probably going to live.

Then, admitting how naive it sounds, he gets to the crazy part: “Maybe the linkage works in both directions. Maybe if you can arrange that we keep hearing from you, you won’t die.” He defends the claim by pointing out that every YC dinner is a kind of deadline. “Staying in regular contact with us will push you to make things happen, because otherwise you’ll be embarrassed to tell us that you haven’t done anything new since the last time we talked.”

Paul Graham: If every startup had such a commitment device the success rate would be 90%.

He even quantifies the power of a commitment device like this, using the example of two startup founders who appeared in Newsweek with ‘the next billionaires’ printed across their chests. It would be unthinkably humiliating [2] for them to fail after that, Graham points out. If every startup had such a commitment device the success rate would be 90%, about which Paul Graham emphasizes he’s not kidding.

Joel Spolsky suggests the same probability that an organic growth (as opposed to VC-funded) company that moves forward every day will eventually win, i.e., become a 10 million dollar a year business in 5-10 years. We’re right on schedule to do that.

Lifehack: Create Your Own Commitment Device

If you’re amenable to a bit of lifehackery, here’s what you should do to give yourself that same 90% chance of success without Newsweek writing an article about you. First, create a public graph [3] like this:

A graph of our public changelog with annotations for notable events

Of course it will take 1000 days for yours to look like that. [4] The “eep!” and “$1000” on our graph mean that we’re skating the edge (as usual) and will owe someone $1000 if we don’t improve Beeminder in some visible way by midnight tonight!

Once you have a graph like that, let your users know (even if you only have a handful of beta users initially) that you’re on the hook to announce a User-Visible Improvement (UVI) once a day on average. We tweet ours to a special Twitter account — @beemuvi — but any public changelog will do.

Ours have ranged from trivial, e.g.,

“Added some padding so the Feedback button doesn’t overlap the text if you make your browser too skinny. HT Judy Soule.”

to epic, like recently:

Android App v2.0! Vastly faster & includes a built in timer app for beeminding how much time you spend on things…”

You can start with a very unambitious definition of “improvement”. When we originally announced our commitment to do this we listed all of the following as things that count:

  • New blog posts
  • Fixing typos
  • Tweaking the layout or the logo or whatever
  • Improvements to how the bot responds to emails
  • Tweaks to the algorithm for generating the yellow brick road
  • Tweets from our main twitter account (@bmndr)
  • Improvements to the log in procedure or payment processing
  • Any new feature or tweak to any feature
  • New tips-of-the-day in the email responses from the bot

Basically, anything that makes Beeminder better, even in the most tiny, trivial way. Over the last 1000 days, we’ve gradually gotten more ambitious. Fixing typos and adding tips of the day and whatnot don’t count anymore. Bugfixes and CSS tweaks still do. And we have a separate graph for blog posts, which you can see in the sidebar, so this article doesn’t count as a UVI.

The real criterion is simply anything we’re not ashamed to publicly announce as our UVI of the day. We have (just) enough pride that we’d rather cough up the $1000 than tweet anything that clearly violates the spirit of the commitment.

Close Calls

In fact, there have been two occasions where we were ready to cough it up. The first was two years ago when we first visited Portland to interview for the Portland Seed Fund. The time zone change threw us off and, although we had deployed an improvement earlier in the day, we forgot to publicly tweet it by official midnight. At that point we had few enough salivating users watching like hawks that no one called us out in the 20 minutes or so before we remembered and got the tweet out. We would definitely have paid if someone had. [5]

“We have coughed up hundreds of dollars to our users on other meta graphs.”

More recently, we deployed our daily UVI and even publicly tweeted it, but forgot to update the actual graph (which we still do manually, but we recommend you set up an IFTTT recipe to auto-update yours!). We actually convened a little tribunal to rule on whether we should cough up the $1000 in that case. The verdict from our users was no, we’re subject to the standard grace period for getting data manually entered. Phew! We have coughed up hundreds of dollars to our users on other meta graphs — recently $270 for not getting a blog post out on time, and $90 each for our “40 hours/week on Beeminder” goals — but not yet thousands.

But at this point, if we do cough up this $1000 it will have been worth it about 100 times over. (And we’d immediately re-up and keep ourselves on the hook.) Even aside from the commitment device, about which more shortly, our change log is super handy for looking up what happened when, like when we compose monthly update emails and whatnot. Or even for debugging sometimes. In theory we have the git history but that’s too cumbersome and noisy.

Signaling Value

This has been immensely important for us as a commitment device to stay focused on forward progress, but we’re extreme in our need for lifehacky tricks like that to keep ourselves on track — which is why we made Beeminder in the first place, i.e., itch-scratching. We’re entirely serious when we say that Beeminder would not exist if it weren’t for beeminding Beeminder this way, particularly our UVI goal forcing inexorable forward progress even when we felt demoralized.

If you think you don’t need to do this to survive, then, first of all, reread Paul Graham’s How Not to Die. Note the part about how demoralizing startups can be. “The low points in a startup are just unbelievably low,” Graham says. “I bet even Google had moments where things seemed hopeless.”

But even if you don’t need to play this kind of trick on yourself, there’s huge value in signaling your commitment to steady progress and improvement. Prospective users place a surprising amount of weight on things like “how long ago was the latest blog post” and other such heuristics for whether you’re still in active development. Probably because it’s so common for startups to peter out despite the front page still being all “everything’s great! sign up now!” Watching your public changelog will mean more to your users than you can imagine, and will be persuasive to new users, not to mention making them more tolerant of your foibles. They know you’re getting better every day!

Addendum: Holy Moly, We Actually Paid Up On This

UPDATE 2013-12-03: We’ve now actually paid the $1000 on this, to Henrik Wist, who was the first one to spot it, per our fine print. Ouch! But of course, at 98 cents per UVI, this has been a steal! Not that we purposefully derailed, or even got especially lackadaisical about it after this article gushing about how great our UVI commitment is. We even said, portentiously, that “at this point, if we do cough up this $1000 it will have been worth it about 100 times over.” So, yeah. Still true. But still silly to actually derail like that!

The actual reason for the derailment was just the confluence of multiple beemergencies. Most notably our Android blog post (our blogging commitment is itself up to $810 at stake). We had some candidate UVIs written down and I was meaning to discuss them with Bethany (aka Queen Bee, aka our CTO) before we tweeted one. But we got wholly distracted by the emergency blog post, coming right down to wire and hitting publish at 23:59:30. We even spent another hour improving the post, forgetting all about the UVI beemergency until I saw the fateful blog comment come in.

Heightening the sting of this, we had actually deployed one of the UVIs and merely failed to tweet it in time. It was tempting to call that close enough. But after a previous close call (where we did tweet it in time and just forgot to update the graph in time) we decided on “tweeted by midnight” as the defining criterion for fulfilling the commitment. So there was simply no weasel room and so we paid up.

We also said, when we wrote this article, “we’d immediately re-up and keep ourselves on the hook,” which is what we’ve done. Another $1000 is on the line with nary a day’s respite! (Which, we should emphasize, is all far harsher than is the norm for Beeminder, but for this flagship Beeminder goal we like the extreme stringency.)

UPDATE: Funny, delightful, and poignant follow-up guest post by Henrik Wist who won our $1000: blog.beeminder.com/bikes.


 

Related Reading and Links


 

Footnotes

[1] What Beeminder looked like 1000 UVIs ago, if you promise not to laugh:

Screenshot of Beeminder.com (then Kibotzer.com) from 1000 days ago

[2] We’re not actually a fan of this style of commitment device, though we do have a competitor that takes that approach. Hollywood teaches us to make grand commitments; real life teaches us that little-and-often wins. We find that the motivation of a single event will wear off, that goals change, and that you need just the right amount of inflexibility.

[3] The graph itself isn’t the most important part in this case — we want to convince you first of all to create a public changelog and commit to changing it, publicly. That alone is a commitment device if you have users watching, though surprisingly easy to get behind on, deluding yourself that you’ll catch back up soon, until it’s hopeless. If you want to really get yourself on the hook and follow our example, sign up for Beeminder and create a new goal — a “Do More” goal — committing to one user-visible improvement per day (7 per week, or maybe start with 5 per week, or even just 3 per week to start out — you can always dial it up later).

[4] Two clarifications: We don’t yet have annotations like that. Those were added in post-production. But we’ll surely have them within 1000 days/UVIs from now! And, two, you don’t have to put $1000 at stake like that off the bat. In fact, you don’t put any money at stake at first, nor do you ever have to as long as you keep all your datapoints on your Yellow Brick Road.

[5] We hold ourselves to a stricter standard for our own meta Beeminder contracts. For our users, you can always just tell us that a derailment was due to a technicality and we’ll believe you and not charge you. For ourselves we have some stringent fine print. But we should also point out that Beeminder contracts are much less scary than they sound. Certainly less so than our esteemed competitor, StickK.com. Part of the beauty of Beeminder is that you can reassess your commitment or end it altogether at any time, with a one week delay. So you can adjust your goal, but you can’t change it out of laziness, unless you’re particularly forward-thinking about your laziness!


 

Tags: , , , , , , , , , , , , , , ,

  • Danny G

    How would things be different if they were user visible or invisible? What about goals like commenting your code so you can understand it later? That’s not user visible.

  • http://beeminder.com Daniel Reeves

    Good question! Repeating from the Hacker News comments:

    We did recently decide to *also* commit to non-visible improvements (refactoring, upgrades, unit tests, etc). We’re calling them infrahancements: http://beeminder.com/meta/infra and will tweet those at http://twitter.com/beeminfra

    Commenting and internal documentation, as you say, is another good infrahancement example.

  • Robert Felty

    If a beeminder employees pays on a goal, where does the money go to? I am assuming you don’t give it to beeminder, since this would be kind of like giving yourself money (once you get to 1000 employees, it won’t be like that so much, but when you only have a few, it would be like that.)

    Since I don’t know the answer yet, here is a suggestion – if a beeminder employee fails on a “meta” goal, then the money should go to a competitor, such as Stickk!

  • http://beeminder.com Daniel Reeves

    Ah, the answer is http://blog.beeminder.com/blogdog

    And note from the post here, “We have coughed up hundreds of dollars to our users on other meta graphs — recently $270 for not getting a blog post out on time, and $90 each for our ’40 hours/week on Beeminder’ goals — but not yet thousands.”

  • Rodrigo Belo

    Regarding the addendum, now can you live with that level of stress for over two years? One of the reasons why I started Beeminder less frequently was the constant stress. And having multiple goals did not help.

  • http://beeminder.com Daniel Reeves

    I guess I view the stress as intrinsic to the process of running a business (or getting a degree or writing a book or any other endeavor). For some of us the alternative to commitment devices like this (including various real-world deadlines, not just the artificial ones Beeminder lets you create) is to fall flat on our faces. Ie, to simply fail at the endeavor.

    So if the thing you’re beeminding is fundamentally worth doing, then it’s worth the stress of beeminding it. And of course if you’re not happy with the stress-to-productivity ratio you’re achieving, you can dial down the steepness of your yellow brick road.

    But more fundamentally, I think beeminding something like this actually reduces stress. If you take it as a given that we needed to get Beeminder to the state it’s in now, it was much easier to do that in 1000 small daily steps!