Quantified Self Talk: Tracking My Personal Reliability

On 2018-09-22 I gave a talk at the Quantified Self conference. This is that talk. You can also see an actual recording of it. I got a lot of encouragement afterwards about how people had a kind of lightbulb moment from it, which was nice to hear, and prompted me to say “I’ll turn it into a blog post”. Which, as you’re about to see, meant I kind of had to do so. So here it is! See also my original blog post about this idea, The “I Will” System.
 
(Abstract: I’ve made it a strict personal rule that every time I utter a statement starting with “I will” to someone, no matter how casually, I log the commitment, with a due date, and keep track of when I follow through. Using a highly over-engineered formula, I turn all that data into a single reliability metric. It’s roughly measuring the probability that any given “I will” statement I make will turn out to be true. But it also accounts for it being kinda true — when I do follow through but not by the deadline I specified.)

Hi, I’m Daniel Reeves, cofounder of Beeminder! This talk is mostly not about Beeminder but the common theme is committing to things. First, in case you don’t know about Beeminder, it’s a tool for ongoing, usually open-ended, quantifiable commitments. Like getting 10,000 steps per day as measured by a Fitbit or 40 hours per week on your job as measured by RescueTime, or staying under 100kg as measured by your Withings scale. Or even number of to-do items marked done in Habitica, or messages in Gmail. Beeminder has dozens of integrations with other Quantified Self tools and less QS-y apps that generate data about you incidentally.

(Oh yeah, and the commitment part is that you put in a credit card and agree to get charged if you don’t keep all your datapoints on track toward your goal every day.)

But what if you want to commit to a specific one-time thing, like “I will send in the slides for this talk”? I noticed that I casually make “I will” statements a lot. Like “I’ll see if I can reproduce that bug” or “I’ll let you know if I can make it to the meetup” or “I’ll send you the photos”.

Pr(dreev does X | dreev says he’ll do X) = 0.968799

It was really bugging me that my future-tense statements could sometimes be falsehoods. So I contrived this system so that instead of making a potentially false statement about what I’ll do I can instead, impeccably truthfully and more informatively, always give an exact probability that I’ll do the thing. My current reliability for doing what I say is about 97%.

Before I explain that, let me give some more background.

If you know much about the behavior change literature then you’ll have heard of implementation intentions, or the much simpler term: trigger-action plans. Here’s one I’m proud of: Any time I hear myself saying I should do something (“I should write more code”, “I should run a marathon someday”, etc) then I take some tiny immediate action that makes the thing more likely to happen.

That’s been working well for me for years, mostly with the effect of me not making “should” statements in the first place and instead rephrasing them like “it might be a good idea to do X but…” and then listing the costs or downsides. I feel like this is still a win, if only because it makes me feel less guilty about all the things I “should” be doing but aren’t.

Last year I had the idea to do something similar with “I will” statements. I started with the following trigger-action plan: Anytime I heard myself make an “I will” statement (even the most casual “cool, I’ll take a look”) I had to explicitly mention a deadline, calendar it, and add a datapoint to a Beeminder graph.

2011: Public launch of Beeminder
2014: The “I Should” system
2017: The “I Will” system
2018: Commits.to (Commit Stew) and promises.to

Initially the Beeminder graph was enforcing that I fulfill 8 promises per week. No stipulation of the denominator, just that I fulfill 8 per week.

That Beeminder graph is still going. But astute observers will notice a few things about it. For one, I backed off from that 8/week and have averaged just over 5/week overall. But more interestingly, there are 364 promises logged here (actually a few less than that because of dummy datapoints each time I derailed and paid Beeminder money) but I only have credit for 348.3 of them. What’s up with that?

Funny you should ask. When I started tracking this I soon realized that although fundamentally I care about the probability, or fraction of commitments fulfilled, I needed a way to quantify fulfilled-but-late. And I wanted to distill it all into a single metric for reliability, because that’s the whole point. So this is what I came up with. I figured if I missed a deadline by mere seconds, that 99.999% counts. If I missed it by some minutes, that still 99.9% counts. Hours late isn’t a big deal, that almost fully counts (99%). Days late is like, well, it got done, that’s the main thing. 90%. Being weeks late kind of half defeats the point so that’s 50% credit. Months late mostly doesn’t count. 10%. And years is like, better late than never, but barely (1%).

So the real nerdery is in defining this function for exactly how much credit you get as a function of how late you are. There are three properties we need: One is strict monotonicity, meaning that you always see your reliability score visibly ticking down second by second whenever you have an overdue commitment. Two is that it asymptotically approaches zero. You’ll always get some epsilon of credit for fulfilling a commitment no matter how late you are. And the clever beehavioral-economic property is that there are sudden drops at every Schelling fence. We don’t want you to feel like, once you’ve missed the deadline, that another hour or day or week won’t matter. That’s a slippery slope to never finishing ever. I can come back to this if people have questions about it but you don’t need to grok this to understand the rest of the system.

alice.commits.to/send_bob_edits_by_tomorrow_5pm

So how do you actually use it? Suppose your name is Alice and your coworker or friend is Bob. Whenever Alice says something like “I’ll send you my edits tomorrow”, she types a URL like so.

As in, she literally types that, on the fly, directly to Bob, manually, when she’s making the commitment to him. We’re assuming they’re talking by email or Slack or whatever. When she or Bob click that URL a commitment is created in the app and an entry is added to Alice’s calendar and ideally a datapoint is sent to Beeminder. The system lets you mark the commitment completed and keeps track of your reliability and shows it off to anyone who follows any alice.commits.to link.

Here’s me telling Steven Jonas that I’d send him my QS slides:

I added a note to remind myself that I promised this in an email to Steven. And again, the UI here was me literally typing the URL in the email to Steven: dreev.commits.to slash send underscore slides etc. Clicking that link logs the commitment which I can then edit or later mark fulfilled.

And if you visit just dreev.commits.to you’ll see this — my gallery of commitments.

Some quick numbers for myself:

That’s my reliability based on 391 commitments, plus 7 that don’t count for or against my score because they’re not due yet. But my reliability is gradually ticking down second by second because I have 10 (11 when I wrote that slide) commitments currently overdue.

Some quick numbers for Commits.to overall: We have about 10 active beta users, with a bit over 1000 commitments in the database altogether. And we’ve been working on this for exactly a year now.

It can be relaxing to put faith in a system that ensures you’ll do what you need to when you need to

I’ve learned a lot from working on this and using it. Like in a way it adds stress yet is weirdly relaxing, I think for a similar reason that people find David Allen’s Getting Things Done system relaxing. But there are definitely failure modes. Of course one lesson, which was part of the point all along, is to promise less. Break the habit of saying you’ll do something unless you really mean it. Here’s our friend Kim making that point:

“Commits.to has given me reason to pause before I rattle off an empty promise. As a people pleaser, I was in the habit of saying I would do things without a real intention to follow through on those statements in a consistent way. Now, I value maintaining my overall reliability instead of my agreeability in the moment.”
        — Kim Harrison, second most hardcore beta user

Another lesson is to promise specifics. I’ve been burned by promising things that seemed straightforward but turned out to be ill-defined. Better to promise a very concrete first step and then make another promise for the next step if it still makes sense. And finally, the idea of commits.to is to be motivated by your reliability score and seeing it tick down when you miss a deadline… but… for me anyway, that’s not enough. My score has been slipping lately and despite those Schelling fences for the individual scores, each individual score doesn’t contribute much to my overall reliability and so I’m still on a slippery slope. Which just means I still need Beeminder, so that’s good.

As with Beeminder, I think it’s only a certain kind of personality who wants to put themself through this kind of thing, but I can only assume that that personality type is overrepresented here. This thing is still in closed beta, but if you want to commit to help building it, you can start using it now. Just try yourname.commits.to and you’ll either see someone’s gallery of commitments if that name is taken or you’ll see instructions for how to learn more and get involved.

Thank you!