That’s right my little bees: we put a commitment device in our commitment device to bring out the commitment flavor of the commitment device.
Does everyone know the soup nazi from Seinfeld? Basically it’s an interesting episode in Seinfeldnomics (and boy howdy do we like economics) where there’s a soup stand that gets really popular, but the guy selling the soup fires customers if they don’t order correctly, and so Elaine and Jerry and the gang start referring to him as “the soup nazi”. Which I guess is funny because he’s refusing to do business with people who want to give him money? But clearly his business is doing just-fine-thank-you. And actually, well, he should clearly just hike the price on his soup until his supply meets the demand without him getting all stressed out and cranky at the customers. But maybe he likes screaming at people? In which case maybe the status quo is just fine from his point of view.
Anyway, I digress. We were talking recently about doing experiments on all of you. Like na[oops, no, we are being told in no uncertain terms that we cannot make this joke]! In particular we were talking about the trade-off between expected value of any particular experiment and expected cost. If something has a small chance of having a large effect, but it’s cheap and fast to do, then why not try it out?
A couple years ago we tried out what would happen if we just killed the free plan — signing up meant coughing up for a premium subscription on day one. Answer: signups plummeted. So we undid that. And it led us to concluding that Beeminder should be pledge-focused. The year before that we tried the commitwall which was an absolutely massive success. Learning things!
We’re not exactly short on problems to solve, but the one that’s been perhaps most salient of late is improving our onboarding gulf. We still get a lot of people who sign up for Beeminder but never get around to creating any goals at all. We’ve got a nice list of Obvious Things To Try, like writing interactive tutorials, a series of onboarding emails that keeps re-prompting users through getting started, office hours, building goal creation into sign up, just overhauling goal creation in general, making clonable goals, etc etc. These are all great ideas. But they’re all pretty high cost to implement.
Sidebar on the ideal solution to people signing up for Beeminder and then failing to create a goal
(Besides improvements to the goal creation process itself — also key!)
We’ve internally been calling this Cantor’s Compromise: You have to create a goal as part of creating a Beeminder account, and if you don’t have something in mind yet then the mandated goal is a metaminder goal to add at least one datapoint per week across your other Beeminder goals. As with the current gateway drug commitment device, that also makes for a one-week countdown to get your first goal created.
Bigger picture, we want to give people the tools to solve their real-world incentive-misalignment and if the gateway drug commitment device ends up feeling aligned (haha) with that then maybe a meta-goal can be even more aligned. If done right, which is a huge if, then it can serve as an immediate intro to how Beeminder works, including showing the user that they’re free to hit archive if they don’t actually want the ongoing commitment to use Beeminder itself.
We don’t want to force the user to do anything they don’t want to force themself to do. We want to hold the new prospective user’s hand and make sure we harness the burst of motivation that has them creating a Beeminder account. We want to make sure that indecision/procrastination/ADHD doesn’t snatch them back out of our claws. If we get them set up with a long-term goal for the thing they actually care about, perfect. If they’re so sold on Beeminder that they’re game to beemind their beeminding off the bat, also perfect. If they’re shilly-shallying then we should… make that very uncomfortable. Possibly to the point of saying you can’t actually create this Beeminder account without being on the hook in some way.
We don’t know yet how that will play out, just that if we end up loving the gateway drug then maybe we’ll want to implement it in an even more beeminder-y way.
So then we came up with a crazier thing to try, that has a smaller chance of being a good idea, but is much simpler to implement than any of those things: Signing up for Beeminder is itself a commitment device.
If you’re here, checking us out, you’re probably into the idea of a commitment device. So the idea is that by signing up, we are putting you on the hook to make a goal after signing up, or we will charge you money. And if you’re not on board with that, you might not be in Beeminder’s target audience anyway. (In case you are wondering, this is related to the soup nazi because it intentionally turns away a subset of potential customers.)
More reasons for soup-naziing signups
Why in tarnation would we want to make it harder to sign up? Let us count the ways!
- Learningness. For example, our current CAPTCHA asks what is the thing you lose when you derail (hint: it rhymes with the gooey stuff bees make). People commonly skim that and say “honey” and then they’re forced to read it more carefully and in the process we introduce them to a bit of Beeminder lingo and make sure they’re clear on the commitment device thing.
- Investment. Like the free-form field where we ask you what you intend to beemind. See Danny’s thoughts on this in the forum regarding Noom, which embraces intentional signup friction to a truly ridiculous degree. (And see the end of that thread for an experiment Duolingo did in which they concluded that it can be good to add intentional friction to signup if it makes you introspect about your goal in using Duolingo.)
- Filtering. Customer support is better and easier and more pleasant if we filter for high-quality users. This also matters for the general quality of the Beeminder community.
- Onboarding. It’s painfully common for a user to sign up, immediately feel overwhelmed, and never create a single goal. Or create a half-assed one but delete it in the first week or make it stupidly easy or fail to think through a good metric to mind or otherwise fail to get themselves meaningfully on the hook. We’re attacking this from a few angles but one is to give the user an additional incentive to push through that overwhelm and figure it out.
We threw some extra clarification into the registration form:
![Screenshot of part of the signup form. The Serious Part. The very act of signing up for Beeminder is itself a commitment device. If you don't start a goal by [date] you'll be charged $5. Questions? To make sure you're on board with that, please say in your own words: (1) at least one Beeminder goal you intend to create, and (2) that you agree to pay $5 if you don't set up a goal by the deadline. Placeholder test: I intend to beemind knitting tiny party hats for bees so I have 100 completed by August. I'm on board with getting that goal created in a week or beeing charged $5! (Error text: Please use the literal string '$5' as part of your explicit consent to the meta commitment device.)](https://user-images.githubusercontent.com/9928/224239126-b4849a62-c5f2-4f60-b9b9-ecccb3cd4514.png "HT Nicky and Madge for the placeholder text")
That “Questions?” link has these additional details in a popup:
![The meta commitment device [moneybag-emoji]. We're 1000% serious about signing up for Beeminder being a commitment device in itself. But we work hard to make sure it's an effective one and that we don't take your money unless you agree we should. * First, you can always email us at support@beeminder.com if you have any trouble! And we'll email you 24 hours before we take your money, so you've got time to let us know if something went wrong. * Another safeguard is that you can delete your whole account instantly in the first week if you're really not liking it. (It's easy after the first week as well, it's just no longer instant since that would defeat the point of the commitment device!) * If I were, hypothetically, to get charged that $5, is it game over? * That really should be hypothetical — it's not actually hard to create a Beeminder goal! But just in case, the answer is no, we'll just plaintively ask you, “if not now, when??” and hope that that finally lights the fire under your butt.](https://user-images.githubusercontent.com/9928/224252758-cd5bd00e-fcdd-45fb-bd3a-79a31bea6a2e.png "Apologies for the screenshot of pure text. We hate when people do that. As penance we dutifully pasted every word of it as plaintext into the alt text (not to be confused with the hover/title text that you're reading now. Also, hey, thanks for reading our hovertext!")
We make you type out what you want to beemind, and agree to the meta commitment device. Once you’re all signed up that intention gets sent to the workerbees and they say howdy and prod you to get on it if you’ve not created a goal already. That opens up a handy channel to talk to a human if you’re confused. And now you’ve got Damocles’s $5 hanging over your head prompting you to actually follow through.
This was super peasy to do because
- we were already collecting the intentions and sending them to the workerbees, who
- already welcome newbees and offer assistance with getting started for people who are stuck.
So pretty much all we had to do with registration was write some words in, and add a new check for the existence of the string “$5” in your “intentions” response. We also added a nice little countdown to the empty gallery page et voila.
Anyhow, it’s not the most uncontroversial thing we’ve tried but mostly no one thought it was too crazy of an idea so, as of last week, we’re trying it!
It has dampened signups a little but that seems to be more than made up for in getting people to actually create goals. Of the people that signed up in the past week since we deployed this, 22% have not created any goals. We don’t have enough data yet on how many people are going to hit that one-week wall and actually get charged — only one person so far! The percentage with no goals at all in the two weeks immediately prior to that was 58%.
If it continues to go well we’ll be tempted to bump up the amount at risk to $10 or $20 instead of the current $5. Or jump to the logical conclusion of all this and require the creation of a meta goal as part of signup. But getting that right would take a lot more thought than this simple gateway drug commitment device did.