This is part 3 of a 3-part blog series! Previously we defined Pareto dominance and what that means for software, and then we made our case for and committed to the Pareto Dominance Principle. This post demonstrates what we have in mind when we say we’re committing to publicly justifying every non-Pareto-dominant change to Beeminder. Except then we ended up Pareto dominating the old version after all. Best of all worlds! (A version of this post previously appeared in the Beeminder forum but this one’s a fair bit more polished.)
Notice anything different about the “Set Up Your Pledge” screen when creating a new goal? (And do you have a very good memory because it’s now been over a month since we changed this, since we decided we needed a whole blog series before announcing this change?) For starters there are a bunch of copy tweaks, like retitling the screen to “Pledge Your Money”, and referring to “starting stakes” and “pledge cap”, neither of which were on that screen before:
You probably don’t care about the UI-copy but conceivably you care about one small functional difference: We replaced that +/- stepper for picking your pledge cap with radio buttons to choose between your default pledge cap (which you can change in your account settings, by the way) and one level higher. Like this:
How is less choice an improvement? Well, I was convinced that people were breezing through goal creation and not being thoughtful about picking a pledge cap. Or not understanding what it even meant. I think a common newbee reaction was something like:
Um, max pledge? I don’t know what’s going on yet so… the smallest possible one I guess?
Conveying all the reasons to stick with a high pledge cap — low starting stakes is of course a-ok — was not realistic in that interface. We really want users to not worry their pretty heads about pledge caps at this point. The standard way for a UI to convey “you can change this if you really want but if you’re just breezing through (of course you’re just breezing through) then just ignore this” is to down-pop it. But, for reasons I’ll describe shortly, we needed to keep the pledge cap fairly in-your-face.
So we decided to paternalistically impose a high pledge cap.
But not really impose it. If you want a lower pledge cap, you can always have it by clicking the dollar amount above your graph (or in settings — we have a separate UI problem that the clickability of things in the graph header is not obvious). It’s not a lot of friction but I think it elegantly ensured that if you’re picking a lower pledge cap, you’re doing it intentionally.
And again, you can also set your global default pledge cap in your account settings to further remove friction on goal creation. Like if you want to never risk more than $5. (And you can still bump pledge caps both up and down any time. Only Beemium subscribers can bump up pledges, and reducing pledges takes a week to take effect, but anyone can bump pledge caps up or down instantly.)
I admit this wasn’t based on hard data, which is very hard to interpret without proper bucket testing (aka randomized controlled trials) anyway. It was mostly based on introspection. I, as a user, would find myself breezing through the interface, stepping the pledge cap down to the minimum just because it was easy and I didn’t want to agonize about how much I should potentially stake on what was often a very experimental goal.
Just Kidding
At this point I asked users (via the daily Beemail) what they thought and got brilliant feedback that changed my mind.
“No amount of putting the pledge cap in a flashing huge font would make users pay attention to it. The more you try, the more users tune it out.”
One bit of feedback I had anticipated was: why bother including a choice of pledge cap at all on that screen? If the place to cap pledges is the goal page, fine, but it’s almost taunting in how little choice it gives you as part of goal creation. The answer was that we had another constraint, learned the hard way: If you don’t make the escalating pledges super obvious at goal creation then some users (have I mentioned how common it is for users to breeze through interfaces without reading or thinking?) would remember clicking $5 but not remember (or have read) any text about the pledge automatically increasing from there.
We used to get some very unhappy responses to legit check emails when people finally noticed the $10 and $30 charges.
So why not just up-pop the pledge cap when creating the goal, you ask? Are you even listening to me here? Users do not read the words in interfaces! No amount of putting the pledge cap in a flashing huge font would make users pay attention to it. The more you try, the more users tune it out. But these radio buttons for pledge cap, nice and parallel with the starting stakes radio buttons, convey exactly what you’re getting yourself into in terms of risking your money.
Or so I rationalized.
It turns out it really was too much of a seeming jerk move to give you a scrap of a choice like that. And the problem of getting you to actually notice the increasing pledge schedule wasn’t that hard to solve. We came up with this:
That seemed to solve the original problem, but it was bugging us that we lost a bit of functionality, namely, full control of your pledge cap directly as part of goal creation. (And God forbid we make a non-Pareto-dominant change and have to write a whole blog post about the whole decision process!) So we decided to update it one more time, like so:
The functional difference is that the pledge cap now has a little dropdown arrow and when you click it you get the original stepper back:
So we’re pretty sure this Pareto dominates the original version. Unless you want to quibble about the one additional click, which, given the whole big deal we made out of the Pareto Dominance Principle, you probably should! Except now we’ve blogged about it so our hands are clean.
In any case, I believe we’ve nicely solved the problem of users breezing through goal creation and picking pledge caps that defeat a lot of the power of Beeminder.
Now we just have to decide how important it is for the vertically laid-out pledge schedule to dynamically adjust when you change the pledge cap. I feel a little silly because this was supposed to be a simple UI tweak with a high bang/buck ratio but, well, fussing begets fussing and here we are.