Bayesian Willpower

Thursday, April 8, 2021
By dreeves

Pr(workout | Beeminder) = Pr(Beeminder | workout) * Pr(workout) / Pr(Beeminder)

A couple weeks ago, Scott Alexander wrote “Toward a Bayesian Theory of Willpower”. This is my recap of the theory, my tentative verdict, and what I think it means for Beeminder and motivation hacking more generally.

Let’s start with defining terms! Akrasia means failing to do something you rationally want to do. Not just something you “should” do. Something you genuinely want to do, and can do, yet somehow fail to do. Akrasia is the name for that “somehow”. Willpower means overcoming that problem via pure introspection.

Scott Alexander’s theory says that your brain collects and weights evidence from different mental subprocesses to determine every action you take. The three subprocesses are:

  1. Do nothing
  2. Do what’s most immediately rewarding
  3. Do what you consciously deem best

These all submit evidence via dopamine to your basal ganglia and the winner determines what action you take. There’s a high prior probability on “do nothing” being best, which can be overridden by high enough anticipation of reward, which can be overridden by high enough evidence from your conscious mind.

In this theory, akrasia — Scott Alexander says “lack of willpower” — is an imbalance in these subprocesses. Physiologically maybe that means insufficient dopamine in your frontal cortex such that the evidence from your conscious brain is underweighted. (Dopaminergic drugs seem to increase willpower so I guess that argues in favor of the theory? I’m so out of my depth here.) Hacking your motivation would mean increasing the evidence supplied by your intellectual/logical brain.

Hyperbolic Discounting

Many decades before that, various psychologists and economists came up with a dynamic inconsistency / hyperbolic discounting theory. Your brain does whatever maximizes the net present value of expected future utility (i.e., you figure out the optimal thing and then do it) except your future-discounting is broken. That’s the akrasia. You over-weight immediate consequences.

Everything Is A Nail?

Under hyperbolic discounting my answer is “use commitment devices to bring long-term consequences near!”. Under the Bayesian theory, my answer is “use commitment devices to drive up the weight on the intellectual evidence!”. This is suspiciously convenient coming from me, speaking for Beeminder.

But if the answer is the same either way, then what good actually is the Bayesian theory? Does it have any practical applications that we don’t already have with the hyperbolic discounting theory? Is it all just a just-so story with neuroscience window dressing?

It’s Actually All Incentive Alignment

My claim is that from Beeminder’s pragmatic perspective all theories of akrasia converge where the rubber meets the road. Hyperbolic discounting says you over-weight immediate consequences. The Bayesian theory says you over-weight the evidence from your pure reinforcement learner — which only cares about immediate consequences. We can construct other theories. Take Kahneman’s “Thinking Fast and Slow”: your System 1 takes precedence in your decision-making. Every conceivable theory (er, that I can conceive of) involves over-weighting immediate consequences in one way or another.

Immediate incentives are inordinately powerful. Beeminder’s philosophy is to find ways to make your immediate incentives match your long-term incentives such that willpower needn’t come into play at all.

Does that just mean commitment devices? Here’s a list of examples of aligning one’s own incentives that arguably don’t count as commitment devices:

“You’re altering the tradeoffs between your possible future actions, making your short-term incentives (sweet, sweet sirens) better match your long-term incentives (not crashing into rocks)”

  1. Keep a salad (or Mealsquares) in the fridge so it’s the path of least resistance when snack-seeking
  2. Lay out your gym clothes the night before
  3. Tell your friends about your goals so they exert social pressure
  4. Buy equipment/materials that make exercise/learning/whatever more fun or easier
  5. Break nebulous, ugh-y tasks into simpler, shorter, concreter tasks
  6. Use Randall Munroe’s trick of inserting a delay for loading websites
  7. Cap your Beeminder pledge at something cheap to induce yourself to only derail for a good reason rather than fully committing yourself?
  8. Graph your progress and cultivate a dopamine hit for adding data to said graph?
  9. Habitica!

If you squint hard enough you can call all of those commitment devices. Or think of it as a continuum. Say you’re Ulysses and your undesired action is steering your ship into rocks due to siren songs. At one extreme — an unambiguous commitment device — you can tie yourself to the mast to directly remove the undesired action from your possible actions. At the other extreme you can get, say, another boat to go in front of you with pictures of puppies on it so you’re less tempted to steer toward the sirens.

The commonality is that you’re altering the tradeoffs between your possible future actions, making your short-term incentives (sweet, sweet sirens) better match your long-term incentives (not crashing into rocks).

Scott Alexander’s description of what’s going on at the neurological level is fine (unqualified as I am to judge it) but in terms of what it means for motivation hacking, I don’t see a difference. Maybe it’s like how relativity is more correct and general than Newtonian physics but at human-scale Newton suffices. The Bayesian theory of willpower may be more correct and general than hyperbolic discounting, I’m not sure, but for all practical motivational purposes they prescribe the same thing.

All that said, if you have any approaches to aligning your own incentives — upping the weight of your intellectual evidence — that only make sense in the Bayesian paradigm, I want to hear about them!


 

Thanks to Adam Wolf, Bee Soule, and Michael Tiffany for reading drafts of this.

Tags: , , , , ,