Our Label Ontology For Issue Tracking

Wednesday, March 1, 2017
By dreeves

Collage of bug labels

We’re pretty transparent about how we do things here in the beehive. We share revenue graphs and anything else you’d care to know. But I know the big question on everyone’s mind has been “what labels do they use for bugs in their issue tracker?” Well here I am to answer that question! We have 14 labels at the moment. Eight are descriptive and six are the ones Joel Spolsky proposed around the turn of the century in his article on bug tracking as ways to mark an issue closed.

Here’s the list of them, followed by some discussion of why we have each one.

BUG Opposite of feature
REQ Feature request
UVI Will count as a User-Visible Improvement
STY Style/polish/CSS, or think of it as in pigsty or an eyesore
MEN Mendoza = need to resolve before deploying, part of MVP
PEA Easy-peasy
SKY Pie in the sky (would be awesome but not necessarily worth the effort)
ADO Consensus needed on what to Actually Do (or “much ado about ∅”)

zap Fixed
nix Won’t fix
zzz Postponed, aka snoozed
cnr Could not reproduce
dup Duplicate
aok Feature, opposite of bug, “by design”

BUG and REQ are self-explanatory; it’s nice to distinguish between things that are actually buggy vs just new hotness we want. Sometimes that line is fuzzy though and you might tag something as both BUG and REQ if you can’t decide which it counts as.

If you’ve been hanging out in Beeminderland long you know about UVIs. It’s handy for us to mark those for collecting user-visible improvements to tweet about. Or if we’re scrambling to ship a UVI in time we might look for issues marked UVI and PEA — easy-peasy improvements. (The UVI label is actually almost ubiquitous so it might make more sense to have an INF label for issues that aren’t visible to users, i.e., infrastructure.)

The STY label is handy to filter out if you have zero aesthetic sense or are allergic to CSS. Or maybe you’re our web designer (Josh Pitzalis) and want to focus on these.

The idea with marking issues as Mendoza (MEN) is that if it’s part of a new feature branch or a new product, it can’t be merged or released until all the Mendozas are done. The Mendoza Line is a baseball term referring to a minimum threshold of acceptability. It’s similar to the notion of a Minimum Viable Product. (Which is what we mean by ‘MVP’, not to be confused with the baseball term ‘MVP’ which means ‘Most Valuable Player’. This is what we get for mixing baseball and software metaphors!)

GitHub pro tip: See all issues that are closed but not fixed with with a filter like so: is:issue is:closed -label:zap

Anyway, marking something Mendoza is kind of like an OMG label for really important stuff. We used to use OMG but we ruined that one with overuse so we settled on the idea of a Mendoza line since it has a more specific meaning than “definitely do this”. Namely, we’re agreeing not to launch this new thing until everything tagged MEN is resolved (or we relent and un-Mendoza it). In the past we’ve also used the Mendoza label after deploying something. We’d mark all the issues that needed to be resolved before we felt we could shift gears and focus on something else. It turns out that’s a slippery slope. It devolves into an endless sea of “definitely do this” items, just like with an OMG label. So we’ve learned not to do that anymore.

SKY is for pie-in-the-sky, which is mostly self-explanatory. It can mark something as especially awesome but also lower-priority because it suggests that it may not be worth the cost of implementing. If you’re looking for something to actually accomplish, you might want to filter out the SKYs. As I mentioned, PEA is for easy-peasy which is a good place to look if you want a quick win. SKY and PEA are mostly mutually exclusive, though you might tag something as both if you’re totally unsure how hard it might be.

Last in the all-caps descriptive labels is the ADO label, which means “not actionable” or “still in the realm of opinion”. In Trello we indicate these with a separate list called “Ideas”. They’re things requiring clarification or more discussion or need splitting up into separate issues before being circumscribed enough to take action on. Definitely filter out the ADOs when looking for something to actually accomplish.

Always Be Closing

That brings us to Joel Spolsky’s six ways to resolve or close a bug. Those labels are lowercase because they’re lower priority. Well, zero priority, being closed and all. It’s a mnemonic, ok?

Joel also recommends that the fixer of a bug should mark the issue fixed/resolved and only the person who actually opened the issue should close it. We take some liberty there and just say that the person who opened it should at least notice when it’s closed and either agree by labeling it “zap” or “nix” or whatnot or else reopen it with a comment.

What Not To Label

The main rule is to only make a label for something because it’s actually helpful to filter on. Don’t make ontologies for the sake of pure epistemic virtue. Like don’t make tags to mark whether something is a proposed new feature vs a modification of an existing one, or whether it involves the API vs the website. It’s easy to make the label ontology overwhelming and then people won’t use the right labels and the whole point will be defeated. So wait till there’s a clear and present need before creating new labels!

That’s actually a pretty general software principle: don’t ever write code or create systems because you anticipate it being useful. Be lazy and put it off till you can’t live without it. Another manifestation of this principle is the Shirk & Turk Principle.

I mentioned that we abandoned our OMG label for indicating high-priority issues. At one time we concocted a whole schema for assigning priorities to issues before deciding there is simply no point to doing this. For one, deciding priorities is agonizing. And it doesn’t even end up helping with triage. Whatever the higest priority is, you gradually end up with an overwhelming number of those, and everything lower ends up being tantamount to “this one isn’t as important” in which case it’s better to close it as “zzz” (postponed). Or maybe it can be classified as SKY (pie in the sky). At best we use our fancy priority schema when something catastrophic happens, as a concise way to convey the relative importance of various emergency response tasks.

The Future

This has been useful for us to write up — and thank you for indulging us! — but in terms of advice for anyone else it’s a bit self-defeating. I’m telling you you shouldn’t ever create a new label without a clear and present need for it. So that certainly includes a set of labels some other startup or project finds useful.

Some of our labels seem maybe sort of universal but you should also work to keep your ontology minimal. So let me end with some open questions for ourselves as our ontology evolves:

  1. Can we jettison the REQ (feature request) label? It seemed like a logical counterpart to the BUG label but do we ever want to isolate the list of feature requests or filter them out? Maybe not badly enough to be worth the added friction when creating issues!
  2. Can we merge the SKY (pie in the sky) and ADO (too nebulous to be actionable) labels? They’re logically different but both amount to “we’re not sure this is ready to be or should be worked on”.
  3. Do we want a new label for things a non-technical person could resolve? (Maybe “ABC” because these will usually involve prose or webcopy tweaks.) Time will tell!
  4. UPDATE: See the comments for debate about nixing the “nix” label.

Tags: , , , , , ,

  • Kenn Hamm

    Heh, I was prepared to complain that you had ruined the label scheme I set up months ago, but it actually looks pretty similar.

    “Category” labels like “ABC” make sense if there are different people or teams who would be interested in or able to work on particular issues. That can change over time as the people and how they are organized changes.

    When would you ever use “nix” rather than a more specific reason for not fixing something? Joel’s article doesn’t explain this either.

  • http://beeminder.com Daniel Reeves

    Oh, yes, thank you for helping us with the ontology! That was dumb of me to forget to mention your help! The only ones from you that I jettisoned were “user confusion” (I figured those should mostly count as bugs) and “integrations/autodata” (YAGNI — https://en.wikipedia.org/wiki/You_aren't_gonna_need_it — another thing I should’ve mentioned in the post!) and “needs triage” (I figured lack of labels indicates that).

    Good question about when to close with “nix” (aka “won’t fix”). Joel does have an example in his article. The equivalent for us might be a bug in one of our autodata integration partners’ products. Reproducible, not by design, but not something we can fix.

    I think you just convinced me to nix “nix” though. You should either decide it’s “aok” or if it’s out of your hands then close it with “zzz” (postponed) so it’s still in the bug database as an issue and maybe in the future you’ll be able to work around it or it will otherwise resolve itself. Technically you might still want to distinguish between “not ideal but not worth fixing now” vs “not ideal and not worth fixing ever” but I think that’s too fine a distinction. Just put all those things to sleep with “zzz” and worry about them later!

    Thanks again for the help with all this!

  • http://beeminder.com Daniel Reeves

    I’m realizing that, at least for Trello and GitHub, we can always close an issue without giving it a label at all. Which is a lot like closing it as “nix”. So all the more reason for nixing “nix”. The only counterargument I can think of is that it makes it more annoying to filter for all the things closed for no reason:

    is:closed -label:zap -label:zzz -label:cnr -label:dup -label:aok