Our Label Ontology For Issue Tracking

Wednesday, March 1, 2017
By dreeves

Collage of bug labels

UPDATE 2018 March: We made some changes! Like changing “REQ” for “feature request” to the semi-standard “RFE” for “request for enhancement” (because the former looked too much like “required” or “requisition”). And adding the “ABC” label for non-technical contributors. See also the comparison with GitHub’s default labels at the end of the post.

We’re pretty transparent about how we do things here in the beehive. We share revenue graphs and anything else you’d care to know. But I know the big question on everyone’s mind has been “what labels do they use for bugs in their issue tracker?” Well here I am to answer that question! We have 15 labels at the moment. Eight are descriptive and six are the ones Joel Spolsky proposed around the turn of the century in his article on bug tracking as ways to mark an issue closed.

Here’s the list of them, followed by some discussion of why we have each one.

BUG Opposite of feature
RFE Request For Enhancement, aka feature request
UVI Will count as a User-Visible Improvement
STY Style/polish/CSS, or think of it as in pigsty or an eyesore
MEN Mendoza = need to resolve before deploying, part of MVP
PEA Easy-peasy
ABC Non-technical, like prose or webcopy tweaks
SKY Pie in the sky (would be awesome but not necessarily worth the effort)
ADO Questions about what to Actually Do (or “much ado about ∅”)

zap Fixed
nix Won’t fix or invalid
zzz Postponed, aka snoozed
cnr Could not reproduce
dup Duplicate
aok Feature, opposite of bug, by design

BUG and RFE are self-explanatory; it’s nice to distinguish between things that are actually buggy vs just new hotness we want. Sometimes that line is fuzzy though and you could tag something as both BUG and RFE if you can’t decide which it counts as.

If you’ve been hanging out in Beeminderland long you know about UVIs. It’s handy for us to mark those for collecting user-visible improvements to tweet about. Or if we’re scrambling to ship a UVI in time we might look for issues marked UVI and PEA — easy-peasy improvements. (The UVI label is actually almost ubiquitous so it might make more sense to have an INF label for issues that aren’t visible to users, i.e., infrastructure.)

The STY label is handy to filter out if you have zero aesthetic sense or are allergic to CSS. Or maybe you’re our web designer (Josh Pitzalis) and want to focus on these.

The idea with marking issues as Mendoza (MEN) is that if it’s part of a new feature branch or a new product, it can’t be merged or released until all the Mendozas are done. The Mendoza Line is a baseball term referring to a minimum threshold of acceptability. It’s similar to the notion of a Minimum Viable Product. (Which is what we mean by ‘MVP’, not to be confused with the baseball term ‘MVP’ which means ‘Most Valuable Player’. This is what we get for mixing baseball and software metaphors!)

GitHub pro tip: See all issues that are closed but not fixed with with a filter like so: is:issue is:closed -label:zap

Anyway, marking something Mendoza is kind of like an OMG label for really important stuff. We used to use OMG but we ruined that one with overuse so we settled on the idea of a Mendoza line since it has a more specific meaning than “definitely do this”. Namely, we’re agreeing not to launch this new thing until everything tagged MEN is resolved (or we relent and un-Mendoza it). In the past we’ve also used the Mendoza label after deploying something. We’d mark all the issues that needed to be resolved before we felt we could shift gears and focus on something else. It turns out that’s a slippery slope. It devolves into an endless sea of “definitely do this” items, just like with an OMG label. So we’ve learned not to do that anymore.

SKY is for pie-in-the-sky, which is mostly self-explanatory. It can mark something as especially awesome but also lower-priority because it suggests that it may not be worth the cost of implementing. If you’re looking for something to actually accomplish, you might want to filter out the SKYs. As I mentioned, PEA is for easy-peasy which is a good place to look if you want a quick win. SKY and PEA are mostly mutually exclusive, though you might tag something as both if you’re totally unsure how hard it might be.

ABC is for prose and copy. Maybe you don’t speak great English and want to steer clear of these, or you don’t speak the programming language and these are the only issues you can tackle.

Last in the all-caps descriptive labels is the ADO label, which means “not actionable” or “still in the realm of opinion”. In Trello we’ve indicated these with a separate list called “Ideas”. They’re things requiring clarification or more discussion or need splitting up into separate issues before being circumscribed enough to take action on. Definitely filter out the ADOs when looking for something to actually accomplish.

Always Be Closing

That brings us to Joel Spolsky’s six ways to resolve or close a bug. Those labels are lowercase because they’re lower priority. Well, zero priority, being closed and all. It’s a mnemonic, ok?

Joel also recommends that the fixer of a bug should mark the issue fixed/resolved and only the person who actually opened the issue should close it. We take some liberty there and just say that the person who opened it should at least notice when it’s closed and either agree by labeling it “zap” or “nix” or whatnot or else reopen it with a comment.

What Not To Label

The main rule is to only make a label for something because it’s actually helpful to filter on. Don’t make ontologies for the sake of pure epistemic virtue. Like don’t make tags to mark whether something is a proposed new feature vs a modification of an existing one, or whether it involves the API vs the website. It’s easy to make the label ontology overwhelming and then people won’t use the right labels and the whole point will be defeated. So wait till there’s a clear and present need before creating new labels!

That’s actually a pretty general software principle: don’t ever write code or create a system because you anticipate it being useful. Be lazy and put it off till you can’t live without it. Another manifestation of this is the Shirk & Turk Principle.

I mentioned that we abandoned our OMG label for indicating high-priority issues. At one time we concocted a whole schema for assigning priorities to issues before deciding there is simply no point to doing this. For one, deciding priorities is agonizing. And it doesn’t even end up helping with triage. Whatever the higest priority is, you gradually end up with an overwhelming number of those, and everything lower ends up being tantamount to “this one isn’t as important” in which case it’s better to close it as “zzz” (postponed). Or maybe it can be classified as SKY (pie in the sky). At best we use our fancy priority schema when something catastrophic happens, as a concise way to convey the relative importance of various emergency response tasks.

The Future

This has been useful for us to write up — and thank you for indulging us! — but in terms of advice for anyone else it’s a bit self-defeating. I’m telling you you shouldn’t ever create a new label without a clear and present need for it. So that certainly includes a set of labels some other startup or project finds useful.

Some of our labels seem maybe sort of universal but you should also work to keep your ontology minimal. So let me end with some open questions for ourselves as our ontology evolves:

  1. Can we jettison the RFE (feature request) label? It seemed like a logical counterpart to the BUG label but do we ever want to isolate the list of feature requests or filter them out? Maybe not badly enough to be worth the added friction when creating issues!
  2. Can we merge the SKY (pie in the sky) and ADO (too nebulous to be actionable) labels? They’re logically different but both amount to “we’re not sure this is ready to be or should be worked on”.
  3. Do we want a new label for things a non-technical person could resolve? (Maybe “ABC” because these will usually involve prose or webcopy tweaks.) Time will tell! UPDATE: A year later, we’re tentatively adding it.
  4. UPDATE: See the comments for debate about nixing the “nix” label.

UPDATE: Comparison with GitHub’s Default Labels

If you create a fresh GitHub repository, you get the following labels by default. Here’s how they map to our own labels:

RFE   (Request For Enhancement)
ADO   (Questions to decide what to Actually Do)
wontfix   nix

good first issue
We use these labels as is because GitHub helps potential
first-time contributors discover issues with these labels
help wanted


Tags: , , , , , ,