Attention conservation notice: This is of no actual relevance to Beeminder users. Except the ones who are software developers, which in fact seems to be most of you. (Also AI seems to be drastically lowering the barrier to entry for creating software, so maybe this is about to be relevant for everyone.) Or maybe you just like these random peeks behind the curtain.
Let me get to the point. I dislike the term “tests” (as in unit tests and test-driven development — code that verifies the correctness of other code). What’s my beef with this perfectly innocuous and perfectly descriptive word? First, it has the wrong connotations. You say something is “just a test” to mean it’s a one-off trial. Your software’s meta-sofware for quality assurance and anti-regression infrastructure is not “just some tests” in that sense.
More pragmatically, the word “test” is just too common and overloaded. You can’t search GitHub for Issues that mention “tests”. The results are overwhelmed by mentions of tests-as-in-experiments, offers to test something out, the Javascript regex checking function, pen-testing, developers with chips on their shoulders saying “don’t test me, motherforker”, … Am I exaggerating? On that last one, yes, but I did test this in Beeminder’s bug database, searching the word “test”, and got back 275 gissues (as we call them, for GitHub Issues).
It’s similar for grepping the codebase for the string “test”: 316 lines. That’s more than twice the number of lines mentioning the thing we actually care about.
How do I know? Because we call them quals. It’s an infinitely better term! “Qual” already means “test” in some contexts, as an abbreviation for qualifier. Like how you have to pass your quals before you’re a Ph.D. candidate. It also evokes quality assurance, which makes sense for software.
Then there’s verbability. With “test” you’ve got “I tested that” but it’s impossibly ambiguous. Did you just try it or did you actually qualsify it? [1]
This is part of my broader jargon rant for a future blog post, about how confusing it can be when technical terms also have everyday meanings. Let me tide you over with this excerpt from our old blog post announcing the terminology change from “yellow brick road” to “bright red line”:
Often it’s worth introducing jargon because the plain English version is, ironically, too understandable. I know that sounds ridiculous but that’s because you’re not understanding me yet. The epiphany for me was the following quote, tenuously attributed to George Bernard Shaw: “The single biggest problem in communication is the illusion that it has taken place.” If you say, for example, “max safety buffer” then users, understanding each of those words, will jump to the conclusion that they understand the phrase. But “autoratchet” means something very specific in Beeminderland and it’s less confusing for users to hear “autoratchet” and know that they don’t know what it means than to hear “max safety buffer” and think they know but be wrong. As Mark Twain supposedly said, it ain’t what you don’t know that gets you into trouble. It’s what you know that ain’t so.
What about “Specs”?
We had a whole debate about this in the forum. I think it’s a fine choice. I’ve come to like “quals” better, mainly because specs are both more specific and overlap with another concept I’m enamored with:
- “Spec” definition 1: The document that states the goals or the problem being solved and how all the code will work from a user’s perspective.
- “Spec” definition 2: Code that verifies that functions given known input produce expected output, or that some sequence of UI interactions yields the expected program behavior.
But it’s not at all crazy to overload the term to just mean both of those. They’re pretty related. One is a document in English specifying how future code will function and the other is code that verifies the behavior of existing code. Robert Perce argues that thinking of those two senses of “spec” as the same thing leads to design and usage insights that improve both kinds of specs.
Uphill battles
“Spec” in this sense even has some traction in the wider software world and so might’ve been less of an uphill battle than “qual”. We’ve gotten used to saying “quals” now and I like it and hope it catches on. But realistically, it’s hard to argue it was worth the effort, especially if it doesn’t catch on anywhere else. Mostly I was just annoyed by ambiguities we’d run into talking about “tests” and was nerd-sniped by the onomastic challenge of finding something better.
Footnotes
[1] In some engineering contexts, “qualify” as a transitive verb already means running a set of checks to demonstrate that something meets its requirements. So, yes, you could be less weird and verbify “qual” as “qualify”, but we are not in the business of being less weird. (Also I think this existing sense of “qualify” is another point in favor of the term “quals” to refer to a software test suite.)