Attention conservation notice: This is of no actual relevance to Beeminder users. Except the ones who are software developers, which in fact seems to be most of you. (Also AI seems to be drastically lowering the barrier to entry for creating software, so maybe this is about to be relevant for everyone.) Or maybe you just like these random peeks behind the curtain.
Let me get to the point. I dislike the term “tests” (as in unit tests and test-driven development — code that verifies the correctness of other code). What’s my beef with this perfectly innocuous and perfectly descriptive word? First, it has the wrong connotations. You say something is “just a test” to mean it’s a one-off trial. Your software’s meta-sofware for quality assurance and regression-prevention infrastructure is not “just some tests” in that sense.
More pragmatically, the word “test” is just too common and overloaded. You can’t search GitHub for Issues that mention “tests”. The results are overwhelmed by mentions of tests-as-in-experiments, offers to test something out, the Javascript regex checking function, pen-testing, developers with chips on their shoulders saying “don’t test me, motherforker”, … Am I exaggerating? On that last one, yes, but I did test this in Beeminder’s bug database, searching the word “test”, and got back 275 gissues (as we call them, for GitHub Issues).
It’s similar for grepping the codebase for the string “test”: 316 lines. That’s more than twice the number of lines mentioning the thing we actually care about.
How do I know? Because we call them quals. It’s an infinitely better term! “Qual” already means “test” in some contexts, as an abbreviation for qualifier. Like how you have to pass your quals before you’re a Ph.D. candidate. It also evokes quality assurance, which makes sense for software.
Then there’s verbability. With “test” you’ve got “I tested that” but it’s impossibly ambiguous. Did you just try it or did you actually qualsify it? [1]
This is part of my broader jargon rant for a future blog post, about how confusing it can be when technical terms also have everyday meanings. (UPDATE: The sequel has arrived! Against Pseudovernacular Jargon.)
What about “Specs”?
We had a whole debate about this in the forum. I think it’s a fine choice. I’ve come to like “quals” better, mainly because specs are both more specific and overlap with another concept I’m enamored with:
- “Spec” definition 1: The document that states the goals or the problem being solved and how all the code will work from a user’s perspective.
- “Spec” definition 2: Code that verifies that functions given known input produce expected output, or that some sequence of UI interactions yields the expected program behavior.
But it’s not at all crazy to overload the term to just mean both of those. They’re pretty related. One is a document in English specifying how future code will function and the other is code that verifies the behavior of existing code. Robert Perce argues that thinking of those two senses of “spec” as the same thing leads to design and usage insights that improve both kinds of specs.
Uphill battles
“Spec” in this sense even has some traction in the wider software world and so might’ve been less of an uphill battle than “qual”. We’ve gotten used to saying “quals” now and I like it and hope it catches on. But realistically, it’s hard to argue it was worth the effort, especially if it doesn’t catch on anywhere else. Mostly I was just annoyed by ambiguities we’d run into talking about “tests” and was nerd-sniped by the onomastic challenge of finding something better.
Footnotes
[1] In some engineering contexts, “qualify” as a transitive verb already means running a set of checks to demonstrate that something meets its requirements. So, yes, you could be less weird and verbify “qual” as “qualify”, but we are not in the business of being less weird. (Also I think this existing sense of “qualify” is another point in favor of the term “quals” to refer to a software test suite.)