The Beatles: Rock Band’s tenth anniversary
The Beatles: Rock Band was released ten years ago today. It’s still my favorite game I’ve ever been a part of.
Since I was still in QA during Beatles development, here’s a little anecdote about our process.
Our QA department assigns each bug a severity and a priority. Severity specifies how serious a bug is: a crash is high severity, for instance, while a UI sound effect that’s too quiet is low severity. Meanwhile, priority specifies how “important” a bug is. Priority is usually negotiated between QA and production, and it determines the order in which developers work on fixes.
Severity certainly influences priority, but they’re not interchangeable. Imagine that you’re a producer and your QA team discovers a crash. If the crash is common and easy to replicate—say, every time you go into the Settings menu the game locks up—then that’s an easy call: high-priority bug.
But what if the crash only happens one every hundred tries? Or one every thousand? The rarer it is, the more likely you are to lower the bug’s priority. In other words, at a certain point you’d rather your developers spend their limited time fixing a bunch of more common issues—even less severe ones!—than continue to chase down a single crash that most people won’t encounter.
The opposite case is more unusual: we don’t often see a bug with a very low severity but a very high priority. On Beatles we did, though, because our partners came by the studio to review our progress and give comments. And by partners, I mean actual relatives of the Beatles.
On one occasion I remember, Yoko Ono had some feedback about John’s appearance. I believe one of her notes was that his nose didn’t seem right, although I don’t recall the specifics.
Technically, that was what we in QA called a suggestion. Suggestions were logged as bugs, but they received the lowest possible severity in our system since they aren’t actual specific defects. Most suggestions end up being waived—that is, ignored—because there are inevitably enough real problems late in development that no one has time to address them.
But if Yoko Ono has suggestions about John’s appearance, you probably want to take them seriously—“defect” or not. Thus: the lowest possible severity, but the highest possible priority. We changed his nose.
Fun story aside, that commitment to quality is one of the reasons I love Beatles so much. There was an enormous amount of pressure from everyone—up to and including the band members’ spouses and children—to get things right, and our standards have never been higher. It shows in the final product, I think. I still keep my plastic Rickenbacker around, just in case.