Home Home

On bug tracking systems

Arseni Mourzenko
Founder and lead developer, specializing in developer productivity and code quality
137
articles
November 15, 2018

There are no good bug tracking systems, and I'm about to explain what makes them not good and why good systems weren't developed yet.

To begin, let's illustrate a journey of two persons working on the same team: one of the programmers and a product owner.

One Monday morning

It's Monday morning. The product owner is back from two weeks vacation. He arrived at the office, marked as read his five hundred and fifty e-mails, and now, he wants to know two things: what was done during his absence and what is the current state of the project. He launches Jira and tries to figure out if there is anything useful he can extract from the board. He stares on the screen for half a minute, locks his machine and goes to the neighbor office to talk to another member of the team who arrived early this morning—this seems to be the only way to get up to date; thanks anyway, Jira.

Shortly after that, the other subject arrives. The programmer remembers that she was working in pair on a ticket with Jeff, but lots of things happened during the weekend, so she would enjoy a little help from someone to quickly find what was the thing she was working on. She, too, stares on the Jira board showing around forty tickets for the current sprint, seven being in the “In development” column, two being assigned to Jeff, none to her. Among the two, both are flagged, and don't look at all like the stuff from Friday evening. That's strange. Oh, now she remembers. There was a ticket assigned to George, and George was pairing with Jeff. Then George was in charge of an issue in production, and Jeff needed some help with MQS, so he called Evelyn. Ticket BWS-5044 it is. Time to do some coding.

Meanwhile, the product owner got the information he was looking for. Back to his office, he checked the Jira board for the second time, and noticed that the ticket assigned to Evelyn and flagged for impediment two and a half weeks ago still has its flag, which contradicts what he just learned from his colleague. Evelyn is not one of those who never updates the board, so there should be a problem; time to go see her.

Evelyn confirms that the API team claimed that they solved the issue with the return values; if this is true, the impediment doesn't exist any longer. However, she couldn't check yet if the resolution of the issue is effective, and, frankly, for the past year, each and every time they claimed they solved something, it appeared that either they didn't, or they did it wrong, or they really did it, but forgot to push their changes to production. Should we remove the flag? Unless we are sure the issue is solved, maybe not—decides the product owner. “Do you think I should remove myself from the ticket?”—asks Evelyn. This would be a negative as well, since she knows what she already have done on this ticket and what exactly was the bug in the API.

Back to the text editor, Evelyn discovers that the change she has to do while working on BWS-5044 is something she already did in the where the MQS was used in a project she worked on two or three months ago. If only she can find what it was, she could easily copy the relevant part of the code or put it in a dedicated library to be shared between two projects. Would Jira help? Let's do a search... Ticket assigned to Evelyn, created... hm... no, finished between those dates, with the name of the MQS product in its body. Five results, but nothing relevant. What if... exactly, she remembers, she was working in pair, but she's practically sure the ticket was assigned to her. Oh, right, the lead reassigned the ticket to the guy from the QA team, like he does for every ticket which is ready for tests. Is it possible to search for a ticket on which Evelyn was working at a given moment? How?...

Expectations

In this short story, when faced with a bug tracking system, the two users had a basic set of expectations. None were fulfilled correctly by the system I gave as an example, given that the competitors are not doing a very good job either.

What were the expectations of the product owner?

  1. Get an overview of the tasks flow for a given period of time. This is a perfectly valid requirement, and a user needs quite often to get this sort of information, either because he's back from a vacation, or because he tries to better understand the life of the project. This sort of things is essential during retrospectives, and also essential when one needs to spot a problem which affects negatively the progression of the project.

  2. Get the current state of the project. This is essentially what bug tracking systems are pretending to be for, but at the same time, they achieve to do a rather poor job in showing clearly who's in charge, what is left, where are the bottlenecks, etc. For instance, you know that another team you rely on is your bottleneck; now, look at your board and try to find how can you figure it out from the board itself. The answer is easy: you can't.

  3. Request details about a flag from another member of the team. Here again, modern bug tracking systems pretend to enhance interactions within a team, but they do it only by forcing people to talk to each other because they have no other ways to get the information they need. They had to do it anyway before they started using a bug tracking system, so what's the deal?

  4. Have a clear indicator about the progress of the impediment. Is it solved? Is somebody from the team checking that it's solved? If not, how urgent is it to do the actual check?

  5. Not having to think about the internals of the bug tracking system. From my experience, there are two types of teams. Those who don't know how to use their bug tracking system effectively and don't give a shit about that, and those who don't know how to use their bug tracking system effectively and who spend hours doing meetings about how they should change their practices. When nobody knows how to use a software product, it doesn't tell much good about the design of the product.

The programmer had her own expectations:

  1. Be able to find within seconds what was the thing she was working on the last time. However, the models used by the operating systems, version control systems or bug tracking systems we use doesn't match at all the way we work. Pair programming doesn't exist in the world of machines. The operating system assumes there is only one person in front of a PC. The version control won't tolerate any co-authorship: it's either you or your pair, you have to chose. The bug tracking system won't accept shared responsibilities either.

    This creates the first problem: the model of the world followed by those systems is completely wrong. The second problem comes to the fact that bug tracking systems are manual, and are time-suckers, making it understandable that most users don't update information in those systems in real-time. Finally, the third problem is that they use ambiguous terminology and make no effort to disambiguate it in any way.

    For instance, when George started to work on a production issue, he possibly marked it somehow in the bug tracking system. Did he? The exact thing he did was that he assigned himself on a ticket while also moving it to “In progress” column. But what exactly does it mean, and what would mean a ticket in progress which is not assigned to anyone, or a ticket which is assigned to somebody but is not yet in progress? If the conjuncture (ticket assigned to someone, ticket in progress) means that someone is actively working on it, then there could be no more than one ticket in progress per group (either a person or a pair or... well... even more persons), which, in turn, would mean that George has to be removed from BWS-5044, which, in turn, would lead to a problem to know how the bug tracking system would know that the ticket is now handled by Evelyn and Jeff. If, on the other hand, the conjuncture means something completely different, such as “I'm somehow related to this ticket, so (1) please don't touch it and (2) if you have questions, contact me,” then bug tracking systems can't possibly show the current status of the team, and so we need a different tool to track that, which possibly means that we won't be needing a bug tracking system after all.

  2. Have a clear indicator about the progress of the impediment. How else would she know she has to come back to a flagged ticket to check whether the flag should be removed or not? Should an additional ticket be created for that, and if yes, WTF is the problem with the guys who created such contraption in the first place?

  3. Not having to think about the internals of the bug tracking system. When it is unclear, like this is the case for Evelyn, if someone has to remove himself from the ticket or not, different persons would act differently, leading to a mess. Those are the elementary rules, and they should be clear, not because a five hours meeting resulted in an official document explaining what to do in this particular case, but because it's just clear by looking at the interface of the application.

  4. Be able to easily find what she, or somebody else, was working on at a given moment. This means having decent visualizations in terms of a graphical timeline which shows which projects were affected over time, but also a search which works well most of the time and can be used by a human being. No, the form which contains potentially about a hundred criteria doesn't qualify. Search forms like that are a leak of the internal logic of the application to the user interface. People who do that suck.

Why do we get what we get, and what do we do to get something better?

The major reason is historical. Originally, bug tracking systems were written by programmers for themselves. There is nothing wrong in that, except that a programmer is not an interaction designer, and so design of those tools is far from optimal.

Later, mature bug tracking systems were developed by teams of dozens or hundreds of members, including interaction designers. The problem is that they had the legacy as an example—example of what they shouldn't do, you'd say, but that they followed in order to be able to sell their product. New systems appeared, and some did a great job at getting rid of terrible approaches used by others (for instance Jira got rid of hierarchical dependency structures used by TFS and a few other tools), but changes couldn't be too radical. The problem is that current systems are so fundamentally flawed that the changes have to be radical. There should be a new system, developed from scratch, without the assumptions of its legacy.

A second reason is that bug tracking systems continue to be developed with a wrong goal. They follow to the letter their own name: they track bugs. This is, by its own, a very good thing, but it forgets about the reasons to track bugs in the first place. In rare cases, bugs are tracked just for the sake of tracking bugs; some companies believe doing such tracking is important by itself. In most cases, however, the tracking is done in order to empower people in understanding and improving their processes. To respond to those needs, software needs to be rethought, as a whole, and designed with people and processes in mind.