On bug tracking systems

There are no good bug track­ing sys­tems, and I'm about to ex­plain what makes them not good and why good sys­tems weren't de­vel­oped yet.

To be­gin, let's il­lus­trate a jour­ney of two per­sons work­ing on the same team: one of the pro­gram­mers and a prod­uct own­er.

One Mon­day morn­ing

It's Mon­day morn­ing. The prod­uct own­er is back from two weeks va­ca­tion. He ar­rived at the of­fice, marked as read his five hun­dred and fifty e-mails, and now, he wants to know two things: what was done dur­ing his ab­sence and what is the cur­rent state of the pro­ject. He launch­es Jira and tries to fig­ure out if there is any­thing use­ful he can ex­tract from the board. He stares on the screen for half a minute, locks his ma­chine and goes to the neigh­bor of­fice to talk to an­oth­er mem­ber of the team who ar­rived ear­ly this morn­ing—this seems to be the only way to get up to date; thanks any­way, Jira.

Short­ly af­ter that, the oth­er sub­ject ar­rives. The pro­gram­mer re­mem­bers that she was work­ing in pair on a tick­et with Jeff, but lots of things hap­pened dur­ing the week­end, so she would en­joy a lit­tle help from some­one to quick­ly find what was the thing she was work­ing on. She, too, stares on the Jira board show­ing around forty tick­ets for the cur­rent sprint, sev­en be­ing in the “In de­vel­op­ment” col­umn, two be­ing as­signed to Jeff, none to her. Among the two, both are flagged, and don't look at all like the stuff from Fri­day evening. That's strange. Oh, now she re­mem­bers. There was a tick­et as­signed to George, and George was pair­ing with Jeff. Then George was in charge of an is­sue in pro­duc­tion, and Jeff need­ed some help with MQS, so he called Eve­lyn. Tick­et BWS-5044 it is. Time to do some cod­ing.

Mean­while, the prod­uct own­er got the in­for­ma­tion he was look­ing for. Back to his of­fice, he checked the Jira board for the sec­ond time, and no­ticed that the tick­et as­signed to Eve­lyn and flagged for im­ped­i­ment two and a half weeks ago still has its flag, which con­tra­dicts what he just learned from his col­league. Eve­lyn is not one of those who nev­er up­dates the board, so there should be a prob­lem; time to go see her.

Eve­lyn con­firms that the API team claimed that they solved the is­sue with the re­turn val­ues; if this is true, the im­ped­i­ment doesn't ex­ist any longer. How­ev­er, she couldn't check yet if the res­o­lu­tion of the is­sue is ef­fec­tive, and, frankly, for the past year, each and every time they claimed they solved some­thing, it ap­peared that ei­ther they didn't, or they did it wrong, or they re­al­ly did it, but for­got to push their changes to pro­duc­tion. Should we re­move the flag? Un­less we are sure the is­sue is solved, maybe not—de­cides the prod­uct own­er. “Do you think I should re­move my­self from the tick­et?”—asks Eve­lyn. This would be a neg­a­tive as well, since she knows what she al­ready have done on this tick­et and what ex­act­ly was the bug in the API.

Back to the text ed­i­tor, Eve­lyn dis­cov­ers that the change she has to do while work­ing on BWS-5044 is some­thing she al­ready did in the where the MQS was used in a pro­ject she worked on two or three months ago. If only she can find what it was, she could eas­i­ly copy the rel­e­vant part of the code or put it in a ded­i­cat­ed li­brary to be shared be­tween two pro­jects. Would Jira help? Let's do a search... Tick­et as­signed to Eve­lyn, cre­at­ed... hm... no, fin­ished be­tween those dates, with the name of the MQS prod­uct in its body. Five re­sults, but noth­ing rel­e­vant. What if... ex­act­ly, she re­mem­bers, she was work­ing in pair, but she's prac­ti­cal­ly sure the tick­et was as­signed to her. Oh, right, the lead re­as­signed the tick­et to the guy from the QA team, like he does for every tick­et which is ready for tests. Is it pos­si­ble to search for a tick­et on which Eve­lyn was work­ing at a giv­en mo­ment? How?...

Ex­pec­ta­tions

In this short sto­ry, when faced with a bug track­ing sys­tem, the two users had a ba­sic set of ex­pec­ta­tions. None were ful­filled cor­rect­ly by the sys­tem I gave as an ex­am­ple, giv­en that the com­peti­tors are not do­ing a very good job ei­ther.

What were the ex­pec­ta­tions of the prod­uct own­er?

1. Get an overview of the tasks flow for a giv­en pe­ri­od of time. This is a per­fect­ly valid re­quire­ment, and a user needs quite of­ten to get this sort of in­for­ma­tion, ei­ther be­cause he's back from a va­ca­tion, or be­cause he tries to bet­ter un­der­stand the life of the pro­ject. This sort of things is es­sen­tial dur­ing ret­ro­spec­tives, and also es­sen­tial when one needs to spot a prob­lem which af­fects neg­a­tive­ly the pro­gres­sion of the pro­ject.

2. Get the cur­rent state of the pro­ject. This is es­sen­tial­ly what bug track­ing sys­tems are pre­tend­ing to be for, but at the same time, they achieve to do a rather poor job in show­ing clear­ly who's in charge, what is left, where are the bot­tle­necks, etc. For in­stance, you know that an­oth­er team you rely on is your bot­tle­neck; now, look at your board and try to find how can you fig­ure it out from the board it­self. The an­swer is easy: you can't.

3. Re­quest de­tails about a flag from an­oth­er mem­ber of the team. Here again, mod­ern bug track­ing sys­tems pre­tend to en­hance in­ter­ac­tions with­in a team, but they do it only by forc­ing peo­ple to talk to each oth­er be­cause they have no oth­er ways to get the in­for­ma­tion they need. They had to do it any­way be­fore they start­ed us­ing a bug track­ing sys­tem, so what's the deal?

4. Have a clear in­di­ca­tor about the progress of the im­ped­i­ment. Is it solved? Is some­body from the team check­ing that it's solved? If not, how ur­gent is it to do the ac­tu­al check?

5. Not hav­ing to think about the in­ter­nals of the bug track­ing sys­tem. From my ex­pe­ri­ence, there are two types of teams. Those who don't know how to use their bug track­ing sys­tem ef­fec­tive­ly and don't give a shit about that, and those who don't know how to use their bug track­ing sys­tem ef­fec­tive­ly and who spend hours do­ing meet­ings about how they should change their prac­tices. When no­body knows how to use a soft­ware prod­uct, it doesn't tell much good about the de­sign of the prod­uct.

The pro­gram­mer had her own ex­pec­ta­tions:

1. Be able to find with­in sec­onds what was the thing she was work­ing on the last time. How­ev­er, the mod­els used by the op­er­at­ing sys­tems, ver­sion con­trol sys­tems or bug track­ing sys­tems we use doesn't match at all the way we work. Pair pro­gram­ming doesn't ex­ist in the world of ma­chines. The op­er­at­ing sys­tem as­sumes there is only one per­son in front of a PC. The ver­sion con­trol won't tol­er­ate any co-au­thor­ship: it's ei­ther you or your pair, you have to chose. The bug track­ing sys­tem won't ac­cept shared re­spon­si­bil­i­ties ei­ther.

   This cre­ates the first prob­lem: the mod­el of the world fol­lowed by those sys­tems is com­plete­ly wrong. The sec­ond prob­lem comes to the fact that bug track­ing sys­tems are man­u­al, and are time-suck­ers, mak­ing it un­der­stand­able that most users don't up­date in­for­ma­tion in those sys­tems in real-time. Fi­nal­ly, the third prob­lem is that they use am­bigu­ous ter­mi­nol­o­gy and make no ef­fort to dis­am­biguate it in any way.

   For in­stance, when George start­ed to work on a pro­duc­tion is­sue, he pos­si­bly marked it some­how in the bug track­ing sys­tem. Did he? The ex­act thing he did was that he as­signed him­self on a tick­et while also mov­ing it to “In progress” col­umn. But what ex­act­ly does it mean, and what would mean a tick­et in progress which is not as­signed to any­one, or a tick­et which is as­signed to some­body but is not yet in progress? If the con­junc­ture (tick­et as­signed to some­one, tick­et in progress) means that some­one is ac­tive­ly work­ing on it, then there could be no more than one tick­et in progress per group (ei­ther a per­son or a pair or... well... even more per­sons), which, in turn, would mean that George has to be re­moved from BWS-5044, which, in turn, would lead to a prob­lem to know how the bug track­ing sys­tem would know that the tick­et is now han­dled by Eve­lyn and Jeff. If, on the oth­er hand, the con­junc­ture means some­thing com­plete­ly dif­fer­ent, such as “I'm some­how re­lat­ed to this tick­et, so (1) please don't touch it and (2) if you have ques­tions, con­tact me,” then bug track­ing sys­tems can't pos­si­bly show the cur­rent sta­tus of the team, and so we need a dif­fer­ent tool to track that, which pos­si­bly means that we won't be need­ing a bug track­ing sys­tem af­ter all.

2. Have a clear in­di­ca­tor about the progress of the im­ped­i­ment. How else would she know she has to come back to a flagged tick­et to check whether the flag should be re­moved or not? Should an ad­di­tion­al tick­et be cre­at­ed for that, and if yes, WTF is the prob­lem with the guys who cre­at­ed such con­trap­tion in the first place?

3. Not hav­ing to think about the in­ter­nals of the bug track­ing sys­tem. When it is un­clear, like this is the case for Eve­lyn, if some­one has to re­move him­self from the tick­et or not, dif­fer­ent per­sons would act dif­fer­ent­ly, lead­ing to a mess. Those are the el­e­men­tary rules, and they should be clear, not be­cause a five hours meet­ing re­sult­ed in an of­fi­cial doc­u­ment ex­plain­ing what to do in this par­tic­u­lar case, but be­cause it's just clear by look­ing at the in­ter­face of the ap­pli­ca­tion.

4. Be able to eas­i­ly find what she, or some­body else, was work­ing on at a giv­en mo­ment. This means hav­ing de­cent vi­su­al­iza­tions in terms of a graph­i­cal time­line which shows which pro­jects were af­fect­ed over time, but also a search which works well most of the time and can be used by a hu­man be­ing. No, the form which con­tains po­ten­tial­ly about a hun­dred cri­te­ria doesn't qual­i­fy. Search forms like that are a leak of the in­ter­nal log­ic of the ap­pli­ca­tion to the user in­ter­face. Peo­ple who do that suck.

Why do we get what we get, and what do we do to get some­thing bet­ter?

The ma­jor rea­son is his­tor­i­cal. Orig­i­nal­ly, bug track­ing sys­tems were writ­ten by pro­gram­mers for them­selves. There is noth­ing wrong in that, ex­cept that a pro­gram­mer is not an in­ter­ac­tion de­sign­er, and so de­sign of those tools is far from op­ti­mal.

Lat­er, ma­ture bug track­ing sys­tems were de­vel­oped by teams of dozens or hun­dreds of mem­bers, in­clud­ing in­ter­ac­tion de­sign­ers. The prob­lem is that they had the lega­cy as an ex­am­ple—ex­am­ple of what they shouldn't do, you'd say, but that they fol­lowed in or­der to be able to sell their prod­uct. New sys­tems ap­peared, and some did a great job at get­ting rid of ter­ri­ble ap­proach­es used by oth­ers (for in­stance Jira got rid of hi­er­ar­chi­cal de­pen­den­cy struc­tures used by TFS and a few oth­er tools), but changes couldn't be too rad­i­cal. The prob­lem is that cur­rent sys­tems are so fun­da­men­tal­ly flawed that the changes have to be rad­i­cal. There should be a new sys­tem, de­vel­oped from scratch, with­out the as­sump­tions of its lega­cy.

A sec­ond rea­son is that bug track­ing sys­tems con­tin­ue to be de­vel­oped with a wrong goal. They fol­low to the let­ter their own name: they track bugs. This is, by its own, a very good thing, but it for­gets about the rea­sons to track bugs in the first place. In rare cas­es, bugs are tracked just for the sake of track­ing bugs; some com­pa­nies be­lieve do­ing such track­ing is im­por­tant by it­self. In most cas­es, how­ev­er, the track­ing is done in or­der to em­pow­er peo­ple in un­der­stand­ing and im­prov­ing their process­es. To re­spond to those needs, soft­ware needs to be rethought, as a whole, and de­signed with peo­ple and process­es in mind.