Measurements as a precursor of culture of quality

When au­dit­ing dif­fer­ent IT com­pa­nies, the no­tice­able pat­tern is that com­pa­nies which per­form bad­ly have prob­lems in every imag­in­able area. I've nev­er seen a com­pa­ny which has out­stand­ing work qual­i­ty but se­vere se­cu­ri­ty prob­lems, or a com­pa­ny which does every­thing very well, but have im­por­tant is­sues with the pro­gram­ming tools in use.

This is un­der­stand­able: if you hire skill­ful, tal­ent­ed peo­ple, they will per­form well in many as­pects of their jobs. A team of tal­ent­ed de­vel­op­ers just can't ig­nore that they are los­ing pro­duc­tiv­i­ty be­cause of the lack of ver­sion con­trol, or the ab­sence of unit tests. On the oth­er hand, if you hire the least de­sir­able coders who don't know what they are do­ing, they will fail in what­ev­er they do.

In this con­text, a ques­tion like “How to in­tro­duce a cul­ture of qual­i­ty into the work en­vi­ron­ment?” and com­ments like this one:

The way you con­vince man­age­ment of any­thing is by mak­ing it a val­ue propo­si­tion. How much mon­ey will it save them by im­ple­ment­ing your sug­ges­tions? Show them the mon­ey.

is worth a dis­cus­sion, es­pe­cial­ly since this opin­ion is very pop­u­lar among IT pro­fes­sion­als. What could be eas­i­er? Just show to the CEO how prof­itable his com­pa­ny will be­come if every­body starts pro­mot­ing the cul­ture of qual­i­ty!

In prac­tice, this doesn't work.

The ba­sic log­ic is flawed

It doesn't mat­ter how many ex­am­ples can you list of com­pa­nies which found unit tests, De­vOps, ver­sion con­trol or pair pro­gram­ming as pre­cious prac­tices which bring more mon­ey: there will al­ways be com­pa­nies which screwed up unit tests, De­vOps, ver­sion con­trol and pair pro­gram­ming, re­sult­ing in mon­ey loss.

As IT pro­fes­sion­als, we of­ten be­lieve that some prac­tices bring mon­ey with 100% con­fi­dence. Take ver­sion con­trol: any de­vel­op­er with a bit of self-es­teem will flee any team which doesn't use one. There is ab­solute­ly no tech­ni­cal rea­son not to use ver­sion con­trol.

How­ev­er, from busi­ness point of view, chang­ing a prac­tice of a team from “Just put your source on FTP” to a use of a ver­sion con­trol sys­tem may pre­sent a sub­stan­tial cost and some risks:

1. How to set­up a ver­sion con­trol serv­er? The us­age of a third-par­ty ser­vice may be eas­i­er, but is also as­so­ci­at­ed with a month­ly cost.

2. How to back­up the repos­i­to­ry?

3. What hap­pens if the serv­er goes down? Is it still pos­si­ble for the team to work dur­ing main­te­nance?

4. How much would it cost to train the team mem­bers to use ver­sion con­trol prop­er­ly?

5. What are branch­es and how do we use them?

6. Our pre­vi­ous branch us­age ap­pears com­plete­ly stu­pid. How do we change it now?

A prac­ti­cal ex­am­ple from my last em­ploy­ers' work­place:

• Coders who worked there lacked ba­sic un­der­stand­ing of ver­sion con­trol, re­sult­ing in a waste of one to four hours per week per de­vel­op­er.

• The guy who was in charge of the TFS serv­er was a mo­ron, un­able to man­age any­thing. The serv­er was of­ten down, re­sult­ing in an­oth­er waste of one to two hours per week per de­vel­op­er for the whole com­pa­ny.

• The same guy con­sid­ered it fun­ny to play with branch­es, while hav­ing a deep mis­un­der­stand­ing of ver­sion con­trol in gen­er­al and TFS in par­tic­u­lar. This of­ten re­sult­ed in a waste of two to ten hours per month per de­vel­op­er.

Add to this the cost of host­ing TFS in-house and the orig­i­nal price of the serv­er, and the choice of us­ing ver­sion con­trol ap­pears not that in­ter­est­ing. Ob­vi­ous­ly, it would be dif­fer­ent if the com­pa­ny was hir­ing peo­ple who know the stuff.

Tools, pat­terns and prac­tices are great when they are used right. When not, the best tool can drag team mem­bers in­stead of em­pow­er­ing them.

Mea­sure the ef­fect of a change be­fore do­ing the change

The bad thing is that com­pa­nies which are in a dif­fi­cult po­si­tion usu­al­ly don't have met­rics. Ei­ther they mea­sure noth­ing, or when they ac­tu­al­ly mea­sure some­thing, they are do­ing it wrong or not us­ing the re­sults.

This makes any im­prove­ment pro­pos­al prob­lem­at­ic. Say you want to in­tro­duce unit test­ing, and you con­vince the man­age­ment that unit test­ing makes the com­pa­ny more prof­itable by re­duc­ing the loss­es re­lat­ed to bug solv­ing. The prob­lem is that no­body in this com­pa­ny knows how much ex­act­ly the com­pa­ny spends for de­bug­ging, or how much bugs are re­port­ed and solved.

So two months lat­er, an­oth­er de­vel­op­er tells to the man­age­ment that unit tests ac­tu­al­ly harm pro­duc­tiv­i­ty and cost mon­ey. He has no data to sup­port his claim, but you have no data ei­ther, which means that the one who wins is not the one who is right, but the one who is clos­er to the man­age­ment.

Mea­sure the im­prove­ment brought by a change and make sure re­sults are pub­lic and tak­en se­ri­ous­ly.

The hard part is to make it pos­si­ble to gath­er the sta­tis­tics. Of­ten, it re­quires the change in the process—a change that a team may not be ready to ac­cept.

Peo­ple don't change; at least the cul­ture

In the end, the most im­por­tant thing is the team and the cul­ture. There are teams of in­ex­pe­ri­enced coders who are will­ing to make progress and are en­cour­aged by the man­age­ment. They will ul­ti­mate­ly find a way to im­prove. But there are also teams which lack the ba­sic com­mu­ni­ca­tion skills, are man­aged by peo­ple who re­sist change, or the mem­bers of the team sim­ply don't both­er about their job.

My job as a con­sul­tant is also to de­ter­mine whether the team and their man­age­ment wants change. If ei­ther the team or the man­age­ment couldn't care less, there is not much I can do.