XY-problem and project estimates

A year ago, I was work­ing as an ar­chi­tect and IT con­sul­tant with a com­pa­ny on a large soft­ware pro­ject. The pro­ject was es­ti­mat­ed by their in-house lead de­vel­op­er, and I was asked to re­view the es­ti­mate to check if it's cor­rect.

The only prob­lem was that I had no freak­ing idea that half of the pro­ject fea­tures ex­ist­ed, and the oth­er half were very vague for me to ac­tu­al­ly es­ti­mate them. Nev­er­the­less, I re­ceived the es­ti­mate and was or­dered to re­view it, and my boss did the same. Dur­ing the meet­ing where we had to ex­press our even­tu­al con­cerns about the es­ti­mate, my boss found that the es­ti­mate lacked a few points, such as pro­ject man­age­ment or test­ing. I agreed.

How­ev­er, this is ex­act­ly an XY-prob­lem. The fact that I was asked to re­view the es­ti­mate was ir­rel­e­vant of the orig­i­nal prob­lem of un­cer­tain­ty of every­one—in­clud­ing the lead de­vel­op­er him­self—that the es­ti­mate made any sense. And that means that it's not the re­sult, but the process which should have been re­viewed.

If I both­ered think­ing a bit about why I was asked to check the es­ti­mate, in­stead of blind­ly try to do what I was asked to do, I would have re­act­ed very dif­fer­ent­ly. In­stead of look­ing at the re­sult, I would have checked how the re­sult was at­tained. The re­sult, i.e. the es­ti­mate it­self, be­comes com­plete­ly ir­rel­e­vant here. I don't even have to look at it. In­stead, I have to talk to the lead de­vel­op­er, and ask him to ex­plain me how he did what he did.

This shift has sev­er­al im­pli­ca­tions:

• When I fo­cused on the re­sult, I couldn't do any­thing use­ful. If I want to check the es­ti­mate it­self, I need to do my own, and then com­pare the re­sults. Since I had ab­solute­ly no de­tails re­quired to do a de­cent es­ti­mate, this was a dead end any­way.

  On the oth­er hand, in­ter­ven­ing at the lev­el of the process could be re­al­ly ben­e­fi­cial for my client. If the lead de­vel­op­er did every­thing right, the au­dit would lead to the con­clu­sion that the es­ti­mate should be cor­rect. If, how­ev­er, the de­vel­op­er didn't know well how to es­ti­mate soft­ware pro­jects, or didn't have enough data to do the es­ti­mate, not only the au­dit would have in­val­i­dat­ed the es­ti­mate, but would been an op­por­tu­ni­ty for me to share my skills with this de­vel­op­er and help him to get a more cor­rect es­ti­mate lat­er.

• If one fo­cus­es on the re­sult, the re­marks that one can do are not par­tic­u­lar­ly valu­able. My boss and I agreed that the es­ti­mate doesn't have any pro­ject man­age­ment or test­ing in it. That's a good ob­ser­va­tion, but it is far from be­ing cru­cial, and ac­counts for 20-25% of pro­ject time. More im­por­tant­ly, it gives no clear and mea­sur­able in­di­ca­tion about nei­ther the pre­ci­sion, nor the ac­cu­ra­cy of the es­ti­mate.

  On the oth­er hand, au­dit­ing the process would not only cov­er the en­tire du­ra­tion of the pro­ject, that is 100% of pro­ject time in­stead of 20-25%, but it will also give a clear and mea­sur­able data about both the pre­ci­sion and the ac­cu­ra­cy of the es­ti­mate, for in­stance by show­ing that the pre­ci­sion is way too high and should be de­creased by a fac­tor of fifty, while the ac­cu­ra­cy is ex­treme­ly low even giv­en the ear­ly stage of the pro­ject.

This case is a good ex­am­ple of a sit­u­a­tion where the orig­i­nal re­quest makes no real sense in its cur­rent form, and the per­son be­ing asked to do the thing should first search for the prob­lem at the ori­gin of the re­quest.