Home Home

XY-problem and project estimates

Arseni Mourzenko
Founder and lead developer, specializing in developer productivity and code quality
October 8, 2016

A year ago, I was working as an architect and IT consultant with a company on a large software project. The project was estimated by their in-house lead developer, and I was asked to review the estimate to check if it's correct.

The only problem was that I had no freaking idea that half of the project features existed, and the other half were very vague for me to actually estimate them. Nevertheless, I received the estimate and was ordered to review it, and my boss did the same. During the meeting where we had to express our eventual concerns about the estimate, my boss found that the estimate lacked a few points, such as project management or testing. I agreed.

However, this is exactly an XY-problem. The fact that I was asked to review the estimate was irrelevant of the original problem of uncertainty of everyone—including the lead developer himself—that the estimate made any sense. And that means that it's not the result, but the process which should have been reviewed.

If I bothered thinking a bit about why I was asked to check the estimate, instead of blindly try to do what I was asked to do, I would have reacted very differently. Instead of looking at the result, I would have checked how the result was attained. The result, i.e. the estimate itself, becomes completely irrelevant here. I don't even have to look at it. Instead, I have to talk to the lead developer, and ask him to explain me how he did what he did.

This shift has several implications:

  • When I focused on the result, I couldn't do anything useful. If I want to check the estimate itself, I need to do my own, and then compare the results. Since I had absolutely no details required to do a decent estimate, this was a dead end anyway.

    On the other hand, intervening at the level of the process could be really beneficial for my client. If the lead developer did everything right, the audit would lead to the conclusion that the estimate should be correct. If, however, the developer didn't know well how to estimate software projects, or didn't have enough data to do the estimate, not only the audit would have invalidated the estimate, but would been an opportunity for me to share my skills with this developer and help him to get a more correct estimate later.

  • If one focuses on the result, the remarks that one can do are not particularly valuable. My boss and I agreed that the estimate doesn't have any project management or testing in it. That's a good observation, but it is far from being crucial, and accounts for 20-25% of project time. More importantly, it gives no clear and measurable indication about neither the precision, nor the accuracy of the estimate.

    On the other hand, auditing the process would not only cover the entire duration of the project, that is 100% of project time instead of 20-25%, but it will also give a clear and measurable data about both the precision and the accuracy of the estimate, for instance by showing that the precision is way too high and should be decreased by a factor of fifty, while the accuracy is extremely low even given the early stage of the project.

This case is a good example of a situation where the original request makes no real sense in its current form, and the person being asked to do the thing should first search for the problem at the origin of the request.