Quantifying time and cost: why do we always get it wrong?

Boss

I have an article. I want to publish it as a web page and let the users comment it.

Programmer

Does this article need to be editable?

Boss

No, I have no intent to modify the article from now.

Programmer

Tell me more about the comments. How are they? Could they be edited or removed?

Boss

Those are ordinary comments, with just a name of the author and the body. Once a person sends a comment, this person cannot modify nor remove it any longer. All the comments are displayed as a long list, from the oldest to the most recent one.

Programmer

What about the visual aspects?

Boss

You do whatever you want. I trust your graphical skills. Keep it easy, I don’t want you to spend too much time on it.

Programmer

OK.

Boss

Do you have quote?

Programmer

Let’s say $150.

Boss

WHAT?! $150 for this? It’s just a simple page with some comments. There is really nothing complicated.

Programmer

OK, maybe $100.

Boss

I have quotes for $30 from PHP developers.

Programmer

Well, I can do it for $75.

Boss

Deal.

Dis­cus­sions like this hap­pen all the time when you work in IT. You are giv­en a sum­ma­ry which bare­ly de­scribe the pro­ject, you have no time think­ing about it, and then you re­ceive a strong pres­sure from your boss be­cause some PHP code mon­key from a doubt­ful coun­try is ready to do low qual­i­ty work for a low amount of mon­ey.

How do those three as­pects in­crease the risk of wrong­ly quan­ti­fy­ing both time and cost? How to avoid them? I’ll try to ask all those ques­tions and il­lus­trate all this by mak­ing live a web app.

Part 1: em­brac­ing the re­quire­ments

There is a rule which works well for free­lancers: when cre­at­ing an ap­pli­ca­tion, for­get about do­ing things in or­der, and pre­sent as soon as pos­si­ble some­thing to show, in or­der to con­vince your cus­tomer that you’re work­ing hard on the pro­ject. Telling that you spent the last two months do­ing the ar­chi­tec­ture is in­ter­pret­ed by most peo­ple as: “I spent two months pro­cras­ti­nat­ing and in­stead of apol­o­giz­ing for my lazi­ness, I’ll rather tell you lots of tech­ni­cal words you don’t un­der­stand.” In­stead, a glance on some­thing ex­cit­ing­ly vi­su­al and graph­i­cal con­vinces your cus­tomer ways more, even if you’ve spent a few hours do­ing the vi­su­al.

From this point of view, we, peo­ple work­ing in IT, are not very dif­fer­ent from our cus­tomers. When pre­sent­ed with a short de­scrip­tion of a pro­ject, we are think­ing about the cool fea­tures and how nice it would be to code them, and are com­plete­ly omit­ting every­thing out­side the de­scrip­tion.

We rarely an­tic­i­pate every small­est part of a pro­ject. In most cas­es, we don’t an­tic­i­pate the most parts. Some re­quire­ments are so ob­vi­ous that we can’t omit them, but we still fre­quent­ly for­get about every oth­er re­quire­ment which is not di­rect­ly vis­i­ble.

Il­lus­tra­tion: how sim­ple it could be?

As an il­lus­tra­tion, let’s imag­ine the eas­i­est web ap­pli­ca­tion we can. It’s just a web page with some pre­de­fined text which nev­er changes; peo­ple can come and add com­ments which are then dis­played to every­one from the old­est to the most re­cent one. They can’t reg­is­ter. They can’t re­move or edit their com­ments. Com­ments are just plain text with a name of the au­thor as a sep­a­rate field.

Heck, I can do it in less than half an hour! It’s so easy!

What we for­got

Ac­tu­al­ly, the pro­ject is not easy at all. It feels easy, be­cause we omit­ted to ask our­selves the most im­por­tant ques­tions:

• What will hap­pen if a bot starts to post Vi­a­gra ads twice per sec­ond?

• What if a user leaves one of the fields emp­ty?

• What if a user leaves an in­flam­ma­to­ry com­ment? Would the only way be to re­move it by man­u­al­ly query­ing the data­base? If the user con­tin­u­ous­ly in­sults all oth­er users, wouldn’t it be pos­si­ble to block this user by IP?

• If a user sub­mits a form with a com­ment be­ing 500 000 char­ac­ters long, how it would be dis­played? What if the user’s name is 500 000 char­ac­ters long?

• Are Chi­nese char­ac­ters sub­mit­ted and dis­played cor­rect­ly? What about back­slash? What about re­served HTML char­ac­ters?

• Is the page com­pat­i­ble with Opera? IE7? Sa­fari? How is it dis­played on a mo­bile de­vice?

• If the same page is re­quest­ed 50 times per sec­ond, will it cause a DOS?

• Isn’t there a way to cache the com­ments (and even the whole page) in­stead of query­ing the data­base at every re­quest?

• If the data­base is down, what would hap­pen?

• In case of an er­ror, will this er­ror, would it be a busi­ness lay­er ex­cep­tion or an SQL er­ror, be dis­played di­rect­ly to the user, in­stead of show­ing a nice er­ror page apol­o­giz­ing that some­thing got wrong?

• Are ex­cep­tions logged?

• How the com­ments in the data­base are backed up? What hap­pens if there is a dis­as­ter which af­fects the serv­er and caus­es a loss of serv­er’s data?

• Is de­ploy­ment process au­to­mat­ed?

• If there are four thou­sand com­ments, how long would it take to dis­play a page? Isn’t it a good idea to im­ple­ment pag­i­na­tion, or, bet­ter, pop­u­late old­er com­ments only on de­mand, hav­ing some­thing sim­i­lar to Google Im­ages?

Who should care?

You may won­der why you have to care. You were giv­en the ba­sic de­scrip­tion of the pro­ject. The cus­tomer nev­er asked you to log ex­cep­tions, make a dis­as­ter re­cov­ery plan or han­dle the case where one of the fields of the form is emp­ty. In fact, it’s not to the cus­tomer to think about it. He doesn’t have to know about dis­as­ter re­cov­ery plans, or about au­dit, or why PBKDF2 must be used in­stead of MD5 to store pass­words. He sole­ly ex­press­es his busi­ness needs, be­cause his area of ex­per­tise is his busi­ness, not the way of stor­ing pass­words or dis­play­ing large amount of com­ments on a page.

If you sim­ply lim­it your­self at trans­lat­ing busi­ness needs into a more tech­ni­cal lan­guage, you are omit­ting the most im­por­tant part of the pro­ject when quan­ti­fy­ing time and cost. You then agree with your cus­tomer for $75, while the real cost is at least $800, i.e. ten times more.

There are re­quire­ments which are most­ly ex­plic­it. If the stake­hold­er tells that an app must let the user to re­move the last file the user cre­at­ed through this app, it is clear that one of the re­quire­ments is to do ex­act­ly that. It is less ob­vi­ous that you should also have a re­quire­ment ex­plain­ing how the app should be­have if the user haven’t cre­at­ed any doc­u­ment yet, or if the doc­u­ment is in use, or if it was al­ready re­moved, or is read only, or if it is on a flash dri­ve which was un­plugged mean­while. Would a but­ton be avail­able for the re­moval? Is there a key­board short­cut for it? Is there a warn­ing that the doc­u­ment will be lost for­ev­er, or is there in­stead an undo fea­ture?

Be­fore giv­ing a quote, you should walk through all those re­quire­ments, not just the ob­vi­ous ones. When you don’t know the an­swer (for ex­am­ple in a pre­vi­ous il­lus­tra­tion, do you need a warn­ing or an undo), you should ask the stake­hold­er first.

Part 2: tak­ing time to think

In the pre­vi­ous part, I’ve shown how easy it is to be fooled by sparse busi­ness re­quire­ments and to quan­ti­fy time and cost more than ten times low­er than the ac­tu­al one.

By more time, I mean, much more. When it comes to mod­i­fy­ing a road for ex­am­ple, calls for bids made by the State con­tain hun­dreds of pages to read, de­scrib­ing pre­cise­ly the con­text, what to do, what to wear when do­ing it, etc. A con­sid­er­able amount of time is need­ed to write those doc­u­ments, and some amount of time is re­quired in or­der to read and un­der­stand them.

You cer­tain­ly don’t need to write hun­dreds of pages when you’re re­quest­ed to add the most ba­sic com­ment­ing fea­ture to a sta­t­ic web page. Still, some pre­lim­i­nary think­ing is re­quired in or­der to avoid the sur­pris­es lat­er dur­ing the im­ple­men­ta­tion.

More im­por­tant­ly, you need time to think. Some re­quire­ments are hard to spot, and are dis­cov­ered only af­ter some elab­o­ra­tion. More you un­der­stand the busi­ness prob­lem, more re­quire­ments you can think about, and more re­quire­ments you think about, more oth­ers you can dis­cov­er.

Elab­o­rate at least some doc­u­men­ta­tion

Doc­u­men­ta­tion is cru­cial even at the ear­ly stages. You can hold a few busi­ness re­quire­ments in your head, but when you start think­ing about the de­tails, it is im­pos­si­ble to re­mem­ber every as­pect of the pro­ject.

It would be too cost pro­hib­i­tive to cre­ate WBS, Gantt charts, a full list of func­tion­al and non-func­tion­al re­quire­ments and sim­i­lar doc­u­ments for every sub­mit­ted pro­ject. It is sim­ply im­pos­si­ble giv­en that:

• Your boss (or your cus­tomer) is un­will­ing to wait for your an­swer too long,

• Not every idea is bril­liant, and you may re­ceive lots of ideas for pro­jects which will nev­er be ever at plan­ning phase,

• Ini­ti­a­tion phase is done in col­lab­o­ra­tion with the stake­hold­ers, and of­ten re­quire the in­volve­ment of many peo­ple with di­verse ar­eas of ex­per­tise,

• All this time will be lost (and prob­a­bly un­paid) if the pro­ject is halt­ed at this ear­ly stage.

Still, some light­weight doc­u­men­ta­tion may be only ben­e­fi­cial. I dis­tin­guish three types of in­for­ma­tion which may be gath­ered at this lev­el:

• Busi­ness re­quire­ments. This in­cludes every­thing the stake­hold­er can com­mu­ni­cate you about the pro­ject it­self from the busi­ness point of view.

• Tech­ni­cal data. This is more about tech­ni­cal as­pects which doesn’t con­cern the stake­hold­ers, but are crit­i­cal to quan­ti­fy time and cost. For ex­am­ple, if the pro­ject con­tains some dark parts and re­quires a few pro­to­types, this should be de­scribed in this part of the doc­u­men­ta­tion.

• In­for­ma­tion for stake­hold­ers. This has a very spe­cif­ic pur­pose of train­ing the stake­hold­ers or ex­plain­ing some­thing about the pro­ject or the over­all process. For ex­am­ple, if the stake­hold­er is imag­in­ing that some­thing is pos­si­ble, while giv­en the spe­cif­ic con­straints, it’s not, this would be in this cat­e­go­ry.

Note that if done cor­rect­ly, this doc­u­men­ta­tion may be reused lat­er. As a free­lancer, I of­ten have to work on pro­jects where the only doc­u­men­ta­tion is the pro­ject sum­ma­ry of sev­er­al para­graphs; not pages—para­graphs. Those pro­jects are of­ten of the same type: they waste peo­ples’ time and mon­ey, and they end up in a ter­ri­ble prod­uct no­body can or would like to use.

Tak­ing a few days ac­tu­al­ly think­ing about the pro­ject and writ­ing the thoughts down in an or­ga­nized way would prob­a­bly help a lot in those cas­es; for some of those pro­jects, think­ing about them for a few hours would be enough to un­der­stand how use­less they are.

Ask ques­tions

There are chances that the cus­tomer doesn’t have a clear un­der­stand­ing of the busi­ness re­quire­ments them­selves. If it is the case, you should help them deep­en their pro­ject by ask­ing ques­tions and pro­vid­ing ba­sic tech­ni­cal as­sis­tance. For ex­am­ple, if you take the pro­ject from the part 1, a pos­si­ble di­a­log would be:

Programmer

You told that all the comments are displayed on the same page. Am I right?

Stakeholder

Yes.

Programmer

What happens if there are many comments? I mean a huge quantity, like several thousands.

Stakeholder

You still display them on the same page.

Programmer

Well, you see, it may cause some performance issues. Moreover, the page would be too big in size, so downloading all those comments at once would be painful on a device with a slow internet access.

Stakeholder

What do you suggest?

Programmer

We may want to implement pagination. This is the basic form, and is simple to do. Users will be able to change pages to view older comments. Tell me, how users will use those comments? Would they be searching them randomly, i.e. be interested in being able to access for example page 9 out of 14, or they would rather read the most recent ones, and then go to the older ones if need?

Stakeholder

I think they will actually read the newest ones only, and maybe then go and read the previous comments.

Programmer

OK. In this case, it would be a better alternative to avoid pagination, and to let the user to display a certain amount of older comments at a time.

Be­fore ask­ing the ques­tion, it was im­pos­si­ble to know if pag­i­na­tion is a good idea. It was so un­clear that the only re­quire­ment is­sued from the orig­i­nal de­scrip­tion of the pro­ject was to dis­play the com­ments from the old­est to the newest one. Af­ter this di­a­log, it ap­pears that the com­ments must be dis­played in the ex­act op­po­site or­der, that not all of them should be dis­played, and that pag­i­na­tion is not an op­tion.

En­cour­age pro­to­typ­ing

Ba­sic doc­u­men­ta­tion and a pletho­ra of ques­tions still isn’t enough to be able to quan­ti­fy pre­cise­ly time and cost of a pro­ject. It would def­i­nite­ly help in the ba­sic ex­am­ple I gave in part 1, but would not work for medi­um or large scale pro­jects.

Ag­ile meth­ods help a lot in this case. If you can’t pos­si­bly know how much will it cost to fin­ish the whole pro­ject, you can still give a pre­cise quote for a small part of it. Cre­at­ing a few pro­to­types is a good idea too, since not only you will be able to quote more pre­cise­ly a small pro­to­type than the whole app, but also pro­to­types are them­selves done to be able to know more pre­cise­ly the over­all cost of a pro­ject by elim­i­nat­ing the un­known from the most ob­scure parts.

From the point of view of cost and time eval­u­a­tion, pro­to­types and ag­ile meth­ods are very dif­fer­ent things. As pre­vi­ous­ly said, pro­to­types tend to bring to light the parts which are un­clear. Pro­to­typ­ing helps rapid­ly iden­ti­fy­ing the prob­lem and the so­lu­tion (or the ab­sence of it), with­out wast­ing too much time writ­ing clean code, de­sign­ing clean ar­chi­tec­ture, etc. Some tasks just can­not be eval­u­at­ed, or, in oth­er words, in or­der to know how dif­fi­cult they are, you need to do them in the first place. Pro­to­types are used pre­cise­ly in those cas­es.

Ag­ile meth­ods, on the oth­er hand, are sim­ply sep­a­rat­ing a large pro­ject into small­er chunks to be able to do each chunk sep­a­rate­ly, re­ceive feed­back from the stake­hold­ers, and con­tin­ue with the re­main­ing chunks. This makes the over­all process very flu­id and pro­gres­sive, and al­lows the stake­hold­ers not only to see the cur­rent state of the pro­ject and its cost, but also to stop the pro­ject if the cost be­comes too high, but still have some­thing func­tion­al.

Ad­mit that you don’t know every­thing

If you’re a de­vel­op­er, you might be able to eval­u­ate the cost of the de­vel­op­ment it­self, but that’s all. It’s up to the DBA to de­ter­mine the cost of de­sign­ing the data­base, or to the in­ter­ac­tiv­i­ty de­sign­er to de­ter­mine how much would it cost to work on in­ter­ac­tiv­i­ty de­sign for a spe­cif­ic ap­pli­ca­tion.

More­over, you as a de­vel­op­er rely on oth­er peo­ple. If the DBA cre­ates a com­plex de­sign for a data­base, it may af­fect your code, re­quir­ing more work to do, which trans­lates into more time and high­er cost. If the graph­ic artist cre­ates fan­cy graph­ics for a web­site, it would re­quire more com­pli­cat­ed HTML and CSS, so high­er cost.

Part 3: avoid­ing be­ing in­flu­enced by stake­hold­ers

Mis­un­der­stand­ing as the root cause

Your boss is in­ter­est­ed in low costs. While he un­der­stands that he gets what he pays for, he finds the busi­ness re­quire­ments pret­ty sim­ple and will not un­der­stand why the cor­re­spond­ing price is so abu­sive­ly high.

This means that from the be­gin­ning, there is a sort of mis­un­der­stand­ing.

Re­cent­ly, I had to work on a pro­ject of a web app. The cus­tomer is a pro­gram­mer who had an idea and tried to make it con­crete through code, with in­tent to cre­ate a com­mer­cial-grade app used all around the world. The most sur­pris­ing part is that this pro­gram­mer doesn’t seem to un­der­stand how much ef­fort should be in­volved in a com­mer­cial-grade app tar­get­ing in­ter­na­tion­al mar­ket, and all the ef­forts of the pro­ject are con­cen­trat­ed on pro­gram­ming it­self. Graph­ics are hor­ri­ble. UX is nonex­is­tent. The in­ter­face is trans­lat­ed in Eng­lish by a French per­son who doesn’t know Eng­lish very well. The data­base schema makes me cry (a phone num­ber is saved in an int field, while a dig­it be­tween one and six is stored as nchar(10); there are no con­straints; every as­pect of the schema is a can­di­date for The Dai­ly WTF). No­body ever pro­filed any part of the app, and giv­en that any page is mak­ing at least twen­ty queries gen­er­at­ed by En­ti­ty Frame­work (you know, the ones which don’t fit in a screen and are to­tal­ly un­read­able), SQL pro­fil­ing is out of ques­tion. There is no cache at all. Pass­words are stored in plain text. The log­ging is nonex­is­tent. There is no doc­u­men­ta­tion. The only com­ments I’ve seen are ei­ther un­nec­es­sary or wrong. When I asked how this per­son rates the code base on a scale of 1 to 10, the per­son an­swered “8”. Ac­tu­al­ly, it would be 2, not 1, be­cause there are some pat­terns used in the code.

In short, the ac­tu­al work rep­re­sents at best 1% of the work re­quired to do some­thing cor­rect. Sad­ly, this is the case of most IT pro­jects.

Be­ing a not very ex­pe­ri­enced pro­gram­mer, this per­son has a deep mis­un­der­stand­ing of every oth­er area in­volved in a soft­ware prod­uct. For ex­am­ple, he prob­a­bly don’t know what in­ter­ac­tion de­sign is about.

A busi­ness per­son has a sim­i­lar vi­sion of the de­vel­op­ment part of the soft­ware prod­uct, but, see, more lim­it­ed. De­vel­op­ment is so un­clear for most peo­ple that they would imag­ine that it’s some­thing com­plex, but they nev­er imag­ine how ac­tu­al­ly com­plex it is in re­al­i­ty.

When it comes to imag­in­ing the prod­uct they need, they imag­ine only the di­rect cost of im­ple­ment­ing the busi­ness re­quire­ment. It’s ac­tu­al­ly eas­i­er to il­lus­trate this by an ex­am­ple of a change of a prod­uct.

Let’s say you’ve de­vel­oped a soft­ware prod­uct which deals with some data stored in a cer­tain way. Lat­er, a stake­hold­er asks you to make a tiny lit­tle change in this data; in re­al­i­ty, it re­quires you to change all the de­sign, all the way you store your data, a large part of the data­base schema, and do some mi­nor UI changes. You tell that the ac­tu­al cost of such a change is $5 000, while the stake­hold­er ex­pect­ed at most $40.

The same ap­plies to new prod­ucts. Since the stake­hold­er can­not pos­si­bly fore­see most of the func­tion­al and non-func­tion­al re­quire­ments, he would be sur­prised by the abu­sive­ly high cost you give to him, and try to con­vince you that the cost should be low­er.

In­com­pe­tence as the cause of mis­un­der­stand­ing

I en­joy down­load­ing and read­ing the doc­u­ments in calls for bids pub­lished by the State ad­min­is­tra­tion. What I en­joy the most is the fact that for the tini­est pro­ject, there are hun­dreds of pages de­scrib­ing un­til the tini­est de­tail what is the work to do, and all those pages are writ­ten by very ex­pe­ri­enced peo­ple who know ex­treme­ly well what they are writ­ing about. They spent years in col­lege learn­ing how to do this stuff, and they have a huge pro­fes­sion­al ex­pe­ri­ence.

I try to imag­ine some­thing sim­i­lar to what hap­pens in IT world:

State representative

I have a bridge over a river and it’s too narrow. I want it larger. Could you do it the next week?

Contractor

Wait, I need some more details…

State representative

Well, it’s a bridge with two lanes, and I need four instead. How much would it cost?

Contractor

I really don’t know…

State representative

Come on! It’s a simple bridge! There is nothing difficult. The bridge is already built; you have just to enlarge it. Do you think $50 000 is enough?

Contractor

You should maybe provide a detailed description…

State representative

I already told you precisely what to do.

Contractor

In all cases, in order to make a bridge larger, it would require a lot of work, so $50 000 is hardly enough even for a small one.

State representative

Seriously?! How much would it cost to add two more lanes?

Contractor

Hm, well, I don’t know… What is the bridge you’re talking about?

State representative

The one which connects La Rochelle to Île-de-Ré¹, it’s just 2.9 km long. Nothing complicated. I’m sure you can do it the next week.

Han­dling the mis­un­der­stand­ing

Ex­plain­ing why your quote is ten times high­er to the ex­pec­ta­tions of your boss is hard. Show­ing to your boss some of the re­quire­ments he didn’t think about and telling that there are more may help, but is not very ef­fec­tive in prac­tice, and re­quires draft­ing the re­quire­ments, which takes too much time.

Tak­ing an ex­am­ple in a do­main oth­er than IT, like the il­lus­tra­tion above, may have an im­pact in some cas­es, but is com­plete­ly in­ef­fec­tive in oth­ers. More­over, busi­ness peo­ple are rarely in­clined to read dis­cus­sion-ori­ent­ed stuff. They sim­ply don’t care.

The way I’ve found which seems to be more ef­fec­tive is to have a re­view process where you ask all ques­tions which come into your mind. They don’t have to be an­swer­able by the stake­hold­er; they don’t even have to be an­swer­able at all.

In Flo­rence² pro­ject, the pro­ject starts by the cus­tomer sub­mit­ting any doc­u­ment she has that is re­lat­ed to the pro­ject. Then, be­fore start­ing to draft the back­log and be­fore even ac­cept­ing or re­ject­ing the pro­ject, any per­son of the com­pa­ny, in­ter­est­ed in this pro­ject, can ask ques­tions. The in­tent is to ob­tain enough in­for­ma­tion from the cus­tomer, but this part can be en­larged to unan­swer­able or tech­ni­cal ques­tions which have the side pur­pose to show to the cus­tomer the real work to do.