How to deadlock a project

In 2014, I wrote the ar­ti­cle Tak­ing hostages with Wa­ter­fall, ex­plain­ing how Ag­ile moves the pow­er of de­ci­sion about the scope and the cost of the pro­ject from the ones who make the soft­ware to the ones who pay for it. For years, I've ob­served a prac­tice by un­scrupu­lous free­lance pro­gram­mers and soft­ware de­vel­op­ment com­pa­nies which con­sist­ed of giv­ing an un­re­al­is­tic price/time es­ti­mate for a giv­en pro­ject in or­der to get a cus­tomer, and then play on the sunk cost fal­la­cy ef­fect to ex­tort mon­ey from a cus­tomer.

The schema works this way. The cus­tomer asks to quote a pro­ject. You find that mak­ing it would cost from $150,000 to $200,000 in the very best case. You tell the cus­tomer that your es­ti­mate is $100,000. This way, if there are com­peti­tors which are hon­est, they would give twice the amount, and the cus­tomer will choose you, and not them. Since there is no way for the prod­uct to be de­vel­oped for less than $150,000, you spend the $100,000 while be­ing open to any change re­quests from the cus­tomer, and once you reached your orig­i­nal es­ti­mate, you ask the cus­tomer what to do. You can ei­ther give to the cus­tomer the source code, giv­en that the prod­uct is com­plete­ly un­fin­ished and can­not work in its cur­rent state, or you can con­tin­ue if the cus­tomer agrees to pay more. Since no­body wants to ac­knowl­edge that he wast­ed $100,000 for a use­less pieces of code, the cus­tomer is like­ly to tell you to con­tin­ue. Nat­u­ral­ly, if the cus­tomer starts blam­ing you for bad man­age­ment, you re­ply that the ex­tra mon­ey is ex­plained by the chang­ing re­quire­ments. Then, you con­tin­ue spend­ing mon­ey with­out de­liv­er­ing any­thing, un­til the cus­tomer re­fus­es to pay. I don't know how things are in the Unit­ed States, but in France such scam is com­plete­ly le­gal, and also very com­mon. My es­ti­mate (not backed up by any re­search) is that 95 to 99% of pro­jects in France hap­pen ex­act­ly like this.

The way to es­cape from this trap is, for a cus­tomer, to re­quire the pro­ject to be de­vel­oped in an Ag­ile way. At the end of every sprint, the cus­tomer should ex­pect to see a work­ing prod­uct, that is, a prod­uct which can and pos­si­bly is used by the end users. If the free­lancer or the IT com­pa­ny is un­able to do that, then they shouldn't be paid. It is as sim­ple as that.

But this ar­ti­cle is not about cus­tomers and free­lancers and hostages and sunken cost fal­la­cy. It is about the ma­jor mis­take I made when de­vel­op­ing the blog en­gine which pow­ers the web­site you're us­ing right now. In fact, I took a lib­er­ty of not us­ing con­tin­u­ous de­ploy­ment for some time for this pro­ject, and cre­at­ed a real man­age­ment dis­as­ter which could lead to a can­cel­la­tion of the pro­ject it­self.

I start­ed to work on the blog en­gine in 2014. Over­all, the de­vel­op­ment was smooth, I was adding fea­tures, and I was re­leas­ing them as soon as I was con­fi­dent enough that they work prop­er­ly. Then, it all changed one day. In 2019, I told my­self that the blog en­gine starts to be quite old. There were a lot of things I dis­liked about it, and so I need­ed to per­form a lot of changes. In­stead of per­form­ing small changes, and re­leas­ing them step by step, I some­how found my­self mak­ing large changes one af­ter an­oth­er, with no re­leas­es what­so­ev­er. More changes crept in, and at some point, I was un­able to re­lease. The en­gine I had on my ma­chine was com­plete­ly dif­fer­ent from the en­gine run­ning on the servers; even data­base schema was dif­fer­ent.

Now, I found my­self in a dif­fi­cult sit­u­a­tion. I could throw away all the changes and start over, but sunk cost fal­la­cy ef­fect pre­vent­ed me from do­ing it. Or I could con­tin­ue the changes, hop­ing one day to be able to mi­grate the data to the new schema and to fi­nal­ly de­ploy the new ver­sion to pro­duc­tion. This is what I was try­ing for some time, but with­out see­ing the ap­pli­ca­tion run­ning out there and be­ing used by real per­sons, and with­out even know­ing if I'll be able to de­ploy it one day, the mo­ti­va­tion was at its low­est, and I stopped work­ing on the pro­ject by the au­tumn 2019.

Fig­ure 1 Num­ber of com­mits re­lat­ed to the blog en­gine.

It is only a few months ago that I forced my­self into fin­ish­ing what I start­ed. This came at a cost. The code base was prac­ti­cal­ly com­plete­ly rewrit­ten. The struc­ture re­mains most­ly the same, but the in­di­vid­ual files have prac­ti­cal­ly noth­ing to do with the old ones. This is com­pa­ra­ble to what would hap­pen if a cus­tomer, be­ing kept hostage by a scam­mer de­cides to go to an­oth­er com­pa­ny to rewrite the soft­ware. About a month lat­er, the pro­ject was in pro­duc­tion once again. From now, I can add fea­tures or fix is­sues, and know that the changes can be seen by the end users with­in min­utes.

The fact is, lots and lots of per­son­al pro­jects get aban­doned like this. There are most­ly two risks which can bring a per­son­al pro­ject to a halt. First, the pro­ject doesn't seem that in­ter­est­ing, af­ter all, or too com­plex, or just less fun than a new one. Sec­ond, the pro­ject dead­lock. For a pro­ject that just starts, the first risk is more im­por­tant. I have hun­dreds of pro­jects I've start­ed, and then nev­er came back to them. It is not a bad thing in it­self—de­vel­op­ers need to be able to play with some ideas, with­out hav­ing to com­mit to any­thing. Nat­u­ral­ly, some de­vel­op­ers tend to jump from pro­ject to pro­ject, nev­er fin­ish­ing any­thing; this is in­deed prob­lem­at­ic. As the pro­ject be­comes more ma­ture, the im­por­tance of the sec­ond risk grows. Ma­ture pro­jects are ma­ture be­cause of one thing: they are use­ful. This means that there are chances for a pro­ject to be main­tained, even if main­tain­ing it is not par­tic­u­lar­ly fun. Dead­locks, how­ev­er, can hap­pen in­de­pen­dent­ly of the use­ful­ness of the pro­ject, and they can kill even a pro­ject which is very valu­able.

The so­lu­tion is, you guessed, con­tin­u­ous de­ploy­ment. From the very ear­ly stage of a pro­ject, you en­sure it can be used to ac­com­plish a spe­cif­ic need, and keep­ing an eye on the time-to-mar­ket (TTM) met­ric (that is, the time it takes for a fea­ture to be de­vel­oped and re­leased) could help.

The thing is, this so­lu­tion works well in a cor­po­rate en­vi­ron­ment. The guy who pays the bills is usu­al­ly not the guy who codes, and that's great. This is great specif­i­cal­ly be­cause no­body cares if you want to re­lease or not: the stake­hold­er re­quires the fea­tures to be re­leased fast, and if you don't do it, you're fired. That's it. For per­son­al pro­jects, this doesn't work this way. One wants to be dis­ci­plined, and one wants to tell his friends that he's re­leas­ing of­ten, but the fact is that it can also be ex­treme­ly tempt­ing to try to push a heavy change or an in­ter­est­ing fea­ture, with­out both­er­ing about the de­ploy­ment and fea­ture tog­gles and data mi­gra­tion. And when you're fo­cus­ing on the ex­act prob­lem, you can miss the fact that your TTM goes from an hour to sev­er­al days, then weeks, then months.

Hon­est­ly, I don't know what would be a real so­lu­tion for per­son­al pro­jects. Maybe bet­ter met­rics could help. Maybe some­thing needs to au­to­mat­i­cal­ly throw away the changes if a giv­en TTM is reached. Do you have a so­lu­tion?