Playing the Russian roulette with your business · the excitement of the randomness

Af­ter spend­ing a few months in an or­di­nary small French soft­ware de­vel­op­ment com­pa­ny, I’m tru­ly amazed at the ef­fort which makes the whole process high­ly un­pre­dictable and in­creas­es the odds of fail­ing at every step.

If work­ing for a well-or­ga­nized large soft­ware de­vel­op­ment cor­po­ra­tion in USA is like dri­ving a new, well-work­ing and care­ful­ly checked and GPS-equipped car on a desert high­way, work­ing for those small com­pa­nies is like dri­ving an old, half-bro­ken mo­tor­cy­cle in the mid­dle of Sa­van­na with no map: you will have lots of oc­ca­sions to be sur­prised dur­ing the trip and can still get at des­ti­na­tion, but you may as well get lost and fin­ish your jour­ney as lion’s break­fast.

The fol­low­ing ar­ti­cle is a rant, but a rant which, in my opin­ion, high­lights quite well the way things work in some com­pa­nies.

Do­ing stuff the wrong way

The hir­ing process is care­ful­ly de­signed to pick peo­ple at ran­dom: in­deed, hir­ing only skill­ful de­vel­op­ers would be bor­ing, since their work is more pre­dictable. Be­gin­ners, on the oth­er hand, can rein­vent the wheel, or can come with some fan­cy de­sign which cor­re­sponds ex­act­ly to the ab­stract fac­to­ry pat­tern, but is not called this way, be­cause, af­ter all, it would be bor­ing to find the same pat­terns with their real names every­where: guess­ing which one is which is much more ex­cit­ing. More­over, be­gin­ners can come with a so­lu­tion which will nev­er cross the mind of any ex­pe­ri­enced de­vel­op­er, be­cause no one with a bit of ex­pe­ri­ence will do a such frag­ile or com­pli­cat­ed thing.

When start­ing the pro­ject, the first thing to do is to avoid writ­ing any soft­ware re­quire­ments, be­cause, frankly, it’s much more en­joy­able to just dig in the de­vel­op­ment process, dis­cov­er the re­quire­ments pro­gres­sive­ly, and nev­er track them nor write them down: it helps con­tra­dic­to­ry re­quire­ments to flour­ish, and means that at any mo­ment, the whole pro­ject can be tear apart, be­cause it doesn’t re­spond to a crit­i­cal re­quire­ment which any cus­tomer would need.

When writ­ing code, most stan­dards and con­ven­tions should be re­ject­ed, be­cause, one, no­body cares—it’s much more in­ter­est­ing to spend five min­utes try­ing to guess the par­tic­u­lar con­ven­tion just to dis­cov­er at the end that there is none, and, two, stan­dards kill cre­ativ­i­ty, and it is ob­vi­ous that any­one who have no idea what clean code is would be the best can­di­date for cre­ative cod­ing.

By the way, the re­jec­tion of stan­dards shouldn’t be lim­it­ed to code. For ex­am­ple, stor­age of pass­words is an ex­cel­lent ground to show the full ar­se­nal of cre­ative ideas. Like a home-made en­cryp­tion tech­niques, be­cause, every­one would agree, us­ing SHA256 is booor­ing! A cus­tom-made al­go­rithm may be a com­plete dis­as­ter, be­cause the per­son has no knowl­edge of cryp­tog­ra­phy or math­e­mat­ics what­so­ev­er, but that’s even bet­ter: it adds the ex­cite­ment to know that the ap­pli­ca­tion can be hacked, and the pass­words stolen.

Take types. Why would you ever use some­thing like a con­cur­rent stack, pro­vid­ed by the frame­work? Booor­ing! Take your cre­ativ­i­ty at the next lev­el and rein­vent the wheel from a sim­ple ar­ray. Bonus points if it be­haves like a hash­set, ran­dom­ly dead­locks on Fri­day ex­cept when it’s rain­ing, and uses a hand-made sin­gle­ton which is called dif­fer­ent­ly and can cre­ate some­times its sec­ond in­stance if you in­sist a bit.

In the same way, why would you ever use a GUID? Isn’t it more in­ter­est­ing to in­vent your own, which ap­pears to have noth­ing unique, but that only in­creas­es its val­ue, since you can nev­er be sure at 100% that a new­ly gen­er­at­ed en­ti­ty has an ID which is al­ready in use?

I’ve for­got­ten test­ing. Wait, what test­ing? Test­ing is in­her­ent­ly bad: it is the rou­tine in­car­nat­ed. Look at those TDD fans: cre­ate test, fail it, im­ple­ment the log­ic, pass the test. Cre­ate, fail, im­ple­ment, pass. Cre­ate, fail, im­ple­ment, pass. Such a mo­not­o­nous task! The worst of all is that tests make your code more pre­dictable, and that’s not fun­ny. What’s fun­ny is to push the new ver­sion to pro­duc­tion and won­der how long would it take to see the first crash.

If, among all test­ing, there is one type which can be kept, it’s un­doubt­ed­ly the man­u­al test­ing: the good old test­ing done by hand, with no for­mal check­list, nor list of pro­ce­dures to fol­low. This method­ol­o­gy is tru­ly the best one, since it en­ables to pass the test once, and then fail it the next time be­cause of a slight, bare­ly no­tice­able dif­fer­ence in the pro­ce­dure.

The lack of pro­ce­dures and au­toma­tion should nev­er skirt the de­ploy­ment ei­ther. Who can imag­ine the plea­sure and the sat­is­fac­tion you feel when you fin­ish the man­u­al de­ploy­ment of a web ap­pli­ca­tion which re­quires to hack the data­base, up­date it near­ly with your bare hands (giv­en that if you wrong­ly in­cre­ment, by hand, the schema ver­sion, some­thing, some­where, will fail), then copy-paste some files to the serv­er in or­der to dis­cov­er two min­utes lat­er that you ac­tu­al­ly copy-past­ed files from serv­er to your lo­cal ma­chine, and by the way, it was even a wrong serv­er, then do some voodoo mag­ic, and, at the end, hack the serv­er con­fig­u­ra­tion and dis­cov­er that all the web­sites of all your cus­tomers went down!

To jus­ti­fy this mad­ness, there is one thing that is men­tioned par­tic­u­lar­ly of­ten:

“Our prod­uct sim­ply works; we don’t have to care about how it’s done.”

Well, it doesn’t. And no, ac­tu­al­ly, you should care.

How does it start?

In or­der to reach the gates of il­lu­sion­is­tic cre­ativ­i­ty and free­dom and achieve the ex­cite­ment of play­ing Russ­ian roulette, one shouldn’t make too much ef­fort: hir­ing with your eyes closed and en­sur­ing work­ing con­di­tions which will make cry any mo­ti­vat­ed de­vel­op­er would do the trick.

What hap­pens is that ran­dom hir­ing process, based on some se­ri­ous sta­tis­ti­cal ap­proach­es and met­rics, like the stu­pid years of ex­pe­ri­ence or the dumb num­ber of pro­jects done or the sil­ly num­ber of lan­guages learnt, has every­thing to en­sure that you get the least skill­ful code mon­keys in your com­pa­ny. The ones who don’t know how to write code. The ones who have nev­er read any book about soft­ware de­vel­op­ment. The ones who think that more LOC is bet­ter than less. The ones who work for mon­ey, not be­cause they en­joy what they do.

This gives a sol­id foun­da­tion of in­com­pe­tent work­ers who, giv­en enough mo­ti­va­tion, would con­stant­ly in­no­vate by bring­ing the crap­pi­est so­lu­tions to the most or­di­nary prob­lems. The risk is that since the hir­ing process is ran­dom, by bad luck, one can hire some­body a bit less… say… cre­ative. Some­body who has read books, learnt stuff, and is a ter­ri­fy­ing il­lus­tra­tion of stan­dards, bore­dom, pre­dictabil­i­ty and rou­tine.

Such per­son can shake the foun­da­tions, or even dare to con­vince oth­ers that TDD could have its ben­e­fits or that con­tin­u­ous in­te­gra­tion may be not so bor­ing, or that PBKDF2 can be used in­stead of “en­code this to Base64, shift all bytes to the right, sep­a­rate them by pairs, mul­ti­ply num­bers in pairs and con­cate­nate their hexa­dec­i­mal rep­re­sen­ta­tion.” Even if such per­son wouldn’t be able to do all that, she could still write clean, pre­dictable code, or even start to add unit tests to the code­base, or par­tial­ly au­to­mate the de­ploy­ment process, or, in gen­er­al, talk too much. So, this per­son must leave.

In or­der for such per­sons to leave, one can cre­ate ap­pro­pri­ate work­ing con­di­tions. Scor­ing zero on Joel test is a good start; if it isn’t enough, The Dai­ly WTF is an ex­cel­lent source of in­spi­ra­tion. Fi­nal­ly, any­body can in­vent some tricks to con­vince those per­sons that this com­pa­ny is not a good place for them. Make changes to the (im­plic­it and nev­er writ­ten) re­quire­ments in a way that it would de­mand mo­not­o­nous work which takes time, and make those changes just a few days be­fore the dead­line. En­force the lack of style guide­lines. En­force stu­pid rules no­body needs. Make them read du­pli­cate code in­side a 1000-LOC func­tion with no com­ments and a log­ic writ­ten by some­body who was on a roller coast­er when writ­ing it. And if they dare to crit­i­cize, tell them:

“Our prod­uct sim­ply works; we don’t have to care about how it’s done.”

With a pre­dictable pat­tern—the only pre­dictable pat­tern out there—of an­chor­ing the un­skilled staff and mak­ing cry any­body who dare to show any de­sire to do how suc­cess­ful cor­po­ra­tions do, a part of the task is done.

An­oth­er part is to main­tain a good en­vi­ron­ment which would cul­ti­vate the Cul­ture of Ran­dom­ness. To do so, met­rics are im­por­tant, be­cause with­out met­rics, there is no way to pro­mote good peo­ple—peo­ple who ac­tu­al­ly sus­tain and boost the fairy cre­ativ­i­ty and free­dom, and pro­tect your com­pa­ny against the Dev­il of Rou­tine.

Those met­rics, one can pick them care­ful­ly. It is im­por­tant to choose well, since a bad met­ric can en­cour­age skill­ful de­vel­op­ers—thing we want to avoid.

In or­der to mit­i­gate such risk, one should se­lect only the met­rics which are list­ed in the lit­er­a­ture as be­ing com­plete­ly mean­ing­less, or—even bet­ter—sim­ply harm­ful.

For ex­am­ple, mea­sure the num­ber of lines writ­ten by pro­gram­mers, and re­ward those who have in­flat­ed the code base the most. Make it clear that those who re­duce the LOC through refac­tor­ing will be fired.

An­oth­er ex­am­ple: in­stead of valu­ing the re­sults, count the num­ber of hours a pro­gram­mer spent at work. Make it clear that some­one who spends eight hours per day in front of his desk, sleep­ing half of this time, is much more valu­able for your com­pa­ny that some­one who can ac­com­plish the same tasks four times as fast, but stays only five hours at the work­place.

Re­mem­ber: you should nev­er mea­sure any­thing of a real val­ue. Don’t even dare to mea­sure some­thing like the code cov­er­age. Oh, wait, true, you can’t, since there are no tests in the first place.

Aside di­rect met­rics, peer sup­port is an ex­cel­lent thing too. For ex­am­ple, sim­plic­i­ty should be dis­cour­aged at all costs. Some­body made an ef­fort of cre­at­ing sim­ple to un­der­stand ar­chi­tec­ture and refac­tored her code enough to make it un­der­stand­able at the first glance? Shame on her! Any­body can do such easy thing, right? Try in­stead to write a 2000-LOC code with thir­ty gotos—that’s the hard­core of pro­gram­mer’s life.

How does it end?

Imag­ine that you achieved to hire ran­dom peo­ple to do your soft­ware prod­uct in a ran­dom way, ran­dom­ly mea­sured us­ing ran­dom cri­te­ria. You end up with a ran­dom­ly work­ing prod­uct which fills a ran­dom mar­ket. Giv­en the lack of com­pe­ti­tion, or the fact that the com­peti­tors are em­brac­ing ran­dom­ness in the same way, you ran­dom­ly suc­ceed and do a damn good amount of mon­ey per year. In­deed, every­thing ran­dom­ly and mag­i­cal­ly works, the re­sults are pos­i­tive, so why would you ever care about how the prod­uct is done?

The first rea­son to care is the fact that your prod­uct might grow. New fea­tures may be re­quest­ed by the cus­tomers, or you may want to fill neigh­bor mar­ket nich­es. Step by step, the code base will be­come big­ger, and at the same time crap­pi­er, be­cause no one would care about refac­tor­ing. It’s pret­ty easy to work with a small and crap­py code base, but work­ing with a large and even crap­pi­er one is a dif­fer­ent sto­ry. Changes will be­come slow­er and cost­lier. Ran­dom bugs, arisen from new changes, will be­come more and more fre­quent. While you start­ed with a gazelle, you’ll end up with an old, fat ele­phant that can bare­ly move and spends his time pee­ing un­der him­self.

When, in­stead of re­spond­ing to your cus­tomers’ needs and the con­stant­ly chang­ing de­mand, you find your­self clean­ing ele­phants’ pee, this is the first op­por­tu­ni­ty to see that some­thing can go very wrong. From there, you have two choic­es: start do­ing things the old way, with all those stan­dards, best prac­tices, Ag­ile, QA and oth­er man­i­fes­ta­tions of the Rou­tine, or keep your head down and do the ram in the di­rec­tion of the Sa­cred Ran­dom­ness, by adopt­ing ran­dom mea­sures to ran­dom­ly iden­ti­fied ran­dom prob­lems which may even nev­er ex­ist. Staff—the one which cre­ates the sol­id foun­da­tion of cre­ative in­com­pe­tence—may also come with cre­ative­ly brain-dam­aged ideas. For ex­am­ple, you may end up rewrit­ing the prod­uct from scratch while us­ing the ex­act same prac­tices to keep the old good ran­dom­ness. No need to say that the sec­ond choice goes to­wards the Russ­ian roulette, giv­ing a com­pet­i­tive ad­van­tage to any com­pa­ny will­ing to steal… well, bet­ter to say “take” your cus­tomers.

The lack of flex­i­bil­i­ty in the prod­uct it­self is not the only el­e­ment which makes the end of the sto­ry nei­ther im­prob­a­ble, nor so ran­dom.

A com­pa­ny which hired a dream team filled with ex­perts who will cre­ate a high-qual­i­ty soft­ware prod­uct can af­ford fail­ing mis­er­ably a pro­ject, or lead­ing it to a dead end, or do any­thing which will cause it to be­come as at­tract­ing to the cus­tomers as a fat old ele­phant that can bare­ly move. The rea­son why the fail­ure is ac­cept­able for such com­pa­ny is that giv­en its po­ten­tial, those ex­perts will eas­i­ly come with great ideas and—this is cru­cial—they will learn from their mis­takes and avoid do­ing the same er­ror again in the new pro­ject.

On the oth­er hand, a com­pa­ny which em­braces in­com­pe­tence and the Bel­lowed Ran­dom­ness will have much more dif­fi­cul­ty to com­plete­ly re­ori­ent it­self in or­der to tar­get a dif­fer­ent mar­ket, since there would be no one to give out­stand­ing in­no­v­a­tive ideas. And even if such re­ori­en­ta­tion would be pos­si­ble, the fact that the cul­ture of the com­pa­ny won’t change will give all the odds for a fail­ure in the new mar­ket as well.

A few decades ago, IBM was a leader in hard­ware and soft­ware pro­duc­tion. To­day, my girl­friend doesn’t even know what IBM is, and nev­er heard the name. De­spite the im­pres­sive move from a leader cor­po­ra­tion known by any­one to an en­ti­ty known only by IT peo­ple and busi­ness­men, IBM stays a suc­cess­ful multi­na­tion­al cor­po­ra­tion, far from be­ing dead. Why? Sim­ply be­cause IBM hires peo­ple who know their job, and do it well, peo­ple who know that the Love­ly Ran­dom­ness has no place in IT, peo­ple who seek im­prove­ment in every­thing they do, peo­ple who have ideas—great ideas. By em­brac­ing the in­dus­try stan­dards and best prac­tices, and par­tic­i­pat­ing in their growth, IBM has small room for fail­ure. They don’t de­liv­er ran­dom prod­ucts with ran­dom de­lays at a ran­dom cost.

By do­ing the op­po­site of what large suc­cess­ful cor­po­ra­tions do, com­pa­nies dig their grave. They do it with all their en­er­gy and won­der­ful cre­ativ­i­ty, seek­ing any op­por­tu­ni­ty to fail in the most ran­dom way, and to fail bad­ly, keep­ing as lit­tle chances as pos­si­ble to sur­vive the bad times and the good com­peti­tor.