Narrative requirements

I was re­cent­ly dis­cussing with a group of fel­low de­vel­op­ers the qual­i­fy of doc­u­ments which are usu­al­ly done to list the prod­uct re­quire­ments. Af­ter com­plain­ing about the low qual­i­ty of most doc­u­ments I've seen (and this com­plaint was shared by the oth­er per­sons), I end­ed up talk­ing about two spec­i­fi­ca­tion doc­u­ments which, in my eyes, were un­usu­al­ly good at their job: the doc­u­ments I wrote for two pro­jects a few years ago. Re­call­ing the dis­cus­sion lat­er, I thought it would be in­ter­est­ing to bring more light on those two doc­u­ments, and what made them spe­cial.

The first one was a small pro­ject which was aban­doned a few months af­ter I start­ed work­ing on it be­cause of the lack of in­ter­est. The doc­u­ment I wrote was an ex­cel­lent ex­am­ple of for­mal meth­ods of re­quire­ments writ­ing. The pro­ject was born dur­ing the pe­ri­od of time where I was read­ing James Tay­lor's Man­ag­ing In­for­ma­tion Tech­nol­o­gy Pro­jects and sim­i­lar books, and I was con­vinced that for­mal re­quire­ments elu­ci­da­tion is the best way to do the job. It looks naive, in ret­ro­spec­tive, but I had my rea­sons to be­lieve it back then: the method­ol­o­gy was well known and well doc­u­ment­ed, and could prob­a­bly be ben­e­fi­cial for many pro­jects, if only busi­ness an­a­lysts would take any in­ter­est in those ap­proach­es.

Giv­en the small size of the pro­ject, the re­quire­ments doc­u­ment con­tained no more than fifty pages. Af­ter a short de­scrip­tion of the pro­ject it­self, the doc­u­ment ac­tu­al­ly list­ed, in a very for­mal way, every func­tion­al and non­func­tion­al re­quire­ment I could think of. This for­mal­ism, by the way, helped me lat­er to have a good grasp on non-func­tion­al re­quire­ments most de­vel­op­ers seem to lack.

When work­ing on the pro­ject it­self, I was re­al­ly proud of my work on re­quire­ments: every­thing was clear and well or­ga­nized and com­plete. Sur­pris­ing­ly, it ap­peared that I haven't missed a lot of as­pects of the sys­tem, in­clud­ing the non-func­tion­al ones--if I had to do the work again to­day, I would miss at least half of the things. A few fea­tures ap­peared dif­fer­ent than I orig­i­nal­ly thought, but keep­ing the re­quire­ments doc­u­ment up to date wasn't dif­fi­cult at all.

It could be per­fect, un­less one thinks about the ac­tu­al us­age. It ap­peared that I was both the busi­ness an­a­lyst and the de­vel­op­er. It was my sys­tem. I had to de­cide every as­pect of it, and I had the over­all view of the sys­tem in every de­tail. In real world, it doesn't work this way. Busi­ness rules are draft­ed by one guy, who is (hope­ful­ly) knowl­edge­able about the busi­ness do­main, but has usu­al­ly no clue about the tech­ni­cal as­pects. Tech­ni­cal rules are draft­ed by an­oth­er guy, who may call him­self an ar­chi­tect and claim to have dozens of years of ex­pe­ri­ence in IT, but who don't nec­es­sar­i­ly code on dai­ly ba­sis. And fi­nal­ly, the doc­u­ment is thrown to a bunch of guys who love their pro­gram­ming lan­guage and their IDE, but have no idea what the doc­u­ment is about, and don't nec­es­sar­i­ly want to read tech­ni­cal stuff. What makes things worse is that usu­al­ly nei­ther the busi­ness an­a­lyst, nor the ar­chi­tect, nor the de­vel­op­ers know the for­mal ap­proach­es for writ­ing re­quire­ments, and even if one of them ap­pear to know this stuff, the oth­ers don't, and couldn't care less.

This sit­u­a­tion leads, ob­vi­ous­ly, to the usu­al re­sult we all know: the doc­u­ment which, even when it is well done, is not read or un­der­stood by the per­sons who ac­tu­al­ly cre­ate the soft­ware prod­uct. This is an un­for­tu­nate fate of any for­mal re­quire­ments doc­u­ment: if the re­quire­ments step suc­ceed­ed for the first pro­ject, it's only be­cause my case was very spe­cif­ic. It couldn't be used as a valid il­lus­tra­tion of a tech­nique which works.

The sec­ond pro­ject used a dif­fer­ent ap­proach. There was ab­solute­ly no for­mal­ism, and it wasn't even look­ing as a re­quire­ments doc­u­ment. There were no lists, and no num­bers, and no cross-ref­er­ences. In­stead, there was plain text, a nar­ra­tive.

The doc­u­ment start­ed by giv­ing an over­all view of the sys­tem. It was a prod­uct which was a com­bi­na­tion of two small­er sys­tems in­ter­act­ing with each oth­er. Any­one who have read the first few pages had a very clear, but also not very de­tailed view of the sys­tem, and could ex­plain the pur­pose of it.

Every suc­ces­sive part of the nar­ra­tive en­tered in more and more de­tails. In or­der to work on a spe­cif­ic part of the sys­tem, one could read the spe­cif­ic part of the doc­u­ment, and there is was, all the in­for­ma­tion need­ed to un­der­stand this part in enough de­tail, and with enough ab­strac­tion from oth­er parts. The dif­fer­ent sec­tions were writ­ten in a way that made them au­tonomous: no ex­haust­ing cross-ref­er­ences, but rather a plain ex­pla­na­tion of all the de­tails one need­ed.

Nar­ra­tive style

In soft­ware in­dus­try, com­mu­ni­ca­tion is­sues rep­re­sent a ma­jor is­sue. Many per­sons in cor­po­rate world don't write and don't read. While it is usu­al­ly the task they are ex­pect­ed to do most of their time, they find it not that im­por­tant, and of­ten don't spend enough time do­ing it cor­rect­ly. They don't read their emails in de­tail, and they of­ten make no ef­fort when it comes to writ­ing.

The di­rect ef­fect is the ex­treme loss of per­for­mance due to the lack of prop­er com­mu­ni­ca­tion skills. Talk­ing about re­quire­ments, most con­tent I re­ceive from some cus­tomers is usu­al­ly un­read­able. Be­ing un­able to for­mu­late the most ba­sic ideas, those per­sons don't even make ef­forts com­mu­ni­cat­ing: they sim­ply draft stuff with­out even read­ing it be­fore dis­patch­ing it. How many times I found my­self read­ing some pa­per de­scrib­ing a fea­ture of a pro­ject, try­ing to get at least a slight­est idea about the thing, and dis­cov­er­ing lat­er when dis­cussing the sub­ject with the au­thor that what I imag­ined has ab­solute­ly noth­ing in com­mon with the ac­tu­al fea­ture!

Now, I be­lieve that it's not the fault of those per­sons that they com­mu­ni­cate so poor­ly. The cul­prits are the medi­um, the habits, the cul­ture, and im­i­ta­tion of ex­ist­ing cor­po­rate emails, doc­u­men­ta­tion, re­quire­ments doc­u­ments and any tech­ni­cal doc­u­men­ta­tion.

Now change the con­text. Tell the per­son that she should write a sto­ry, a book, a nov­el. The fo­cus is shift­ed from im­i­tat­ing oth­er poor­ly writ­ten doc­u­men­ta­tion to im­i­tat­ing ac­tu­al au­thors and real books. While not every book is equal, book au­thors write still slight­ly bet­ter than an av­er­age busi­ness an­a­lyst who cre­ates the list of re­quire­ments. The bet­ter the ref­er­ence to im­i­tate, the bet­ter is the re­sult.

This doesn't mean that every busi­ness an­a­lyst is able to write prop­er­ly and clear­ly only by chang­ing the style of the doc­u­ment. What this means, on the oth­er hand, is that through the shift in ref­er­ences, the av­er­age lev­el could only im­prove.

Read­ing

Tech­ni­cal doc­u­men­ta­tion is usu­al­ly bor­ing to read. The prac­tice is very dif­fer­ent from read­ing a nov­el: in­stead of fol­low­ing the nar­ra­tor, one has to jump through cross-ref­er­ences, los­ing one's track, go­ing back, read­ing again, try­ing to es­tab­lish links which were missed by the au­thor, and try­ing to keep the whole pic­ture in all its de­tails in one's head.

But it doesn't have to be bor­ing. Or, if it is bor­ing, it doesn't have to be writ­ten in the first place. Why would you write some­thing no­body would read? Or the ones who will be forced to read it, will suf­fer? It doesn't make sense.

If, on the oth­er hand, the doc­u­men­ta­tion is writ­ten as a nov­el, more peo­ple could en­joy read­ing it. And shar­ing it as well. Once the doc­u­ment is read and shared, it is also main­tained. What could be more un­for­tu­nate than spend­ing time on a doc­u­ment no­body reads, and which gets out­dat­ed very quick­ly any­way?

Struc­ture

A list of re­quire­ments, as usu­al­ly pre­sent­ed, has a struc­tur­al flaw: it is lin­ear. What I mean is that all re­quire­ments are at the same lev­el and are ex­pect­ed to be very gran­u­lar. This gran­u­lar­i­ty is prob­lem­at­ic on any or­di­nary pro­ject: go­ing down to every de­tail is not very dif­fer­ent from sim­ply writ­ing source code di­rect­ly.

Adopt­ing the nar­ra­tive style helps stay­ing close to the ac­tu­al process of re­quire­ments elu­ci­da­tion. One starts by get­ting an over­all view, and then adds de­tails, ob­tain­ing a high gran­u­lar­i­ty on the parts which seem clear at the mo­ment of writ­ing, and re­main­ing su­per­fi­cial on parts which look too vague at the ear­ly stage. This very dif­fer­ence be­tween parts de­scribed in de­tail and parts which are not is an es­sen­tial re­flec­tion of any busi­ness pro­ject.

This makes it also pos­si­ble to let the de­vel­op­ers fill the gaps. When the busi­ness an­a­lyst or, more prob­a­bly, the ar­chi­tect, can­not de­scribe every de­tail, de­vel­op­ers may adapt their im­ple­men­ta­tions and even­tu­al­ly get cre­ative, which is ex­act­ly what one should ex­pect from them.