Workflows

Im­por­tant note: this ar­ti­cle fo­cus­es specif­i­cal­ly on Share­Point work­flows. What is writ­ten here doesn't ap­ply to Win­dows Work­flow Foun­da­tion (WF) and sim­i­lar work­flow tools, which have a per­fect­ly valid us­age and em­pow­er de­vel­op­ers with the abil­i­ty to make some busi­ness rules eas­i­er to change by non-tech­ni­cal users. While WF and source code are com­ple­men­tary, work­flows for Share­Point sub­sti­tute to the code. This means that their na­ture is very dif­fer­ent from gen­er­al work­flow tools.

I spent a week on a Nin­tex train­ing ses­sion. For those who don't know, Nin­tex is a com­pa­ny which cre­at­ed an al­ter­na­tive for work­flows in a con­text of Share­Point. The goal:

• Be­ing sim­ple to use, so that any­one can cre­ate and change work­flows.

• Be­ing much more pow­er­ful com­pared to Share­Point work­flows.

While I agree with the fact that the tool is more pow­er­ful than its free lit­tle broth­er, I'll try to show in the fol­low­ing ar­ti­cle how wrong the ap­par­ent sim­plic­i­ty is. Try­ing to cre­ate a tool which is sim­ple and easy by be­ing vi­su­al, Nin­tex only achieved some­thing which wastes your time and kills your pro­duc­tiv­i­ty while en­cour­ag­ing you to bang your head and ul­ti­mate­ly killing your­self try­ing to solve any prob­lem which can be solved writ­ing five lines of code in any pro­gram­ming lan­guage.

Edit­ing

The most im­por­tant draw­back seems to be the poor user ex­pe­ri­ence which leads to slow­er work. Pro­gram­ming is about text, and this is what makes it par­tic­u­lar­ly pow­er­ful, as well as fast. In any 3GL cou­pled with a com­pre­hen­sive frame­work, a few key­strokes are enough to cre­ate a lot of busi­ness log­ic. With work­flows, the sim­plest log­ic be­comes a pain, trans­lat­ed into dozens of clicks, drag & drops and lots and lots of forms to fill.

Let's take a sim­ple ex­am­ple. We want to load the list of prod­ucts to pre­order and email them to a spe­cif­ic per­son. We start by load­ing the data from a list and ex­clud­ing the du­pli­cates. If the re­sult­ing se­quence con­tains some­thing, we con­cate­nate the names (a com­ma and a space are in­sert­ed in be­tween) and pre­fix them with “Prod­ucts to pre­order.” If the re­sult­ing se­quence is emp­ty, just put a log mes­sage say­ing that, well, there is noth­ing to pre­order.

This is the re­sult us­ing Nin­tex Work­flows:

For each step, there were plen­ty of clicks and plen­ty of in­for­ma­tion to fill in forms. A com­plete night­mare, giv­en the ex­treme sim­plic­i­ty of the task. In­deed, if we com­pare that to source code, the re­sults are fun­ny to look at:

var uniqueNames = this.Data.ProductsQueue.Select(p => p.Name).Distinct();
if (uniqueNames.Any())
{
    var message = "Products to preorder: " + string.Join(", ", uniqueNames);
    this.Messaging.Send(this.Messaging.ToDefault, message);
}
else
{
    this.LogInfo("There are no products to preorder.");
}

In six lines of code, you can see the whole log­ic at once. No clicks re­quired.

The fact that we deal with text makes it pos­si­ble to em­pow­er de­vel­op­ers with many nice tools, such as auto-com­ple­tion, syn­tax high­light­ing and refac­tor­ing. On Lin­ux, grep and oth­er pow­er­ful tools can be used as well. Fi­nal­ly, text ed­i­tors such as vim or Vi­su­al Stu­dio make it ex­treme­ly easy to cut and paste, undo, move blocs of code, search through code and auto-for­mat it to make the life of a pro­gram­mer as easy as pos­si­ble. When deal­ing with work­flows, even a task as easy as copy-past­ing the log­ic of a con­di­tion (for ex­am­ple when switch­ing from an if/else block to an execute-if block) is im­pos­si­ble: if you have five con­di­tions match­ing vari­ables to field val­ues, this means twen­ty form fields to change by hand.

Those are not the only ben­e­fits of plain text ap­proach­es. We, de­vel­op­ers, rely on dai­ly ba­sis on many oth­er tools: back­ground com­pi­la­tion which dis­cov­ers the er­rors near­ly in­stant­ly as we write them, sta­t­ic analy­sis which pre­vents us from in­tro­duc­ing sub­tle and dif­fi­cult to find bugs, for­mal­ized re­views which help us to be­come bet­ter at what we are do­ing, while find­ing even more bugs be­fore they reach pro­duc­tion.

Plain text also im­plies the ben­e­fits of a ver­sion con­trol: ver­sion­ing with diff, blame, branch­ing, op­ti­mistic/pes­simistic con­cur­ren­cy con­trol. With Nin­tex work­flows, don't even think about work­ing on a work­flow at the same time as your col­league. It's like stor­ing source code in a shared fold­er and try­ing to change it in par­al­lel: you'll see fun­ny things hap­pen­ing all the time.

No­tice that the things I'm talk­ing about here are very ba­sic. It's not like I was blam­ing a work­flow tool for not sup­port­ing as­pect ori­ent­ed pro­gram­ming (AOP) or not hav­ing de­sign pat­terns. Could you list five oth­er things you have in your fa­vorite lan­guage and which is im­pos­si­ble to do with Nin­tex work­flows?

De­bug­ging

When it comes to de­bug­ging, it looks like the de­sign­ers of the work­flow tools made every­thing they could to avoid this part of the de­vel­op­ment process. The only tool which is ac­tu­al­ly im­ple­ment­ed in Nin­tex Work­flows is the abil­i­ty to put a mes­sage in a log. In oth­er words, if I want to be able to trace the ex­e­cu­tion of a work­flow in de­tails, I need to dou­ble the quan­ti­ty of work­flow en­tries. Ex­cept that I wouldn't, be­cause it ap­pears that in most cas­es, such log­ging is dis­abled in pro­duc­tion. Nice.

Even worse, Nin­tex Work­flows will be very po­lite with you and will nev­er blame you for mak­ing mis­takes, un­like com­pil­ers do. In fact, if you do make a mis­take, Nin­tex Work­flows will usu­al­ly skip over, pro­duc­ing a com­plete­ly in­con­sis­tent re­sult, and let you fig­ure out what hap­pened. You want to load a se­quence into a num­ber and then do a forEach on this num­ber? With Nin­tex Work­flows, you can! (“For God's sake, why isn't it loop­ing, while there are el­e­ments in my col­lec­tion?!”)

The fun­ni­est part is that it's one thing where work­flows could ex­cel. Every step could log its in­puts and out­puts, and even­tu­al­ly its in­ter­nal work­ings spe­cif­ic to the type of ac­tion. For in­stance, a loop could log the time it took to ex­e­cute each it­er­a­tion.

Now com­pare this to code. In pro­gram­ming, we have step-by-step de­bug­ging, we have break­points, we have ex­cep­tions with a stack trace. Would you ever work with a pro­gram­ming lan­guage which lacks those de­bug­ging fun­da­men­tals?

Pri­or to de­bug­ging per se, you of­ten have the ben­e­fit of sta­t­ic check­ers (which, let's be hon­est, find a lot of er­rors which would be­come bugs which might be dif­fi­cult to spot dur­ing de­bug­ging) and some­times (IDEs for C# or Java, for in­stance), you also ben­e­fit of back­ground com­pi­la­tion which makes it pos­si­ble to cor­rect bugs in near real-time.

Test­ing

Should I men­tion that there is no au­to­mat­ed test­ing for work­flows? Right, you test it by hand, and you dis­cov­er re­gres­sions through phone calls of your an­gry users.

Pro­fil­ing

When code is slow, one has to run a pro­fil­er and to find the bot­tle­neck. This doesn't re­solve the prob­lem, but at least makes it pos­si­ble to de­ter­mine pre­cise­ly where is the prob­lem.

Once the prob­lem is lo­cat­ed, one may run bench­marks and try dif­fer­ent op­ti­miza­tions. Ul­ti­mate­ly, a de­vel­op­er re­moves the bot­tle­neck through a bunch of tri­al and er­ror steps.

Nin­tex Work­flows shows how long every step takes with­in a giv­en run of a work­flow. This might help (al­though it's not gran­u­lar enough), but, se­ri­ous­ly, I have to hov­er over every step and com­pare them by hand? Any pro­fil­ing tool I've seen shows me the hot path, as well as the re­main­ing data in a for­mat I can ac­tu­al­ly use.

Also, what about the CPU load or mem­o­ry used? Nin­tex Work­flows con­sid­ers that I don't need this in­for­ma­tion.

Con­vo­lut­ed ap­proach of a prob­lem which doesn't ex­ist in the first place

For me, those work­flow tools rep­re­sent a con­vo­lut­ed ap­proach of a prob­lem which doesn't ex­ist in the first place. Com­pared to code, we are ex­pect­ing three ben­e­fits from work­flows:

• Abil­i­ty for non-de­vel­op­ers to deal with busi­ness log­ic. Good in the­o­ry, it doesn't work this way in prac­tice. I've ex­plained how com­pli­cat­ed and er­ror prone work­flow tools are. This means that non-tech­ni­cal per­sons who will try to use work­flows will quick­ly be over­whelmed by the user-hos­tile as­pect of the thing. Those per­sons will end up ei­ther ask­ing a pro­gram­mer to cre­ate a work­flow, and then nev­er change it, or hire a pro­gram­mer to main­tain and mod­i­fy the work­flow.

• Vi­su­al­iza­tion. Work­flows are nice. You im­me­di­ate­ly see what is go­ing on. Wait, are you? Ac­tu­al­ly, when I look at a work­flow oth­er than the most el­e­men­tary one, I can't see shit. I need to scroll hor­i­zon­tal­ly and ver­ti­cal­ly all the time. I need to click every­where, open dozens of pop­ups. If there is hell for pro­gram­mers and in hell, they have night­mares, I imag­ine that in those night­mares, they have to de­bug a work­flow.

  The fun­ny thing is that we do have enough ex­cel­lent vi­su­al­iza­tion tools for main­stream lan­guages. For .NET, there is NDe­pend, and Vi­su­al Stu­dio Ul­ti­mate con­tains vi­su­al­iza­tion tools as well. One can note that those vi­su­al­iza­tion tools work at a lev­el of meth­ods, not con­trol state­ments. There is a good rea­son for that: there is no rea­son to have a vi­su­al tool for con­trol state­ments for any sane (and so, short) method, be­cause such method will fit on screen, and text will be the most ap­pro­pri­ate vi­su­al­iza­tion.

  So for short meth­ods, text vi­su­al­iza­tion doesn't need to be sub­sti­tut­ed by any­thing else. Large sys­tems with sane meth­ods have vi­su­al­iza­tion tools al­ready. So what's the point of the work­flows? They only per­vert the vi­su­al­iza­tion of sim­ple log­ic, and more com­pli­cat­ed log­ic can­not be vi­su­al­ized any­way with them.

• Ex­e­cu­tion stand­by. Work­flows are good at in­ter­act­ing with users. They may re­quest a per­son to do a task, and then wait for hu­man ac­tion as long as it will take. They are de­signed for that. The prob­lem is that we don't need work­flows to wait for user ac­tions. Asyn­chro­nous pro­gram­ming ex­ist­ed long be­fore work­flows were in­vent­ed, and every main­stream lan­guage (OK, ex­cept PHP) sup­ports asyn­chro­nous tasks. Work­flows are in no way bet­ter than pro­gram­ming lan­guages at stay­ing in­ac­tive for long pe­ri­ods of time, and wak­ing up when a per­son fi­nal­ly re­spond­ed. API-ori­ent­ed de­sign makes it huge­ly easy to im­ple­ment the thing.

So what is the al­ter­na­tive?

The al­ter­na­tive

It's ex­treme­ly sim­ple. Take a mes­sage queue ser­vice (MQS), an API and a Python script. Yes, that's all you need. The Python script in­ter­acts with both the API and the MQS. Let's see:

• Abil­i­ty for non-de­vel­op­ers to deal with busi­ness log­ic. To­tal­ly. Give me a week, and I can train any non-tech­ni­cal per­son to deal with Python bet­ter than I can deal with work­flows now, af­ter a week of train­ing. As I ex­plained pre­vi­ous­ly, work­flows can­not be used by a non-tech­ni­cal per­son any­way. Python, on the oth­er hand, can eas­i­ly be learned and the per­son can use those skills lat­er for writ­ing short scripts and oth­er stuff.

• Vi­su­al­iza­tion. To­tal­ly. Python be­ing an el­e­gant lan­guage, its code is easy to read and un­der­stand. Func­tions make it pos­si­ble to very eas­i­ly break the code into parts and re­or­ga­nize and refac­tor it lat­er. Agreed, this comes with ex­pe­ri­ence, but so does the use of the work­flows. Well, is it so? I'm still wait­ing to see a work­flow which is easy to read and un­der­stand, when I don't need to do hun­dreds of clicks to open lots of pop­ups.

• Ex­e­cu­tion stand­by. To­tal­ly. With web in­fra­struc­ture and a MQS, the code is ex­e­cut­ed when need­ed and can re­act to per­son's ac­tions. It don't even need to NOP around. It just ex­e­cutes a giv­en flow, and then stops, to be called once again when there is some­thing else to pro­ceed.

Some oth­er ben­e­fits:

• It's free. Those thou­sands of dol­lars you throw away every year for a work­flow so­lu­tion, well, you can keep it or spend for some­thing ac­tu­al­ly use­ful.

• It's ex­ten­si­ble by de­vel­op­ers. They can ex­tend the API to pro­vide ad­di­tion­al func­tion­al­i­ty which is too com­plex to im­ple­ment for non-tech­ni­cal users. In turn, those users will use the API to call the spe­cif­ic func­tions in a very sim­ple way.

• You have all the ben­e­fits of a pro­gram­ming lan­guage. Syn­tax high­light­ing, ver­sion con­trol, de­bug­ging, pro­fil­ing,... you name it.

• The skills the non-tech­ni­cal per­sons ac­quire learn­ing Python can be used else­where to ac­com­plish ba­sic script­ing tasks.

In my hum­ble opin­ion, such work­flow tools are based pure­ly on mar­ket­ing, while pro­vid­ing ab­solute­ly no val­ue what­so­ev­er (if only could they be less harm­ful!) I wish things were dif­fer­ent, and I hope this overview of the sub­ject will help my fel­low de­vel­op­ers to de­fend their pro­ject against the in­tro­duc­tion of work­flows.