Packages, dependencies and interaction between teams

Re­cent­ly, a ques­tion of Pro­gram­mers.SE passed un­no­ticed, be­cause of its length. This is un­for­tu­nate, be­cause the ques­tion high­lights a bunch of mis­un­der­stand­ings with­in many teams on a broad range of sub­jects, from Con­tin­u­ous In­te­gra­tion to de­pen­den­cies man­age­ment to col­lab­o­ra­tion be­tween teams with­in a com­pa­ny.

This is one of those sub­jects which, once you un­der­stand the idea, ap­pear ex­treme­ly sim­ple to im­ple­ment, but un­til you do, are very com­pli­cat­ed even to grasp. My an­swer on Pro­gram­mers.SE is not par­tic­u­lar­ly well writ­ten and I don't want to be more de­tailed there, so I'll try to do a bet­ter job here.

The sto­ry of a team

It all starts by a team de­vel­op­ing a soft­ware prod­uct in C#. They thought a lot about the ar­chi­tec­ture, and made a de­ci­sion to have a main ap­pli­ca­tion, a con­sole vari­ant, and three li­braries:

• The one deals with a cur­rent prob­lem which is en­coun­tered by many C# de­vel­op­ers across many pro­jects,

• The oth­er one deals with the specifics of the com­pa­ny, giv­en that oth­er teams with­in the same com­pa­ny could ben­e­fit from it,

• The third one is in­her­ent to the prod­uct it­self, and has very lim­it­ed val­ue out­side the scope of the pro­ject.

The pro­ject grows, and so does the team. Few months lat­er, a ded­i­cat­ed team is cre­at­ed in or­der to main­tain the first two li­braries. This team is split in two few months lat­er. This leads to nu­mer­ous mis­un­der­stand­ings and con­flicts be­tween the teams. As they com­mit to the same repos­i­to­ry, merges are painful to solve, and teams spend more time blam­ing each oth­er for break­ing the build than do­ing ac­tu­al work. A few at­tempts to set up ver­sion­ing fail, be­cause of the dif­fi­cul­ty of han­dling de­pen­den­cies.

This hap­pens a bit too of­ten. From what I've seen when au­dit­ing com­pa­nies, every time mul­ti­ple teams with­in the com­pa­ny try to in­ter­act, bad things hap­pen. Bad man­agers will start blam­ing peo­ple: lazy pro­gram­mers make use of the dif­fi­cult sit­u­a­tion to trans­fer re­spon­si­bil­i­ties to oth­ers and spend time do­ing noth­ing. We won't go this way, and will rather blame the process.

Blam­ing the process

In this case, blam­ing the process is easy, be­cause we have an ex­cel­lent ex­am­ple where, in a very sim­i­lar con­text, a dif­fer­ent process is ap­plied and works pret­ty well. This con­text is the ex­ter­nal de­pen­den­cies.

What hap­pens when dif­fer­ent teams are not work­ing to­geth­er, are not even work­ing in the same com­pa­nies, and may not even know each oth­er per­son­al­ly and had no meet­ings or oth­er in­ter­ac­tions what­so­ev­er?

Some will as­sert that they won't be able to build a prod­uct. Well, think again.

How many times you've used En­ti­ty Frame­work? How of­ten was you blam­ing Mi­crosoft's de­vel­op­ers of break­ing your build af­ter they re­leased a new ver­sion of their ORM?

It is very in­tu­itive to build the in­ter­ac­tion we get with teams we don't know much about. Would it be En­ti­ty Frame­work, Json.NET, ASP.NET MVC or any oth­er third-par­ty pack­age. When the third-par­ty de­liv­ers a new ver­sion of the pack­age, we have a choice be­tween mov­ing to the new ver­sion or stay­ing with the one we used be­fore. This means that what­ev­er they did af­fects us through the ex­plic­it choice of mi­grat­ing to a new ver­sion of the pack­age, which also means test­ing that every­thing still works in this new ver­sion. De­vel­op­ers of Json.NET, in turn, don't have to know who uses their pack­age. They sim­ply push a new ver­sion, and keep it up to you to move to it.

The change in the process is cru­cial both in terms of think­ing about the work­flow and in terms of ab­strac­tion lev­el. You don't have to know any­thing about the team of the third-par­ty prod­uct. You don't ac­tu­al­ly care if de­vel­op­ers of jQuery use Git, SVN or TFS, and you don't care about the struc­ture of their team of the way they work. The pack­age be­comes the prod­uct it­self.

Back to our teams who are still shout­ing and blam­ing each oth­er, we can't left un­no­ticed how their work­flow and or­ga­ni­za­tion are use­less­ly com­pli­cat­ed. The prob­lem is that every team knows too much about the oth­ers, and this makes it prac­ti­cal­ly im­pos­si­ble to de­vel­op any­thing, in­stead of mak­ing things eas­i­er. If they stop hurt­ing each oth­er and in­ter­act only in terms of pack­ages, each team can pick their own ver­sion con­trol and work ac­cord­ing to their specifics, while be­ing un­ob­tru­sive for oth­er teams.

• The first pro­ject can be moved to GitHub and re­ceive con­tri­bu­tions from oth­er com­pa­nies, while be­ing dis­trib­uted through NuGet,
• The sec­ond pro­ject can re­side on in­ter­nal ver­sion con­trol serv­er and be de­ployed through the cor­po­rate NuGet serv­er which can be ac­cessed by staff and part­ners only,
• The third pro­ject can still be de­vel­oped to­geth­er with the two ap­pli­ca­tions, with no changes in the work­flow or the in­fra­struc­ture.

This makes it easy to ap­proach a com­pa­ny as a se­ries of ser­vices. Host­ing a ver­sion con­trol serv­er, for in­stance, moves from a core el­e­ment of the com­pa­ny to a sim­ple ser­vice with a bunch of char­ac­ter­is­tics such as the re­li­a­bil­i­ty, the back­ups, the per­for­mance and the ad­di­tion­al fea­tures. One part of a com­pa­ny may de­ploy an SVN serv­er, an­oth­er part may pro­vide Git, and every team could freely chose be­tween those ser­vices, de­pend­ing on the qual­i­ty of ser­vice, even­tu­al­ly the in­ter­nal cost (if the com­pa­ny is large enough), and es­sen­tial­ly on the pref­er­ences of the mem­bers of the team.

The best part is that it makes com­plete sense to ap­proach the prob­lem in terms of ser­vices. Clas­si­cal­ly, com­pa­nies tend to cen­tral­ize things like ver­sion con­trol in or­der to re­duce the costs of main­te­nance and sup­port. This might work, but may also cre­ate an in­ter­nal mo­nop­oly; an un­skilled sys­tem ad­min­is­tra­tor may cause every de­vel­op­er in the com­pa­ny to suf­fer from bad ser­vice, with no good rea­son. Com­pe­ti­tion, at this lev­el, may lead to low­er costs and bet­ter ser­vice in the same way an open mar­ket does at a larg­er scope.

As a side note, I suf­fered from that for a year at my pre­vi­ous em­ploy­er's com­pa­ny. We had a mo­ron in charge of TFS, so you may imag­ine the num­ber of is­sues we got from a bad­ly con­fig­ured serv­er which was ran­dom­ly re­boot­ing or sim­ply freez­ing from time to time. Us­ing this TFS serv­er was manda­to­ry for every de­vel­op­er, which meant that when the serv­er was down, no­body could ac­tu­al­ly work (es­pe­cial­ly giv­en the in­tru­sive na­ture of TFS which prac­ti­cal­ly re­quires you to be on­line per­ma­nent­ly.)

So we were fifty guys wast­ing our time and hav­ing no oth­er choic­es? WTF?! Why? Wouldn't it be much eas­i­er for a ded­i­cat­ed per­son an­noyed by this prob­lem to set up an­oth­er TFS serv­er, or even a free Git/SVN vari­ant us­ing Lin­ux, and en­cour­age every one to move to it? Why would any CEO de­cide to throw mon­ey and low­er the pro­duc­tiv­i­ty of his em­ploy­ees?

What about bidi­rec­tion­al in­ter­ac­tions?

A nat­ur­al re­mark is that the con­text of in­ter­nal pro­jects, uni­di­rec­tion­al in­ter­ac­tion of a provider—the in­ter­ac­tion we seem to have with the de­vel­op­ers of jQuery or En­ti­ty Frame­work—doesn't work well in a com­pa­ny, where a giv­en pack­age may be used by two or three “cus­tomers”, not hun­dreds of thou­sands.

The mis­take here is to think that our in­ter­ac­tions with third-par­ty pack­age cre­ators are pure­ly uni­di­rec­tion­al. It's more sub­tle than that. If you en­counter a bug, you can open a bug re­port. If you need a fea­ture, you can ask for it. This means that the re­la­tions are pure­ly bidi­rec­tion­al, de­spite the ap­pear­ances.

What might hap­pen is that the third-par­ty doesn't care about the bug you found in their prod­uct, or that they find your fea­ture re­quest use­less and stu­pid. But isn't that ex­act­ly what hap­pens be­tween teams in the same com­pa­ny? When in­ter­act­ing to­geth­er, those teams will of­ten fight over the fact that a bug isn't re­al­ly a bug, or that a fea­ture is not that use­ful and is, be­lieve me, very, very dif­fi­cult to im­ple­ment. If teams dis­agree, the usu­al way to solve the dis­crep­an­cy is to es­ca­late to a com­mon su­per­vi­sor. With pack­ag­ing sys­tem, the very same es­ca­la­tion could hap­pen as well. The only dif­fer­ence is that there would be a bias to­wards the provider, which may not be that bad af­ter all, since it re­duces fea­ture creep and over­all func­tion­al in­sta­bil­i­ty.

Mi­croser­vices

The idea of mi­croser­vices is very sim­i­lar when it comes to build­ing ab­strac­tions of the world in or­der to fo­cus on a small part of the sys­tem, but many pack­ages can­not be pre­sent­ed as mi­croser­vices. For in­stance, a li­brary such as Json.NET makes no sense as a mi­cro-ser­vice: if I let a ser­vice se­ri­al­ize and de­se­ri­al­ize data, how would I be able to use the re­sult with­out do­ing se­ri­al­iza­tion and de­se­ri­al­iza­tion once again?

Pack­ages have a tremen­dous ben­e­fit of be­ing very low lev­el. This makes their in­te­gra­tion with­in the code very straight­for­ward and free from ad­di­tion­al lay­ers of redi­rec­tion. With a mi­croser­vice, you'll have to use a high lev­el com­mu­ni­ca­tion chan­nel such as HTTP, which makes it a bad choice for a bunch of sit­u­a­tions (such as a li­brary which se­ri­al­izes and de­se­ri­al­izes JSON).

This is also the draw­back of pack­ages: since they are low lev­el, it's up to you, the de­vel­op­er, to en­sure they have the re­quired en­vi­ron­ment to work prop­er­ly. You can't sim­ply put an URI of a ser­vice and ex­pect every­thing to work mag­i­cal­ly; in­stead, you have to know some of the in­ter­nals of the pack­age in or­der to use it prop­er­ly.