Official version control, competing systems and free market

The way ver­sion con­trol ser­vice is or­ga­nized with­in many cor­po­ra­tions is that there is an op­er­a­tions de­part­ment which is in charge of the of­fi­cial sys­tem, and, de­pend­ing on the com­pa­ny, de­vel­op­ers are ei­ther forced to use this sys­tem, or are sim­ply high­ly en­cour­aged to do it.

For in­stance, if SVN is the ver­sion con­trol of choice, then all teams should use SVN for all pro­jects.

The rea­son­ing at the ori­gin of those poli­cies is, I ad­mit it, rather sol­id. Hav­ing one tool used by every­one is ex­pect­ed to lead to:

• Bet­ter qual­i­ty of ser­vice (QOS). Since the ser­vice is man­aged by a ded­i­cat­ed team of spe­cial­ists who know ex­act­ly what they are do­ing, it makes it pos­si­ble for them to fo­cus on pro­vid­ing the best qual­i­ty and re­li­a­bil­i­ty for all em­ploy­ees.

• More sav­ings. If the SVN serv­er is used by ten per­sons and costs $2 000 per month to main­tain, it has a cost of $200 per month per user. If the same serv­er is used by a thou­sand of per­sons, its cost drops down to $2 per month per user—a sig­nif­i­cant dif­fer­ence. Or we can do it the oth­er way around: if the com­pa­ny is will­ing to spend $20 per month per de­vel­op­er for ver­sion con­trol with­out fo­cus­ing on sav­ings, it means that the op­er­a­tions team would get $200 per month if the serv­er is used by ten de­vel­op­ers, or $20 000 per month if the serv­er is used by all thou­sand em­ploy­ees. More mon­ey means more skill­ful per­sons spend­ing more time im­prov­ing the ser­vice, mean­ing a bet­ter QOS.

• Bet­ter in­te­gra­tion. If source code moves from one pro­ject to an­oth­er or is shared be­tween pro­jects, the process is pain­less: svn mv is what it gets the job done.

• Bet­ter au­dit. The com­pa­ny can han­dle all the source code in one place, track who can ac­cess sen­si­tive code, and block ac­cess to source code to a dis­grun­tled em­ploy­ee re­al­ly fast be­fore fir­ing him.

• Bet­ter back­up man­age­ment. This is sim­ply QOS ap­plied to back­ups.

All this should be true in some com­pa­nies, but it's not the case every­where. What I've seen in prac­tice is that cen­tral­ized ver­sion con­trol is poor­ly man­aged, lead­ing to reg­u­lar in­ter­rup­tions of ser­vice. The most ter­ri­ble case I've heard of from a col­league was a com­pa­ny where TFS serv­er was down at least once per week and at least for a few hours. This was so an­noy­ing that he end­ed up cre­at­ing a client which would take source code and share it through Win­dows shared net­work fold­ers (se­cu­ri­ty poli­cies wouldn't per­mit hav­ing any cus­tom serv­er ser­vice with­in the com­pa­ny and most ports were closed any­way) with his pairs. The app in­clud­ed ba­sic ver­sion­ing and merg­ing, so it could ac­tu­al­ly be an el­e­men­tary re­place­ment for TFS. Un­sur­pris­ing­ly, a few months lat­er, the in­ter­nal use of the tool was for­bid­den by the boss “for se­cu­ri­ty rea­sons”, so the de­vel­op­ers were back to send­ing code through e-mails when TFS was down.

Most cas­es I've seen were not that ex­ces­sive, but still, the QOS was far from per­fect. If ver­sion con­trol is down for even an hour per month, pre­vent­ing one hun­dred em­ploy­ees from work­ing, this would mean not only 100 paid hours wast­ed, but also loss of fo­cus, frus­tra­tion and, in­evitably, de­creased pro­duc­tiv­i­ty. As I re­mem­ber, when dis­cussing those is­sues with de­vel­op­ers, they were usu­al­ly ad­mit­ting that “if only they could host ver­sion con­trol them­selves”, things would get bet­ter.

I'm not sure if they are right. Host­ing ver­sion con­trol is not an easy thing, and I be­lieve that frus­trat­ed de­vel­op­ers were large­ly un­der­es­ti­mat­ing the com­plex­i­ty of the task. It is one thing to host SVN on a per­son­al serv­er, and it is a very dif­fer­ent thing to en­sure high QOS and re­li­able data pro­tec­tion for dozens of de­vel­op­ers. But the fact that the task is dif­fi­cult doesn't mean it can't be done.

What I sug­gest is that in­stead of re­ly­ing on a sin­gle ver­sion con­trol, com­pa­nies should au­tho­rize teams to set up their own ver­sion con­trol ser­vices. They prob­a­bly should be ad­vised and/or au­dit­ed on reg­u­lar ba­sis, es­pe­cial­ly when it comes to se­cu­ri­ty and back­ups, but they should at least have the pos­si­bil­i­ty to be able to com­pete with the of­fi­cial, or bet­ter to say “his­tor­i­cal” ver­sion con­trol.

This would lead to sev­er­al ben­e­fits:

• De­vel­op­ers could chose a prop­er sys­tem based on the speci­fici­ty of their pro­ject. For in­stance, if the of­fi­cial ver­sion con­trol is SVN, it could make per­fect sense to use Git for a ge­o­graph­i­cal­ly dis­trib­uted team where some mem­bers work on reg­u­lar ba­sis with­out in­ter­net con­nec­tion (for in­stance on a plane). It could also make sense for a team work­ing on a .NET app to pick TFS which has a slight­ly bet­ter in­te­gra­tion with Vi­su­al Stu­dio.

• De­vel­op­ers who used their own ver­sion con­trol could pro­mote it to oth­er teams. At this state, this be­comes re­al­ly in­ter­est­ing. In­stead of hav­ing a ded­i­cat­ed team of sys­tem ad­min­is­tra­tors who don't re­al­ly care a lot about the qual­i­ty, we now have mul­ti­ple sys­tems which com­pete with each oth­er. This, in turn, en­cour­ages every­one to de­liv­er a bet­ter ser­vice to a low­er cost.

• With reg­u­lar au­dits, any­one could make an in­formed choice based on the cen­tral au­thor­i­ty sys­tem which pub­lish­es trust­ed in­for­ma­tion about the char­ac­ter­is­tics of dif­fer­ent ver­sion con­trol sys­tems avail­able with­in the com­pa­ny. For in­stance, such sys­tem could in­di­cate that Per­force serv­er has an av­er­age re­sponse time of 250 ms. and has dai­ly on-site and off-site back­ups, while SVN1 has a re­sponse time of 900 ms. but hourly on-site and off-site back­ups and SVN2 has a re­sponse time of 50 ms. but only week­ly on-site only back­ups. Giv­en those met­rics, it makes sense to use SVN2 for pro­to­types, but one would rather use Per­force or SVN1 for any­thing else, even if it comes at the price of slow­er re­sponse times.

  Those are just ex­am­ples. The cen­tral au­thor­i­ty may con­tain more met­rics, or may pre­sent them in a dif­fer­ent man­ner. I haven't though about the sub­ject, but I think the fo­cus should be on clear vi­su­al­iza­tions for easy com­par­i­son, cou­pled with ver­bose tech­ni­cal data and in­for­ma­tion about the way those sta­tis­tics were gath­ered, in or­der to en­sure trans­paren­cy and let nerds ac­tu­al­ly spend hours talk­ing about the im­por­tance of five mil­lisec­onds dif­fer­ence in the mean re­sponse time.

But what about in­te­gra­tion, that is the is­sue of mi­grat­ing a piece of code from one ver­sion con­trol to an­oth­er? For me, this is a non-is­sue. If the team wants to change their ver­sion con­trol, many pop­u­lar sys­tems have tools which make it pos­si­ble to mi­grate the full his­to­ry of changes from oth­er pop­u­lar sys­tems. If, on the oth­er hand, teams want to sim­ply share com­mon code which is reused in sev­er­al pro­jects, I think the ap­proach is wrong: in­stead, there should be a com­mon li­brary which is then dis­trib­uted through the pack­ag­ing sys­tems such as npm, pip or NuGet.

A few years ago, I would find the idea of com­pet­ing ver­sion con­trol sys­tems rather strange. For Pel­i­can De­sign & De­vel­op­ment, my choice was clear­ly SVN for a few good rea­sons, and I couldn't imag­ine why would I let an­oth­er sys­tem be used to­geth­er with the of­fi­cial SVN. To­day, I would see such com­pet­ing sys­tems more as an op­por­tu­ni­ty to pro­vide bet­ter and cheap­er ser­vices which are bet­ter suit­ed for spe­cif­ic pro­jects and teams. For in­stance, if a team loves dis­trib­uted ver­sion con­trol sys­tems, why forc­ing them to use SVN?

In a sim­i­lar way, I find it per­fect­ly rea­son­able for a team to pick a lan­guage which is not of­fi­cial­ly sup­port­ed with­in Pel­i­can De­sign & De­vel­op­ment, and to add this sup­port. Mi­croser­vices ap­proach I cur­rent­ly use makes the in­ter­op­er­abil­i­ty seam­less. The only thing is to en­sure that there is a good sup­port for test­ing and au­toma­tion, es­pe­cial­ly in the con­text of con­tin­u­ous de­ploy­ment.