Ratings in a CV

Heard to­day a dis­cus­sion of two col­leagues about a pos­si­ble im­prove­ment of a CV by adding for every tech­nol­o­gy a rat­ing, show­ing the mas­tery lev­el of the per­son in this par­tic­u­lar tech­nol­o­gy.

I'm sor­ry about those col­leagues. They have a deep mis­un­der­stand­ing of the hir­ing process and of the val­ue and the form of data. This idea is to­tal­ly flawed; it's bro­ken be­yond re­pair.

Be­fore ex­plain­ing why is it wrong to rate one's own mas­tery of a tech­nol­o­gy (or to ask a per­son to rate him­self on a CV), let me high­light one of the is­sues I high­light re­peat­ed­ly over the past five years: the overem­pha­sis on tech­nolo­gies. It's very sim­ple to ex­plain. There is no such a thing as a Java soft­ware en­gi­neer, or a Ruby soft­ware en­gi­neer. There are Vi­su­al Ba­sic pro­gram­mers and there are PHP coders. If you are look­ing for a code mon­key whose job is to pro­duce lines of code, good for you; keep fo­cus­ing on tech­nolo­gies when read­ing a CV. On the oth­er hand, if you are try­ing to hire a per­son whose job would be to pro­duce val­ue through in­for­ma­tion tech­nol­o­gy, in­di­vid­ual tech­nolo­gies shouldn't care that much. Great soft­ware en­gi­neers have a great learn­ing ca­pac­i­ty; a pro­fes­sion­al soft­ware en­gi­neer who nev­er wrote a line of code in Python could be­come quite flu­ent in this lan­guage in a mat­ter of weeks. How­ev­er, a mediocre Python coder would re­quire years of con­stant ef­fort to achieve the lev­el of a pro­fes­sion­al soft­ware en­gi­neer.

This means that the list of tech­nolo­gies shouldn't the main part of the CV. It is still nec­es­sary to put this list on a CV, but the em­pha­sis should rather be on the things that ac­tu­al­ly mat­ter. This makes the dis­cus­sion about the im­prove­ment of that list not that im­por­tant. But since we're here, let's dis­cuss.

Per­son­al­ly, I'm strong­ly con­vinced that I have zero out of five in Java, four out of five in C#, three out of five in Python. What does it mean? Se­ri­ous­ly, what do you know about me that you didn't knew be­fore? How could you use this new data if you were about to de­cide whether you want to hire me or not?

Imag­ine you are look­ing for peo­ple to work on one of the three pro­jects. One is writ­ten in Java, an­oth­er one in C#, and an­oth­er one in Python. What would you sug­gest me if you were hir­ing me, based on the in­for­ma­tion above? C# pro­ject? Failed. What you didn't know is that I have a heavy pref­er­ence for Python, and I want to learn Java more.

Dif­fer­ent sce­nario. You're look­ing for some­one to work on a C# pro­ject. You have my CV, but also a CV of an­oth­er per­son. His pro­file seems very close to mine, but he rat­ed him­self five out of five for C#. Who would be your pri­ma­ry choice based ex­clu­sive­ly on the CV? And here, you failed again, and you made the whole sys­tem fail. By pre­fer­ring the one who rat­ed him­self high­er, you don't take in ac­count the Dun­ning-Kruger ef­fect. It hap­pened that in the past, I knew C# damn well, much more than any of my col­leagues. Since I haven't worked too much with Mi­crosoft's tech­nolo­gies since then, my lev­el de­creased, while the loss is re­cov­er­able. More­over, if I put five-star rat­ing to Jon Skeet, how could I pos­si­bly put my­self more than three stars? But if I put three stars, peo­ple from hu­man re­sources would fil­ter my CV, so I should ar­ti­fi­cial­ly in­crease the num­ber of stars, just to avoid be­ing fil­tered. That oth­er guy did the same thing; how­ev­er, he didn't use Jon Skeet fac­tor, so he went from four to five stars, even if ob­jec­tive­ly, his lev­el is low­er then mine. See how mis­er­able it gets?

The ter­ri­ble thing about those rat­ings is that they pre­sent sub­jec­tive data as if it was ob­jec­tive. By do­ing that, they mis­lead the read­er.

Star­ing sys­tem is all about num­bers. 4.7/5 is slight­ly bet­ter than 4.6/5, which is much bet­ter than 2.1/5. This works pret­ty well as soon as sta­tis­tics and big data sets are in­volved. On Net­flix, if a movie is rat­ed 4.7/5 by twen­ty thou­sand users, it should be pret­ty pop­u­lar. If it's rat­ed 2.1/5 by twen­ty thou­sand users, some­thing is not right with this movie. This doesn't mean that I will en­joy the first movie and will hate the sec­ond one—this is plain mis­in­ter­pre­ta­tion of sta­tis­tics—but it has an ac­tu­al, ob­jec­tive mean­ing in terms of pop­u­lar­i­ty, made by av­er­ag­ing in­di­vid­ual rat­ings of a large num­ber of users.

When the data set de­creas­es, the ob­jec­tiv­i­ty de­creas­es as well. Imag­ine the same Net­flix rat­ings based not on twen­ty thou­sand users, but only three per­sons. The rel­e­vance of such rat­ing, and es­sen­tial­ly its val­ue, would be close to zero.

Back to CV, the rat­ings we get here are based on the judge­ment of a sin­gle per­son. This es­sen­tial­ly means that sta­tis­ti­cal­ly speak­ing, the ob­jec­tiv­i­ty is zero, even if one ex­cludes the ar­ti­fi­cial change of rat­ings based on per­son­al con­sid­er­a­tions of the per­son, such as the ar­ti­fi­cial in­crease dis­cussed sev­er­al para­graphs above. How­ev­er, pre­sent­ed in a form of rat­ings, the in­for­ma­tion makes look it­self as some­thing as ob­jec­tive as Net­flix movie rat­ings.

Pro­vid­ing mis­lead­ing in­for­ma­tion to at­tract the at­ten­tion on less im­por­tant parts of the CV? I don't see the point, un­less it's done to dis­tract from the skills that re­al­ly mat­ter—one of those skills be­ing the abil­i­ty to pre­sent in­for­ma­tion cor­rect­ly.

If self-eval­u­a­tion there is, it should be in a form which shows sub­jec­tiv­i­ty. “Poor,” “Av­er­age,” “Good,” “Great” four-stra­ta sys­tem is much bet­ter in this way. It shows what it shows: that the per­son es­ti­mates his skill lev­el in PHP as “Av­er­age” and his lev­el in JavaScript as “Great.” There is no at­tempt to pass sub­jec­tive in­for­ma­tion as ob­jec­tive here. In this sense, “ex­pert” term should be avoid­ed, be­cause of Mal­colm Glad­well's 10,000 hour rule. Here once again, the term has an ob­jec­tive con­no­ta­tion, with a pre­cise met­ric as­so­ci­at­ed with it.

Let's pre­tend no­body will be fooled by the form of the data, and every­body un­der­stands that those rat­ings are only in­dica­tive of how skill­ful the per­son thinks to be in a lan­guage. Let's pre­tend that there were no ar­ti­fi­cial ad­just­ments ei­ther. Now, wouldn't it be re­al­ly valu­able to know what the per­son thinks of his own skills? OK, let's see. My C# skills as seen by my­self:

“For a few years, I was much skill­ful in C# than all my col­leagues. One el­e­ment which makes me think that is the fact that I en­sured nu­mer­ous train­ing ses­sions in prac­ti­cal­ly every pro­ject, teach­ing both the ba­sics of the lan­guage and the frame­work, in­clud­ing func­tion­al pro­gram­ming as­pects, as well as more ad­vanced stuff, such as for­mal proof through Code con­tracts. An­oth­er el­e­ment is that I al­ways found my­self ei­ther di­rect­ly work­ing or ac­tive­ly help­ing oth­ers on the most chal­leng­ing parts of the pro­ject, where tech­ni­cal ex­per­tise in mul­ti­ple as­pects such as IL op­ti­miza­tions was re­quired. In the last few years, I aban­doned C# for my in-house pro­jects, and used it pro­fes­sion­al­ly only on sim­ple, non-chal­leng­ing pro­jects, which means that I for­got a bunch of things I knew be­fore, and I'm com­plete­ly un­aware of all the new things which ap­peared in C# and .NET Frame­work in those past years.”

This is as short as it gets, and it's a per­fect­ly fine for an in­ter­view. I have no idea how this amount of in­for­ma­tion could be trans­formed in a num­ber from zero to five, while still re­main­ing any val­ue.

A fun­ny ex­er­cise is to gen­er­al­ize the rat­ing sys­tem to in­clude first the non-lan­guages, and then non-tech­nolo­gies as well. Non-lan­guages are all the tech­nolo­gies which are not the pro­gram­ming lan­guages, such as LDAP or AWS. Could you rate your­self on mi­croser­vices? On De­vOps? On scal­a­bil­i­ty? As for the non-tech­nolo­gies, they in­clude all the skills re­quired from a de­vel­op­er, but which are not re­lat­ed to a spe­cif­ic tech­nol­o­gy. For in­stance, could you give your­self a rat­ing at com­mu­ni­ca­tion with pairs? Or the un­der­stand­ing of the busi­ness do­main? Or the abil­i­ty to use lat­er­al think­ing to come with out of the box so­lu­tions to a prob­lem?

Back to lan­guages, the fun­ny thing is that it's not that dif­fi­cult to pro­vide ob­jec­tive, mean­ing­ful in­for­ma­tion. A chart with the num­ber of changed LOC in a giv­en lan­guage over a pe­ri­od of ten years, com­pared to sim­i­lar charts for oth­er lan­guages, tells so much more than some stu­pid rat­ing one puts on a CV, based on per­son­al ex­pec­ta­tion of some out­come which may not even be true. We live at an age of data; do you a ser­vice, and start ex­ploit­ing this data ac­cord­ing­ly in­stead of pro­vid­ing wrong in­for­ma­tion through wrong meth­ods.