Home Home Posts Rants about IT DevOps Stuff I'm working on

Relevance of tags on Careers

Arseni Mourzenko
Founder and lead developer, specializing in developer productivity and code quality
117
articles
August 15, 2014

Careers profiles includes a list of tags the person is top 10%, top 20% and top 30% in. Surprisingly (maybe not), those tags are followed religiously by some recruiters. They make a big mistake.

Relevance of tags from Stack Overflow

If we want tags to be relevant, i.e. to have a correlation between the actual profile and skills of the person and the tags, we should be able to verify all of the following elements:

  1. The questions have all relevant tags. For example, if the question concerns design patterns, it has design-patterns tag.

  2. The questions have only relevant tags. For example, if the question is tagged design-patterns, it is strongly related to design patterns; in other words, a person who has no skills in design patterns couldn't answer correctly the question in a predictable way.

  3. Canonical tags are always available. For example, if the question is about Python 3, it is tagged python and python3.

  4. The answers are upvoted based exclusively on their quality and the expertise and insight they provide. For example, a highly technical, very detailed answer will be highly upvoted, while an answer which is short, nicely written and gives only an overview of the subject, doesn't have the same insight and doesn't use the exact terminology will be upvoted much less.

  5. The upvotes are proportional to the expertise the person provides, and independent of whether the question is a wiki or not or the popularity of the question.

Here's the problem: in order for tags to be relevant, all of those five elements should be verified, but in practice, none is. First, as any system, Stack Overflow voting can be gamed. Repwhores do exactly that and succeed. Second, even without gaming, it doesn't work:

  1. The questions don't necessarily have relevant tags. I was recently surprised to discover that on Programmer.SE, polymorphism tag has only 53 questions. Seriously, 53 questions on a site which is exactly for subjects such as polymorphism? Either some polymorphism-related questions are missing the relevant tag, or Programmers.SE is mostly about tools or project management or history; not that I dislike those subjects, but they are not really whiteboard-type questions.

  2. The questions have irrelevant tags too. For example, why a question about factory pattern have tags such as java and c#? In my life, I've written no more than 50 lines of Java, but I can answer the question and get a few irrelevant java points.

  3. The lack of canonical tags don't need an illustration. Look at the number of questions tagged sql-server-2008 which are in no way specific to this version of SQL Server but lack sql-server tags, or the questions tagged c#-4.0 which apply as well to C# 5 and lack c# tag.

  4. The fact that answers are not upvoted based exclusively on their expertise and insight is obvious as well and intrinsic to human nature. Nobody wants to read a five page-scrolls long answer which uses technical terms. Short, layman variants are upvoted more because they attract more attention. They are easier and faster to read; only a few technical persons actually interested in the subject will read difficult and long answers.

  5. The upvotes don't depend of providen expertise either; the most influential aspect is the popularity of the question. You can write an excellent answer to a question which is closed just after you send your answer, and receive no upvotes. On the other hand, a mediocre answer to a question with 6K visits have lots of chances to be upvoted.

But what if despite all this theory, the tag system was working in reality, presenting relevant tags to the recruiters on Careers? A glimpse to the tags I have in my profile shows that it's not the case. Currently, the tags I have are the following:

  • Top 10%: php, html, css.
  • Top 20%: c#, .net, asp.net, html5, css3, xhtml.
  • Top 30%: javascript, linq, design-patterns.

The first place for php tag in my Careers profile looks like a joke. I hate PHP. I don't want to be hired for any job related to PHP. I don't even want to be a user experience designer in a PHP-oriented company. I do answer PHP-related questions from time to time, because they are easy to answer, and because those questions are usually written by very inexperienced persons and sometimes, I have an opportunity to encourage the person to improve as a programmer.

The position of html5 and css3 is funny too. I don't know those two well enough. I mean, I can develop HTML 5 websites, but I don't have a huge experience similarly to web developers who spend all their time writing HTML 5 and CSS 3 for the last two years.

Finally, those tags give an impression that I'm a programmer. This is very annoying. Including sites other than Stack Overflow would help giving a better picture, but Stack Exchange team doesn't seem particularly interested in this improvement.

Relevance of cross-site rating

Who am I according to the tags from other sites? Programmers.SE tells us that I'm proficient in:

  1. programming-practices: they got it right.

  2. c#: maybe, but on this site, this tag is mostly irrelevant, unlike on Stack Overflow.

  3. interview: they got it right.

  4. web-development: I may indeed have some bonus points in architecture and design of web-based solutions compared to desktop software.

  5. programming-languages: the tag is irrelevant for Careers.

  6. javascript: maybe, although it's irrelevant here just like c#.

  7. php: I hate tags!

  8. unit-testing: great.

  9. html: irrelevant.

  10. testing: great.

  11. coding-style: great, but it's not even in the top 10 of my tags; that's disappointing, given that I focus a lot of my work on coding style and answer a lot of style-related questions on Programmers.SE.

  12. productivity: finally, they got it. I would see it in the first 3, but at least, it is mentioned.

  13. source-code: I have no idea what this tag is about.

  14. learning: the same.

  15. security: my reputation on http://security.stackexchange.com/ would be more relevant; but I don't want to discuss my 110 points on Security.SE. Subject closed. Thank you.

  16. development-process: great, but 16th, really?

  17. refactoring: you have to kill me. Why isn't it in the first three?

  18. .net: irrelevant.

  19. css: irrelevant.

  20. tdd: ok (but not at this position).

The actually relevant tags, such as code-quality, design-patterns, design, project-management, naming, self-improvement, agile, teamwork, architecture, code-smell, optimization, functional-programming, coding-standards, waterfall, development-methodologies, team or employee-relations didn't found their place in top 20, and most weren't even on the first page. That's disappointing.

This was Programmers.SE. I won't do the same thing for other sites, but limit myself to the overall reputation. Here are the top 5 sites in my Stack Exchange profile:

  • Programmers (58 711 points).
  • Stack Overflow (11 064 points).
  • User Experience (2 515 points).
  • Photography (1 437 points).
  • The Workplace (1 368 points).

This is nearly relevant, if we omit two aspects:

  • I'm much more skillful in user experience than in photography, and I'm much more skillful in programming than the results show. If those points were relevant, I would have, say, 45K on Programmers, 35K on Stack Overflow, 35K on Workplace, 30K on User Experience and 1.5K on on Photography.

  • The last one in top 5 shouldn't be photography, but either Project Management or Code Review or Graphic Design or Freelancing. I should definitively have a lot of reputation on Project Management where I have zero points for now.

So, instead of being relevant of my expertise, the list of sites sorted by reputation is more relevant of how I interact with the different communities. For example, I had no idea there is Project Management site on Stack Exchange. Another example: The Workplace; it looks like the website is great, but I usually forget it exists, and spend my time on Programmers instead. The last example: Super User; I hate this community, but still ask questions there (even if most of my questions stay there for a week, unseen, before I remove them) and gave a few answers.

Conclusion

Even if Careers team include tags from other sites, the list of top tags would still be irrelevant for a recruiter. Reputation just can't correlate with expertise, making it a weak reference for pre-interview selection.

What would be the relevant cross-site tags for a profile like mine based exclusively on skills? Maybe something like that:

  1. productivity, obviously.

  2. code-quality, obviously.

  3. refactoring, obviously.

  4. automation: I'm a well-known promoter of full automation of the overall infrastructure as well of testing automation.

  5. testing/unit-testing/regressions: I'm a known promoter of extensive testing.

  6. workplace/working-conditions: This is strongly related to productivity, and I studied the subject for a long time.

  7. development-methodologies/agile/tdd: basically project management, but from a perspective of a theorician.

  8. architecture/design/design-patterns: because I'm not a programmer.

  9. project-management/development-process: because I'm that too.

  10. .net/c#: I enjoy .NET a lot and spent 7 years working it. C#, .NET and Visual Studio is my favorite ecosystem and C# alone is my favorite language, even if I have find Python excellent as well and enjoy Node.js a lot.

  11. teamwork/team/interpersonal-communication/employee-relations: essentially everything related to team organization in a light of productivity and "get the things done" philosophy (which has nothing to do with productivity, by the way).

  12. devops/continuous-integration: because who can live without while specializing in productivity, automation and development processes?

  13. functional-programming: While I haven't used any purely functional language too much, I make a great deal of using the full potential of functional paradigms in C#, Python and JavaScript.

  14. technical-debt: I built a model which measures technical debt. If this doesn't count, I don't know what counts.

  15. user-experience: worked a lot on UX to forget mentioning it.

  16. communication: extremely important to ensure highest possible productivity, and also a very interesting subject too.

  17. performance/profiling: I do that too, a lot.

  18. coding-style/coding-standards: because I become a jerk when it comes to style.

  19. interview: I have interviewed enough people and worked on the theoretical part of the subject long enough.

  20. html/css: because everybody should mention that, even developers working on embedded software.

  21. photography, video-production: to show that I don't do only development-related stuff in my life.

This list gives a good overview of my profile and domains I know better than others. As you can see, it has nothing to do with any tags shown on Stack Exchange for my profile.


I just received java tag on Programmers.SE. By the way, in my whole life, I've written no more than 100 LOC in Java.