Relevance of tags on Careers
Careers profiles includes a list of tags the person is top 10%, top 20% and top 30% in. Surprisingly (maybe not), those tags are followed religiously by some recruiters. They make a big mistake.
Relevance of tags from Stack Overflow
If we want tags to be relevant, i.e. to have a correlation between the actual profile and skills of the person and the tags, we should be able to verify all of the following elements:
The questions have all relevant tags. For example, if the question concerns design patterns, it has
The questions have only relevant tags. For example, if the question is tagged
design-patterns, it is strongly related to design patterns; in other words, a person who has no skills in design patterns couldn't answer correctly the question in a predictable way.
Canonical tags are always available. For example, if the question is about Python 3, it is tagged
The answers are upvoted based exclusively on their quality and the expertise and insight they provide. For example, a highly technical, very detailed answer will be highly upvoted, while an answer which is short, nicely written and gives only an overview of the subject, doesn't have the same insight and doesn't use the exact terminology will be upvoted much less.
The upvotes are proportional to the expertise the person provides, and independent of whether the question is a wiki or not or the popularity of the question.
Here's the problem: in order for tags to be relevant, all of those five elements should be verified, but in practice, none is. First, as any system, Stack Overflow voting can be gamed. Repwhores do exactly that and succeed. Second, even without gaming, it doesn't work:
The questions don't necessarily have relevant tags. I was recently surprised to discover that on Programmer.SE,
polymorphismtag has only 53 questions. Seriously, 53 questions on a site which is exactly for subjects such as polymorphism? Either some polymorphism-related questions are missing the relevant tag, or Programmers.SE is mostly about tools or project management or history; not that I dislike those subjects, but they are not really whiteboard-type questions.
The questions have irrelevant tags too. For example, why a question about factory pattern have tags such as
javaand c#? In my life, I've written no more than 50 lines of Java, but I can answer the question and get a few irrelevant javapoints.
The lack of canonical tags don't need an illustration. Look at the number of questions tagged
sql-server-2008which are in no way specific to this version of SQL Server but lack sql-servertags, or the questions tagged c#-4.0which apply as well to C# 5 and lack c#tag.
The fact that answers are not upvoted based exclusively on their expertise and insight is obvious as well and intrinsic to human nature. Nobody wants to read a five page-scrolls long answer which uses technical terms. Short, layman variants are upvoted more because they attract more attention. They are easier and faster to read; only a few technical persons actually interested in the subject will read difficult and long answers.
The upvotes don't depend of providen expertise either; the most influential aspect is the popularity of the question. You can write an excellent answer to a question which is closed just after you send your answer, and receive no upvotes. On the other hand, a mediocre answer to a question with 6K visits have lots of chances to be upvoted.
But what if despite all this theory, the tag system was working in reality, presenting relevant tags to the recruiters on Careers? A glimpse to the tags I have in my profile shows that it's not the case. Currently, the tags I have are the following:
- Top 10%:
php, html, css.
- Top 20%:
c#, .net, asp.net, html5, css3, xhtml.
- Top 30%:
The first place for
The position of
Finally, those tags give an impression that I'm a programmer. This is very annoying. Including sites other than Stack Overflow would help giving a better picture, but Stack Exchange team doesn't seem particularly interested in this improvement.
Relevance of cross-site rating
Who am I according to the tags from other sites? Programmers.SE tells us that I'm proficient in:
The actually relevant tags, such as
This was Programmers.SE. I won't do the same thing for other sites, but limit myself to the overall reputation. Here are the top 5 sites in my Stack Exchange profile:
- Programmers (58 711 points).
- Stack Overflow (11 064 points).
- User Experience (2 515 points).
- Photography (1 437 points).
- The Workplace (1 368 points).
This is nearly relevant, if we omit two aspects:
I'm much more skillful in user experience than in photography, and I'm much more skillful in programming than the results show. If those points were relevant, I would have, say, 45K on Programmers, 35K on Stack Overflow, 35K on Workplace, 30K on User Experience and 1.5K on on Photography.
The last one in top 5 shouldn't be photography, but either Project Management or Code Review or Graphic Design or Freelancing. I should definitively have a lot of reputation on Project Management where I have zero points for now.
So, instead of being relevant of my expertise, the list of sites sorted by reputation is more relevant of how I interact with the different communities. For example, I had no idea there is Project Management site on Stack Exchange. Another example: The Workplace; it looks like the website is great, but I usually forget it exists, and spend my time on Programmers instead. The last example: Super User; I hate this community, but still ask questions there (even if most of my questions stay there for a week, unseen, before I remove them) and gave a few answers.
Even if Careers team include tags from other sites, the list of top tags would still be irrelevant for a recruiter. Reputation just can't correlate with expertise, making it a weak reference for pre-interview selection.
What would be the relevant cross-site tags for a profile like mine based exclusively on skills? Maybe something like that:
This list gives a good overview of my profile and domains I know better than others. As you can see, it has nothing to do with any tags shown on Stack Exchange for my profile.
I just received