I have often pondered what makes a Senior Developer “Senior” and what makes a Junior Developer “Junior”. Invariable such titles are bestowed upon software developers based upon where they are in their career and how many years of experience they have.

The idea that the number of years you have been doing a job is in my view quite an old fashioned idea. In some industries this may be valid; clearly I would prefer to be operated on by a heart surgeon with 15 years experience as opposed to a surgeon who is newly qualified – that said even in this context the newly qualified Dr may be versed in more modern techniques that the seasoned surgeon may not have knowledge.

I’m not suggesting that experience shouldn’t be considered but I would suggest that we are more interested in the skills that an individual has. Experience means that the individual has had longer to gain those skills but it in no way guarantees that the individual has used the time wisely.

With this in mind recently I’ve been giving some thought to how we should measure the performance of our developers and how to support them in building skills that benefit us as a company.

One of the tools that I have developed to help with this is a Developer Skills Matrix. The concept is really simple – across the top I list all of the skills that I want my developers to have and down the side I specify levels of knowledge (No Knowledge, Basic, Good, Great, Superstar).

Skills

When specifying the skills its important to drill down to a granular level of knowledge; being to broad brush doesn’t give you enough detail. In the example above you might be tempted to group all of the skills under the heading “React” – this would limit the benefit of the matrix as it is perfectly feasible for a developer to have no-knowledge of Redux and still be a great developer.

The idea is to create headings which cover areas that a developer might have knowledge or could be expected to have knowledge in order to effectively do their job.

Ratings

In order to make best use of the ratings its important to provide clarity on what level of knowledge you would expect an individual to have in order to achieve the rating.

I use the following guidance but it can be adjusted to suit your own context.

No Knowledge

You might have heard of the technology but you’ve never used it.

Basic

You have a working understanding and are able to make basic use of the concept or technology. You would usually ask a lot of questions and rely heavily on others to support you in any implementation work.

Good

You are comfortable using the concept or technology as part of your everyday work but struggle with some of the advanced concepts. Sometimes its good to get help from someone who has more experience than you.

Great

You are really comfortable with the concept / technology and feel you have a great working knowledge of most of the more advanced concepts. You can identify when people are incorrectly using the concept or technology and will offer support to assist them.

Superstar

You know this concept / technology inside out and are a subject matter expert. You are able to support, help and teach others at all levels. If a stack overflow question were asked related to this area you would jump right in and answer confidently.

Once you’ve set up your matrix ask your Developers to complete it. I like to ask Engineers to justify their ratings in each area – this in turn leads to good 1:1 conversations and also helps normalise ratings across the development team.

Scoring the Matrix

When the matrix has been completed by the whole development team I like to score the matrix. No Knowledge = 0, Basic = 1, Good = 2, Great = 3, Superstar = 4.

When view across the whole of the development team this scoring allows you to identify areas of team weakness, where perhaps you have limited knowledge as a company in a particular area. This can then inform your recruitment strategy by identifying the areas you want to particular focus on when interviewing potential new joins.

The scoring can also be used to identify weaker team members or to help inform performance conversations. I would caution that this is not a silver bullet and should not be used as the only way to measure an individuals performance but it does open up some opportunities to recognise your staff and to inform clear and quantifiable objectives.

Ways to use the data

There are many ways to use the data to help your teams with continuous improvement such as:

Objective Settings

Identify 3 skill areas where a developer has no knowledge. Set an objective that by the next performance review the developer will have basic knowledge in all 3 areas and ask the individual to provide evidence of their achievement. 

The knowledge can be gained in a variety of ways such as independent research, an online course or pair programming. The method isn’t important but the commitment to improvement will benefit both the individual and your company. You know this concept / technology inside out and are a subject matter expert. You are able to support, help and teach others at all levels. If a stack overflow question were asked related to this area you would jump right in and answer confidently.

Company / Team OKRs

If you use Objective Key Results within your organisation then this data can be used as a quantifiable measure of performance improvement.

Objective: As an Engineering Team we will continually improve the skills required to build our products:

Key Result: The average score for every skill should improve each quarter.

Common Pitfalls

The skills Matrix is not a silver bullet – it can be used as one tool to help with your overall performance management strategy.

When I first considered introducing this tool into my team I discussed with other Engineering Managers in my network and I heard horror stories of similar frameworks being abused – in one case engineers had nicknamed the tools “The redundancy framework”.

Like all tools its success is very much dependent upon its implementation – if implemented well as part of a wider performance strategy the framework can help identify areas of weakness within the team and help guide areas of focus for improvement over time.

Have you used a framework like this in your organisation? If you have I would be really keen to hear from you – please comment below or reach out to me on Twitter @stevewestgarth.