1.1.1. Lines of Code 1.1.3. Story Points
a) Technical ability
1.3. Considerations for Promotion
1. Understanding and Assessing Engineers
Having already spoken a lot about engineers’ motivations in the leadership section, I want to spend some time now on some ways to get a gauge of how good your engineers are, and how to develop them. It’s important to identify weaknesses of course but you have to be specific in what you’re looking for. Developers that aren’t performing at a high enough level are not necessarily a lost cause. Save a very small number of people in my past who just wanted to slide by, I don’t know many developers that didn’t want to be great engineers. Almost all of them pride themselves on their abilities. There is a broad spectrum of developers out there, and most want to get higher up that spectrum as they go.
1.1. How Not to Assess an Engineer
Assessing an engineer can be a struggle for any manager, technical or otherwise. Non-technical managers are in a weaker position to quantitively assess technical skill, and technical managers that lean more engineer than manager may be weaker at fairly and accurately assessing people skills. In my time I’ve seen a small handful of common measures that are consistently adopted almost by default, but in my view, they outright miss the mark, are highly context dependent, cannot be fairly compared between engineers, and they do not represent the whole engineer.
1.1.1. Lines of Code
The “lines of code” metric is amongst the most commonly used when trying to get a handle on engineer productivity. In a greenfield scenario this metric can be an interesting indication of how quickly an engineer can bootstrap a new component, but it doesn’t take long before that value declines once they get something roughed in. In a brownfield scenario this metric loses meaning even more quickly. For one, an engineer that is extending existing code will generally write less code because the framework is already there, and the changes may require comparatively few lines of code. Secondly, a developer could spend a week working to identify a bug and end up making only a single character code change, and so in that case assessing productivity on lines of code is clearly inappropriate. Yet again, if an engineer is refactoring a portion of the code base, this metric becomes even more difficult to meaningfully interpret because much of that effort will be reorganisation and test development, and as many lines of code may be changed as deleted.
Lastly, personal style and level of adherence to code quality and design standards will affect the lines of code metric too. I for one like to split function calls with more than 2 arguments across multiple lines for readability, and this artificially inflates the number of lines of code I’ve written. Also, the process of correctly splitting functions down into smaller chunks creates longer files and shorter functions as it adds more braces, more variable declarations (and more state management in general), and more function argument lists. Building more smaller, single purpose classes instead of monolithic ones drives this value up too. The longer files this creates doesn’t in and of itself infer a more productive developer when looked at as a metric in isolation. A very good developer may also be excellent at expressing ideas concisely, so for the same functional expectation they may write less code than another developer.
As a final thought, irrespective of how much code they’ve written to express a functional need, if the code isn’t good code you’ll incur downstream impacts in terms of bugs, post-release support, issues in knowledge management, and more. These all impact overall productivity and predictability.
If there is any merit to this metric at all, it may actually be in how many lines of test code they’ve written as compared to functional code. The fact that the code is test code doesn’t relieve it of the same considerations for functional code, but it does serve to demonstrate a commitment to building quality in from the ground up. Even then though, as it relates to tests there are much better metrics to go by that are a lot less subjective and require a lot less interpretation of context to determine comparative value.
1.1.2. Bug Count
The incidence of escaped bugs (which is to say bugs caught in a release candidate build or in production) is a reflection on code quality as much as developer proficiency, and thus taken alone, bug count may not be a useful metric in all circumstances. For greenfield code this metric can be meaningful up to a point because the context of a bug is known; it is related specifically to the developer that wrote that new code. However, as soon as you introduce dependencies across other teams and components (classes, interfaces, APIs), this can break down quickly as now the delineation of responsibility is not as clear cut. Besides, bugs happen - they’re a fact of life.
Especially in a brownfield scenario, bugs can occur in code that is problematic to begin with, and even your most conscientious developers can create bugs in overly complex, poorly organised, hard to read code. Bug count can be as much a function of code organisation than the proficiency of the developer who last touched it (notwithstanding that I’d advocate in 90% of cases that they should have organised the code before extending it, but if there is inadequate test coverage this can be really risky and ill-advised and a test-covered micro-refactor might have been considered first).
If your department has a SWAT team for handling escaped bugs, this metric is even less meaningful as all they’re doing is triaging issues created by other developers. These developers are doing different work, are likely creating only very small spot-changes (if they’re not directing them to the offending developer), and they need to be assessed differently as it relates to this metric. Even then, as it relates to the rest of your developers, you’re comparing apples to oranges and so this doesn’t serve as a fair comparison point for developer productivity.
1.1.3. Story Points
By definition, the sprint velocity as measured in story points shouldn’t be used to compare velocity between teams because each team could well point their stories differently for an equivalent amount of delivered work. It follows then that point values should not be compared across developers, either, and for this reason I don’t categorise this as a metric. As fundamental an assertion as this is, I once saw C-level leadership in a very large organisation looking to this measure to help answer questions about department productivity. This was a leadership team that didn’t have a better construct within which to work, and engineering leadership wasn’t succeeding in crafting a better one.
1.2. A Meaningful Approach to Assessing Engineers
While you are organisation building it will behoove you to be as quantitative and detailed about your engineers as possible so that you’re not introducing risk by leaving the wrong person in place or putting the wrong person in charge through failure to recognise their weaknesses. You’re also exposed to risk when you don’t recognise someone’s strengths as they may not feel objectively observed or appreciated, and naturally this creates potential for turnover. You owe it to yourself and your engineers to do more than rank them on a general, low-resolution scale; something comprehensive to use as the basis for adjustments in placement and compensation is much better.
There are many purely qualitative dimensions of knowledge, performance, and attitude, but being explicit about which ones you’re viewing can help create detailed, meaningful, and evidence-based pseudo-quantitative results. The following list could be used as the basis for an assessment of these traits. In the interests of not overthinking scores, I’ve historically rated engineers on a 1 to 5 scale:
a) Technical ability:
- Authority - their ability to represent a technical position with justifiable conviction
- Knowledge of their stack
- Ability to conceive a design independently
- Ability to recognise and fix design antipatterns
- Platform knowledge for their area of responsibility
- Achieves their goals with limited assistance
b) Attitudinal factors:
- Positivity – do they create a good feeling or are they a locus of negative energy
- Creativeness – do they present ideas when solutioning
- Curiosity and tenacity – do they stick with a problem until it’s fixed
- Teamwork – respectful and collaborative with others
- Ability to listen/receptiveness
- Reliability – do they follow through on their commitments
- Confidence – do they have it and do they instil it
- Takes ownership
c) Commitment to general quality and code quality (CQ):
- Takes an active role in creating appropriate tests for their code
- Has a natural sense of CQ
- Recognises smelly code or designs and implements received coaching advice
- Consistency in high quality commits as illuminated by CodeScene
- Ratio of bug commits to non-bug commits - a quantitative measure, meaningful only when compared to members of their team or other members playing in that same part of the platform
- Consistently creates pure CQ-only commits - for their code or someone else’s, as measured per sprint
You’ll need to be up-front with what the assessment criteria are for an engineer at a given level, and of course the higher the level the higher their numbers should be. I consider each of these traits to be highly actionable; if they understand what’s expected, they’ll be able to proactively manage these dimensions of their performance. Involve them in this conversation in working out where they want to focus their self-improvement efforts next so that they feel they have some degree of self-determination; it is their progress, after all.
If they are on the same page with their goals, they’ll likely be very responsive to feedback. Again, by being proactive in setting these expectations, giving tougher feedback is easier, too. You also don’t need to wait for any formal review period to let an engineer know they’ve improved or regressed in any area either; you can share this with them as a visual score card to help reinforce their progress. This will demonstrate that you’re engaged in their career development, and this can in turn drive loyalty. Feedback gets implemented faster, and your organisation will derive benefits from it faster, too.
1.3. Considerations for Promotion
The previous section describes mostly the “whats” of the job for your average engineers, and the attitudinal measures peak to some of the “hows”, too. There is also a different set of “hows” that ever more senior engineers need to exhibit that will distinguish management material from the “rest”. The age-old adage that the good stuff tends to float to the top is especially true amongst engineers – especially in more dynamic growth-stage companies. Opportunities for advancement present themselves frequently, and the right opportunity given to the right person can allow them to flourish and develop function at higher levels. If you’ve the budget to create more senior positions in your org chart, it helps to know what kind of management material you’ve already got if your intent is to promote from within (and almost always should be). This section will speak to those traits. Sometimes though you need to promote engineers when you don’t really want to.
The high turnover that is so common to engineering organizations can put an engineering manager in a tough spot, especially the more senior the resource is. Without external hiring, the manager may only have so many internal qualified candidates to back-fill them, and if you need to promote from within it is often true that more junior engineers are simply not ready for that responsibility. Promoting the wrong person can create team dysfunction and performance issues for a person that may not (yet) be well suited to the role.
Either way it goes, if you’ve got a rubric for how to assess management readiness, a conversation can be had openly and honestly with your candidates, and you can lay the groundwork for a plan to develop them into what you need (and hopefully what they want). Your candidates don’t have to have all the desirable traits to the ideal level desired by any means, but you’ve got to believe they have the potential to get there, with eyes-wide-open about the traits they may never fully develop; you’ll need to have a plan to cover those gaps.
In the same way we addressed measures on technical skill, attitude, and quality-mindedness, desirable traits in the management dimension can be identified and ranked on a 1-5 scale. Ideally, your candidates should be at or near the thresholds you set for each trait in the management dimension. This makes expectations explicit, objective, and actionable:
- Has an ownership mindset
- Contributes to or initiates process improvements
- High hevel of engagement in meetings
- Ability to think critically, especially in ambiguous situations
- What they bring to the team in terms of realising technical opportunities
- Their effectiveness in coaching other engineers from a code quality and design patterns perspective
- Must demonstrate some architectural acumen or have proven ability to collaborate in a productive way with the architecture team, as demonstrated by their respect for and actual implementation of the architectural advice given
- Demonstrated fidelity to the execution of processes as defined
- They can state their views on fellow engineer’s performance in an honest and objective way
- Is quality-minded
- Holds others accountable, and especially doesn’t let issues persist once identified (to the extent they can within their power), being that they attempt to resolve them themselves or raise them to management
1.4. Contributor Type Should Modulate Evaluations
Be mindful that engineers tend to be on a spectrum between team contributors on one end and individual contributors at the other. There is a place for engineers at each of this spectrum, as long as your balance is right for the mission you have. Some types of engineering problem are less well suited to a larger team approach, and it can be useful to have one or two engineers deep-diving into a specific problem set. Some engineers are just built this way, and they tend to be capability builders in technical realms such as frameworks; they’re less functionality minded. There is nothing wrong with not being as team-oriented when used in the right way, and if you have a mismatch of expectations here it may abrade their work experience.
Acknowledging a person as an individual contributor is important not just from the standpoint of aligning them with the right kind of tasks, but also because this knowledge needs to modulate your expectations when you’re assessing them, and because you need to emphasize oversight on certain activities with this kind of individual more than you might with a team contributor.
Specifically, individual contributors need to be highly active in managing knowledge transfer by placing emphasis on documentation and diagrams. By virtue of the very fact that there is less interaction with team members on their work, there are fewer built-in accountability mechanisms, so you need to manage them a little more tightly. It’s also important to ensure they remember that they’re still part of a larger operation and they need to play nicely when it comes to common interests such as build process, versioning, code hygiene, and quality-mindedness.
As it relates to assessing their leadership and management acumen, it’s not unusual to see a difference in their preparedness to step into higher level management roles and it’s something they might do only reluctantly if asked to do so. This shouldn’t be used to penalise them; they have their place and it’s important to be realistic about their nature and what’s meaningful to them.
No comments:
Post a Comment