Best Practices to Measure Software Engineering Performance: It’s time to help your board understand engineering

Updated on June 9th, 2023 | Author: Greg Ceccarelli | Read time: ~ 5 mins 

Measuring software engineer performance is a complex task that goes beyond simple quantitative metrics. While traditional methods like counting lines of code have been used in the past (or abused), they fail to provide an accurate representation of a developer's true value. In fact, focusing solely on lines of code leads to unintended consequences, such as encouraging bloated and inefficient code or discouraging engineers from refactoring and optimizing their work. While those of us in the technical realm can attest to the ineffective and damaging results of a myopic managerial focus on singular metrics; for some reason, executive teams continue to misinterpret, mismanage, and scrutinize engineering on an ineffective and damaging basis. Navigating such an executive team dilemma can be incredibly stressful at best, career endangering at worst.

The engineering function has long been a black-box due to a lack of objective data about what’s happening within the team. There’s also a lack of proactive education across businesses specifically around how engineering effectiveness should be defined. This has led to un-informed efforts to only “track” engineering team activity – or worse yet individual activity sans a balanced approach. Once they understand the problem they can take action to fix it by:

1. Nipping this “lines of code thing” in the bud. Do we evaluate salespeople by how long their slide decks are? Do we evaluate copywriters by how many words they write? 

2. Educating your executives and your board with objective, defensible data. Don’t wait for them to ask. 

3. Building a culture of healthy accountability. Are all engineers good? No. Will half-baked performance management help? Hell no. Invest in your developer experience. Developer toil is bad for them, bad for you.

1. Educate your executives and your board

2. Take Action Now and get ahead of the “Lines of Code” conversation

3. Build a culture of healthy accountability

4. Invest in the developer experience

   Conclusion

Your board is clearly after something—and their ask for lines of code measures is simply a symptom of their desire to better understand how engineering is performing. 

We’ve implemented a balanced scorecard that provides a holistic view of engineering without focusing on a single, poor approximation of developer ‘effort.’ Further, these metrics provide the needed insight to empower and unblock your team. We have found time and time again that productivity gaps are caused by systemic issues, not individual performance.  

Productivity: Focused on team throughput

• What is each team’s throughput? 

• How much time is spent writing and rewriting code? 

• How efficiently is new code written?

Predictability: Focused on sprints & releasing

• How predictable is delivery relative to expectations? 

• What is the totality of engineering effort allocated across projects?

Reliability: Focused on how collaboration can improve reliability

• How do quality issues trend over quarters?

• How often are defects impacting customers?

• How thoroughly are PRs being reviewed?

• What percentage of PRs have no reviews?

Delivery: Focused on DevOps

• How often do teams deploy code?

• What is the lead time to make a change?

• When incidents or defects occur, how long does it take to recover?

• How often do deployments degrade service?

1. Send an email to your CEO today. 

2. Explain why LoC is a terrible way to conduct performance reviews

3. Schedule time to explain how engineering works, with good metrics.

4. Educate non-engineering leadership on the pitfalls, consequences, and fallout that will result from improperly measuring engineering productivity.

We’re currently wrapping up a research study on how developers thrive in their work—and some of the early insights shed light on what healthy accountability really looks like. 

87% of developers in our research agreed that tracking parts of their coding process would increase the level of visibility and value they felt their technical work had. They want this tracking to feel authentic and thoughtful: developers who were on teams that tracked work in what developers called "the right way" were more productive compared to developers on teams that didn't.

Yet only 24% of developers agreed, "my manager and teammates have the right level of visibility into my work." And less than one out of every four developers in our research said they were on a team that was consistently measuring work in a way that seemed "right for us." And across industries, roughly one out of every four developers says they aren't even sure IF their team's engineering effort is tracked and reported at the org level.

Developers want to feel like their engineering effort is understood, and that it helps the business make better decisions. A manager in our research said:

"It doesn't matter how well you manage your team if you're not aligned or bring visibility into the place that your team or department has within the organization. Those pieces are critical... healthy accountability is about helping [developers] grow and be successful from their own personal ends, and then helping them contribute to the business in a way that the business can see as well.”

You implement a balanced scorecard and you find areas of opportunity. What do you do next? We recommend two courses of action that will improve the developer experience on your team. 

1. Dive deeper with objective metrics. With tools like Flow, you can drill into the insights identified on your scorecard to see where your developers experience the most friction. Importantly, you can quantify the impact those frictions have on the business so you can advocate for the needed investment. Feel confident asking for what you need with the numbers to back you up. 

2. Ask your developers what they need. Quantitative data is extremely valuable, but we’ve also found that self-reported metrics complete the picture. We’ve partnered with a company called DX that has built a research-backed, purpose built developer experience management program. DX captures 75 perceptual and behavioral measures of developer experience and productivity to amplify the voice of your developers. 

Improving the engineering function through optimizing the developer experience is an effort best tackled with a scalpel, not a sledgehammer. Combining the objective insights and data from Flow while amplifying the voice of the developer helps ensure you are precise, constructive, and effective.  

If you’ve reacted the same way we have to the seeming surge in concerning performance tracking attempts, take it as an opportunity to own the narrative around engineering effectiveness. No one knows your team better than you do, but it’s your job to help everyone understand what makes a good engineering team tick. 

If you’d like to explore how we can help, reach out! We’ve worked with hundreds of incredible engineering teams to implement thoughtful, impactful metrics that have driven their business forward and helped them earn the backing and investment from empowered executives, investors, and board members.