Best Practices to Measure Software Engineering Performance: It’s time to help your board understand engineering
Updated on February 16, 2023 | Author: Greg Ceccarelli | Read time: ~ 5 mins
Salesforce's recent news around engineering performance reviews creates a dilemma for engineering leaders, and a great opportunity for engineers.
“The company is implementing much stricter performance measurements for engineering…This could include performance reviews based on the quantity of code produced for engineers, a flawed way to measure engineering productivity, which encourages quantity over quality."
Apparently the power of Elon Musk continues to influence the strategies and views of activist investors and board members, making us all cringe at the idea of who or what could be next. (See Elon code reviews)
Where this recession-influenced efficiency myopia gets particularly dangerous is when activist investors begin to define how engineering should be evaluated without an intrinsic understanding of the domain.
Why is this an engineering dilemma and how can it be turned into an engineering opportunity?
The engineering function has long been a black-box due to a lack of objective data about what’s happening within the team. There’s also a lack of proactive education across businesses specifically around how engineering effectiveness should be defined. This has led to un-informed efforts to only “track” engineering team activity – or worse yet individual activity sans a balanced approach. Once they understand the problem they can take action to fix it by:
Nipping this “lines of code thing” in the bud. Do we evaluate salespeople by how long their slide decks are? Do we evaluate copywriters by how many words they write?
Educating your executives and your board with objective, defensible data. Don’t wait for them to ask.
Building a culture of healthy accountability. Are all engineers good? No. Will half-baked performance management help? Hell no. Invest in your developer experience. Developer toil is bad for them, bad for you.
4 Best Practices to Measure Software Engineering Performance
Your Opportunity: Educate your executives and your board.
Your board is clearly after something—and their ask for lines of code measures is simply a symptom of their desire to better understand how engineering is performing.
We’ve implemented a balanced scorecard that provides a holistic view of engineering without focusing on a single, poor approximation of developer ‘effort.’ Further, these metrics provide the needed insight to empower and unblock your team. We have found time and time again that productivity gaps are caused by systemic issues, not individual performance.
Productivity: Focused on team throughput
What is each team’s throughput?
How much time is spent writing and rewriting code?
How efficiently is new code written?
Predictability: Focused on sprints & releasing
How predictable is delivery relative to expectations?
What is the totality of engineering effort allocated across projects?
Reliability: Focused on how collaboration can improve reliability
How do quality issues trend over quarters?
How often are defects impacting customers?
How thoroughly are PRs being reviewed?
What percentage of PRs have no reviews?
Delivery: Focused on DevOps
How often do teams deploy code?
What is the lead time to make a change?
When incidents or defects occur, how long does it take to recover?
How often do deployments degrade service?
Take Action Now and Get ahead of the “Lines of Code” conversation:
Send an email to your CEO today.
Explain why LoC is a terrible way to conduct performance reviews
Schedule time to explain how engineering works, with good metrics.
Educate non-engineering leadership on the pitfalls, consequences, and fallout that will result from improperly measuring engineering productivity.
Build a culture of healthy accountability
We’re currently wrapping up a research study on how developers thrive in their work—and some of the early insights shed light on what healthy accountability really looks like.
87% of developers in our research agreed that tracking parts of their coding process would increase the level of visibility and value they felt their technical work had. They want this tracking to feel authentic and thoughtful: developers who were on teams that tracked work in what developers called "the right way" were more productive compared to developers on teams that didn't.
Yet only 24% of developers agreed, "my manager and teammates have the right level of visibility into my work." And less than one out of every four developers in our research said they were on a team that was consistently measuring work in a way that seemed "right for us." And across industries, roughly one out of every four developers says they aren't even sure IF their team's engineering effort is tracked and reported at the org level.
Developers want to feel like their engineering effort is understood, and that it helps the business make better decisions. A manager in our research said:
"It doesn't matter how well you manage your team if you're not aligned or bring visibility into the place that your team or department has within the organization. Those pieces are critical... healthy accountability is about helping [developers] grow and be successful from their own personal ends, and then helping them contribute to the business in a way that the business can see as well.”
Invest in the Developer Experience
You implement a balanced scorecard and you find areas of opportunity. What do you do next? We recommend two courses of action that will improve the developer experience on your team.
Dive deeper with objective metrics. With tools like Flow, you can drill into the insights identified on your scorecard to see where your developers experience the most friction. Importantly, you can quantify the impact those frictions have on the business so you can advocate for the needed investment. Feel confident asking for what you need with the numbers to back you up.
Ask your developers what they need. Quantitative data is extremely valuable, but we’ve also found that self-reported metrics complete the picture. We’ve partnered with a company called DX that has built a research-backed, purpose built developer experience management program. DX captures 75 perceptual and behavioral measures of developer experience and productivity to amplify the voice of your developers.
Improving the engineering function through optimizing the developer experience is an effort best tackled with a scalpel, not a sledgehammer. Combining the objective insights and data from Flow while amplifying the voice of the developer helps ensure you are precise, constructive, and effective.
If you’ve reacted the same way we have to the seeming surge in concerning performance tracking attempts, take it as an opportunity to own the narrative around engineering effectiveness. No one knows your team better than you do, but it’s your job to help everyone understand what makes a good engineering team tick.
If you’d like to explore how we can help, reach out! We’ve worked with hundreds of incredible engineering teams to implement thoughtful, impactful metrics that have driven their business forward and helped them earn the backing and investment from empowered executives, investors, and board members.
Optimize software delivery.
Build happier, healthier teams.
Leverage workflow data to optimize software delivery and build more meaningful connections with your team members.
5 keys to successful organizational design
How do you create an organization that is nimble, flexible and takes a fresh view of team structure? These are the keys to creating and maintaining a successful business that will last the test of time.Read more
Why your best tech talent quits
Your best developers and IT pros receive recruiting offers in their InMail and inboxes daily. Because the competition for the top tech talent is so fierce, how do you keep your best employees in house?Read more
Technology in 2025: Prepare your workforce
The key to surviving this new industrial revolution is leading it. That requires two key elements of agile businesses: awareness of disruptive technology and a plan to develop talent that can make the most of it.Read more