GitPrime elevates engineering leadership with objective data. In this interview series, engineering leaders talk about how to build high performing teams.

When startups are first setting out, the initial team is small and usually solving relatively discrete (albeit central) problems. But at scale, the quantity and size of the problems grow larger than a single manageable team can handle.

That’s when you hit the first growing pains of scaling, both organizationally and technologically. And that’s the stage where many of us don’t know what to do to usher in the next era of our company.

“How do you get from one team, where everybody fits around a conference table, to the Amazon-style or Google-sized organization?” asks Randy Shoup, the VP of Engineering at WeWork.

Shoup himself has worked as an engineering leader with several companies of various sizes during periods of intense scaling over the last three decades. He was the VP of Engineering at Stitch Fix, Director of Engineering for the Google App Engine, the CTO and cofounder of Shoplilly, and the Chief Engineer at eBay, among other leadership positions.

In this exclusive interview, Shoup unpacks several strategies that organizations can undertake as they grow in order to scale efficiently, profitably, and as painlessly as possible.

Divide your organization organically into smaller teams.

There’s a few ways to handle the inevitable circumstance of teams outgrowing the problems they’re working on (as well as the problems outgrowing the teams working on them), but only one way that both Shoup and the industry at large have found to work well.

“The Amazons and the Googles and the Netflixes of the world have all co-evolved to essentially the same model,” Shoup says, “which is individual full-stack, two-pizza teams, each of which is responsible for a service or a small element of an application. Then you form your organization out of many of them.”

Shoup calls this a fractal organization. At one level of granularity, you have the big-picture feature or service. At higher levels of magnification, that product subdivides into smaller teams—teams with more targeted focus. Zoom in from there, and you have individuals with an even tighter focus.

Teams are like cellular organisms—they need to divide to grow.

Whether you’re scaling up from one team to two, or from dozens to hundreds, consider your organization as a microscopic organism. Shoup likes to analogize scaling teams to cellular mitosis.

“People maybe remember from high school biology that there's a cell and that cell subdivides into two cells, and then each of those cells subdivides into two more cells, et cetera, et cetera,” he says. “Sooner or later you have a human or a monkey or a blue whale or something like that, but we all start as one cell. Organizations, healthy organizations, work that same way.”

At early-stage startups, everyone works as a single team with a single set of goals. Once the team becomes large enough—say, ten or twelve people—it needs to undergo mitosis. You might split it into the buyer side and the seller side, or the driver side and the rider side.

Those teams will continue to grow, and at some point each one must divide again—say, as an example, into the app side and the administrative side.

Allow the teams to show you their organic divisions.

Rather than stress about how to divide the teams when they grow large enough, you can rely on the teams to show you their natural points to cleave into two or three distinct teams.

“When you're one team that's a bit over-large, you're not actually all working on one thing,” Shoup points out. “You're actually working on two or three different things anyway. So typically the natural points of cellular mitosis, where we divide the cell, those just come naturally. I've never found a problem where a team was too big to be one team, but it wasn't obvious how to subdivide it.”

When a team becomes too unwieldy, that almost by definition shows you that the team needs to be split, and the division will make itself so clear that you don’t need to worry about where to draw the lines.

Build cross-functional teams that own their work from end to end.

The traditional enterprise software development approach distinguishes product people from engineers, and engineers from quality control, and all of them from customer service.

“What we learned over the last thirty years is that doesn’t work very well,” Shoup says. “It's much better to have in one team, the people who have the idea, the people who do the typing, the people who are responsible for the quality, and the people that operate the software. A bunch of the places where I've worked, Google for example and Stitch Fix, we had full-stacked teams, cross-functional teams that own a particular piece of software, like an application or a service, and own it completely, end to end, from the idea of it all the way through to the operation of it in the real world.”

It doesn’t matter who does what, so long as the team has all the right skills.

What exact roles belong on each individual team depends on what that team’s goals and purpose are. The roles may be split among distinct individuals, or each person may touch several aspects of that process. The key idea, Shoup says, is that all of the necessary skill sets exist within that team’s boundaries.

He also stresses that the idea of a “full-stack” team is a bit of a misnomer. “Nobody really means all the way down to the silicon,” he says. “Not the people who rack and stack the hardware in the data center. Not the people who build chips. It’s all the skill sets you need to get the job done.”

End-to-end ownership reduces the feedback cycle.

The fundamental goal of building teams with complete end-to-end ownership is to reduce the feedback cycle. When you’re not having to communicate between teams to do your primary work, you can have ideas, test them, deploy them, and learn from them in a much tighter cycle.

“The way we have as an industry to do that, is by co-locating all those skill sets together in a single team and turning a waterfall pipeline process into a more cross-functional team where everybody's all working together,” Shoup says.

Optimize for providing as much customer value as quickly as possible.

The meta-reason for reducing the feedback cycle with single-purpose, full-stack teams is that you are optimizing your organization to provide as much customer value as possible, as quickly as possible.

“When your team is too large, it’s slowed down,” Shoup says. “We’re not able to provide the same amount of value as quickly by being one team, as by being two independent teams.”

This is where organizational culture comes into play. If an organization is reluctant to divide unwieldy teams, odds are its first-order concern is not producing the most customer value in the least amount of time. Ditto for organizations that spend undue time building foundations without releasing and iterating on its work.

Aim for releasing the minimum viable product.

To scale your teams and technology successfully, you need a culture built on the three legs of minimum viable products. You need a product to release; the product needs to be viable; and you should release it when it has the minimum necessary to actually be viable.

“We’re going to get some feedback from our initial set of customers,” Shoup explains. “We're going to iterate and continue to refine that product. Hopefully we’ll get more customers, hopefully we will start to scale and have more and more and more customers.”

By focusing your culture around getting customers value quickly, you release products that may not be perfect—but that’s the only way you get valuable input from your existing customers. Shoup sees successful cultures building their teams around these feedback loops, so that they can respond to feedback efficiently and productively.

Iterate to the solution.

Shoup elaborates on this cultural predicate of minimum viable products: “One of the main cultural values is the idea of minimalism and the idea of iterating to the solution,” he says. “The idea that we're not going to get it right the first time and we shouldn't try, but at the same time we should try to solve it in a most minimal and straightforward way. We recognize, as a team, at an organization, as a business, we're going to iterate on it and make it better.”

He references the old idea that if you’re not a little embarrassed by your release, you waited too long to release it. You can problem-solve only so much in the safety of development; however, if you release your work into the wild, you can iterate to the solution much more quickly (and therefore create more value) because you’re getting valuable responses from actual users.

“If what they're saying is, ‘It's fine, but these things could be better,’ awesome. That's exactly the regime you want to be in,” Shoup says. “Now you can iterate and improve those things that were bad.”

Build around the problems you’re trying to solve.

If you’re in fact building teams to own a single feature end-to-end, and that team is equipped to address customer input efficiently, then odds are you’re already building your teams around the problems they’re trying to solve.

But it’s still worth delving into what those problems really are. Sometimes, Shoup says, your first-order problems aren’t your real rooted issues. And a lot of deep-seated problems are products of having silos instead of truly cross-functional teams.

“The more fundamental problem about siloing is that the organization isn't aligned to customer needs,” he explains. “There's one part of the organization that cares about customers, but maybe isn't involved in engineering, and we call those people product or business. Then there's this other part of the organization, which is the engineers and the operators of the software.”

Subdivide according to the organization you want to create.

Instead of growing larger sub-organizations to handle each growing facet of a business, Shoup reminds us to keep subdividing the problems in such a way so that each problem is roughly team-sized.

In so doing, Shoup reforms the organization while invoking Conway’s Law: that the products organizations develop reflect those organizations’ communication structures.

“If you want to have an organization that's built in small components, one of the first things that you can do is to construct your organization out of these small teams, because organizational structure and the architecture tend to be duals or reflections of each other,” he says. “Actually, I'm actively doing right now with some of my teams at WeWork—reforming the organization in a way that more directly reflects the software that we build, or the software that we want to build.”

Utilize managers to make their teams as productive as possible.

In a healthy, generative organization, an engineering manager's job is to remove road blocks from their engineers and team members.

That’s essentially it, Shoup says.

“The job is to help to make as productive as possible the existing people in the team,” he says. “And also, frankly, to grow the team. In high-growth organizations, like WeWork for example, one of the huge jobs of management is actually to recruit and hire and grow the team overall.”

Shoup stresses that managers aren’t there to tell their teams what to do. They are there to enable the team members to do what they already know how to do.

Incorporate product managers into the team.

In addition to engineering managers in Shoup’s model, teams also contain a dedicated product management role. And unlike a people manager, which is pretty much always a single person, he has seen a couple different ways to make product management work well.

“One way to do that is to have a person whose full-time job it is to be a product manager,” he explains. “Another totally legitimate solution to that problem is to have the team members in that boundary share that responsibility in some way. They together come up with what the roadmap is and what the prioritization is, and so on.”

Which way work better depends on how each particular team or organization is constructed. Shoup would not claim that one approach is better than the other.

“I’m agnostic about how that responsibility is allocated to the people,” he says. “The thing that I think is the invariant, is that within the team boundary there is a product management function.”

Make the shift from monolith to microservices—but only if you need to.

As your organization and your technology scale, you’ll start to suspect that you need to shift from a monolith to microservices.

You may—and you may well not. “My strong belief and my experience is that different phases of a product, and different phases of a company, tend to be better served by different types of software,” Shoup says. “I want to say that probably ninety percent of applications in the world are best served by a monolithic application. It’s the exception rather than the rule when you need to build a distributed system with what we call microservices.”

He uses an S-curve diagram to demonstrate these general phases of a scaling organization: starting phase, scaling phase, optimizing phase.

Every startup starts out pretty small, with a few initial customers and slight growth. Then at some point, the successful companies hit a scaling phase where customers come rapidly. After that, you’ll often see the optimizing phase—the top part of the S-curve, where it flattens out and growth is slower.

“In that starting phase, you really are just trying to meet the near-term needs of your small set of customers,” Shoup explains. “So you want to iterate really rapidly. You’re trying to optimize for rapid developer productivity and for iteration. What you are not trying to optimize for is massive scale. And it's totally legit to build a monolith in that phase.”

Look for warning signs that your monolith is running out of gas.

At some point, a startup may scale enough that it outgrows its monolith. Either it’s experiencing feature velocity and its teams are unable to make forward progress from stepping on each other’s toes, or its monolith doesn’t physically scale from a perspective of load or latency. Customer growth might grow so much, what Shoup calls “load-related scaling,” that the monolith can’t handle it. Perhaps you need to be able to load different parts of the system independently of others, or you want different parts of the system to evolve at difference rates.

Really, only once you start experiencing these needs do you want to break the monolith into smaller pieces.

“Microservices are a solution to the scaling phase, not the starting or steady state initial phase,” Shoup says. “It's only when you're doing that rapid scaling that you need them.”

Have open, mature conversations about everything.

The recurring theme in our interview with Shoup is that having open, up front, and overt conversations is massively helpful to scaling both organizations and technology.

“I could not agree more that in these healthy, generative organizations, the ability to have these conversations transparently and maturely is a key aspect,” he says.

“If you’re not able to talk to your business partners and explain, ‘Here's why the system isn't performing well. Here's why it's not reliable. Here's what I would do to fix it. Here are the investments that we should make,’ and if your business partners aren't receptive to that conversation, that's bad. A lot of terrible software has been built in organizations that weren't able to have mature conversations about long-term engineering viability.”

Each of Shoup’s seven strategies for scaling successfully benefits from open communication:

• Divide your organization organically into smaller teams.
• Build cross-functional teams with end-to-end ownership.
• Optimize for providing customer value as quickly as possible.
• Build your architecture around the problem you’re trying to solve.
• Utilize managers to make their teams as productive as possible.
• Transition from a monolith to microservices—only when you need to.
• Have honest and transparent conversations… even about having honest and transparent conversations.

By learning from each of these tactics, you’ll equip your organization to become more nimble as it reaches more customers more efficiently. Your teams will be constructed to learn as they grow, ultimately contributing to a productive and impactful scaling process.