Engineering @ Pluralsight: Creating Our Product Collaboratively

At Pluralsight we’ve put a lot of thought and effort into developing the engineering practices we use every day. These practices are based on principles that we’ve come to value. The principles themselves are more important than the practices, but the practices are what help us achieve the things we value. In order to both preserve and evolve our principles and practices, we had our engineers nominate some of our most respected engineers to codify them; this resulted in our Engineering at Pluralsight document.

If you take a look at that document, you’ll notice that on the left side of each page there is a principle that we value, and on the right side is a list of items that we Do, Encourage, and Avoid. The Do section lists items that we have agreed all engineers will do at Pluralsight. The Encourage section lists practices that we encourage as a best practice, but we recognize that they might not be appropriate in all cases. So, while we do value them and encourage teams to use them, a team may choose alternative approaches if they still accomplish the value on the left. And, lastly, the Avoid section lists things we all agree to avoid at Pluralsight.

While there are several principles in this document, this post is going to focus on the second: We Create Our Product Collaboratively.

What We Do

To support creating our product collaboratively, here are the practices that we ask all engineers at Pluralsight to adopt:

We work on cross-functional teams. In order to facilitate collaboration, each of our 40+ teams are a cross-functional mix of individuals that include: A product manager, a product designer, a devops engineer, and a few software engineers (usually 4-6). These cross-functional teams work on full-vertical product experiences – meaning they own everything from the front-end to the back-end. Organizing ourselves into these cross-functional teams allows a team to have complete control over the product they ship from customer research and design, to development, to production. Because our cross-functional teams are also responsible for quality, we don’t have a separate QA department. All of this facilitates an uninterupted flow of value to our customers. Ideally, our teams have everything they need to deliver value without the overhead of having to step outside of their team.

We ensure that at least two people review all code before it ships. We are comfortable working on code with which we are familiar; collaborating on the code we write helps spread knowledge about our codebase. Most of our teams at Pluralsight either pair program or mob program in order to accomplish this goal, but in order to respect the autonomy of our teams we allow them to choose any approach that accomplishes this goal. In addition to spreading knowledge around the team, having multiple people collaborating on code also increases architectural and code quality. Often having a second (or third, or fourth) person looking at our code helps us identify mistakes or blind spots that we are less likely to notice on our own.

We integrate early and often within and across teams. We’re not big fans of long-lived feature (or other) branches. Our preference, as discussed in Engineering @ Pluralsight: Continuously Delivering Value, is to use trunk-based development. We do allow for teams to autonomously choose how to achieve this goal including using short-lived feature-branches, but if the development of a feature in a branch is going to take longer than a day or two, we want to be sure we’re integrating this code into the trunk (i.e. master) branch frequently. This presents the challenge if integrating code that is incomplete or not yet ready for production. In order to allow incomplete features to be integrated with the master branch, we may need to wrap the code in a feature toggle. Feature toggles allow you to integrate incomplete code with other code that is being developed and ship it all to production together even when the features are incomplete. By integrating early and often, we are able to detect integration problems before we go too far in one direction with a feature. This leads to additional collaboration within and across teams which helps our engineering culture and helps us more holistically deliver value to our customers.

We facilitate integration by documenting as we go. While we are not a particularly documentation heavy organization, we do ensure we provide documentation for integration points and complex processes. With so many teams at Pluralsight, we often need to integrate with other teams – typically to send or retrieve data from another team’s systems. Most of this communication is done asynchronously via pub/sub messaging but also occasionally via synchronous API calls. In either case when we create a new integration point, we make sure to document it where our team and other teams can find it and understand how to integrate. As we work, we also identify any complex processes and architectures that aren’t easily discovered or understood by just looking at the code, and we document those processes and architectures as we go. We avoid creating the documentation up-front and prefer to create all of this documentation as we create the code – this allows our code and architecture to evolve as we learn lessons while we are coding.

We make decisions guided by the decision delegation framework. While it is true that we currently make decisions guided by the decision delegation framework, we’ll likely update this soon to say that we make decisions guided by the RAPID framework as we move to be more consistent with the rest of our organizations at Pluralsight. Either way, the point is to identify who has the right and responsibility for different types of decisions. Adding clarity to who provides input and recommendations and who makes final decisions helps avoid confusion and chaos in an organization. Additionally we believe in the subsidiarity principle, which suggests that decisions should be dealt with at the most immediate (or local) level that is consistent with their resolution. Or in other words, we prefer to push decisions down to the teams and individuals where possible as opposed to pushing them up to managers and executives. There are decisions, however, that need to be delegated to all these different levels, and we like to be clear about where those decisions should be made.

What We Encourage

To support creating our product collaboratively, here are the practices that we encourage all engineers at Pluralsight to adopt. They are important to us and we believe that they will bring additional value, but we also respect the autonomy of our teams and engineers and so, while we believe that these are valuable, we allow some flexibility here based on the context in which each team is working.

We encourage pair and mob programming We believe that the most effective way to create our product collaboratively is to do so via pair or mob programming. In pair programming, you have two engineers working together on all production code. In mob programming, the whole team of engineers codes together. This results in a highly collaborative process of software development. Together, engineers make decisions as they code and can help create architecture together. We believe that, in most situations, this creates a better-architected system and is more effective at spreading knowledge than working individually with pull requests. When you solve problems together, you more deeply understand the reasoning behind why code is in the state that it is in. Code reviews provide some of this, but often lack the depth that you get when you work together to face a problem and brainstorm solutions. Pairing and mobbing provides a shorter feedback loop than code reviews since you decide together, in the moment, what to create. We believe that, in most scenarios, this creates better architecture and cleaner code.

What We Avoid

This is essentially the opposite of the What We Do section. We ask all engineers at Pluralsight to avoid these practices. We believe these distract from our goal of collaboratively creating our product.

We do not create knowledge silos. We actively avoid having someone (or even a pair) work extensively or for long periods (more than a few days) in one area of our product. Knowledge silos create problems when someone else needs to work in that area and doesn’t understand the code or the motivation behind architectural and coding decisions. Working in a silo can also lead to large designs that the team later determines to be an unfit or subpar solution. Mobbing and rotating people through pairs helps to avoid these silos.

We do not work in isolation. Sometimes it’s tempting to go off on our own to solve a complex problem. At times it might be helpful to do this to deeply understand a problem, but if we do this, it is viewed as research, and we don’t create production code this way. We may occasionally do some research in isolation, but the goal is to bring the learnings back to the team and then have the team move forward together on a solution. This helps avoid the same problems discussed above when talking about knowledge silos.

We do not hand off work to another group to own. You can see this in a few different ways at Pluralsight. Mainly where you might see work traditionally hand off to another department. Firstly, we don’t have a QA department; our cross-functional teams completely own quality from design all the way to production. We ensure this quality with a robust suite of automated tests and we also verify functionality in ourselves in our staging environment. Once we verify it is good to go, we ship it! Speaking of shipping it, we don’t hand that off either. Our cross-functional teams have a devops mindset and a dedicated devops engineer. Ideally the software engineers on the team can self-service all the devops work themselves, but we also have the devops engineer there to help us along the way. Once we have validated the changes are ready for production, we (typically the software engineers) just push a button and off it goes. And that happens many times every day across all our teams. Zero-downtime, daytime deploys are another thing we value, but that’s a topic for another day. We also try to avoid handing code off from one team to another. While I can’t say we’ve never done this, it is something we actively try to avoid. We’re comfortable with our own code, and working in code that we didn’t help create causes fear. When you fear changing code, the code eventually decays because we are afraid to do the type of refactoring that is necessary to keep code healthy. And, of course, making changes to code we didn’t create is time consuming and fraught with peril because we lack context around the decisions that were previously made and the unintended impact our changes may have on this mysterious codebase.

Summary

Creating our product collaboratively is one of the many principles that we value at Pluralsight and have documented in Engineering at Pluralsight. These guiding principles and the corresponding practices have helped us to scale engineering at Pluralsight while communicating, maintaining and evolving our values and practices as we grow. It has been valuable for us to separate values and principles from practices – this allows us to tailor our practices to support what we truly value. Practices are a means to an end. The values (or principles) define the goals, and the practices are our current best understanding of how to achieve those goals. Have you given thought to the things that you value in your organization? If you were to create something like Engineering at Pluralsight, what would it look like for your organization?