035 - Determining your cloud maturity

In this audio guide, Jeremy Morgan will walk you through the process of determining which phase of cloud maturity your organization resides in. He’ll break down the cloud migration process, establish where you are and help identify the direction you want to go in.

• The Pluralsight Cloud Maturity Matrix

If you enjoy this episode, please consider leaving a review on Apple Podcasts or wherever you listen.

Please send any questions or comments to podcast@pluralsight.com.

00:00:06.7 Daniel Blaser

Hello and welcome to All Hands on Tech, where today's leaders talk tomorrow's technology. I'm Daniel Blaser. 

Migrating to the cloud is one of the biggest challenges organizations face today. You need systems that are faster, more secure, and scalable. It's difficult to know where to start or how to find the best way forward. You need a solid plan. In this audio guide Jeremy Morgan will walk you through the process of discovering which phase of cloud maturity your organization resides in. He'll break down the cloud migration process. determine where you are, and where you want to go.

00:00:42.8 Jeremy Morgan

No question, the era of cloud computing is here. What was once considered a "Nice to have" or "Something to shoot for" is now a foundation of any technology organization. The big question is why? Why should organizations spend money, resources, and people's time to move everything to the cloud? Is it so we can chase the latest fads or tell everyone we're on the cutting edge of tech? No. There are very practical reasons for cloud migration. Reason one: moving faster. With the emergence of DevOps and changes to how we deliver software, speed has increased. Customers aren't waiting six months or a year for a batch of features anymore. They expect constant improvement. If your competition is delivering software features faster, you're at a disadvantage. Cloud technologies are designed around the concept of continuous delivery. 

Another reason is decreasing spend. Lower costs have always been important to any business. Reducing your expenses means more cushion for unexpected events and the ability to increase your investments and grow. Cloud services enable you to only pay for what you need and, long-term, they provide cost savings that traditional infrastructures just can't match. 

Also, improving security. Security risk increases by the hour these days. Moving everything in our lives to the internet gives motivation to bad actors to access your data. Cloud services provide security and peace of mind that's expensive to implement with traditional infrastructure. 

Finally, driving scalability. In today's world your business could double the amount of customers in a day. When that happens, you don't want to be lugging servers into your data center and plugging them in. With the cloud, you'll be moving a slider or having automated scripts that add that capacity for you. Migrating your infrastructure is an investment in your organization. An important thing to understand is, organizations aren't changing to adapt to the cloud. The cloud is adapting to changes to the way we develop software. Most cloud services are designed as a reaction to a changing process. For instance, engineers aren't moving to virtualized servers because they're available on the cloud. Many organizations have been using virtualized servers for years because it improves the software development process and makes your infrastructure more scalable. Cloud providers are now offering virtual servers at a lower cost with automation tools to assist in provisioning. 

Long story short, it's the organizations themselves that are driving change and cloud providers are providing the best solutions in reaction to those changes. Today we're going to talk about migrating your organization to the cloud. 

00:03:18.2 Jeremy Morgan

Let's talk about cloud maturity. We'll explore the idea of cloud maturity and what it means for your organization. First, we'll talk about where you are now, how you can determine where your organization is currently in their cloud migration, then we'll talk about where you want to be. Very few organizations are living a hundred percent in the cloud and many don't need to be. We'll look at how to assess that. Finally, we'll talk about migrating your infrastructure and what that looks like, some common pain points, and how to address them. 

What is cloud maturity? It's a benchmark that determines where you're at in your transformation from on-premise infrastructure to cloud-based infrastructure. It's far from a straight line path. It's full of pitfalls and lessons. At one side, we have an infrastructure we're all familiar with. Our servers are hosted in the building, in the data center. We have servers set up and they're named after Greek gods or Disney characters and we have a smart group of Engineers keeping them running 24/7. We have web servers, database servers, file servers, you name it. They're all right down the hall in that noisy room with the dim lights and raised floor, or maybe it's a repurposed office or broom closet. 

This is what's known as a hundred percent on premises, which we refer to as on-prem. This model served us well for decades and frankly, it still serves some companies well, but this is one end of the spectrum. On the other end of the spectrum we have everything in the cloud. We have very few servers, per the old definition, because most of our applications run in services. We don't have database servers. We have a data service that scales up or down when we need it. There isn't a single machine in our building because it's all in the cloud. That's the other end of the spectrum. 

Now, our goal in cloud transformation is to move from the on-premise model to the cloud model as quickly as possible, right? Well, spoiler alert. Very few organizations out there are a hundred percent in the cloud, but we need to determine where on the spectrum our organization resides. As I mentioned, it's not a straight line from on-prem to the cloud. 

Pluralsight has a Pluralsight Cloud Index which represents where you're at on a particular vector of cloud adoption. Each point on this map represents a step moving to the cloud. It goes off in all directions. I'll break it down into some of the phases so you can determine where you're at. You can be really far in one direction, say applications, and be at ground zero with another one, like continuous integration. 

00:05:45.7 Jeremy Morgan

Here are the phases of cloud maturity when you're going from an on-premise solution to the cloud. The first step is experimentation and foundation. This is where you are just starting to consider a move to the cloud and planning your strategy. 

Now, phase two is when you're in the process of starting to move things over. Phase 3 is repetition and expansion. You've moved things over, you have a system in place, and you're operational in the cloud. Now you're just migrating small parts of your system over. Phase 4 is when you've made all the moves you need to make, and you observe and maintain the cloud-based infrastructure. Let's determine where we're at.

If you're in Phase 1, you don't have anything in the cloud. You want to move to the cloud and you're making plans to do so. Here are some of the things you need. The first vector is executive buy-in. It's a critical vector, but it's a wild card due to organization size. It could be that you're the Head of IT and Development and you're the Executive, or you may be in a spot where there are many layers of permissions needed. At one end of the spectrum we have "Not approved". Nobody has approved any movement into the cloud. Then it goes to "Proposed". There's a plan in place that needs approval. Then we're "Under review". A plan has been submitted. Next, we're "Approved for a hybrid of on-prem and cloud". Finally, "Approved for full cloud". This part of the phase involves getting permission to move into the cloud and getting executive buy-in isn't always easy. But the move to the cloud can be fairly expensive. Not only are you incurring new costs from the cloud provider, you're paying Engineers to work on the transition and you're throwing away expensive hardware in the data center when you're done. Decision makers want to know if this is the right move to make. Most organizations go through a try and see approach here. You need to run experiments and do a proof of concept. If you're at the very beginning of this phase, a good tactic is to take a service or application with minimal dependencies and create a plan to move that to the cloud. Now, I call out dependencies rather than risk, or how heavily used the application is, for a reason. An application with a lot of dependencies will take time and resources to move and it's more likely to fail. You don't want your most problematic application or service to determine whether or not you get permission to move forward. That application that only runs on Windows Vista and churns out a report with some long-abandoned software is not the one you want to choose for your proof of concept. Do something a little more portable. 

00:08:12.1 Jeremy Morgan

Now, let's talk about hardware. In Phase 1, you have all of your servers on-prem. They're either in a data center on site, or in a building, or colocation facility. Your servers are bare metal servers mounted on racks. You may have database servers, file servers, mail servers. If they're hosted off-site you may have a VPN to access them. If they're on site they're on your internal network. You may have web servers as well, that are exposed through the firewall or located in the DMZ. This is how we've done things for decades. 

Now what does your data situation look like? In Phase 1, all of your data is on-prem. It's on file servers and database servers that reside on site on physical machines. Ideally, when moving to the cloud, you'll move towards virtual hosted database servers, and eventually database and file services. Now, your virtualization is another big determiner of maturity. For decades we had the physical servers with names and assignments and that worked great. Then came much larger machines that host small servers inside of them. Now we have virtualization in the cloud, which allows you to spec out a server with a web interface and press build.

So, where are you on this spectrum? Are you still using bare metal servers? Where are your applications hosted? Are they traditional web-based or client-server architecture apps running on site? At this stage of maturity, it's likely the case, and you haven't moved any applications to the cloud. 

How is your continuous integration? When Engineers commit code, what happens? This can be entirely independent of the cloud. You could be in Phase One and be a hundred percent on-prem and still have full integration with source code. It's one of the indicators of maturity. How about continuous deployment? Are your applications still being delivered by manually copying artifacts to production servers or do you have all of this automated? Some organizations are still slow to automatically deploy software, and with good reason. If you haven't established a resilient system with easy fallbacks, you need a human to be a gatekeeper to keep software from breaking your system. 

Finally, how's observability and monitoring? Are you just gathering logs, or do you have servers that monitor uptime, performance, and security? This is another vector that can be a hundred percent done on premises, but you'll need third-party applications to do it rather than relying on observability of cloud providers, which are often far cheaper. In the first stage of cloud maturity, you'll likely have no application monitoring happening and you'll be reliant on logs and sharp Engineers to keep an eye on things. 

One common thing you'll see here is experimentation. If I had to narrow down one thing that is required to be successful in a cloud transformation, it's experimentation. You have to try things, observe them, and make future decisions based on the results over and over again. This doesn't stop after you've moved to the cloud. It's permanent. If you know where you are now, you have to determine where you want to be. Phase One is basically square one. You likely have all your servers on premise, everything lives there, the deployment is manual, and you don't have a lot of monitoring or observation. Where do you go from there? 

00:11:23.9 Jeremy Morgan

Here are your first steps. First, you need to establish a proof of concept. This is a small project to sample a bit of cloud. Something like a small application that's running on a server that you can move to a cloud, or a website. This serves two purposes. You can prove it to yourself and your team that you're able to make the step and you can also prove it to management. 

The next step is, establish an innovation team, or you can name it whatever you like. Take a group of Engineers and task them with being the go-to folks to move to the cloud. Just the act of creating a team around it makes it real, both to the engineers and others in the organization. There's a team dedicated to this, it's serious. If possible, do a cost-benefit analysis. Did you save costs by moving to the web? Accrue more? Is it worth more money to make the switch? Don't forget to factor in the total cost of ownership. 

00:12:20.3 Jeremy Morgan

Phase Two is the migration and standardization phase. You've proved that you can move something into the cloud and you're looking into the future. In this phase you'll likely have a hybrid infrastructure. You have some things that are cloud-based while most of your applications are still hosted on-prem.

A common pattern here is when you have a company with significant internal infrastructure, they will move the public website to the cloud while leaving the rest on-prem. This allows you to test the waters and see how things are going to go. Your deployment in this phase is still pretty manual. Most of your artifacts are being moved manually from development to staging to production. Your buy-in at this point is increased, but management is still watching to see how things go and they're not yet fully committed to the cloud. 

To move out of this phase you'll need to do the following: look for patterns. What I mean by this is look for things that work. For example, if you discover a way to script and automate a database transfer from your on-prem server to the cloud, take notes. Tweak the process and use it on the next service. You'll learn the nuances between on-prem and cloud and keep notes of all that. As you migrate products or groups of applications to the cloud you want each one to be easier than the last. 

Which leads me right into my next point which is increase your automation. This is the time to automate. Think about it. When you want to automate your software deployment, for example, if there's no pressing need you aren't going to spend a lot of time on it. Fires come up, priorities shift, and your Engineers have to work on other things. However, if you need to manually move a piece of software from on-prem to the cloud, it's the perfect time to create that automation. Dust off the PowerShell or Bash skills and start automating your push. You can use this as part of your CI/CD process later. This is also the prime time to increase observability. That's another thing that gets put off by software teams. Why bother setting up observation when we have work to do? The good news is whichever cloud provider you choose it will have tools for observing your servers and applications. After you install and switch over to them, make observation part of your workflow. You'll thank yourself for it later. 

00:14:34.1 Jeremy Morgan

Now we're in Phase Three. This is my favorite phase to be in. Repetition and expansion. At this point, you are fully hybrid. You have a mix of applications on-prem and some in the cloud, including production applications and mission-critical stuff. You'll notice some renewed excitement from your Innovation Team in this phase. I called this repetition and expansion because with every new project you're repeating all the stuff that worked in the past and you're expanding the infrastructure. Each project keeps going faster and faster. You and your Innovation Team look like rock stars. You've conducted experiments and taken notes. You have the best practices in place. Naturally, you've set up some automation as part of your adoption to the cloud. Your CI and CD systems are likely maturing at this point as well. You may even have a clear path from committing to trunk and having the changes pushed out to production automatically. 

In this phase, leadership is committed to the cloud. There's no talk of going back and they're only asking when everything will be in the cloud. The expansion may stop at this point. You may move over most of your applications to the cloud and have a few that stay behind. Many organizations stay in this phase for a long time and that's okay. You may not ever go 100% cloud and you may not need to. If you're able to operate reliably and deliver features and fixes to customers fast, there's no reason to stress out about a few applications that are still hosted on-prem. A few manual processes here and there also might be okay. Your results will vary.

To move out of this phase you'll need to do the following. Make your delivery fully continuous. Deployment, integration, everything. When something's merged into main, it should pop into production shortly after. Getting this locked down and solidified allows you to focus on other things. There should be very little human intervention for delivering software. Everything that's migrated should be well documented somewhere. Just because you moved it to the cloud, doesn't mean you'll never touch it again. The more you document, the more you'll allow new Engineers coming on to understand the system and it will make life easier if you decide to move that product to another cloud service. Experiment, over and over. Try new things. Will moving from a database server to a database service run faster? Does it scale better? Is it cheaper? These are the questions to ask. You want to optimize what you have in the cloud as much as possible to increase the value of the investment. 

00:17:08.3 Jeremy Morgan

Now we're at Phase Four: Observation and optimization. Very few organizations can really say they're on the edge of each of these vectors, but it's possible. Here, we are in a cloud first environment. All of your applications live in the cloud. Any new application is created in the cloud. All of your software is delivered through continuous integration and deployment and it's done at a rapid pace. Testing is automated and baked into the workflow. Leadership is fully invested in the cloud and your job now is to maintain and improve that investment by optimizing continuously and providing full visibility into operations. 

Where do we go from here? We're in Phase Four, fully cloud, so we're done, right? Not even close. Now your job is to keep this machine running successfully. You must improve and focus on observability. Leave no stone unturned. Every part of your process should be monitored and reported on, even if you rarely use it. Profilers, testing tools, chaos generators, all of these things should be a part of your everyday life now. 

Focus on building self-healing systems. Build systems that recover from errors and fix themselves. Have systems that inject chaos into the infrastructure, have it shut down services, and see how long it takes to repair it. This is called MTTR, or mean time to recovery. Focus on scaling. What happens if you go viral overnight? What happens if you acquire another company and you have to move that traffic under your infrastructure? Focus on finding smart ways to scale up.

00:18:41.1 Jeremy Morgan

Use the following factors to determine what your level of cloud maturity is. 

Hardware infrastructure: does it live on site in a data center, fully in the cloud, or somewhere in between? 

Data: is your data on-prem with servers, all in the cloud, or in between? 

Applications: are they hosted on a traditional web server, hosted in a cloud service, or both? 

Virtualization: are you using conventional bare-metal machines, all virtual, or both? 

Deployment integration: is it fully manual, fully automated, or a mix? 

Finally, Executive buy-in: is management reviewing proposals or giving you the green light for a hundred percent in the cloud? 

Find out where you are and where you want to be. Here are the main takeaways that I've touched on here. These are the things that you'll need in order to be successful in any cloud transformation. Experiment, try new things, see what happens when you convert a service or find a new way to automate something. Then, fail often and learn. If everything you do is succeeding the first time you try it, you probably aren't innovating fast enough, and your competition will. Take risks and when you fail, learn from it. Blameless post-mortems are one of the best ways to keep moving forward. If people aren't afraid to fail, they'll innovate more. 

Finally, create a culture of learning. What I mean by this is encourage your team to experiment and fail often, as we just mentioned. When they fail, don't chastise them, or threaten to fire them. Encourage them to learn from mistakes and move forward. Encourage experiments. Any member of your team can contribute greatness. It may be your architect with 30 years’ experience or a college intern with a great idea. Listen and consider everyone and make them feel like it's safe to contribute. 

Thank you for listening to this podcast. I love the fact that people are excited as I am about things like cloud migrations. Go out and conquer this.

00:20:45.5 Daniel Blaser

Thank you for listening to All Hands on Tech. To see show notes and more info, visit pluralsight.com/podcast. 

We'll also include a link in the show notes to the Cloud Maturity Matrix that Jeremy referenced in today's episode. Thank you and have a great rest of your day.