Cloud optimization projects come in many shapes and sizes and often have different goals depending on who is accountable for the outcome. Some leaders may be focused on cost savings, while others demand higher performance. However, teams that dive into the project's technical components without proper planning can become overwhelmed by the abundance of configuration options and fail to deliver on the project’s goals.
This article explores a five-step strategy to run cloud optimization projects with business goals in mind. We look at ways to tightly scope the teams and workloads involved and find opportunities for continuous improvement and future projects. Once your team is confident in the project’s strategy, it’s easier to focus on the intricacies of the individual cloud components that make up your infrastructure.
Five steps to cloud optimization—summary
| Step | Description |
|---|---|
| Understand business goals first |
Optimization projects should begin with business goals such as:
|
| Define the organizational scope | Cloud services often span multiple teams. Tightly scope projects to increase agility and reduce intercommunication requirements. |
| Document the “must haves” | Your industry may require explicit standards. Begin projects by documenting the non-negotiables, then focus on other opportunities. |
| Individually review cloud services before checking the entire system | You can leverage cloud services in countless ways, and there is no one way to accomplish a goal. By focusing on each service first, it is easier to optimize the final system design. |
| Analyze the results | Ensure that your optimization project either met or exceeded the business goals and that your “must haves” are in place. |
#1 Understand business goals
Some businesses define themselves as technology companies and utilize cloud services to deliver their core competencies. Others simply view cloud providers as another data center or backup facility. The nature of your company’s relationship with cloud providers can determine your opportunities for cloud optimization. For example, if a cloud environment was set up a couple of years ago and never touched again, there are likely several opportunities for resource optimization.
The most common business goals for optimization projects include reducing costs, tightening security boundaries, cleaning up rogue resources, or ensuring business continuity during a disaster. Regardless of the goal, there is likely a focus on increasing revenue or reducing costs, and these metrics should be captured. Some costs, such as security breaches or natural disasters, are harder to measure but should still be quantified with high-level estimates. Additionally, engineering time and agility should not be underestimated, and an optimization project that leads to time savings for developers should be quantified. Remember that if you can’t measure it, you can’t improve it.
The most important goal of an optimization project should be to move quickly. By tightly scoping business goals, you can ensure agility by reducing the intercommunication required between groups in your organization. It is better to run multiple smaller optimization projects in weekly sprints with a handful of clearly defined goals than attempting one large optimization project that spans a year and attempts dozens of business outcomes.
Common business goals of cloud optimization
#2 Define organizational scope
After the business goals have been defined, organizational requirements are next. In smaller companies, an entire optimization project may belong to a single IT team with input from finance or operations. However, in large organizations, cloud services often span organizational boundaries and require involvement from security, networking, and engineering teams. Almost every cloud product is configurable in multiple dimensions, so having the correct teams giving input is important.
The best way to build teams around optimization projects is to avoid focusing on specific cloud resources, projects, or services. Instead, map them back to existing organizational boundaries in your company. This can also help estimate timelines. For example, if including a security or enterprise architecture team in an initiative is challenging, you can document that dependency and get them involved early and often.
Engineers are typically the most expensive resources in an organization and understand cloud environments better than others. That being said, if you are starting an optimization project that includes various non-technical tasks, it should consist of a dedicated project manager or team member ready to coordinate cross-departmental meetings, segment non-technical from technical tasks, and document the outcomes so the project doesn’t become a net drain on engineering time.
Additional tips
Once key teams and individuals have been defined for a project, outline a RACI model for everyone involved. This should be a collaborative exercise to ensure that each individual knows their role in the project.
When defining a team’s size or involvement, less is more. The best projects focus on needing fewer responsible and accountable people to move milestones forward, but also include a strong network of individuals that can be consulted when necessary.
A useful estimation method called the group intercommunication formula is written as: n (n – 1) / 2. For every person who is added to an optimization project, the communication complexity increases. A project with 10 people has 45 individual lines of potential communication. With 11 people, that number increases to 55. The smaller the team, the more likely the success of effective communication between the team members and completing a project on schedule.
It can be tempting to include additional team members “just in case” they are needed, but it’s better to build a core team of necessary inputs over edge cases. Likewise, if you find additional optimization opportunities during a project, document them for a later project instead creating additional work on a small project that is being executed by a nimble team.
Communication points between people and project
#3 Identify non-negotiable requirements
Once you have a well-defined team with clear business goals, next comes the “must haves.” Documenting the “must haves” first and then focusing on the “nice to haves” in any cloud optimization project is crucial to eliminating scope creep and ensuring the original goals are met.
If you operate in a highly regulated industry, such as financial services, there are likely third parties that dictate some of your requirements. These range from data retention policies to non-negotiable security boundaries between cloud environments. Before embarking on an optimization project, ensure these requirements are documented so optimization activities do not result in non-compliance or create additional work later.
Example of must-have requirements
A common example is PCI compliance, which requires you to maintain documents and audit several requirements regularly. If you’re not in a regulated industry but still want to implement best practices across your cloud environments, reviewing recommendations from the Cloud Security Alliance and CIS benchmarks is an excellent starting point.
Another common example of a hard requirement is maintaining availability, such as uptime or p99 response times, to meet a contractually defined SLA. If you’re embarking on a project that may put an SLA at risk, such as reducing infrastructure costs, ensure testing and documentation are in place before the project begins.
Example of secondary requirements
Secondary requirements may include bin packing VMs or Kubernetes clusters more efficiently, but they may not necessarily be a definition of success or a meaningful outcome for your project. You may find that the incremental improvements of running a machine slightly hotter do not lead to the cost savings from simply deleting a different VM. It is important to remember that nothing is 100% perfect in the cloud, but it is often just a series of tradeoffs. The constant innovation in the space also indicates that many of these services are dynamic resources with evolving functionality.
#4 Review individual cloud components
At this point, you have everything you need to embark on a cloud optimization project. With business goals, an empowered team, and documented requirements at the ready, you can finally dive into the details of your cloud services.
Map team members to individual services
It’s important to review each cloud service individually and then the system as a whole. Make a list of each service in the scope of your optimization project, and then assign dedicated individuals to become experts in each service that falls within their purview. Document the architecture of how these services are interconnected to ensure you’re not creating a single point of failure or bottleneck during your optimizations.
Pick a “lens” to view each service based on the RACI model outlined above. For example, if you’re running a security-focused project with individuals from a security team, focus on the best practices related to security. It’s unlikely that the security personnel will care how data replication happens inside a SQL database, but they will certainly care who can access that database.
Examples of analysis for cloud optimization
Cloud services are often described as a “LegoⓇ box” that can be assembled however the end users see fit. By focusing on each component individually, your team can understand the features and limitations of each, along with the control you can exert over them. For example, if an optimization goal is ensuring your operating systems are up to date, you may learn that an IaaS service provides much more control over patching operating systems while your PaaS service limits access to the underlying OS.
Most vendors provide an overview of best practices for each of their services. For example, Google Cloud provides specific best practices for running Windows Server on a VM, while AWS offers best practices for EC2. In Azure, you may need to optimize for running SQL Server on a VM.
#4 Analyze your cloud optimization project results
After the cloud optimization project is complete, reviewing the results is important. Returning to the first step of our project of defining business goals, ensure that your optimization project either met or exceeded those goals and that your “must haves” are in place. If a goal wasn’t met or a requirement was not met, perform a retrospective analysis to uncover the issues that prevented success. If the team discovers additional optimization opportunities during their project, review them and plan the next sprint with them in mind.
This final step is to determine what went well, what went poorly, and set aside improvement opportunities for the next project. Performing this retrospective analysis within two weeks after the project is completed is best.
If the project’s business goals include quantitative metrics that can be reviewed periodically, build them into your retrospective process. Some metrics may take days or months to measure, but back to the project's original goal, if you can’t measure it, you can’t improve it.
Tightly scope sprints and retrospectives
|
Platform
|
Provisioning Automation |
Security Management |
Cost Management |
Regulatory Compliance |
Powered by Artificial Intelligence |
Native Hybrid Cloud Support
|
|---|---|---|---|---|---|---|
|
Cloud Native Tools |
✔
|
✔
|
✔
|
|
|
|
|
CoreStack
|
✔
|
✔
|
✔
|
✔
|
✔
|
✔
|
Conclusion
Following this five-step process for cloud optimization projects is a structured way to meet your business goals while ensuring your cloud environments comply with regulations. Projects should be tightly scoped, each individual should know their role in the project, and the team should be ready to dive into the complexities of each cloud service given their role. CoreStack includes several tools to help with each step, including measuring costs, application performance, and security postures.
Leveraging purpose-built tooling like CoreStack for optimization projects drastically reduces the time to value and increases agility for the teams involved. It can also provide augmented expertise in individual cloud services so the project team can focus on what they do best without needing to become experts in multiple platforms. You can deliver cloud optimization projects that ensure agility, compliance, and diligence throughout.
