Reduce errors, improve consistency, and deploy with confidence using codified IaC best practices¶
Business Problem¶
Enterprises with complex information technology infrastructures struggle to reproduce infrastructure consistently for applications and services. Among the common challenges are manual processes for deploying and configuring infrastructure and overly simplified automated processes that lack required validation steps. Available guidance is fragmented, and a lack of applied best practices can result in infrastructure downtime, security vulnerabilities, and inefficient resource utilization.
Key DevOps practices and engineering fundamentals such as infrastructure as code (IaC), continuous integration/continuous deployment (CI/CD), and testing ensure infrastructure resources deploy consistently across all environments. Using tools to automate the application of these practices minimizes the use of error prone manual tasks in the deployment phase, leading to increased reliability and security.
Solution¶
To overcome common infrastructure provisioning and deployment challenges, enterprises use components from a repository, Symphony, to manage aspects of their IaC deployment lifecycle within their cloud environments. The Symphony repository encompasses codified IaC best practices and workflows that developers can apply to IaC projects to achieve process automation and end to end validation before deploying to production. Symphony's purpose is to help organizations improve availability and reduce remediation time across on-premises and multi-cloud environments.
Symphony includes templates and workflows to help with:
- Automating deployment of resources using IaC
- Incorporating development lifecycle practices for IaC
- Resource validation
- Dependency management
- IaC test automation and execution
- Security scanning
Depending on the use case, Symphony can be used to manage the full deployment lifecycle or improve the reliability of existing IaC deployments.
Challenges Addressed by Symphony¶
Manual provisioning and deployment tasks - Organizations that provision and deploy cloud resources manually run the risk of accidentally deleting cloud resources during mission-critical production deployments. Adopting IaC with CI/CD best practices ensures correctness and consistency across production deployments. Using Symphony to automate best practice application reduces the need for error prone manual steps and shortens time to re-deploy a cloud infrastructure.
Serial cloud deployment - Serial cloud deployments require an entire infrastructure deployment for even the most minor change. When setting up IaC environment structure, using Symphony to modularize deployments enables parallelized deployments. A modular IaC deployment will reduce deployment time as teams would be able to independently deploy modules instead of the entire infrastructure at once.
Lack of infrastructure testing - Testing is a fundamental software practice. Automated testing of infrastructure code enables IaC scripts validation before deploying to production. Lack of testing and best practices results in inefficient deployments, redundancies, and a complex deployment environment. Implementing an automated IaC testing strategy with Symphony simplifies the testing process and accelerates deployment.
Value Proposition¶
Managing IaC deployment lifecycle with Symphony -
- Improved consistency and repeatability of infrastructure provisioning
- Increased confidence in the reliability of each IaC module, and the solution as a whole
- Enforced security best practices
- Improved readability and maintainability of IaC
- Improved collaboration and onboarding of new team members
Applying IaC and infrastructure deployment test automation with Symphony -
- Reduced risk of errors and inconsistencies in IaC scripts
- Early internal consistency errors detection
- Enforced coding standards and organizational practices
- Reduced time spent on code reviews and detection of issues
Logical Architecture¶
The following logical architecture diagram illustrates how the Symphony Pull Request (PR) workflow is used to provision and orchestrate infrastructure deployments. This workflow ensures best practices are applied in IaC repos in the development phase. The PR workflow provides a set of validations in the code review process. Validations will vet the PR branch code changes. Changes to the IaC are validated at the code quality level. Changes are also tested on a real deployed environment to determine impact of resource changes.
Validate¶
This stage ensures code readiness of the pull request changes. It executes pre-validated events, runs validations and linting tools, scans code for possible cred leaks, and executes any unit tests. Stage steps are executed in the following sequential order.
Preview & Deploy¶
This stage plans the execution of the IaC code and estimates the scope of the changes. The steps include:
- initializes the IaC tool selected,
- runs plan/what-if commands to detect the changing scope,
- executes pre_deploy events,
- runs deploy commands to update the resources,
- executes post_deploy events, and
- ensures successful resource updates.
Test¶
This stage executes the integration or end-to-end tests against the recently deployed/updated resources. Successful tests ensure the configurations/changes are reflected and resources are working as expected. It then publishes the results of the tests and drops them as artifacts for future reference.
Destroy¶
This stage destroys the deployed IaC resources of an environment and report any failures for easier cost and resource management. Details of stage execution may vary based on features available on the orchestrator's IaC tool.
In this workflow, a set of input variables can be updated at execution time. While the inputs could vary based on the selected IaC tool, one common input variable across all is the environment name, which defines what environment configurations to be used.
Report¶
This stage generates the needed scripts to repro the deployments if available, publish the created reports, and backup state files if necessary.
Implementations¶
You will need to consider the following when using/building implementations for the solution:
- Choice of IaC tool: Terraform is multi-cloud and platform-agnostic. Bicep, however, is Azure-specific. If you are working exclusively with Azure, Bicep might provide a more streamlined and simplified experience. But if you're working with multiple clouds or planning to in the future, Terraform might be a better choice due to its broader compatibility.
- Expertise: Terraform has its own DSL, HCL, which might require some learning if you or your team are not familiar with it. Bicep, on the other hand, is designed to be simpler and more readable than ARM templates.
- Integration with CI/CD tools: Both GitHub Actions and Azure DevOps have strong integration with Terraform and Bicep. Consider the rest of your tech stack, as well as your team's familiarity and comfort with these tools.
- Security: IaC can inadvertently lead to security issues if not managed carefully. Sensitive data might accidentally get exposed, or insecure configurations might get deployed.
- Version control: IaC goes hand-in-hand with version control. Make sure all infrastructure changes are tracked in a version control system like Git. This allows you to see who made changes, what the changes were, and when they were made.
- State management: With Terraform, consider where and how you will store your state files. Remote state storage with locking is usually a good idea for team environments. Bicep doesn't have state files, as the state is managed by Azure Resource Manager.
Terraform Deployment with Symphony¶
This Symphony implementation features GitHub Actions as an orchestrator and Terraform as the IaC tool. Symphony configures the IaC repository with pipelines and settings needed to achieve CI/CD for IaC.
To provision your new repository with Symphony using GitHub Actions and Terraform, the following command is executed:
This reference implementation contains the output of what to expect in your environment:
Bicep Deployment with Symphony¶
This Symphony implementation features GitHub Actions as an orchestrator and Bicep as the IaC tool. Symphony configures the IaC repository with pipelines and settings needed to achieve CI/CD for IaC.
To provision your new repository with Symphony using GitHub Actions and Bicep, the following command is executed:
This reference implementation contains the output of what to expect in your environment:
Using Symphony with other well-known Azure deployment services¶
While Symphony can be used in isolation, a service like Azure Deployment Environments (ADE) can be used to provide isolated, secure, and repeatable environments to test and validate Azure infrastructure before it's deployed to production. Symphony and ADE can be used together to create a robust deployment pipeline.
In a typical use case, you use Symphony to define and orchestrate your infrastructure as code. These definitions are then deployed to an ADE for testing and validation. Once validated, the same definitions are used to deploy the infrastructure to your production environment. This way, ADE and Symphony complement each other, combining to deliver a powerful, automated, and reliable infrastructure deployment pipeline.