Build a CI/CD pipeline using AWS developer tools
Building and delivering software isn't easy. Everyone wants to have a consistent quality and to be able to deliver new functionality quickly. In this write-up, I will demonstrate Continuous Integration and Continuous Delivery for a Simple .NET application using AWS Developer tools.
Assumptions - I am assuming that you have prior working knowledge of AWS EC2, IAM roles, IAM policies and S3 before checking out this post. I will try and share reference links for items that are outside the scope of this particular write-up. You are here because you are looking to build a pipeline for a .Net Application on Windows Server with IIS installed. I will not be talking about any Linux stuff here. I will probably do a post for Linux in the near future.
What AWS Services will be used to build the CI/CD pipeline?
What sample application are we going to use for the demonstration? For this demo, we are going to use a
Simple Hello World.Net MVC Application. The solution can be cloned from the public GIT repository that I have created for this blog post. You can use visual studio to view the contents and debug the application locally.
Example - If I were to run 100 Builds on a Windows, build.general1.medium instance per day each lasting 2.6 mins each then I will be charged around (3*100*0.018) $5.4 per day.
For more information on pricing, please check the official AWS documentation
How do I set up and configure CodeBuild? Open the AWS CodeBuild console at
https://console.aws.amazon.com/codebuild/. Create a project using the CodeCommit Repository. See below for the configuration details:
What is the buildspec.yml file? A
build spec is a collection of build commands and related settings, in YAML format, that AWS CodeBuild uses to run a build. Without a build spec, AWS CodeBuild will not be able to successfully convert your build input into build output, nor will it be able to locate the build output artifact in the build environment to upload to your output bucket. Below is the build spec file from the sample project:
How do I test my build? Open the AWS CodeBuild console at
https://console.aws.amazon.com/codebuild/. Select your previously created project and start the build.
If everything goes smoothly then you build will succeed and you will see the screen below output screen.
Now you can move to the S3 portal and check your published code in the bucket you picked for uploading your artifacts. This code is ready for deployment and next, we will be deploying this to an EC2 Server using CodeDeploy.
Note - The build will not always go smoothly. I had an issue with the below error:
Error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in the <Import> declaration is correct and that the file exists on disk. This is a very common issue on build servers where you don't have Visual Studio installed. I actually added the package using NuGet in the Solution and updated .csproj to import the path in the package directory.
Now, we have our build running successfully we will look at the deployment phase.
Setup and configuration
This is what we will have once we have our pipeline configured and functional: There are a lot of steps that are involved in setting up the continuous deployment pipeline. I have broken these steps into different parts and will go over them one by one to make things easier to understand.AWS CodeCommit
To start with, we will 1. migrate the existing code from Github to AWS CodeCommit. AWS CodeCommit is a version control service hosted by Amazon Web Services that you can use to privately store and manage assets in the cloud. CodeCommit is simply like any other version control system available out there. It is as secure as any code you will put into a CodeCommit repository, and will be automatically be encrypted by AWS KMS service without having to do any additional configuration. How much will CodeCommit cost? The first five active users (IAM users that have permissions to manage CodeCommit) can operate for a month for absolutely no cost provided they don't cross the preset limits. Any additional user will be charged $1/user/month. First five users are entitled to:- Unlimited repositories
- 50 GB-month of storage
- 10,000 Git requests/month
- Unlimited repositories
- 10 GB-month of storage per active user
- 2,000 Git requests/month per active user
- $0.06 per GB-month
- $0.001 per Git request
How do I set up users for AWS CodeCommit?
Step 1. You will need to log in to the AWS console and then switch to IAM. Here you can create a user with following options: Access type - Programmatic Access (if you want to assign console access then you can feel free to do that as well) Policy - AWSCodeCommitFullAccess (since this is a developer environment, I will keep things simple and grant my developer full access to manage CodeCommit) Remember to save access keys and secret keys generated at IAM user creation time. You will not be able to retrieve these keys at a later stage. If you are not aware of how to create an IAM user then please refer, AWS documentation to create AWS IAM User.How do I set up the local machine to interact with CodeCommit?
Step 2. On the Windows machine that you will be using to develop the application, normally your own desktop machine, do the following:- Install the AWS-CLI with all default options. You can find the install guide and installer here.
- Install Git on your desktop. You can download it for Windows from the official download link.
- Install AWSToolsAndSDKForNet.msi so that you can configure the Credential Helper to use with HTTPS Communication Protocol for GIT.
- Open cmd
- Move to cd "C:\Program Files (x86)\AWS Tools\CodeCommit"
- run command "git-credential-AWSS4.exe". This will create a .config file to configure our credentials
- run command "git config --global --edit". Take a look at .config file contents.
- Clone the GIT repository to the local machine:
- [code lang="bash"]git clone https://github.com/pythianarora/proj-aws-dev-tools.git temp-local-proj-aws-dev-tools[/code]
- Get the CodeCommit Repository URL to push to:
- [code lang="bash"]aws codecommit get-repository --repository-name proj-helloworld[/code]
- Push this to AWS CodeCommit repository:
- [code lang="bash"]git push https://git-codecommit.us-east-1.amazonaws.com/v1/repos/proj-helloworld --all[/code]
- Delete temp folder on local.
- Clone CodeCommit repository to local:
- [code lang="bash"]git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/proj-helloworld local-proj-helloworld[/code]
AWS CodeBuild
We will now configure 2. CodeBuild to build our application code. AWS CodeBuild is a fully managed build service that compiles source code, runs tests, and produces software packages that are ready to deploy. CodeBuild does integrate with Github as well, but I wanted to do do this write-up with all services in AWS. How much will CodeBuild cost? With CodeBuild you are charged based on the duration it takes for your build to execute. Build duration is calculated in minutes, from the time you submit your build until your build is terminated, rounded up to the nearest minute.Compute instance type | Memory (GB) | vCPU | Linux price per build minute ($) | Windows price per build minute ($) |
---|---|---|---|---|
build.general1.small | 3 |
2 | 0.005 | N/A |
build.general1.medium | 7 |
4 | 0.010 | 0.018 |
build.general1.large | 15 |
8 | 0.020 | 0.036 |
AWS CodeDeploy
Now we will 3. deploy the code to an EC2 Server using AWS CodeDeploy. AWS CodeDeploy is a deployment service that automates application deployments to Amazon EC2 instances, on-premises instances, or serverless Lambda functions. I am going to perform a quick, in-place deployment to demonstrate the concept but AWS CodeDeploy presents a lot of deployment options including Blue-Green Deployment. If you are interested in learning more about CodeDeploy, check the official documentation. How much will CodeDeploy cost? It is free for any deployments to EC2/Lambda. You pay $0.02 per on-premises instance update using AWS CodeDeploy. Where is the deployment going to happen? Create an EC2 Server with following settings:- AMI - Microsoft Windows Server 2016 Base
- Instance Type - t2.micro
- Instance Details - Leave everything default as this is a Lab environment and I am not interested in setting up VPC for the same. You will need to need to attach an EC2 role to this instance that will give S3 access to this instance. For details on creating the role for EC2 instance to access S3, refer this AWS documentation here.
- Storage - Leave this to default i.e. 30 GB EBS root volume for windows.
- Add Tags - This is important because this is what we are going to use when Deploying toy instance from CodeDeploy. Create a Nametag
- Name: proj-hello-world-ec2-instance
- Security Groups - Add RDP and HTTP ports for source - 0.0.0.0/0 - For lab setup so I don't need to lock this down.
- Key Pair - Launch and Create a Key Pair to connect to the instance.
- Navigate to "deploy-proj-helloworld" application and deploy the new revision.