Using AWS CodePipeline, AWS CodeCommit and AWS CodeBuild to always keep your Golden AMIs up to date

Introduction

Adoption of DevOps principles is fundamental for any company that wants to deliver solutions at great speed and agility. These principles allow you to deploy solutions in an auditable, consistent and repeatable manner.

At a minimum, you should have two environments – non production and production. Your non production environment will be used to innovate and to fix bugs. After the artifacts have been properly tested in non production, they can be promoted to the production environment, where it can be deployed using an appropriate change management process. This ensures that your production environment always runs a stable version of the artifacts and is guarded from any issues and outages.

To ensure all deployments are consistent, repeatable and are auditable, deployment pipelines must be used to deploy and update environments.

The pipelines can be configured to provision Amazon EC2 instances using the AWS supplied Amazon Machine Images (AMIs) and as part of the provisioning process, you would install additional packages and customise it to suit the use case. This is a good solution, however you will lose significant time during your deployment process since each server will have to spend additional time during this customisation process (depending on the number of additional packages and customisations, it can take anywhere from a few minutes to a couple of hours). The other problem with this approach is that since each of the servers are being customised individually, they might end up running different versions of the packages (you could version pin the packages to get around this).

A better option is to use a golden AMI for your Amazon EC2 instance deployments. A golden AMI is created using a pipeline that is different from your deployment pipeline, by installing all the additional packages and customisations on the AWS supplied AMI. Your deployment pipeline will then use the golden AMI instead of the AWS supplied AMI. This will make your deployments quicker since you won’t have to spend time on customisations.

To ensure our environments have a good security posture, we must regularly patch them with the latest security updates. In the cloud, you have the option of installing the updates in place or creating a new golden AMI that includes these updates and then refreshing your environments with this new golden AMI. I am a big advocate of immutable infrastructure and so always back the latter.

Most of you might be aware that AWS refreshes their AMIs frequently. The new AMIs include the latest security bug fixes at the time of release. You could then use these to create your golden AMIs.

Checking for when AWS releases a new AMI can be a daunting task, of which I am not a fan of. That is why I created a solution that would check daily if AWS had released a newer version of the AMI that I use, and then it will automatically build a new golden AMI using it. This ensured that my golden AMI was always up to date.

In this blog, I will take you through the solution I developed.

High Level Architecture

The diagram below shows the high level architecture of the solution. Each of the key components is denoted by a number and its explanation is provided underneath the diagram.

Below is the explanation of the key components (items marked with a number).

1. An Amazon EventBridge rule runs daily at 18:00 UTC (04:00 UTC+10). This rule invokes an AWS Lambda function.
2. The AWS Lambda function checks if a new Amazon Linux 2 AMI has been released by AWS since it last ran (the golden AMI uses Amazon Linux 2 as its base). The AMI id of the last known Amazon Linux 2 AMI is stored in an AWS Systems Manager Parameter Store Parameter. The AWS Lambda function queries AWS for the latest Amazon Linux 2 AMI and then compares its ami id with what is stored in AWS Systems Manager Parameter Store Parameter. If they are same, then no new Amazon Linux 2 AMI was released by AWS, however if they differ then a new Amazon Linux 2 AMI was released since the last time the AWS Lambda function ran.
3. If the AWS Lambda function finds that a new Amazon Linux 2 AMI had been released, it updates the AWS Systems Manager Parameter Store Parameter with the new AMIs ami id.
4. The AWS Lambda function then triggers the AWS CodePipeline pipeline which will build a new golden AMI.
5. The AWS CodePipeline pipeline starts by retrieving the latest committed code from the AWS CodeBuild repository. This repository contains all the information that the AWS CodeBuild project will need to build a new golden AMI.
6. The AWS CodePipeline pipeline passes the artifacts from AWS CodeCommit to the AWS CodeBuild project. The AWS CodeBuild project uses the buildspec file contained in the artifacts to build the golden AMI. As part of this build process, it downloads Hashicorp Packer and uses it to carry out the image creation process. The ami id stored in the AWS Systems Manager Parameter Store Parameter is used as the base for the golden AMI. When the build is successful, the AWS CodeBuild project updates a parameter in AWS Systems Manager Parameter Store with the ami id of the latest golden AMI. If the build fails, the AWS CodeBuild project reverts the value of the base ami in AWS Systems Manager Parameter Store to its previous value. This enables the pipeline to retry this base ami on next invocation, giving time to troubleshoot and fix the issue that caused the failure.
7. Amazon EventBridge rules are triggered whenever the AWS CodeBuild project starts, succeeds or fails.
8. When the Amazon EventBridge notification rule is triggered, it sends the message to an Amazon Simple Notification Service Topic.
9. The Amazon Simple Notification Service sends an email message with the notification to all email addresses that are subscribed to it.

Understanding the code

Now that we know how the solution has been put together, lets go through the code to see how all of that is actually achieved.

The solution is written using an AWS Serverless Application Model (AWS SAM) template. It uses these AWS services – AWS CodeCommit, AWS CodeBuild, AWS CodePipeline, Amazon Simple Notification Service, AWS Systems Manager Parameter Store, AWS Lambda function and Amazon EventBridge rules (scheduled and pattern). The AWS Lambda function is written in Python 3.7. The AWS CodeBuild project uses HashiCorp’s Packer for creating the golden AMI.

1. First, download the code from my GitHub repository. To do this, run the following command.
   1. git clone https://github.com/nivleshc/blog-create-evergreen-golden-amis.git
2. Open template.yaml in your favorite IDE.
3. This file is divided into sections – Parameters, Resources and Outputs. Lets go through the Parameters first. Think of these as variables and constants, which will be used in this AWS SAM template. Below is a snippet of this section.

	Parameters:
	ProjectName:
	Type: String
	Description: This is the name of the project. This will be used to prefix resource names where unique names are required
	EmailForNotifications:
	Type: String
	Description: The email address to which all notifications regarding then Golden AMI builds will be sent to.
	VPCId:
	Type: String
	Description: The id for the VPC where the packer temporary EC2 instance will be created.
	CodePipelinePipelineName:
	Type: String
	Description: Name to be used for the AWS CodePipeline pipeline
	CodePipelineArtifactStoreS3Bucket:
	Type: String
	Description: Amazon S3 bucket used by AWS CodePipeline for storing artifacts
	CodeCommitRepoName:
	Type: String
	Description: Name to be used for the AWS CodeCommit repository that will be created for this project
	CodeCommitBranchName:
	Type: String
	Description: Name of the AWS CodeCommit repository branch that will be used to trigger the CodePipeline pipeline
	CodeBuildProjectName:
	Type: String
	Description: Name to be used for the AWS CodeBuild project.
	CodeBuildCWLogGroupName:
	Type: String
	Description: CloudWatch Logs log group name that will be used by AWS CodeBuild for logging purposes
	CodeBuildCWLogStreamName:
	Type: String
	Description: CloudWatch Logs log group stream name that will be used by AWS CodeBuild for logging purposes
	AmiNamePrefix:
	Type: String
	Description: The prefix to use for the AMI name for the AMI that Packer will create.
	PackerTemplateFilename:
	Type: String
	Description: Name of the Packer template file to use when creating an AMI using packer. This file must exist in packer_files/template folder
	BaseAmiSSMParameterName:
	Type: String
	Description: AWS SSM Parameter Store Parameter Name for where the latest base ami's ami id will be stored
	BaseAmiSSMParameterDesc:
	Type: String
	Description: The description for the AWS SSM Parameter Store Parameter Name that has the latest base ami's ami id.
	GoldenAmiSSMParameterName:
	Type: String
	Description: AWS SSM Parameter Store Parameter Name for where the latest golden ami's ami id will be stored
	GoldenAmiSSMParameterDesc:
	Type: String
	Description: The description for the AWS SSM Parameter Store Parameter Name that has the latest golden ami's ami id.

view raw blog-create-evergreen-golden-amis-template-parameters.yaml hosted with ❤ by GitHub

The description underneath each of the parameters provides a good explanation about what it will be used for. The values for these parameters are defined in the Makefile and will be passed to the AWS SAM template when when the deploy target is called. We will cover the values in the Implementation section later on.

4. Next, lets move on to the Resources section. This defines all the resources that will be provisioned by this AWS SAM template, together with their configuration.

The first resource definition is for the AWS IAM role that will be used by the AWS CodePipeline pipeline. When you create your first AWS CodePipeline pipeline, you get an option to have the wizard create this IAM role for you. It populates the appropriate permissions that will be required. Just remember that this is not an AWS managed role, but a user managed role. This means that as you add more resources to your AWS CodePipeline pipeline, you will have to update the permissions in this service role accordingly. Since we are not going to be using the wizard, I will create this IAM role in this AWS SAM template.

To get the correct permissions, I first used the wizard to create the service role and then translated the resulting service role’s permissions into a resource in this AWS SAM template. I must admit, the policy is quite lengthy.

	# IAM role that will be used for AWS CodePipeline pipeline
	CodePipelineRole:
	Type: AWS::IAM::Role
	Properties:
	Description: Policy used in trust relationship with CodePipeline
	AssumeRolePolicyDocument:
	Version: '2012-10-17'
	Statement:
	– Effect: Allow
	Principal:
	Service:
	– 'codepipeline.amazonaws.com'
	Action:
	– 'sts:AssumeRole'
	Path: /
	Policies:
	– PolicyName: !Join [ '-', [ !Ref 'AWS::StackName', !Ref ProjectName, 'CodePipeline-Policy' ] ]
	PolicyDocument:
	Version: '2012-10-17'
	Statement:
	– Effect: Allow
	Action:
	– iam:PassRole
	Resource: '*'
	– Effect: Allow
	Action:
	– codecommit:CancelUploadArchive
	– codecommit:GetBranch
	– codecommit:GetCommit
	– codecommit:GetRepository
	– codecommit:GetUploadArchiveStatus
	– codecommit:UploadArchive
	Resource: '*'
	– Effect: Allow
	Action:
	– codedeploy:CreateDeployment
	– codedeploy:GetApplication
	– codedeploy:GetApplicationRevision
	– codedeploy:GetDeployment
	– codedeploy:GetDeploymentConfig
	– codedeploy:RegisterApplicationRevision
	Resource: '*'
	– Effect: Allow
	Action:
	– codestar-connections:UseConnection
	Resource: '*'
	– Effect: Allow
	Action:
	– elasticbeanstalk:*
	– ec2:*
	– elasticloadbalancing:*
	– autoscaling:*
	– cloudwatch:*
	– s3:*
	– sns:*
	– cloudformation:*
	– rds:*
	– sqs:*
	– ecs:*
	Resource: '*'
	– Effect: Allow
	Action:
	– lambda:InvokeFunction
	– lambda:ListFunctions
	Resource: '*'
	– Effect: Allow
	Action:
	– opsworks:CreateDeployment
	– opsworks:DescribeApps
	– opsworks:DescribeCommands
	– opsworks:DescribeDeployments
	– opsworks:DescribeInstances
	– opsworks:DescribeStacks
	– opsworks:UpdateApp
	– opsworks:UpdateStack
	Resource: '*'
	– Effect: Allow
	Action:
	– cloudformation:CreateStack
	– cloudformation:DeleteStack
	– cloudformation:DescribeStacks
	– cloudformation:UpdateStack
	– cloudformation:CreateChangeSet
	– cloudformation:DeleteChangeSet
	– cloudformation:DescribeChangeSet
	– cloudformation:ExecuteChangeSet
	– cloudformation:SetStackPolicy
	– cloudformation:ValidateTemplate
	Resource: '*'
	– Effect: Allow
	Action:
	– codebuild:BatchGetBuilds
	– codebuild:StartBuild
	– codebuild:BatchGetBuildBatches
	– codebuild:StartBuildBatch
	Resource: '*'
	– Effect: Allow
	Action:
	– devicefarm:ListProjects
	– devicefarm:ListDevicePools
	– devicefarm:GetRun
	– devicefarm:GetUpload
	– devicefarm:CreateUpload
	– devicefarm:ScheduleRun
	Resource: '*'
	– Effect: Allow
	Action:
	– servicecatalog:ListProvisioningArtifacts
	– servicecatalog:CreateProvisioningArtifact
	– servicecatalog:DescribeProvisioningArtifact
	– servicecatalog:DeleteProvisioningArtifact
	– servicecatalog:UpdateProduct
	Resource: '*'
	– Effect: Allow
	Action:
	– cloudformation:ValidateTemplate
	Resource: '*'
	– Effect: Allow
	Action:
	– ecr:DescribeImages
	Resource: '*'
	– Effect: Allow
	Action:
	– states:DescribeExecution
	– states:DescribeStateMachine
	– states:StartExecution
	Resource: '*'
	– Effect: Allow
	Action:
	– appconfig:StartDeployment
	– appconfig:StopDeployment
	– appconfig:GetDeployment
	Resource: '*'

view raw blog-create-evergreen-golden-amis-template-resources-codepipelinerole.yaml hosted with ❤ by GitHub

The AWS CodeCommit repository will contain the buildspec.yaml file and all the scripts that will be used to create a golden AMI. Whenever these files change, this should trigger the creation of a new golden AMI.

This will be accomplished using an Amazon EventBridge rule (formally known as AWS CloudWatch rule). However, for this Amazon EventBridge rule to work, it needs to have appropriate permissions in place. The next resource definition creates an AWS IAM role that will be used by this Amazon EventBridge rule.

	# IAM role that will be used by the AWS EventBridge rule to start the AWS CodePipeline pipeline
	AmazonEventBridgeEventRole:
	Type: AWS::IAM::Role
	Properties:
	AssumeRolePolicyDocument:
	Version: 2012-10-17
	Statement:
	–
	Effect: Allow
	Principal:
	Service:
	– events.amazonaws.com
	Action: sts:AssumeRole
	Path: /
	Policies:
	–
	PolicyName: !Join [ '-', [ !Ref ProjectName, 'Pipeline-Execution-Policy' ] ]
	PolicyDocument:
	Version: 2012-10-17
	Statement:
	–
	Effect: Allow
	Action: codepipeline:StartPipelineExecution
	Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref EvergreenAMIPipeline ] ]

view raw blog-create-evergreen-golden-amis-template-resources-amazoneventbridgerole.yaml hosted with ❤ by GitHub

The next resource definition creates an AWS IAM role that will be used by the AWS CodeBuild project. These permissions are required to create the golden AMI.

	# IAM role that will be used by the AWS CodeBuild project
	AWSCodeBuildRole:
	Type: AWS::IAM::Role
	Properties:
	AssumeRolePolicyDocument:
	Version: 2012-10-17
	Statement:
	–
	Effect: Allow
	Principal:
	Service:
	– codebuild.amazonaws.com
	Action: sts:AssumeRole
	Path: /
	Policies:
	– PolicyName: !Join [ '-', [ !Ref 'AWS::StackName', !Ref ProjectName, 'CodeBuild-Policy' ] ]
	PolicyDocument:
	Version: '2012-10-17'
	Statement:
	– Effect: Allow
	Action:
	– logs:CreateLogGroup
	– logs:CreateLogStream
	– logs:PutLogEvents
	Resource:
	– !Join [ '', [ 'arn:aws:logs:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':log-group:/aws/codebuild/', !Ref CodeBuildProjectName ] ]
	– !Join [ '', [ 'arn:aws:logs:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':log-group:/aws/codebuild/', !Ref CodeBuildProjectName, ':*' ] ]
	– Effect: Allow
	Action:
	– s3:PutObject
	– s3:GetObject
	– s3:GetObjectVersion
	– s3:GetBucketAcl
	– s3:GetBucketLocation
	Resource:
	– !Join [ '', [ 'arn:aws:s3:::', !Ref CodePipelineArtifactStoreS3Bucket ] ]
	– !Join [ '', [ 'arn:aws:s3:::', !Ref CodePipelineArtifactStoreS3Bucket, '/*' ] ]
	– Effect: Allow
	Action:
	– codecommit:GitPull
	Resource: !GetAtt EvergreenAMICodeCommitRepo.Arn
	– Effect: Allow
	Action:
	– ssm:DescribeParameters
	Resource: '*'
	– Effect: Allow
	Action:
	– ssm:PutParameter
	– ssm:DeleteParameter
	– ssm:GetParameterHistory
	– ssm:GetParametersByPath
	– ssm:GetParameters
	– ssm:GetParameter
	– ssm:DeleteParameters
	Resource: !Join [ '', [ 'arn:aws:ssm:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':parameter/', !Ref ProjectName, '/*' ] ]
	– Effect: Allow
	Action:
	– codebuild:CreateReportGroup
	– codebuild:CreateReport
	– codebuild:UpdateReport
	– codebuild:BatchPutTestCases
	– codebuild:BatchPutCodeCoverages
	Resource: !Join [ '', [ 'arn:aws:codebuild:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':report-group/', !Ref CodeBuildProjectName, '-*' ] ]
	– Effect: Allow
	Action:
	– logs:CreateLogGroup
	– logs:CreateLogStream
	– logs:PutLogEvents
	Resource:
	– !Join [ '', [ 'arn:aws:logs:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':log-group:', !Ref CodeBuildCWLogGroupName ] ]
	– !Join [ '', [ 'arn:aws:logs:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':log-group:', !Ref CodeBuildCWLogGroupName, ':*' ] ]
	– PolicyName: !Join [ '-', [ !Ref 'AWS::StackName', !Ref ProjectName, 'CodeBuild-Packer-Policy' ] ]
	PolicyDocument:
	Version: '2012-10-17'
	Statement:
	– Effect: Allow
	Action:
	– ec2:AttachVolume
	– ec2:AuthorizeSecurityGroupIngress
	– ec2:CopyImage
	– ec2:CreateImage
	– ec2:CreateKeypair
	– ec2:CreateSecurityGroup
	– ec2:CreateSnapshot
	– ec2:CreateTags
	– ec2:CreateVolume
	– ec2:DeleteKeyPair
	– ec2:DeleteSecurityGroup
	– ec2:DeleteSnapshot
	– ec2:DeleteVolume
	– ec2:DeregisterImage
	– ec2:DescribeImageAttribute
	– ec2:DescribeImages
	– ec2:DescribeInstances
	– ec2:DescribeInstanceStatus
	– ec2:DescribeRegions
	– ec2:DescribeSecurityGroups
	– ec2:DescribeSnapshots
	– ec2:DescribeSubnets
	– ec2:DescribeTags
	– ec2:DescribeVolumes
	– ec2:DetachVolume
	– ec2:GetPasswordData
	– ec2:ModifyImageAttribute
	– ec2:ModifyInstanceAttribute
	– ec2:ModifySnapshotAttribute
	– ec2:RegisterImage
	– ec2:RunInstances
	– ec2:StopInstances
	– ec2:TerminateInstances
	Resource: '*'

view raw blog-create-evergreen-golden-amis-template-resources-awscodebuildrole.yaml hosted with ❤ by GitHub

This brings us to the last AWS IAM role that is defined in the AWS SAM template. This AWS IAM role will grant the Amazon EventBridge (Amazon CloudWatch) service permissions to publish messages to the Amazon Simple Notification Topic (SNS). The Amazon EventBridge rules will publish notifications in regards to the status of the AWS CodeBuild project, for example, when it starts, with it completes successfully or when it fails.

	# IAM role that will be attached to the AWS SNS Topic. This will allow AWS EventBridge events to publish messages to it
	EvergreenAMISNSTopicPolicy:
	Type: AWS::SNS::TopicPolicy
	Properties:
	PolicyDocument:
	Version: '2012-10-17'
	Statement:
	– Sid: __default_statement_ID
	Effect: Allow
	Principal:
	"AWS": "*"
	Action:
	– sns:GetTopicAttributes
	– sns:SetTopicAttributes
	– sns:AddPermission
	– sns:RemovePermission
	– sns:DeleteTopic
	– sns:Subscribe
	– sns:ListSubscriptionsByTopic
	– sns:Publish
	Resource: !Ref EvergreenAMISNSTopic
	Condition:
	StringEquals:
	"AWS:SourceOwner": !Ref 'AWS::AccountId'
	– Sid: Allow_CloudWatchEvents_To_Publish
	Effect: Allow
	Action:
	– sns:Publish
	Principal:
	"Service": "events.amazonaws.com"
	Resource: !Ref EvergreenAMISNSTopic
	Topics:
	– !Ref EvergreenAMISNSTopic

view raw blog-create-evergreen-golden-amis-template-resources-amazonsnstopicpolicy.yaml hosted with ❤ by GitHub

To create the golden AMI, I am using Packer by HashiCorp. This is an awesome tool, feature rich and easy to use.

However, there is one thing that I don’t like about it. To customise the base AMI, it creates a temporary Amazon EC2 instance using it, and then runs the customisations, as specified in a template. Out of the box, it attaches a security group that allows any public IP address to connect to its SSH port (from 0.0.0.0/0). I must admit, it is not too bad, since even if you could find the IP address of this temporary Amazon EC2 instance and connect to it, you would still need the SSH keys that only Packer has.

I have worked at a number of Enterprises and I can tell you that, even though it doesn’t sound too bad, you will still raise the eyebrows of the security folks, if you were to propose attaching a security group that allows 0.0.0.0/0 as its source.

To mitigate this issue, the next resource definition will create a security group that only allows SSH from the AWS CodeBuild service running in Sydney (ap-southeast-2) region. This security group will be passed onto Packer, and it will attach this security group, instead of its overly permissive security group, to the temporary Amazon EC2 instance.

	EvergreenAMIPackerSecurityGroup:
	Type: AWS::EC2::SecurityGroup
	Properties:
	GroupDescription: Security Group used by Packer to connect to the temporary Amazon EC2 instance
	GroupName: !Join [ '-', [ !Ref ProjectName, 'packer-sg' ] ]
	SecurityGroupEgress:
	– CidrIp: 0.0.0.0/0
	Description: Allow all outbound traffic
	FromPort: -1
	IpProtocol: -1
	ToPort: -1
	SecurityGroupIngress:
	– CidrIp: 3.26.127.24/29
	Description: Allow inbound from AWS CodeBuild Servers
	FromPort: 22
	IpProtocol: tcp
	ToPort: 22
	– CidrIp: 13.55.255.216/29
	Description: Allow inbound from AWS CodeBuild Servers
	FromPort: 22
	IpProtocol: tcp
	ToPort: 22
	VpcId: !Ref VPCId
	Tags:
	– Key: Name
	Value: !Join [ '-', [ !Ref ProjectName, 'packer-sg' ] ]

view raw blog-create-evergreen-golden-amis-template-resources-packersecuritygroup.yaml hosted with ❤ by GitHub

If you have configured your AWS CodeBuild project in another region, you will need to update the CidrIp addresses for the two ingress rules above to match the AWS CodeBuild service IP addresses from that region. You can find the IP addresses of all AWS services at https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html.

The next resource definition creates an AWS Systems Manager Parameter Store Parameter. This will be used to store the base AMIs ami id. The resource definition below sets the value to the ami id of the latest Amazon Linux 2 AMI available at the time of writing this blog. In the Implementation section, I will guide you into changing this value to the ami id of the latest Amazon Linux 2 that is available at your time of deployment.

	EvergreenAMIBaseAMIParameterStore:
	Type: AWS::SSM::Parameter
	Properties:
	Description: !Ref BaseAmiSSMParameterDesc
	Name: !Ref BaseAmiSSMParameterName
	Tier: Standard
	Type: String
	Value: ami-0c641f2290e9cd048

view raw blog-create-evergreen-golden-amis-template-resources-awsssmparameterbaseami.yaml hosted with ❤ by GitHub

As previously mentioned, an AWS CodeCommit repository will be used to host the buildspec.yaml and all the templates and scripts necessary to create the golden AMI. The next resource definition will create an empty AWS CodeCommit repository. In the Implementation section, I will provide instructions on how to populate the above mentioned files into it.

	# this is the AWS CodeCommit repository that will be created
	EvergreenAMICodeCommitRepo:
	Type: AWS::CodeCommit::Repository
	Properties:
	RepositoryName: !Ref CodeCommitRepoName
	RepositoryDescription: This is a repository for the Evergreen AMI project.

view raw blog-create-evergreen-golden-amis-template-resources-awscodecommitrepo.yaml hosted with ❤ by GitHub

The next resource definition will create an AWS CodeBuild project that will be used in our AWS CodePipeline pipeline to create the golden AMI.

	# this is the AWS CodeBuild project
	EvergreenAMICodeBuildProject:
	Type: AWS::CodeBuild::Project
	Properties:
	Artifacts:
	Type: NO_ARTIFACTS
	BadgeEnabled: false
	Description: AWS CodeBuild project to build artefacts for Evergreen AMI project.
	Environment:
	ComputeType: BUILD_GENERAL1_SMALL
	Image: aws/codebuild/amazonlinux2-x86_64-standard:4.0
	PrivilegedMode: true
	Type: LINUX_CONTAINER
	LogsConfig:
	CloudWatchLogs:
	GroupName: !Ref CodeBuildCWLogGroupName
	Status: ENABLED
	StreamName: !Ref CodeBuildCWLogStreamName
	Name: !Ref CodeBuildProjectName
	ResourceAccessRole: !GetAtt AWSCodeBuildRole.Arn
	ServiceRole: !GetAtt AWSCodeBuildRole.Arn
	Source:
	GitCloneDepth: 1
	Location: !Join [ '', [ 'https://git-codecommit.', !Ref 'AWS::Region', '.amazonaws.com/v1/repos/' , !Ref CodeCommitRepoName ] ]
	Type: CODECOMMIT
	TimeoutInMinutes: 60
	Visibility: PRIVATE

view raw blog-create-evergreen-golden-amis-template-resources-awscodebuildproject.yaml hosted with ❤ by GitHub

The next resource definition is the heart and brain of the entire solution. It will provision our AWS CodePipeline pipeline. This pipeline will retrieve the artifacts from the AWS CodeCommit repository and then pass it to the AWS CodeBuild project. It will orchestrate the entire golden AMI creation process.

	# this is the AWS CodePipeline pipeline
	EvergreenAMIPipeline:
	Type: AWS::CodePipeline::Pipeline
	Properties:
	RoleArn: !GetAtt CodePipelineRole.Arn
	Name: !Ref CodePipelinePipelineName
	RestartExecutionOnUpdate: False
	ArtifactStore:
	Type: S3
	Location: !Ref CodePipelineArtifactStoreS3Bucket
	Stages:
	–
	Name: Source
	Actions:
	–
	Name: Source
	ActionTypeId:
	Category: Source
	Owner: AWS
	Provider: CodeCommit
	Version: 1
	InputArtifacts: []
	OutputArtifacts:
	– Name: !Join [ '-', [ !Ref 'AWS::StackName', 'SourceArtifact' ] ]
	Namespace: SourceVariables
	Configuration:
	RepositoryName: !GetAtt EvergreenAMICodeCommitRepo.Name
	BranchName: !Ref CodeCommitBranchName
	PollForSourceChanges: 'false'
	RunOrder: 1
	–
	Name: Build
	Actions:
	–
	Name: Build
	ActionTypeId:
	Category: Build
	Owner: AWS
	Provider: CodeBuild
	Version: 1
	Configuration:
	BatchEnabled: False
	CombineArtifacts: False
	ProjectName: !Ref CodeBuildProjectName
	EnvironmentVariables:
	!Join
	– ''
	– – '[{"name": "AWS_AMI_NAME_PREFIX",'
	– ' "value": "'
	– !Ref AmiNamePrefix
	– '", "type": "PLAINTEXT" }, '
	– '{"name": "AWS_BASE_AMI_ID", '
	– '"value": "'
	– !Ref BaseAmiSSMParameterName
	– '", "type": "PARAMETER_STORE"}, '
	– '{"name": "AWS_BASE_AMI_SSM_PARAM_NAME", '
	– '"value": "'
	– !Ref BaseAmiSSMParameterName
	– '", "type": "PLAINTEXT"}, '
	– '{"name": "AWS_BASE_AMI_SSM_PARAM_DESC", '
	– '"value": "'
	– !Ref BaseAmiSSMParameterDesc
	– '", "type": "PLAINTEXT"}, '
	– '{"name": "AWS_GOLDEN_AMI_SSM_PARAM_NAME", '
	– '"value": "'
	– !Ref GoldenAmiSSMParameterName
	– '", "type": "PLAINTEXT"}, '
	– '{"name": "AWS_GOLDEN_AMI_SSM_PARAM_DESC", '
	– '"value": "'
	– !Ref GoldenAmiSSMParameterDesc
	– '", "type": "PLAINTEXT"}, '
	– '{"name": "PACKER_TEMPLATE_FILENAME", '
	– '"value": "'
	– !Ref PackerTemplateFilename
	– '", "type": "PLAINTEXT"}, '
	– '{"name": "PACKER_SECURITY_GROUP_ID", '
	– '"value": "'
	– !GetAtt EvergreenAMIPackerSecurityGroup.GroupId
	– '" , "type": "PLAINTEXT"}]'
	InputArtifacts:
	– Name: !Join [ '-', [ !Ref 'AWS::StackName', 'SourceArtifact' ] ]
	OutputArtifacts:
	– Name: !Join [ '-', [ !Ref 'AWS::StackName', 'BuildArtifact' ] ]
	RunOrder: 2

view raw blog-create-evergreen-golden-amis-template-resources-awscodepipelineyaml hosted with ❤ by GitHub

When the AWS CodePipeine pipeline triggers the AWS CodeBuild project, it also passes along the following environment variables to it. These are required for creating the golden AMI.

Below is the list of these environment variables that are passed.

• AWS_AMI_NAME_PREFIX – the prefix to use when naming the new golden ami.
• AWS_BASE_AMI_ID – the ami id of the base ami id to use when creating the golden ami.
• AWS_BASE_AMI_SSM_PARAM_NAME – the AWS SSM Parameter Store Parameter name where the base ami’s ami id is stored.
• AWS_BASE_AMI_SSM_PARAM_DESC – the description for the AWS SSM Parameter Store Parameter that contains the ami id of the base ami. This is used when the value of the Parameter is reverted to its previous value when an AWS CodeBuild project’s build fails.
• AWS_GOLDEN_AMI_SSM_PARAM_NAME – the AWS SSM Parameter Store Parameter’s name where the newly created golden AMIs ami id should be stored.
• AWS_GOLDEN_AMI_SSM_PARAM_DESC – the description to use for the AWS SSM Parameter Store Parameter that will contain the ami id of the newly created golden AMI.
• PACKER_SECURITY_GROUP_ID – the security group that packer will use for the temporary Amazon EC2 instance when creating the golden AMI (this is the restrictive security group that was created above)

As mentioned previously, this solution will check if a new Amazon Linux 2 AMI has been released by AWS. If a new AMI was recently released by AWS, this solution will automatically create a new golden AMI using this new base AMI.

This solution will use an AWS Lambda function to find out if a new Amazon Linux 2 AMI has been released. As you already are aware, the ami id of the latest known Amazon Linux 2 AMI will be stored in an AWS SSM Parameter Store Parameter. When the AWS Lambda function is invoked, it will query for the latest released Amazon Linux 2 AMI, and compare its ami id with what is stored in the AWS SSM Parameter Store Parameter. If these don’t match, this means since the last time the AWS Lambda function was invoked, AWS has released a new Amazon Linux 2 AMI. When this happens, the AWS Lambda function will update the AWS SSM Parameter Store Parameter that stores the base AMI’s ami id with the ami id of the latest Amazon Linux 2 AMI. It will then trigger the AWS CodePipeline pipeline, which will generate a new golden AMI using the new base AMI.

By default, this AWS Lambda will run daily at 18:00 UTC (04:00 UTC+10).

The next resource definition is for the above mentioned AWS Lambda function.

	DetectNewBaseAMIFunction:
	Type: AWS::Serverless::Function
	Properties:
	Handler: src/check_if_new_base_ami_released.lambda_handler
	Runtime: python3.7
	Timeout: 300
	MemorySize: 128
	Events:
	CloudWatchEventsSchedule:
	Type: Schedule
	Properties:
	Schedule: 'cron(0 18 * * ? *)'
	Name: !Join ['-', [ 'Check-if-new-base-ami-released-', !Ref 'AWS::StackName' ]]
	Description: Check if a new base ami matching our filter pattern has been released by AWS
	Enabled: True
	Environment:
	Variables:
	REGION: !Ref 'AWS::Region'
	SSM_PARAMETER_NAME_BASE_AMI_ID: !Ref BaseAmiSSMParameterName
	CODEPIPELINE_PIPELINE_NAME: !Ref CodePipelinePipelineName
	Policies:
	– Statement:
	– Sid: ReadUpdateSSMParameterStoreValues
	Effect: Allow
	Action:
	– ssm:PutParameter
	– ssm:DeleteParameter
	– ssm:GetParameterHistory
	– ssm:GetParametersByPath
	– ssm:GetParameters
	– ssm:GetParameter
	– ssm:DeleteParameters
	Resource: !Join [ '', [ 'arn:aws:ssm:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':parameter/', !Ref ProjectName, '/*' ] ]
	– Sid: StartCodePipelinePipelineExecution
	Effect: Allow
	Action:
	– codepipeline:StartPipelineExecution
	Resource: !Join ['', ['arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref CodePipelinePipelineName ]]
	– Sid: CheckLatestAWSAmis
	Effect: Allow
	Action:
	– ec2:DescribeImages
	Resource: '*'

view raw blog-create-evergreen-golden-amis-template-resources-awslambda-detectnewbaseami.yaml hosted with ❤ by GitHub

The next resource definition will create an Amazon Simple Notification Service (SNS) Topic. This will be used to send notifications regarding the status of the AWS CodeBuild project. By default, the email address that is provided in the Makefile (EMAIL_FOR_NOTIFICATIONS) will be subscribed to this Amazon SNS Topic.

Don’t forget to click on the approval link in the confirmation email that AWS will send to this email address. Notifications will not be sent to that email address until that link is clicked to approve Amazon SNS to send messages to it.

	EvergreenAMISNSTopic:
	Type: AWS::SNS::Topic
	Properties:
	DisplayName: !Join [ '-', [ !Ref ProjectName, 'notifications' ] ]
	Subscription:
	– Protocol: email
	Endpoint: !Ref EmailForNotifications
	Tags:
	– Key: Name
	Value: !Join [ '-', [ !Ref ProjectName, 'notifications' ] ]
	TopicName: !Join [ '-', [ !Ref ProjectName, 'notifications' ] ]

view raw blog-create-evergreen-golden-amis-template-resources-amazonsnstopic.yaml hosted with ❤ by GitHub

The next resource definition will create an Amazon EventBridge rule. This rule will trigger the AWS CodePipeline pipeline whenever a commit is pushed to the AWS CodeCommit repository, in the branch name referred to by the variable CODECOMMIT_BRANCH_NAME (this variable is defined in the Makefile and defaults to main). This rule ensures that a new golden AMI is created whenever there is a change to the buildspec.yaml or any of the files that are used to create the golden AMI.

	EventBridgeRuleToTriggerCodePipeline:
	Type: AWS::Events::Rule
	Properties:
	Description: This Amazon EventBridge rule will automatically trigger the Evergreen AMI Codepipeline pipline when changes are detected in the Evergreen CodeCommit repository in the monitored branch
	EventBusName: default
	EventPattern:
	source:
	– aws.codecommit
	detail-type:
	– CodeCommit Repository State Change
	resources:
	– !GetAtt EvergreenAMICodeCommitRepo.Arn
	detail:
	event:
	– referenceCreated
	– referenceUpdated
	referenceType:
	– branch
	referenceName:
	– !Ref CodeCommitBranchName
	Name: !Join [ '-', [ !Ref ProjectName, 'Pipeline-Trigger' ] ]
	State: ENABLED
	Targets:
	–
	Arn:
	!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref EvergreenAMIPipeline ] ]
	RoleArn: !GetAtt AmazonEventBridgeEventRole.Arn
	Id: !Join [ '-', [ !Ref ProjectName, 'Pipeline-Trigger' ] ]

view raw blog-create-evergreen-golden-amis-template-resources-amazoneventbridgerule-trigger-pipeline.yaml hosted with ❤ by GitHub

The next resource definition creates another Amazon EventBridge rule. This rule will detect when the AWS CodeBuild project finishes successfully, and it will send a notification to the AWS SNS Topic.

	EventBridgeRuleForCodeBuildSuccess:
	Type: AWS::Events::Rule
	Properties:
	Description: This Amazon EventBridge rule will monitor and send notifications for successful CodeBuild project runs
	EventBusName: default
	EventPattern:
	source:
	– aws.codebuild
	detail-type:
	– CodeBuild Build State Change
	detail:
	project-name:
	– !Ref CodeBuildProjectName
	build-status:
	– SUCCEEDED
	Name: !Join [ '-', [ !Ref ProjectName, 'CodeBuild-Project-Success-Notifications' ] ]
	State: ENABLED
	Targets:
	–
	Arn: !Ref EvergreenAMISNSTopic
	InputTransformer:
	InputPathsMap:
	"build-id": "$.detail.build-id"
	"project-name": "$.detail.project-name"
	"build-status": "$.detail.build-status"
	InputTemplate: |
	"AWS CodeBuild Project '<project-name>' with Build Id '<build-id>' has completed successfully. Build Status: '<build-status>'."
	Id: !Join [ '-', [ !Ref ProjectName, 'CodeBuild-Project-Success-Notifications' ] ]

view raw blog-create-evergreen-golden-amis-template-resources-amazoneventbridgerule-detect-successful-codebuild.yaml hosted with ❤ by GitHub

The last resource definition creates another Amazon EventBridge rule. This rule detects when the AWS CodeBuild project either starts, stops or fails. When this happens, it will send a notification to the AWS SNS Topic.

	EventBridgeRuleForCodeBuildNonSuccess:
	Type: AWS::Events::Rule
	Properties:
	Description: This Amazon EventBridge rule will monitor and send notifications for all Evergreen AMI CodeBuild project state changes except for SUCCEEDED.
	EventBusName: default
	EventPattern:
	source:
	– aws.codebuild
	detail-type:
	– CodeBuild Build State Change
	detail:
	project-name:
	– !Ref CodeBuildProjectName
	build-status:
	– IN_PROGRESS
	– STOPPED
	– FAILED
	Name: !Join [ '-', [ !Ref ProjectName, 'CodeBuild-Project-NonSuccess-Notifications' ] ]
	State: ENABLED
	Targets:
	–
	Arn: !Ref EvergreenAMISNSTopic
	InputTransformer:
	InputPathsMap:
	"build-id": "$.detail.build-id"
	"project-name": "$.detail.project-name"
	"build-status": "$.detail.build-status"
	InputTemplate: |
	"Status for AWS CodeBuild Project '<project-name>' has changed. Build '<build-id>' has reached the Build Status of '<build-status>'."
	Id: !Join [ '-', [ !Ref ProjectName, 'CodeBuild-Project-NonSuccess-Notifications' ] ]

view raw blog-create-evergreen-golden-amis-template-resources-amazoneventbridgerule-detect-failed-codebuild.yaml hosted with ❤ by GitHub

The last section of this AWS SAM template lists all the outputs that will be shown after the provisioning has finished. As you would have noticed, there are quite a lot of calculated variable names that are being used in this template, for example the name of the AWS CodeCommit Repository, the AWS CodeBuild project name. The outputs section will display all these values, so that they can used to find the appropriate resources. After deployment, you can view the information from the Outputs in the Outputs tab of the AWS CloudFormation stack that AWS SAM creates. You can find the name of this AWS CloudFormation stack in the Makefile. The format of the name is sam-${PROJECT_NAME} where PROJECT_NAME is what you had provided in the Makefile.

Below is a snippet of the Outputs section.

	Outputs:
	BaseAmiSSMParameterName:
	Description: The Base AMI SSM Parameter Store Parameter Name.
	Value: !Ref BaseAmiSSMParameterName
	GoldenAmiSSMParameterName:
	Description: The Golden AMI SSM Parameter Store Parameter Name.
	Value: !Ref GoldenAmiSSMParameterName
	CodeCommitRepoName:
	Description: The name of the AWS CodeCommit Repository.
	Value: !Ref CodeCommitRepoName
	CodeCommitBranchName:
	Description: The name of the AWS CodeCommit Repository branch where the code is to be committed.
	Value: !Ref CodeCommitBranchName
	CodeBuildProjectName:
	Description: The name of the AWS CodeBuild Project.
	Value: !Ref CodeBuildProjectName
	CodeBuildCWLogGroupName:
	Description: The Amazon CloudWatch Logs Group that will be used by the AWS CodeBuild Project for loggging purposes.
	Value: !Ref CodeBuildProjectName
	CodeBuildCWLogStreamName:
	Description: The Amazon CloudWatch Logs Group Stream name that will be used by the AWS CodeBuild Project for loggging purposes.
	Value: !Ref CodeBuildProjectName
	CodePipelinePipelineName:
	Description: The name of the AWS CodePipeline pipeline.
	Value: !Ref CodeBuildProjectName

view raw blog-create-evergreen-golden-amis-template-outputs.yaml hosted with ❤ by GitHub

Implementation

Now that we have gone through the code, lets proceed to deploying this solution into your AWS Account.

1. Open the file named Makefile in your favorite IDE.
2. Update the following variables (found at the top of the file)
   • aws_profile – this should be set to the AWS profile that you have configured locally to provision resources into your AWS Account
   • aws_s3_bucket – this Amazon S3 bucket will be used by AWS SAM to store artefacts for the AWS CloudFormation stack that it will create. This Amazon S3 bucket must exist.
   • PROJECT_NAME – give a meaningful name for your project. Do not include any spaces. This will be used to prefix the names of the resources that are created. This will ensure that these resources have a unique name, should you do multiple deployments of this solution in the same AWS Account.
   • EMAIL_FOR_NOTIFICATIONS – provide an email address to which notifications regarding the status of the AWS CodeBuild project will be sent to. Ensure that you have access to this email address’s inbox since AWS will send a confirmation email containing a link that needs to be clicked on.
   • VPCID – this is the id of the Amazon Virtual Private Cloud (AWS VPC) where Packer will create the temporary Amazon EC2 instance, from which the golden AMI will be created. The default subnet in this AWS VPC must be a public subnet. Packer will create the temporary Amazon EC2 instance in this default subnet and the AWS CodeBuild servers will attempt to connect to it via SSH. For simplicity, you can use the default VPC. If you use a non default VPC, update the packer template file to add subnet_id and provide the subnet id of your public subnet in this non default VPC. You can read more about it here https://www.packer.io/plugins/builders/amazon/ebs#subnet_id.
   • CODEPIPELINE_ARTIFACTSTORE_S3_BUCKET – this is the Amazon S3 bucket where the AWS CodePipeine pipeline will store its artifacts. Use a different Amazon S3 bucket than what was provided for aws_s3_bucket above.
   • Leave the values for the rest of the variables as they are.
3. Run the following command. AWS SAM will generate package.yaml and then upload it to the Amazon S3 bucket specified in aws_s3_bucket variable in the Makefile.make package
4. Next, run the following command to deploy the solution into your AWS Account. It will use the AWS profile specified in the aws_profile variable in the Makefile.make deployAWS SAM will show the change set for what will be deployed. You will be prompted with a Y/N to continue. Type Y and press enter to continue. You will be able to see the progress of the deployment from AWS SAM. You can also track the progress via the AWS CloudFormation stack that AWS SAM creates (the name of the AWS CloudFormation stack is contained in the variable aws_stack_name , as defined in the Makefile).
5. After the deployment is successful, we need to update the value of the AWS SSM Parameter Store Parameter that contains the ami id of the base AMI that will be used to create the golden AMI. By default, it contains the ami id of the latest Amazon Linux 2 AMI that was available at the time this blog was written.
   • Using the AWS Management Console, open the EC2 dashboard. From the left-hand side menu, under Images click on AMI Catalog. Then, in the search box, enter Amazon Linux 2 and press Enter.
   • From the search results, select the AMI with the prefix Amazon Linux 2 AMI (HVM) – Kernel and which has the highest kernel number. Copy the x86 ami id of this. For example, at the time of writing this blog, the latest Amazon Linux 2 HVM AMI name is Amazon Linux 2 (HVM) – Kernel 5.10, SSD Volume Type and its x86 ami-id is ami-0c641f2290e9cd048.
   • If the ami id of the latest Amazon Linux 2 HVM AMI that you found is different from ami-0c641f2290e9cd048 then continue on to update the AWS SSM Parameter Store Parameter value. Otherwise, skip to step 6.
   • Open the AWS Systems Manager Portal and from the left-hand side menu, under Application Management click Parameter Store.
   • Under My parameters locate the Parameter name that corresponds to that for your base ami. This name was part of the AWS SAM outputs (the format of its name is /$PROJECT_NAME/base_ami_id). Once you have located it, click on it.
   • On the top right-hand side of the screen, click Edit and then replace the contents under Value with the ami id that you have for the latest Amazon Linux 2 HVM AMI. Press Save changes.
6. Next, we have to copy the files required by the AWS CodeBuild project to the AWS CodeCommit repository. However, before we proceed, we need to take care of a few prerequisites. Do not proceed until the following have been completed.
   • Setup an IAM user with access to the newly created AWS CodeCommit repository. Use HTTPS as the protocol for connecting to it. The instructions for getting this done are available at https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html.
   • Clone your newly created AWS CodeCommit repository to your local computer. For authentication, use the IAM user that you provided access to in step A above. Use the instructions provided at https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-connect.html to clone the AWS CodeCommit repository. You will get a message saying that you are cloning an empty repository. – that is ok. The name of your AWS CodeCommit repository is shown in the AWS SAM outputs. The name has the format ${PROJECT_NAME}Repo.
   • Use the following command to create a new branch for your AWS CodeCommit repository. The branchname was specified in the Makefile and defaults to main.
     • git checkout -b <branchname>
   • From the GitHub repository that you had previously cloned the code for this solution, copy the contents of the folder CodeBuildFiles to the root of where you cloned your AWS CodeCommit repository. DO NOT COPY THE FOLDER CodeBuildFiles. Instead just copy its contents, including all the subfolders inside it. You should have the buildspec.yml file AND packer_files folder at the root of your AWS CodeCommit repository folder.
   • Next, use the following command to add all the files to the staging area.
     • git add *
   • After that, use the following commands to commit the files and push them to your AWS CodeCommit repository.
     • git commit -m "initial files for AWS CodeBuild project"
     • git push --set-upstream origin <branchname>
   • The above command will create a new branch in your AWS CodeCommit repository that will have the same name as the value of the variable branchname (as defined in the Makefile) with the files that were inside the folder CodeBuildFiles. You can confirm this by logging onto the AWS Management Console and opening the AWS CodeCommit Portal.
7. After the push to your AWS CodeCommit repository, within a few minutes your AWS CodePipeline pipeline will get triggered automatically by the Amazon EventBridge rule. You should receive an email when the AWS CodeBuild project starts, stating that the AWS CodeBuild project is in state IN_PROGRESS. You can monitor the progress of the AWS CodePipeline pipeline and the AWS CodeBuild project by logging onto the AWS Management Console and accessing the AWS CodePipeline and AWS CodeBuild Portal. The name of the AWS CodePipeline pipeline is included in the AWS SAM outputs. The name has the format ${PROJECT_NAME}_Pipeline.
8. Once the AWS CodeBuild project has finished, you will receive an email with the status of the build. If the build was successful, login to the AWS Management Console and then open the AWS SSM Portal and then go to the Parameter Stores screen. Under My parameters you should now see a new Parameter. The name of this Parameter will be of the format /{PROJECT_NAME}/latest_golden_ami_id (the name is part of the AWS SAM outputs). This Parameter will contain the ami id of the newly created golden AMI.
9. If the AWS CodeBuild project build had failed, login to the AWS Management Console and then open the AWS CodeBuild Portal. Use the project build history logs to troubleshoot the issue.

Some more information

You might be wondering what customisations are being done to the base AMI and how you could modify it to add your own customisations to it.

The customisations are included in the file called customise_and_install_packages.sh. This file is in the GitHub repository, inside the CodeBuildFiles/packer_files/scripts folder.

Below is the content of the above file.

	#!/bin/bash
	set -ex
	sudo yum update -y
	sudo /usr/sbin/update-motd –disable
	echo 'No unauthorized access permitted' | sudo tee /etc/motd
	sudo rm /etc/issue
	sudo ln -s /etc/motd /etc/issue
	sudo yum install -y elinks screen
	sudo yum install git -y

view raw blog-create-evergreen-golden-amis-customise_and_install_packages.sh hosted with ❤ by GitHub

As part of the implementation, you might remember that the contents of the above folder, CodeCommitFiles had been committed to your AWS CodeCommit repository. This is what your AWS CodeBuild project uses to customise the golden AMI. To locate this file in your own AWS CodeBuild repository, look at the path packer_files/scripts/customise_and_install_packages.sh in the branch referred to by the branchName variable (as defined in the Makefile).

To add your customisations to your golden AMI, update the file and then push it to your AWS CodeCommit repository in the branchName branch.This will trigger the AWS CodePipeline pipeline, which will use the AWS CodeBuild project to create a new golden AMI using the customisations that you provided.

Summary

My philosophy in life is to automate processes as much as possible. This ensures they are repeatable, auditable and free from any human errors.

The automation described in this blog will ensure that your golden AMI is always kept up-to-date, since it will always be using the latest version of the base AMI from AWS. You will also be informed when a new golden AMI is available since it will send a notification of the build process. You can then adjust your deployment pipelines to consume this new golden AMI.

In the next blog, I will show you how to extend the current solution, so that the golden AMI is automatically tested after being built. This will ensure that there are no surprises when you use it in your deployment pipelines.

I hope you found this blog beneficial and till the next time, stay safe.