How to remove a resource before creating it with the CDK

Resource creation failures

In the previous article we looked into a case where a resource was created implicitly, preventing the CDK from creating it. In that case, that was a Log Group that AppSync created, but the same problem pops up in several places: an IoT Core domain configuration can not be created when one already exists for the domain, for example. Whenever a resource has to be unique, CloudFormation's behavior, the service that the CDK uses under the hood, will lead to undeployable stacks.

Related

The problems with implicit AWS resources

When a simple change leads to broken deployments

In this article, we'll look into the Log Group example we introduced in the previous article: the resource is already created by AppSync and adding it to the CDK stack fails the deployment.

Example baseline

The stack defines an AppSync API:

const api = new aws_appsync.GraphqlApi(this, "Api", {
	name: "test-api",
	definition: {
		schema: aws_appsync.SchemaFile.fromAsset(path.join(__dirname, "schema.graphql")),
	},
	authorizationConfig: {
		defaultAuthorization: {
			authorizationType: aws_appsync.AuthorizationType.IAM,
		}
	},
	logConfig: {
		fieldLogLevel: "ALL",
	},
});

This automatically configures an IAM role that allows it to create a Log Group and then put its messages there. As a result, the first invocation will create resources in the AWS account.

The name of the Log Group is fixed for an AppSync API: it is always /aws/appsync/apis/<api id>. Because of this, you can think of it as a resource that has to be unique: the CDK can't create a new Log Group for this API without first deleting the existsing one.

This is demonstrated by an error when the Log Group resource is added to the stack:

const logs = new aws_logs.LogGroup(this, "AppSyncLogGroup", {
	logGroupName: `/aws/appsync/apis/${api.apiId}`,
	retention: aws_logs.RetentionDays.TWO_WEEKS,
	removalPolicy: RemovalPolicy.DESTROY,
});

Deployment fails:

DeleterCustomResourceStack: deploying... [1/1]
DeleterCustomResourceStack: creating CloudFormation changeset...
[··························································] (0/4)

10:29:21 AM | CREATE_FAILED        | AWS::Logs::LogGroup         | AppSyncLogGroup
Resource handler returned message: "Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/aws/appsync/apis/veope6joxnfodacgbgusg7ti7y"}' alr
eady exists." (RequestToken: 0a680121-5b94-d3e4-71b3-08e2f04ff836, HandlerErrorCode: AlreadyExists)
10:29:21 AM | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack  | DeleterCustomResourceStack
The following resource(s) failed to create: [AppSyncLogGroup25FD6293].
10:29:21 AM | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack  | DeleterCustomResourceStack
The following resource(s) failed to create: [AppSyncLogGroup25FD6293].

An obvious solution here is to go to the AWS console and delete the resource before deploying the stack. After all, missing some logs is usually not a big problem. But there are two problems with this approach.

First, in the case of logging, AppSync will recreate the Log Group whenever a new message arrives. This can be overcome by removing its logs:CreateLogGroup permission.

But the second problem is that it does not work in other situations. For example, if you have a custom domain configured that was added outside the CDK then removing that would render the API unavailable. Because of this, you can't do it in advance, but has to be done as part of the deployment. And a manual action defeats the point of IaC.

Deleter custom resource

CloudFormation supports a mechanism to fill in the gaps with custom code: custom resources. This is a Lambda function that already exists or is deployed by the same stack and will be called during the deployment. This offers enormous flexibility: since the Lambda code can be anything, a custom resource can potentially touch any part of the AWS account and even outside.

The idea is simple: deploy a custom resource that deletes the Log Group when it is created. By adding a dependency to it from the Log Group resource, the CDK deployment will automatically clear the group before creating it again.

While the two things happen in the scope of a single deployment, there is a race condition here: the AppSync API can recreate the resource on its own. Because of this, it's useful to configure it without this permission:

const logsRole = new aws_iam.Role(this, "LogsRole", {
	assumedBy: new aws_iam.ServicePrincipal("appsync.amazonaws.com"),
});
logsRole.addToPolicy(new aws_iam.PolicyStatement({
	effect: aws_iam.Effect.ALLOW,
	resources: ["arn:aws:logs:*:*:*"],
	actions: [
		// no "logs:CreateLogGroup"
		"logs:CreateLogStream",
		"logs:PutLogEvents",
	],
}));
*/

const api = new aws_appsync.GraphqlApi(this, "Api", {
	// ...
	logConfig: {
		role: logsRole,
		fieldLogLevel: "ALL",
	},
});

Then the custom resource is a simple call to the AWS SDK:

const logGroupRemover = new custom_resources.AwsCustomResource(this, 'AssociateVPCWithHostedZone', {
	onCreate: {
		service: "@aws-sdk/client-cloudwatch-logs",
		action: "DeleteLogGroupCommand",
		parameters: {
			logGroupName: `/aws/appsync/apis/${api.apiId}`,
		},
		physicalResourceId: custom_resources.PhysicalResourceId.of(`/aws/appsync/apis/${api.apiId}`),
		ignoreErrorCodesMatching: "ResourceNotFoundException",
	},
	policy: custom_resources.AwsCustomResourcePolicy.fromSdkCalls({
		resources: custom_resources.AwsCustomResourcePolicy.ANY_RESOURCE,
	}),
});

The above code takes advantage of CDK's wrapper construct that makes it simpler to write custom resources without all the boilerplate. But there is nothing preventing writing a proper Lambda function that does more complex cleanup procedure. For example, to clean up an AppSync custom domain, you'll need a series of operations: delete the domain configuration, delete the ACM certificate, then remove the CNAME record from the domain.

To make sure that the custom resource is deployed before the Log Group, add a dependency:

const logs = new aws_logs.LogGroup(this, "AppSyncLogGroup", {
	logGroupName: `/aws/appsync/apis/${api.apiId}`,
	retention: aws_logs.RetentionDays.TWO_WEEKS,
	removalPolicy: RemovalPolicy.DESTROY,
});
logs.node.addDependency(logGroupRemover);

With this, the Log Group can be moved to be managed by the CDK.