CloudFront Updates Are No Longer Soul Destroying

2020-03-16 Much less idle cloud gazing

I’ve mentioned CloudFront’s speed, or lack thereof, several times in my past posts.

In my post on converting Lambda@Edge from JavaScript to Python I wrote:

…the painful slowness of CloudFront updates.
…
This was a slow exercise.

In my post on cfn-lint I wrote:

Creating and deleting resources can take anywhere from a few seconds (such as SNS topics) to a soul-sucking 30 minutes (CloudFront, I’m looking at you!).

Indeed, compared to other CDNs, CloudFront has gained the reputation as the “slow one.” I know I’ve shyed away from using it on one or two projects simply because I couldn’t face the update times.

Well, good news, this is a thing of the past! And oddly, AWS have made no official announcement… yet.

Update (2020-03-20): Woodrow Arrington at AWS wrote an official announcement, with pretty graphs and a link back to this post. Thanks Woodrow!

I discovered it through Last Week in AWS, that CloudFront had already rolled it out and semi-announced it in a Twitter reply:

We recently made changes that reduced the time to ~5 min consistently. We’re not done yet as this continues to be a priority for us this year. When we say a config is deployed we reference p100 metrics; however, most edge locations are updated in seconds. —[@cloudfront on Twitter](https://mobile.twitter.com/cloudfront/status/1235650978126258176)

I had to try it for myself. I decided to make a “small” change and upgrade my Lambda@Edge function from Python 3.7 to 3.8.

Due to Lambda changes, this caused a bit of churn in my codebase. The ability to inline some code with the Code.ZipFile parameter is no longer available for the Python 3.8 runtime.

This had been really convenient for simple Lambda use cases. Previously I had the 30 lines of Python code inline in my CloudFormation template. Now I have it in a separate file, and 30 extra lines spread between Ansible and CloudFormation to create an S3 bucket, zip the code, upload it, and then refer to it. Oh well.

After that kerfuffle, I deployed. My ansible run has the profile_tasks callback active, so you can see the relative timestamps:

...
TASK [cloudformation stack] ***************************************************
Monday 16 March 2020  10:33:36 +0000 (0:00:01.314)       0:00:06.521 **********
changed: [localhost]

TASK [site install] ***********************************************************
Monday 16 March 2020  10:39:30 +0000 (0:05:53.938)       0:06:00.459 **********
changed: [localhost]
...

Wow, 6 minutes minus 6 seconds = 5 minutes 54 seconds. That’s like five times faster!

And changing a Lambda function is a relatively expensive operation, since it copies the Lambda function out to all regions.

I also made a mistake in my first iteration. I forgot to update my Lambda function definition to point to the new module name.

This lead to my site being down with a “503 The Request could not be satisfied”:

And I found the error in the Lambda@Edge logs:

[ERROR] Runtime.ImportModuleError: Unable to import module 'index': No module named 'index'

Thankfully it was only another iteration to fix it, so my site wasn’t down very long. This one took less than 5 minutes!

Fin

Well done to the CloudFront team for speeding this up. I’m looking forward to the official announcement - maybe they’re waiting until they’ve sped things up even further.

This will certainly make me much less reluctant to use and recommend CloudFront.

—Adam

Read my book Boost Your Git DX to Git better.

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: aws, cloudfront, cloudformation