Implementing a Serverless Flask REST API using AWS SAM
By
Introduction
In this article I demonstrate how to build a serverless REST API using the AWS Serverless Application Model (SAM) framework along with the popular Python based Flask micro web framework. SAM is a fantastic developer productivity booster which serves to wrap and extend CoudFormation as well as provide a CLI on top of the standard AWS CLI. Collectively SAM enables smooth, repeatable, and automated dev-test-deploy cycles within the AWS Cloud.
The complete code for this tutorial's project is on my GitHub account available in this repo.
Setup of AWS IAM, CLI and SAM
The first necessary to get started is to create an AWS IAM user with programmatic access and sufficient privilege policies to run the SAM CLI. A simple way to do this is to create a IAM user with AdministratorAccess policy. However, this is arguably a heavy handed approach so, please act in accordance with your organization's policies and procedures. The IAM user I'll be using for this article is named serverless-admin.
Aside from the note about creating an appropriate IAM user for running your SAM CLI to interact with your AWS account the SAM Docs on AWS do a very thorough job of describing how to setup the standard AWS CLI along with the SAM CLI. Please consult the linked docs for the remainder of the initial setup then rejoin in the next section where I walk through scafolding out the SAM project.
Building Out the SAM Project Structure and Template
AWS Serverless Application Model (SAM) utilizes simplified templating Infrastructure as Code (IoC) abstraction ontop of the standard CloudFormation specification as described in the SAM docs. In this section I build out the basic structure of my SAM project structure by piggy backing off the sam init command, specifying a valid Python3 runtime I have on my local machine(Python 3.8), give the app a name of lol-api, then select the Hello World Example template in the interactive terminal session as shown below.
$ sam init --runtime python3.8 --name lol-api
Which template source would you like to use?
1 - AWS Quick Start Templates
2 - Custom Template Location
Choice: 1
Cloning app templates from https://github.com/awslabs/aws-sam-cli-app-templates.git
AWS quick start application templates:
1 - Hello World Example
2 - EventBridge Hello World
3 - EventBridge App from scratch (100+ Event Schemas)
4 - Step Functions Sample App (Stock Trader)
5 - Elastic File System Sample App
Template selection: 1
-----------------------
Generating application:
-----------------------
Name: lol-api
Runtime: python3.8
Dependency Manager: pip
Application Template: hello-world
Output Directory: .
Next steps can be found in the README file at ./lol-api/README.mdNow if I inspect the directory structure that was created withthe help of the unix tree command I see the contents shown below.
$ tree lol-api
lol-api
├── README.md
├── events
│ └── event.json
├── hello_world
│ ├── __init__.py
│ ├── app.py
│ └── requirements.txt
├── template.yaml
└── tests
└── unit
├── __init__.py
└── test_handler.py
4 directories, 8 filesThe Hello World exmaple provides a pretty good functional example and structure for my project. If you are following along feel free to take a quick look at the files but, I'll be quickly moving along to customize this boiler plate starter project to serve my end goal of having a Python/Flask serverless REST API.
To start I rename the hello_world directory to flask_api then I update the requirements.txt to include the dependencies specific to building a Flask based Lambda API which are Flask-Lambda, requests and boto3 as shown below.
# requirements.txt
flask-lambda
boto3
requestsFor the moment I ignore the contents of the app.py file and instead move on to the template.yaml, meant to contain the IaC for the app, replacing it's contents with that which are shown next.
# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
This is a SAM template for lol-api which is a Serverless API for
composing and managing Lots of Lists.
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 10
Parameters:
ExecEnv:
Type: String
AllowedValues:
- local
- dev
- stage
- prod
Default: prod
LotsOfListsTableName:
Type: String
Default: lots-of-lists
Resources:
LotsOfListsFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: flask_api/
Handler: app.app
Runtime: python3.8
Environment:
Variables:
TABLE_NAME: !Ref LotsOfListsTableName
REGION_NAME: !Ref AWS::Region
EXEC_ENV: !Ref ExecEnv
Policies:
- DynamoDBCrudPolicy:
TableName: !Ref LotsOfListsTable
Events:
FetchLists:
Type: Api
Properties:
Path: /lists
Method: get
CreateList:
Type: Api
Properties:
Path: /lists
Method: post
FetchList:
Type: Api
Properties:
Path: /lists/{id}
Method: get
UpdateList:
Type: Api
Properties:
Path: /lists/{id}
Method: put
DeleteList:
Type: Api
Properties:
Path: /lists/{id}
Method: delete
LotsOfListsTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: !Ref LotsOfListsTableName
AttributeDefinitions:
- AttributeName: listId
AttributeType: S
- AttributeName: userId
AttributeType: S
KeySchema:
- AttributeName: userId
KeyType: HASH
- AttributeName: listId
KeyType: RANGE
ProvisionedThroughput:
ReadCapacityUnits: 2
WriteCapacityUnits: 2
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
LotsOfListsApi:
Description: "API Gateway endpoint URL for Prod stage for Lots Of Lists function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/lists"
LotsOfListsFunction:
Description: "Lots Of Lists Lambda Function ARN"
Value: !GetAtt LotsOfListsFunction.Arn
LotsOfListsFunctionIamRole:
Description: "Implicit IAM Role created for Lots Of Lists function"
Value: !GetAtt LotsOfListsFunctionRole.Arn
One of the best things about both the SAM templating syntax and the CloudFormation spec that it is built on is the clear expressiveness and self documenting nature of Infrastructure as Code templates. I feel like anyone with a modest understanding of common AWS Services can read a SAM / CloudFormation template and generally understand what the corresponding stack will produce. That being said, I will not being into the details of the syntax other than to say that the SAM templating spec allows for direct usage of standard CloudFormation so, if there is something that the regular SAM Resource specification does not cover that you require then simply drop down to the standard CloudFormation template language and you are set.
Coding the Flask REST API
With the infrastructure defined in the template.yaml along with the basic project file structure and Python dependencies specified what remains is the fun part. Writing App Code!
Heading over to a suitable code editor I open flask_api/app.py and delete all the initial content provided by the Hello World SAM initialization template and move on by importing dependencies, instantiating a instance of Flask-Lambda then define a few constants representing the environment variables I specified in in the last section's template.yaml file. After that its really just implementing the standard Flask API for building REST API and interacting with AWS DynamoDB via the AWS Python Boto3 SDK as shown below. It's worth meantioning at this point that FlaskLambda is rather simple wrapper that simply intercepts and shuffles around requests and responses between formats that are compatible with Lambda functions and the popular Flask web microframework to provide developers the ability to use the familar API of Flask.
import os
import uuid
import boto3
from boto3.dynamodb.conditions import Key
from flask import request, jsonify
from flask_lambda import FlaskLambda
EXEC_ENV = os.environ['EXEC_ENV']
REGION = os.environ['REGION_NAME']
TABLE_NAME = os.environ['TABLE_NAME']
app = FlaskLambda(__name__)
if EXEC_ENV == 'local':
dynamodb = boto3.resource('dynamodb', endpoint_url='http://dynamodb:8000')
else:
dynamodb = boto3.resource('dynamodb', region_name=REGION)
def db_table(table_name=TABLE_NAME):
return dynamodb.Table(table_name)
def parse_user_id(req):
'''When frontend is built and integrated with an AWS Cognito
this will parse and decode token to get user identification'''
return req.headers['Authorization'].split()[1]
@app.route('/lists')
def fetch_lists():
try:
user_id = parse_user_id(request)
except:
return jsonify('Unauthorized'), 401
tbl_response = db_table().query(KeyConditionExpression=Key('userId').eq(user_id))
return jsonify(tbl_response['Items'])
@app.route('/lists', methods=('POST',))
def create_list():
list_id = str(uuid.uuid4())
try:
user_id = parse_user_id(request)
except:
return jsonify('Unauthorized'), 401
list_data = request.get_json()
list_data.update(userId=user_id, listId=list_id)
tbl = db_table()
tbl.put_item(Item=list_data)
tbl_response = tbl.get_item(Key={'userId': user_id, 'listId': list_id})
return jsonify(tbl_response['Item']), 201
@app.route('/lists/<string:list_id>')
def fetch_list(list_id):
try:
user_id = parse_user_id(request)
except:
return jsonify('Unauthorized'), 401
tbl_response = db_table().get_item(Key={'userId': user_id, 'listId': list_id})
return jsonify(tbl_response['Item'])
@app.route('/lists/<string:list_id>', methods=('PUT',))
def update_list(list_id):
try:
user_id = parse_user_id(request)
except:
return jsonify('Unauthorized'), 401
list_data = {k: {'Value': v, 'Action': 'PUT'}
for k, v in request.get_json().items()}
tbl_response = db_table().update_item(Key={'userId': user_id, 'listId': list_id},
AttributeUpdates=list_data)
return jsonify()
@app.route('/lists/<string:list_id>', methods=('DELETE',))
def delete_list(list_id):
try:
user_id = parse_user_id(request)
except:
return jsonify('Unauthorized'), 401
db_table().delete_item(Key={'userId': user_id, 'listId': list_id})
return jsonify()
After the code is written what remains is to then build and deploy the SAM application.
To build I use the following build command of SAM CLI to build the project using a AWS CLI profile configured for my serverless-admin IAM user and destined for the US-East-2 Ohio region of the AWS Cloud.
sam build --profile serverless-admin --region us-east-2Then to deploy I use the deploy SAM CLI command in guided mode. This is an interactive command that allows me to save my inputs for automated deployment parameters in subsequent deployments.
sam deploy --stack-name lol-api \
--profile serverless-admin \
--region us-east-2 \
--guidedNow on subsequent changes of the template or code the same build and an abbreviated deploy command shall be used as shown below.
sam build --profile serverless-admin --region us-east-2
sam deployTesting with the HTTPie Client
One of the outputs from this build command will be the endpoint for the REST API which for me is https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists but, this endpoint will go away at then of this tutorial when I clean up the CloudFormation stack so please don't rely on it being available and instead create your own.
I can try out the REST API using the a CLI program such as HTTPie or a GUI program like PostMan as well as the API GateWay dashboard in the AWS Console.
Below I use the httpie program to generate some list data for a mock user represented by a bare Authorization token value representing a userId of 12345. Note that this is just a weird stop-gap implementation in the REST API that will be changed once the frontend app is built and integrated with AWS Cognito service will issue a suitable JWT token.
To create a list for mock user with id 12345.
http -j POST https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists \
"Authorization:bearer 12345" \
name=Groceries \
items:='[{"name":"milk", "completed":false},{"name":"cookies", "completed":false}]'Example output.
HTTP/1.1 201 Created
Connection: keep-alive
Content-Length: 167
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:43:44 GMT
Via: 1.1 61729b32280fd6715c2a3b0dbb7e571a.cloudfront.net (CloudFront)
X-Amz-Cf-Id: pacrTjVome7UGBq8vQL6zdCKOk3APZSA2DBQj8pGnuh-OnSP757oVA==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72990-405ddd7e4e1444ca3eba9437;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6NuhFVQiYcFcYA=
x-amzn-Remapped-Content-Length: 167
x-amzn-RequestId: c16e1614-3702-4177-bd31-62348daeb570
{
"items": [
{
"completed": false,
"name": "milk"
},
{
"completed": false,
"name": "cookies"
}
],
"listId": "160471cf-67a6-4e8f-aefb-512100cfd0ab",
"name": "Groceries",
"userId": "12345"
}To fetch lists for a mock user of id 12345.
http -j GET https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists "Authorization:bearer 12345"Example output.
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 169
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:44:51 GMT
Via: 1.1 f1234553b388306d833e1a4591227882.cloudfront.net (CloudFront)
X-Amz-Cf-Id: KKQ5L6pfbyGHfZiOLPbcDFivED2Fw6X8BEGNfZdt1gNhZTCJCXeLzQ==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc729d3-64b0bcdd073a6aa11fed2831;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6N5GHG-iYcFe-w=
x-amzn-Remapped-Content-Length: 169
x-amzn-RequestId: cc0423ed-0763-40aa-a35f-e5f190e8d19d
[
{
"items": [
{
"completed": false,
"name": "milk"
},
{
"completed": false,
"name": "cookies"
}
],
"listId": "160471cf-67a6-4e8f-aefb-512100cfd0ab",
"name": "Groceries",
"userId": "12345"
}
]To update a list item the previous list, assuming it has an id of 160471cf-67a6-4e8f-aefb-512100cfd0ab.
list_id=160471cf-67a6-4e8f-aefb-512100cfd0ab
http -j PUT https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists/$list_id \
"Authorization:bearer 12345" \
items:='[{"name":"milk", "completed":true},{"name":"cookies", "completed":true}]'Example output.
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 3
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:49:53 GMT
Via: 1.1 afb9be97319013ab1a18f338fce40f2a.cloudfront.net (CloudFront)
X-Amz-Cf-Id: ypQRoQKRsOq8qN184L7BwdxbgroiU_QkdtnLR6Vj8kpNqoxbNWBozg==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72aff-622f1960600fa6b21342b352;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6On9HYuiYcFd1A=
x-amzn-Remapped-Content-Length: 3
x-amzn-RequestId: 165ca484-1d96-4b72-b532-2828f066f531
{}Similarly, to fetch a single list with id 160471cf-67a6-4e8f-aefb-512100cfd0ab for a mock user with id 12345.
list_id=160471cf-67a6-4e8f-aefb-512100cfd0ab
http -j GET https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists/$list_id \
"Authorization:bearer 12345"Example output.
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 165
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:50:32 GMT
Via: 1.1 89d40f9555bb19bc571952b32ca87399.cloudfront.net (CloudFront)
X-Amz-Cf-Id: yejrLrOX9uavz8beTyG8NyJt5qhWy46EGPDvbkIn_FDj_aNXE5MXBw==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72b28-57faa3394fde74a876e8fe0e;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6OuWHKgCYcFTJw=
x-amzn-Remapped-Content-Length: 165
x-amzn-RequestId: b08737c8-bebf-4ed8-a102-4262a55e6f00
{
"items": [
{
"completed": true,
"name": "milk"
},
{
"completed": true,
"name": "cookies"
}
],
"listId": "160471cf-67a6-4e8f-aefb-512100cfd0ab",
"name": "Groceries",
"userId": "12345"
}
Then to delete the same list item.
list_id=160471cf-67a6-4e8f-aefb-512100cfd0ab
http -j DELETE https://93dl1r3gzk.execute-api.us-east-2.amazonaws.com/Prod/lists/$list_id \
"Authorization:bearer 12345"Example output.
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 3
Content-Type: application/json
Date: Wed, 02 Dec 2020 05:51:05 GMT
Via: 1.1 32814ee4b53f3642b74e20a0ba5944f7.cloudfront.net (CloudFront)
X-Amz-Cf-Id: FPL4JIZJw9Cwx4J_iY2uTUFtcKrISjHX_XdtUY7qi6IyGNZFu2L2Jg==
X-Amz-Cf-Pop: SFO5-C1
X-Amzn-Trace-Id: Root=1-5fc72b49-1ca904da3daadd3b4fb9b143;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: W6OzhFmbCYcF0SQ=
x-amzn-Remapped-Content-Length: 3
x-amzn-RequestId: a89553fc-41c9-4993-9e5c-e33307a620da
{}Cleaning Up a SAM Project's Resources (aka Removing the CF Stack)
To tear down all of a SAM applications resources you can either delete it in the CloudFormation dashboard with the AWS Console or use the AWS CLI and it's cloudformation subcommand like so.
aws cloudformation delete-stack \
--stack-name lol-api \
--profile serverless-admin \
--region us-east-2Running SAM Project Locally to Shorten the Development Test Fix Feedback Loop
Often times the functionality that we are asked to build as developers is not quite so straight forward as the simple CRUD functionality of this sample app which simply shuffles data between request to the database and back. As complexity grows so does iterative cycle of develop a piece of code, run the piece of code in the system to test the change, fix or modify then repeat. Even though we have the ability to deploy to a dev or staging environment in the AWS cloud in a pretty simple and straight forward way it is often best to shorten the feedback loop and run all or portions of the system locally. Using SAM along with Docker we can do just that.
For simple Lambda functions and REST APIs that do not interact with a database directly (ie, operates on the data its given or sources data from another API) the SAM CLI provides the following two commands.
# run a function by logical ID
sam local invoke FunctionName
# run an API locally
sam local start-apiFor the demo app I've presented in this tutorial I have a dependency for a DynamoDB table so running the SAM REST API locally requires a bit more setup work. I have to pull down and run the Amazon DynamoDB Local docker image which enables me to interact with a one or more DynamoDB tables locally without incuring costs and at a faster development pace.
Luckily the setup for doing this is fairly easy.
First I create a docker network to run the Amazon DynamoDB local docker image along with the SAM REST API project in as well.
docker network create dynamodb-localAfter that I fire up the Amazon DynamoDB docker image running it within the dynamodb-local network just created.
docker run --network dynamodb-local --name dynamodb -p 8000:8000 amazon/dynamodb-local
With the DynamoDB docker container running on my system I then create my lots-of-lists table in it as specified in the template.yaml template to be used for local running and development of my SAM project. For this I can use the AWS CLI or one of the available Higher Level Programming SDKs like boto3. For this project I use the AWS CLI and specify the --endpoint-url to be the localhost and port I forwarded traffic to when starting up the DynamoDB container.
aws dynamodb create-table --table-name lots-of-lists \
--attribute-definitions AttributeName=userId,AttributeType=S AttributeName=listId,AttributeType=S \
--key-schema AttributeName=userId,KeyType=HASH AttributeName=listId,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=2,WriteCapacityUnits=2 \
--endpoint-url=http://localhost:8000Then I can issue the following SAM CLI command to run the project locally within a docker container connected to the same network I created and am running the DynamoDB container on. The output of this command will indicate that the REST API is available on localhost port 3000
sam local start-api --parameter-overrides ExecEnv=local \
--docker-network dynamodb-localYou'll notice the --parameter-overrides flag being used in the above command which sets the parameter ExecEnv to local effectively overriding the default value of prod specified in the template.yaml file. This value is then referenced in the LotsOfListsFunction as an environment variable EXEC_ENV and used to dictate which DynamoDB resource to connect to via the Boto3 SDK as shown in the code snippet below from app.py
import os
import uuid
import boto3
from boto3.dynamodb.conditions import Key
from flask import request, jsonify
from flask_lambda import FlaskLambda
EXEC_ENV = os.environ['EXEC_ENV']
REGION = os.environ['REGION_NAME']
TABLE_NAME = os.environ['TABLE_NAME']
app = FlaskLambda(__name__)
if EXEC_ENV == 'local':
print('Configuring local dynamodb access')
dynamodb = boto3.resource('dynamodb', endpoint_url='http://dynamodb:8000')
else:
dynamodb = boto3.resource('dynamodb', region_name=REGION)What remains is to hop over in yet another terminal I can again use the HTTPie client to test out the locally running SAM Flask REST API.
$ http -j POST http://localhost:3000/lists \
"Authorization:bearer 12345" \
name=Groceries \
items:='[{"name":"milk", "completed":false},{"name":"cookies", "completed":false}]'
HTTP/1.0 201 CREATED
Content-Length: 167
Content-Type: application/json
Date: Thu, 03 Dec 2020 03:18:04 GMT
Server: Werkzeug/1.0.1 Python/3.8.6
{
"items": [
{
"completed": false,
"name": "milk"
},
{
"completed": false,
"name": "cookies"
}
],
"listId": "72dddc16-0543-4489-83e9-1c8071800936",
"name": "Groceries",
"userId": "12345"
}Resources to Learn More About Serverless Architecture and Computing
- For more thorough coverage of using Flask with AWS SAM please checkout my course Ultimate Guide to Deploying Flask to AWS which presents how to deploy Flask using both Traditiona EC2 based Server and AWS SAM Serverless Architectures and Methods
- One of the best practical guide to the AWS Serverless Application Model is Gojko Adzic's book Running Serverless
- For a walk through of how to still benefit from code reuse in a Serverless Application see related article Keeping Python AWS Serverless Apps DRY with Lambda Layers
- Deployments in a Serverless World can be difficult to wrap your head around, luckily thecodinginterface.com has covered this topic so have a read of Using Lambda Versions, Aliases and Safe Deployments with AWS Serverless Application Model
- Serverless isn't just for APIs and Microservices, it can be a very power component of modern Data Engineering Pipelines as demonstrated in Serverless Conversions From GZip to Parquet Format with Python AWS Lambda and S3 Uploads
thecodinginterface.com earns commision from sales of linked products such as the book suggestions above. This enables providing high quality, frequent, and most importantly FREE tutorials and content for readers interested in Software Engineering so, thank you for supporting the authors of these resources as well as thecodinginterface.com
Conclusion
In this article I have demonstrated the ease and simplicity of using the AWS Serverless Application Model framework for developing a serverless Python based Flask REST API. In articles to follow I will be rounding out the Lots of Lists application by similarly demonstrating how to build and deploy a complete React web application capable of registering and authenticating users with the AWS Cognito service as well as utilize the REST API built in this article.
As always, thanks for reading and please do not hesitate to critique or comment below.