Over the past two years TechConnect has had an increasing demand for creating “Serverless” API backends, from scratch or converting existing services running on expensive virtual machines in AWS. This has been an iterative learning process for us and I feel many others in the industry. However, it feels like each month pioneers in the field answer our cries for help by creating or extending Open-source projects to make our “serverless” lives a little easier.
There are quite a few options for creating serverless applications in AWS (Serverless Framework, Zappa, etc..). However, In this blog post, we will discuss using AWS SAM (Serverless Application Model, previously known as Project Flourish) to create a CORS enabled API. All templates and source code mentioned can be found in this GitHub repository. I heavily recommend having this open in another tab, along with the AWS SAM project.
API Design First with Swagger
Code or Design first? One approach is not necessarily better than the other, but at TechConnect we’ve been focusing on a design first mentality when it comes to building APIs for our clients. We aren’t the users of the APIs we build and we aren’t the front-end developers who might build a website off of it. Instead our goal when creating an external API is to create a logical and human readable API contract specification. To achieve this we use Swagger, the Open API specification to build and document our RESTful backends.
In the image below, we have started to design a simple movie ratings API in YAML using the Open API specification. In its current state, it is just an API contract showing the requests and responses. However, it will be further modified to become an AWS API Gateway compatible and AWS Lambda integrated document in future steps.
Code Structure
Our API is a simple CRUD that will make use of Amazon DynamoDB to create, list and delete movie ratings of a given year. This could all easily reside in a single Python file, but instead we will split it up to make it a little more realistic for larger projects. As this is a small demo, we’ll be missing a few resources that would usually be included in a real project (tests, task runners, etc..), but try having a look at The Hitchhiker’s Guide to Python for a nice Python structure for your own future APIs.
- template.yaml
- swagger.yaml
- requirements.txt
- movies
- api
- __init__.py
- ratings.py
- core
- __init__.py
- web.py
- __init__.py
Our Python project movies contains two sub-packages; api and core. Our AWS Lambda handlers are located in api.ratings.py , where each handle will; process the request from API Gateway, interact with DynamoDB (using a table name set by an environment variable) and return an object to API Gateway.
movies.api.ratings.py
...
from movies.core import web
def get_ratings(event, context):
...
return web.cors_web_response(200, ratings_list)
CORS in Lambda Responses
In the previous step you might have noticed we were using a function to build an integration response. The object body is serialized into a JSON string and the headers Access-Control-Allow-Headers, Access-Control-Allow-Methods and Access-Control-Allow-Origin are enabled for Cross-Origin Resource Sharing (CORS).
movies.core.web.py
def cors_web_response(status_code, body):
return {
'statusCode': status_code,
"headers": {
"Access-Control-Allow-Headers":
"Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token",
"Access-Control-Allow-Methods":
"DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT",
"Access-Control-Allow-Origin":
"*"
},
'body': json.dumps(body)
}
CORS in Swagger
Previously in our Lambda code, we built CORS headers into our responses. However, this is only one half of the solution. Annoyingly we must add an OPTIONS HTTP method to every path level of our API. This is to satisfy the preflight request done by the client to check if CORS requests are enabled. Although it uses x-amazon-apigateway-integration, it is a mocked response by API Gateway. AWS Lambda is not needed to implement this.
swagger.yaml
paths:
/ratings/{year}:
options:
tags:
- "CORS"
consumes:
- application/json
produces:
- application/json
responses:
200:
description: 200 response
schema:
$ref: "#/definitions/Empty"
headers:
Access-Control-Allow-Origin:
type: string
Access-Control-Allow-Methods:
type: string
Access-Control-Allow-Headers:
type: string
x-amazon-apigateway-integration:
responses:
default:
statusCode: 200
responseParameters:
method.response.header.Access-Control-Allow-Methods: "'DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT'"
method.response.header.Access-Control-Allow-Headers: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Origin: "'*'"
passthroughBehavior: when_no_match
requestTemplates:
application/json: "{\"statusCode\": 200}"
type: mock
Integrating with SAM
Since AWS SAM is an extension of CloudFormation, the syntax is almost identical. The snippets below show the integration between template.yaml and swagger.yaml. The AWS Lambda function GetRatings name is parsed into the API via a stage variable. swagger.yaml integrates the Lambda proxy using x-amazon-apigateway-integration. One important thing to note is that the Swagger document is not required to create an API Gateway resource in AWS SAM. However, we are using it due to our design first mentality and it being required for CORS preflight responses. The AWS SAM team are currently looking to reduce the need for this in CORS applications. Keep an eye out for the ongoing topic being discussed on GitHub.
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
DefinitionUri: swagger.yaml
StageName: v1
Variables:
GetRatings: !Ref GetRatings
...
GetRatings:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./build
Handler: movies.api.ratings.get_ratings
Role: !GetAtt CrudLambdaIAMRole.Arn
Environment:
Variables:
RATINGS_TABLE: !Ref RatingsTable
Events:
GetRaidHandle:
Type: Api
Properties:
RestApiId: !Ref ApiGatewayApi
Path: /ratings/{year}
Method: GET
...
swagger.yaml
paths:
/ratings/{year}:
get:
...
x-amazon-apigateway-integration:
responses:
default:
statusCode: 200
responseParameters:
method.response.header.Access-Control-Allow-Origin: "'*'"
uri: arn:aws:apigateway:REGION:lambda:path/2015-03-31/functions/arn:aws:lambda:REGION:ACCOUNT_ID:function:${stageVariables.GetRatings}/invocations
passthroughBehavior: when_no_match
httpMethod: POST
type: aws_proxy
Deploying SAM API
Now that all the resources are ready, the final step is to package and deploy the SAM application. You may have noticed in the template.yaml the source of the Lambda function was listed as ./build. Any AWS Lambda function that uses non-standard Python libraries will require them to be included in the deployment. To demonstrate this, we’ll send our code to a build folder and install the dependencies.
$ mkdir ./build
$ cp -p -r ./movies ./build/movies
$ pip install -r requirements.txt -t ./build
Finally, you will need to package your SAM deployment to convert it to a traditional AWS CloudFormation template. First your will need to make sure your own account id and desired region are used (using sed). You will also need to provide an existing S3 bucket to store the packaged code. If you inspect the template-out.yaml you will notice that the source of each AWS Lambda function in an object in S3. This is what is used by aws cloudformation deploy. One final tip is to remember to include --capabilities CAPABILITY_IAM in your deploy if you are creating any roles during your deployment.
$ sed -i "s/account_placeholder/AWS_ACCOUNT_ID/g" 'swagger.yaml'
$ sed -i "s/region_placeholder/AWS_REGION/g" 'swagger.yaml'
$ aws cloudformation package --template-file ./template.yaml --output-template-file ./template-out.yaml --s3-bucket YOUR_S3_BUCKET_NAME
$ aws cloudformation deploy --template-file template-out.yaml --stack-name MoviesAPI --capabilities CAPABILITY_IAM