SAM Readme¶

planttracer-webapp¶

This project contains source code and supporting files for a serverless application that we deploy with the SAM CLI. It includes the following files and folders.

deploy - Code for the application’s Lambda function.
deploy/Dockerfile - creates the container, installing all packages in deploy/requirements.txt
events - Invocation events that one day will be used to test invoking the function
tests - Unit tests that will one day be used to test the application code.
template.yaml - The template that defines the application’s AWS resources.

Deploy the application¶

The Serverless Application Model Command Line Interface (SAM CLI) is an extension of the AWS CLI that adds functionality for building and testing Lambda applications. It uses Docker to run your functions in an Amazon Linux environment that matches Lambda. It can also emulate your application’s build environment and API.

To use the SAM CLI, you need the following tools.

SAM CLI - Install the SAM CLI
Python 3 installed
Docker - Install Docker community edition

To build and deploy your application for the first time, run the following in your shell:

sam build --use-container
sam deploy --guided

The first command will build the source of your application. The second command will package and deploy your application to AWS, with a series of prompts:

Stack Name: The name of the stack to deploy to CloudFormation. This should be unique to your account and region, and a good starting point would be something matching your project name.
AWS Region: The AWS region you want to deploy your app to.
Confirm changes before deploy: If set to yes, any change sets will be shown to you before execution for manual review. If set to no, the AWS SAM CLI will automatically deploy application changes.
Allow SAM CLI IAM role creation: Many AWS SAM templates, including this example, create AWS IAM roles required for the AWS Lambda function(s) included to access AWS services. By default, these are scoped down to minimum required permissions. To deploy an AWS CloudFormation stack which creates or modifies IAM roles, the CAPABILITY_IAM value for capabilities must be provided. If permission isn’t provided through this prompt, to deploy this example you must explicitly pass --capabilities CAPABILITY_IAM to the sam deploy command.
Save arguments to samconfig.toml: If set to yes, your choices will be saved to a configuration file inside the project, so that in the future you can just re-run sam deploy without parameters to deploy changes to your application.

You can find your API Gateway Endpoint URL in the output values displayed after deployment.

Use the SAM CLI to build and test locally¶

Build your application with the sam build --use-container command.

planttracer-webapp$ sam build --use-container

NOTE:

This only works if your systems is configured for AWS. (e.g. $HOME/.aws/credentails) are problerly configured.

planttracer-webapp$ sam local invoke HelloWorldFunction --event events/event.json

The SAM CLI can also emulate your application’s API. Use the sam local start-api to run the API locally on port 3000.

planttracer-webapp$ sam local start-api
planttracer-webapp$ curl http://localhost:3000/

The SAM CLI reads the application template to determine the API’s routes and the functions that they invoke. The Events property on each function’s definition includes the route and method for each path.

      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get

Fetch, tail, and filter Lambda function logs¶

To simplify troubleshooting, SAM CLI has a command called sam logs. sam logs lets you fetch logs generated by your deployed Lambda function from the command line. In addition to printing the logs on the terminal, this command has several nifty features to help you quickly find the bug.

NOTE: This command works for all AWS Lambda functions; not just the ones you deploy using SAM.

planttracer-webapp$ sam logs  --tail

Or, more specifically:

planttracer-webapp$ sam logs -n HelloWorldFunction --stack-name "planttracer-webapp" --tail

For more information, see:

SAM CLI Documentation.

Tests¶

Tests are defined in the tests folder in this project. Use PIP to install the test dependencies and run tests.

planttracer-webapp$ pip install -r tests/requirements.txt --user
# unit test
planttracer-webapp$ python -m pytest tests/unit -v
# integration test, requiring deploying the stack first.
# Create the env variable AWS_SAM_STACK_NAME with the name of the stack we are testing
planttracer-webapp$ AWS_SAM_STACK_NAME="planttracer-webapp" python -m pytest tests/integration -v

Cleanup¶

To delete the sample application that you created, use the AWS CLI. Assuming you used your project name for the stack name, you can run the following:

sam delete --stack-name "planttracer-webapp"

Educational Resources¶

SAM Programmer notes¶

Handling binary types¶

https://chatgpt.com/share/676099eb-30a8-8010-841a-f2ed04a8dcff

You:

I have a bottle python application running in lambda. I’m getting utf-8 decoding errors when static files are served. One is a .mp3 file. One is favicon.ico. One is a JSON object. These didn’t happen when I ran under Zappa but they are now happening after I switched to SAM. Here is the error; how do I fix? 2024/12/16/[$LATEST]38f3b8b8966f4b0385db87164f2683d8 2024-12-16T21:18:20.299000 LAMBDA_WARNING: Unhandled exception. The most likely cause is an issue in the function code. However, in rare cases, a Lambda runtime update can cause unexpected function behavior. For functions using managed runtimes, runtime updates can be triggered by a function change, or can be applied automatically. To determine if the runtime has been updated, check the runtime version in the INIT_START log entry. If this error correlates with a change in the runtime version, you may be able to mitigate this error by temporarily rolling back to the previous runtime response[“body”] = self.body.getvalue().decode(“utf-8”)pig_responseition 0: invalid start byteml

ChatGPT:

The issue lies in the encoding assumption when serving static files or generating responses in your Bottle application running on AWS Lambda. Specifically, your application attempts to decode binary content (e.g., .mp3, favicon.ico) or incorrectly handles content types like JSON, leading to utf-8 decoding errors. Here’s how to fix it:

Update Static File Handling Ensure that static files are served as binary content without attempting UTF-8 decoding:

from bottle import static_file

@app.route('/static/<filename>')
def serve_static(filename):
    return static_file(filename, root='./static', mimetype='application/octet-stream')

For specific file types like .mp3 and .ico, ensure the mimetype is correctly set (e.g., audio/mpeg for .mp3 or image/x-icon for .ico).2. Fix JSON Encoding JSON responses must be explicitly encoded and returned as UTF-8. Instead of returning plain JSON strings, set the Content-Type header and ensure the response body is serialized correctly:

from bottle import response
import json

@app.route('/data')
def json_response():
    data = {"key": "value"}
    response.content_type = 'application/json'
    return json.dumps(data)

If json.dumps generates binary output, it’s safe to return it without decoding to UTF-8.3. Avoid Automatic UTF-8 Decoding in Lambda Handler AWS Lambda’s integration with API Gateway or ALB might try to decode the response body. Ensure the response uses the correct isBase64Encoded flag for binary content:

Update your Lambda function to include this flag in the response for binary files:

import base64

def lambda_handler(event, context):
    # Example for serving binary files
    binary_content = load_binary_file('path/to/favicon.ico')  # Replace with your binary file loader
    return {
        "statusCode": 200,
        "headers": {"Content-Type": "image/x-icon"},
        "isBase64Encoded": True,
        "body": base64.b64encode(binary_content).decode('utf-8')
    }

In template.yaml for SAM, declare binary media types:

Resources:
  Api:
    Type: AWS::Serverless::Api
    Properties:
      BinaryMediaTypes:
        - image/x-icon
        - audio/mpeg
        - application/json

Update Dependencies

Ensure you are using a compatible version of bottle, and update any middleware or custom handlers introduced for Lambda or SAM deployment.

pip install --upgrade bottle

If you’re using additional packages for Lambda integration (e.g., bottle-lambda), ensure they handle binary content correctly.

Debugging Add logging to your application to verify the Content-Type headers and binary file serving logic:

import logging

@app.route('/debug/<filename>')
def debug_file(filename):
    logging.info(f"Serving file: {filename}")
    return static_file(filename, root='./static')

Use CloudWatch logs to inspect these values.

Summary:¶

Serve static files with proper mimetype and avoid UTF-8 decoding.
Handle JSON explicitly using json.dumps and set Content-Type: application/json.
Use the isBase64Encoded flag in Lambda responses for binary files.
Declare binary media types in the SAM template for API Gateway.

If these steps don’t resolve the issue, provide more context about your SAM template and Lambda handler for further debugging.

You:

what does “isBase64Encoded”: True, do?

ChatGPT:

The "isBase64Encoded": True flag in an AWS Lambda response tells API Gateway (or other AWS services handling the response) that the body field contains binary data encoded as a Base64 string.

Why It’s Needed¶

AWS API Gateway transmits responses as JSON objects to clients. Since JSON does not natively support raw binary data, binary content must be encoded in a way that is safe to include in a JSON string. Base64 encoding is commonly used for this purpose because it converts binary data into a text-based format. When the isBase64Encoded flag is set to True:

API Gateway knows to treat the body field as Base64-encoded binary data.
API Gateway decodes the Base64 string back into binary before sending the response to the client. If the flag is not set, API Gateway assumes the body is plain text and tries to interpret or handle it as such, which can lead to issues when dealing with binary files like images, audio, or other non-textual data.

Example Usage¶

Lambda Function:¶

import base64

def lambda_handler(event, context):
    # Load binary file content
    with open('/path/to/image.png', 'rb') as f:
        binary_data = f.read()

    # Encode binary data to Base64
    base64_encoded_data = base64.b64encode(binary_data).decode('utf-8')

    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "image/png"
        },
        "isBase64Encoded": True,  # Indicates the body contains Base64-encoded data
        "body": base64_encoded_data
    }

Resulting Response Sent to Client¶

When the client makes a request, API Gateway:

Decodes the Base64-encoded data back into binary.
Sends it to the client as raw binary data with the appropriate Content-Type header. Without isBase64Encoded: TrueIf the flag is omitted or set to False, the body is treated as a plain text string. For binary content, this results in corruption (e.g., images won’t display, audio files won’t play).

When to Use¶

Use "isBase64Encoded": True when:

Your response includes binary content (e.g., .mp3, .ico, .png, .jpg).
You’re serving files or other binary data through Lambda and API Gateway.

SAM and Binary Media Types¶

For this to work seamlessly, you must also configure API Gateway to support binary media types, as shown in your SAM template.yaml:

Resources:
  ApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      BinaryMediaTypes:
        - image/png
        - image/jpeg
        - image/x-icon
        - audio/mpeg

This configuration ensures API Gateway knows which content types should use the isBase64Encoded flag for binary responses.

You:

Why was this not a problem running under zappa?

ChatGPT:

The difference in behavior between Zappa and AWS SAM when serving static files or handling binary responses likely arises from how Zappa preconfigures API Gateway and integrates with Lambda compared to SAM.

Key Differences Between Zappa and SAM:¶

Zappa Automates Binary Media Type Configuration

Zappa : By default, Zappa automatically sets up API Gateway to support common binary media types (e.g., images, audio, and others). It adds these types to the BinaryMediaTypes configuration in API Gateway, so files like .mp3 or .ico are treated correctly.
SAM : You must explicitly declare BinaryMediaTypes in the template.yaml file. If this step is skipped, API Gateway won’t handle binary responses properly, leading to UTF-8 decoding errors.

Response Handling

Zappa : It includes built-in middleware to handle binary file serving. Zappa transparently encodes binary data as Base64 when required and sets the isBase64Encoded flag in the Lambda response.
SAM : SAM-generated Lambda functions don’t include this middleware by default. If your Lambda function sends binary data without setting isBase64Encoded or if API Gateway isn’t configured for binary media types, it will attempt to decode the response as UTF-8, causing errors.

Static File Handling

Zappa : Zappa has mechanisms to serve static files (like .mp3 or .ico) from the /static path or an associated S3 bucket. It manages the correct Content-Type and binary handling automatically.
SAM : SAM doesn’t provide direct support for static files. You need to either:
- Serve static files directly from S3 via a public bucket.
- Implement the file-serving logic in your Lambda function, including setting the correct headers and handling binary data.

Built-In Middleware in Zappa Zappa includes middleware that handles:

Automatic Base64 encoding for binary responses.
Correct content-type inference for static files.

With SAM, you’re responsible for implementing this functionality in your Lambda function, or you must configure your API Gateway properly.

Why This Didn’t Occur with Zappa¶

Binary Media Types Already Configured : Zappa preconfigured API Gateway to handle binary media types correctly.
Built-In Base64 Encoding : Zappa’s middleware automatically Base64-encoded binary content and set the isBase64Encoded flag.
Static File Handling : Zappa simplified serving static files, ensuring correct MIME types and response handling out of the box.
Default Assumptions About Application Needs : Zappa was built specifically for deploying Python WSGI applications, so it handled common issues like binary responses automatically.

Fixing This with SAM¶

To replicate Zappa’s behavior in SAM, ensure the following:

Add Binary Media Types to API Gateway : Update your template.yaml:

Resources:
  ApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      BinaryMediaTypes:
        - image/x-icon
        - audio/mpeg
        - application/json
        - application/octet-stream

Encode Binary Responses in Your Lambda Function : Explicitly encode binary data as Base64 and set isBase64Encoded to True in your Lambda response.
Serve Static Files From S3 : Offload static file serving to S3, where files are served directly with proper MIME types and without encoding issues.
Use Middleware for Binary Handling : Include middleware (like mangum for ASGI apps) if needed to manage Base64 encoding and API Gateway integration. For Bottle:

pip install mangum

Update your Lambda handler:

from mangum import Mangum
from bottle import Bottle

app = Bottle()

# Define routes...
handler = Mangum(app)

By replicating these behaviors in SAM, you can match the functionality you previously had with Zappa.

SAM Readme¶

planttracer-webapp¶

Deploy the application¶

Use the SAM CLI to build and test locally¶

Fetch, tail, and filter Lambda function logs¶

Tests¶

Cleanup¶

Educational Resources¶

SAM Programmer notes¶

Handling binary types¶

Summary:¶

Why It’s Needed¶

Example Usage¶

Lambda Function:¶

Resulting Response Sent to Client¶

When to Use¶

SAM and Binary Media Types¶

Key Differences Between Zappa and SAM:¶

Why This Didn’t Occur with Zappa¶

Fixing This with SAM¶

Plant Tracer Web Application

Navigation

Related Topics