Efficiently Fetching SSM and Secrets in AWS Lambda with Pydantic Models

Introduction

Fetching configuration values or secrets from AWS SSM Parameter Store or Secrets Manager is a common pattern in Lambda functions. However, fetching these values multiple times per invocation can lead to increased latency, higher costs, and hitting AWS API rate limits. In this post, we'll discuss why you should fetch secrets only once per Lambda execution and how to use Pydantic to manage your settings efficiently.

Why Fetch SSM/Secrets Only Once?

• Performance: Each call to SSM or Secrets Manager is a network request, which adds latency.
• Cost: AWS charges per API call; unnecessary calls increase your bill.
• Rate Limits: Excessive calls can hit AWS API rate limits, causing throttling or failures.
• Best Practice: Fetch once, reuse throughout the Lambda invocation.

Using Pydantic for Settings Management

Pydantic's BaseSettings class makes it easy to define and validate configuration values. You can fetch secrets once at cold start and reuse them across your Lambda handler and other modules.

Example: Centralized Settings with Pydantic

from typing import Union
from pydantic import BaseSettings

class _Settings(BaseSettings):
    SSM_RDS_HOST: Union[str, None] = None
    ENVIRONMENT: str = None

    @property
    def ssm_rds_host(self):
        if self.SSM_RDS_HOST:
            return self.SSM_RDS_HOST
        else:
            # Fetch from SSM Parameter Store
            return self.get_ssm_value("/dev/rds_host")
    def get_ssm_value(self, key: str) -> str:
        # Logic to fetch value from SSM Parameter Store
        ...

# Instantiate settings at module level (cold start)
settings = _Settings()

Now, you can import settings in any other module or Lambda handler:

from .settings import settings

def lambda_handler(event, context):
    db_host = settings.SSM_RDS_HOST or settings.get_ssm_value("rds_host")
    # ...rest of your logic...

Conclusion

Fetching SSM parameters or secrets only once per Lambda execution is a simple optimization that can improve performance and reliability. Using Pydantic for settings management keeps your code clean and maintainable. Adopt this pattern to make your AWS Lambda functions faster and more robust!