Secrets Manager

How AWS Secrets Manager is used for secure token and credential storage.

PayPulse Cloud uses AWS Secrets Manager as the single store for all OAuth tokens and sensitive credentials. No secret values are written to source control — they are injected at deploy time via terraform.tfvars or written dynamically at runtime when users connect their Gmail accounts.

Secret types

Two categories of secrets are used by the Gmail OAuth integration:

gmail/user/{user_id}

Per-user OAuth token bundle. Written at runtime when a user connects their Gmail account. Read by every Lambda that accesses the Gmail API.

Google-OAuth-Client-ID

The iOS OAuth client ID from Google Cloud Console. Defined in Terraform and deployed once. Read by Lambda functions when building OAuth credentials for token refresh.

gmail/user/{user_id}

This secret is created or updated by the gmail_store_tokens Lambda each time a user connects (or reconnects) their Gmail account. It is also updated automatically whenever an access token is refreshed.

The secret stores a JSON object with the following shape:

example secret value

{
  "access_token": "ya29.a0AfB_...",
  "refresh_token": "1//0gABC...",
  "expires_at": "2024-11-01T11:00:00.000000",
  "expires_in": 3600,
  "scope": "https://www.googleapis.com/auth/gmail.readonly ...",
  "token_type": "Bearer",
  "created_at": "2024-11-01T10:00:00.000000",
  "google_user_id": "123456789012345678901",
  "google_email": "[e-mail]",
  "google_name": "Jane Smith",
  "google_verified_email": true
}

Google-OAuth-Client-ID

This secret holds the iOS OAuth client ID string. It is defined in Terraform and its value is supplied via terraform.tfvars.

Terraform configuration

The Secrets Manager resources for credentials are defined in secrets.tf:

secrets.tf

# Google OAuth client ID for Gmail API access
resource "aws_secretsmanager_secret" "google_oauth_client_id" {
  name        = "Google-OAuth-Client-ID"
  description = "Google OAuth client ID for Gmail API access via iOS app."
}

resource "aws_secretsmanager_secret_version" "google_oauth_client_id_value" {
  secret_id     = aws_secretsmanager_secret.google_oauth_client_id.id
  secret_string = var.google_oauth_client_id
}

The var.google_oauth_client_id variable is sourced from terraform.tfvars, which is listed in .gitignore. Per-user OAuth secrets (gmail/user/{user_id}) are not defined in Terraform — they are created and managed entirely at runtime by the application.

Never commit actual secret values to terraform.tfvars or any other file in the repository. The terraform.tfvars file is gitignored for this reason. Set secret values by running terraform apply locally with your own terraform.tfvars, or by updating the secret value directly in the AWS Console or via the AWS CLI.

Reading and writing secrets from Python

All Secrets Manager operations go through secretsmanager_utils.py in the common Lambda layer.

Storing OAuth tokens

store_oauth_tokens is called by the gmail_store_tokens Lambda after a user grants consent. It updates the secret if it already exists, or creates it if this is the user’s first connection:

secretsmanager_utils.py

def store_oauth_tokens(
    user_id: str,
    access_token: str,
    refresh_token: Optional[str],
    expires_in: int,
    scope: str,
    region: str,
    google_user_info: Dict[str, str] = None
):
    from utils.oauth_utils import prepare_oauth_secret_data

    secret_name = f"gmail/user/{user_id}"
    oauth_data = prepare_oauth_secret_data(
        access_token, refresh_token, expires_in, scope, google_user_info
    )

    session = boto3.session.Session()
    client = session.client(service_name='secretsmanager', region_name=region)

    try:
        # Try to update existing secret first
        client.update_secret(
            SecretId=secret_name,
            SecretString=json.dumps(oauth_data)
        )
        logging.info(f"Updated OAuth tokens for user {user_id}")

    except client.exceptions.ResourceNotFoundException:
        # Secret doesn't exist, create it
        client.create_secret(
            Name=secret_name,
            SecretString=json.dumps(oauth_data)
        )
        logging.info(f"Created OAuth tokens for user {user_id}")

Retrieving OAuth tokens

get_oauth_tokens reads the token bundle and annotates it with is_expired: True if the access token has passed its expires_at timestamp:

secretsmanager_utils.py

def get_oauth_tokens(user_id: str, region: str) -> Dict:
    secret_name = f"gmail/user/{user_id}"

    session = boto3.session.Session()
    client = session.client(service_name='secretsmanager', region_name=region)

    try:
        get_secret_value_response = client.get_secret_value(SecretId=secret_name)
    except ClientError as e:
        raise SecretsManagerError(
            f"Error retrieving OAuth tokens for user {user_id}"
        ) from e

    secret = get_secret_value_response['SecretString']
    oauth_data = json.loads(secret)

    # Check if token is expired and needs refresh
    from datetime import datetime
    if 'expires_at' in oauth_data:
        expires_at = datetime.fromisoformat(oauth_data['expires_at'])
        if datetime.utcnow() >= expires_at:
            logging.warning(f"Access token for user {user_id} has expired")
            oauth_data['is_expired'] = True

    return oauth_data

Updating tokens after refresh

update_oauth_tokens is called by create_gmail_service after a successful token refresh. It preserves the existing scope and Google user info while writing a new access_token and expires_at:

secretsmanager_utils.py

def update_oauth_tokens(
    user_id: str,
    access_token: str,
    refresh_token: str,
    expires_in: int,
    region: str
):
    from utils.oauth_utils import prepare_oauth_secret_data

    secret_name = f"gmail/user/{user_id}"

    # Get existing OAuth data to preserve scope and Google user info
    try:
        existing_data = get_oauth_tokens(user_id, region)
        scope = existing_data.get('scope', 'https://www.googleapis.com/auth/gmail.readonly')
        google_user_info = {
            'google_user_id': existing_data.get('google_user_id', ''),
            'google_email': existing_data.get('google_email', ''),
            'google_name': existing_data.get('google_name', ''),
            'google_verified_email': existing_data.get('google_verified_email', False)
        }
    except SecretsManagerError:
        scope = 'https://www.googleapis.com/auth/gmail.readonly'
        google_user_info = None

    oauth_data = prepare_oauth_secret_data(
        access_token, refresh_token, expires_in, scope, google_user_info
    )

    session = boto3.session.Session()
    client = session.client(service_name='secretsmanager', region_name=region)

    client.update_secret(
        SecretId=secret_name,
        SecretString=json.dumps(oauth_data)
    )
    logging.info(f"Successfully updated OAuth tokens for user {user_id}")

Deleting tokens

delete_oauth_tokens is called when a refresh token is found to be expired or revoked, forcing the user to reconnect. It uses ForceDeleteWithoutRecovery=True to bypass the 30-day recovery window:

secretsmanager_utils.py

def delete_oauth_tokens(user_id: str, region: str):
    secret_name = f"gmail/user/{user_id}"

    session = boto3.session.Session()
    client = session.client(service_name='secretsmanager', region_name=region)

    try:
        client.delete_secret(
            SecretId=secret_name,
            ForceDeleteWithoutRecovery=True
        )
        logging.info(f"Successfully deleted OAuth tokens for user {user_id}")

    except client.exceptions.ResourceNotFoundException:
        logging.warning(f"No OAuth tokens found for user {user_id}")

    except ClientError as e:
        raise SecretsManagerError(
            f"Error deleting OAuth tokens for user {user_id}"
        ) from e

Initial secret values with terraform.tfvars

The Google-OAuth-Client-ID secret value is injected at deploy time via a terraform.tfvars file:

terraform.tfvars (example — do not commit real values)

google_oauth_client_id = "123456789-abc123.apps.googleusercontent.com"

This file is listed in .gitignore and must never be committed to the repository. Each developer or CI environment provides its own copy of terraform.tfvars when running terraform apply.

Per-user secrets (gmail/user/{user_id}) are not created by Terraform. They are written at runtime by the gmail_store_tokens Lambda the first time a user connects their Gmail account, and are updated automatically whenever tokens are refreshed.