Adding vendors

How to add new retail invoice vendors to PayPulse Cloud.

Adding a new vendor requires two steps: creating a JSON config file and optionally adding a prompt config for the Gemini parser. No Lambda code changes are needed.

1. Create the vendor config JSON

   Create a new file at vendor_configs/{sub_type}/{vendor_id}.json. The vendor_id must be lowercase and unique across all vendors — it is used as the DynamoDB partition key and as the S3 path segment.

   {
     "vendor_id": {"S": "vendor_id"},
     "vendor_name": {"S": "Vendor Name"},
     "invoice_category": {"S": "retail"},
     "invoice_sub_type": {"S": "clothing"},
     "default_email_patterns": {"SS": ["[e-mail]"]},
     "default_subject_keywords": {"SS": ["Your order"]},
     "parser_type": {"S": "html"},
     "active": {"BOOL": true},
     "supports_pdf": {"BOOL": false},
     "supports_html": {"BOOL": true},
     "logo_url": {"S": ""},
     "created_at": {"S": "2026-01-01T00:00:00Z"},
     "updated_at": {"S": "2026-01-01T00:00:00Z"}
   }
   

   Place the file in the subdirectory that matches the invoice_sub_type value (e.g. vendor_configs/clothing/ for clothing vendors).

2. Upload the config to DynamoDB

   Run the upload script from the vendor_configs/ directory. The script iterates over all */*.json files and uploads each one using aws dynamodb put-item with a attribute_not_exists(vendor_id) condition, so existing records are never overwritten.

   cd vendor_configs
   ./upload_vendors.sh
   

   Example output:

   =========================================
   Uploading vendor configs to DynamoDB
   Table: VendorConfig
   =========================================
   
   Uploading dominos.json ... SUCCESS
   Uploading foodora.json ... ALREADY EXISTS
   Uploading my_new_vendor.json ... SUCCESS
   
   =========================================
   Upload Summary
   =========================================
   Total files:      3
   Successful:       2
   Already existed:  1
   Failed:           0
   =========================================
   

   The script exits with code 1 if any upload fails.

3. Add a prompt config (optional)

   To improve parsing accuracy, add an entry to lambda_layers/gemini_parsers/python/vendor_prompt_configs/{sub_type}.py.

   "vendor_id": {
       "vendor_desc": "Short description of the vendor",
       "is_email_in_swedish": False,
       "items_desc": "what the items are (e.g. electronics)",   # optional
       "header_info": 'Look for "Order confirmation" header.',   # optional
       "field_translations": {                                    # optional, Swedish vendors only
           "Ordernummer": "Order number",
           "Totalt": "Total",
       },
   },
   

   The vendor_id key must exactly match the vendor_id value in the JSON config. This step can be skipped — the parser will still run, but with less context.

4. Test the new vendor

   Trigger a retail invoice fetch for your account by calling the ingest endpoint. You can scope the request to a specific date range to avoid re-fetching old data:

   curl -X POST https://{api-id}.execute-api.eu-west-1.amazonaws.com/v1/invoices/retail/ingest \
     -H "Authorization: Bearer {your_jwt_token}" \
     -H "Content-Type: application/json" \
     -d '{
       "start_date": "2026-01-01",
       "end_date": "2026-03-21"
     }'
   

   A successful response looks like:

   {
     "message": "Retail invoices fetched successfully",
     "data": {
       "totalEmailsFound": 3,
       "byVendor": {
         "my_new_vendor": 3,
         "zalando": 0
       },
       "dateRange": {
         "start": "2026/01/01",
         "end": "2026/03/21"
       },
       "errors": []
     }
   }
   

   Check CloudWatch Logs at /aws/lambda/fetch_retail_invoices for per-email processing details and any errors.

Vendor config field reference

Field	DynamoDB type	Required	Description
vendor_id	S	Yes	Unique lowercase identifier. Used as DynamoDB PK and S3 path segment.
vendor_name	S	Yes	Human-readable display name shown in the iOS app.
invoice_category	S	Yes	Always retail.
invoice_sub_type	S	Yes	One of: clothing, food-delivery, food_delivery, miscellaneous, subscriptions, technology, travel, grocery, utility. Must match the subdirectory name.
default_email_patterns	SS	Yes	String Set of sender email addresses. At least one value is required — the vendor is skipped if this is empty.
default_subject_keywords	SS	Yes	String Set of subject-line keywords used to narrow the Gmail search.
parser_type	S	Yes	Email format to parse. Use html.
active	BOOL	Yes	true to include in fetch runs, false to disable.
supports_pdf	BOOL	Yes	Whether this vendor sends invoices as PDF attachments.
supports_html	BOOL	Yes	Whether this vendor sends invoices as HTML email bodies.
logo_url	S	No	Public S3 URL to the vendor logo. Leave as "" if not yet uploaded.
created_at	S	Yes	ISO 8601 creation timestamp.
updated_at	S	Yes	ISO 8601 last-update timestamp.

The upload_vendors.sh script

#!/bin/bash

# Script to upload vendor configurations to DynamoDB VendorConfig table
# Usage: ./upload_vendors.sh

# Color codes for output
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

TABLE_NAME="VendorConfig"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

echo "========================================="
echo "Uploading vendor configs to DynamoDB"
echo "Table: $TABLE_NAME"
echo "========================================="
echo ""

# Counter for statistics
total=0
success=0
already_exists=0
failed=0

# Iterate over all JSON files in subdirectories
for config_file in "$SCRIPT_DIR"/*/*.json; do
  # Check if any JSON files exist
  if [ ! -f "$config_file" ]; then
    echo -e "${YELLOW}No JSON files found in $SCRIPT_DIR subdirectories${NC}"
    exit 1
  fi

  filename=$(basename "$config_file")
  vendor_id=$(echo "$filename" | sed 's/.json$//')

  total=$((total + 1))

  echo -n "Uploading $filename ... "

  # Run the put-item command with condition expression
  if aws dynamodb put-item \
    --table-name "$TABLE_NAME" \
    --item "file://$config_file" \
    --condition-expression "attribute_not_exists(vendor_id)" \
    --no-cli-pager \
    2>&1 | grep -q "ConditionalCheckFailedException"; then

    echo -e "${YELLOW}ALREADY EXISTS${NC}"
    already_exists=$((already_exists + 1))

  elif [ ${PIPESTATUS[0]} -eq 0 ]; then
    echo -e "${GREEN}SUCCESS${NC}"
    success=$((success + 1))
  else
    echo -e "${RED}FAILED${NC}"
    failed=$((failed + 1))
  fi
done

echo ""
echo "========================================="
echo "Upload Summary"
echo "========================================="
echo "Total files:      $total"
echo -e "${GREEN}Successful:       $success${NC}"
echo -e "${YELLOW}Already existed:  $already_exists${NC}"
echo -e "${RED}Failed:           $failed${NC}"
echo "========================================="

# Exit with error code if any failed
if [ $failed -gt 0 ]; then
  exit 1
fi

The script uses a conditional write (attribute_not_exists(vendor_id)) so running it multiple times is safe — existing records are never silently overwritten. If you need to update an existing vendor’s configuration, use aws dynamodb update-item or delete and re-upload the record.

Real vendor examples

These are the current vendor configs in the repository.

Dominos (food-delivery)

{
  "vendor_id": {"S": "dominos"},
  "vendor_name": {"S": "Dominos"},
  "invoice_category": {"S": "retail"},
  "invoice_sub_type": {"S": "food-delivery"},
  "default_email_patterns": {"SS": [
    "[e-mail]"
  ]},
  "default_subject_keywords": {"SS": [
    "Tack for din beställning"
  ]},
  "parser_type": {"S": "html"},
  "active": {"BOOL": true},
  "supports_pdf": {"BOOL": false},
  "supports_html": {"BOOL": true},
  "logo_url": {"S": "https://paypulse-vendor-logos.s3.eu-west-1.amazonaws.com/svgs/dominos.svg"},
  "created_at": {"S": "2025-10-06T10:30:00Z"},
  "updated_at": {"S": "2025-10-06T10:30:00Z"}
}

Foodora (food-delivery)

{
  "vendor_id": {"S": "foodora"},
  "vendor_name": {"S": "Foodora"},
  "invoice_category": {"S": "retail"},
  "invoice_sub_type": {"S": "food-delivery"},
  "default_email_patterns": {"SS": [
    "[e-mail]"
  ]},
  "default_subject_keywords": {"SS": [
    "Orderbekräftelse"
  ]},
  "parser_type": {"S": "html"},
  "active": {"BOOL": true},
  "supports_pdf": {"BOOL": false},
  "supports_html": {"BOOL": true},
  "logo_url": {"S": "https://paypulse-vendor-logos.s3.eu-west-1.amazonaws.com/pngs/foodora.png"},
  "created_at": {"S": "2025-10-06T10:30:00Z"},
  "updated_at": {"S": "2025-10-06T10:30:00Z"}
}

Zalando (clothing)

{
  "vendor_id": {"S": "zalando"},
  "vendor_name": {"S": "Zalando"},
  "invoice_category": {"S": "retail"},
  "invoice_sub_type": {"S": "clothing"},
  "default_email_patterns": {"SS": [
    "[e-mail]"
  ]},
  "default_subject_keywords": {"SS": [
    "Thanks for your order"
  ]},
  "parser_type": {"S": "html"},
  "active": {"BOOL": true},
  "supports_pdf": {"BOOL": false},
  "supports_html": {"BOOL": true},
  "logo_url": {"S": "https://paypulse-vendor-logos.s3.eu-west-1.amazonaws.com/svgs/zalando.svg"},
  "created_at": {"S": "2025-10-06T10:30:00Z"},
  "updated_at": {"S": "2025-10-06T10:30:00Z"}
}

Jack & Jones (clothing)

{
  "vendor_id": {"S": "jack&jones"},
  "vendor_name": {"S": "Jack & Jones"},
  "invoice_category": {"S": "retail"},
  "invoice_sub_type": {"S": "clothing"},
  "default_email_patterns": {"SS": [
    "[e-mail]"
  ]},
  "default_subject_keywords": {"SS": [
    "Orderbekräftelse"
  ]},
  "parser_type": {"S": "html"},
  "active": {"BOOL": true},
  "supports_pdf": {"BOOL": false},
  "supports_html": {"BOOL": true},
  "logo_url": {"S": ""},
  "created_at": {"S": "2025-10-18T10:30:00Z"},
  "updated_at": {"S": "2025-10-18T10:30:00Z"}
}

Anthropic (subscriptions)

{
  "vendor_id": {"S": "anthropic"},
  "vendor_name": {"S": "Anthropic"},
  "invoice_category": {"S": "retail"},
  "invoice_sub_type": {"S": "subscriptions"},
  "default_email_patterns": {"SS": [
    "[e-mail]"
  ]},
  "default_subject_keywords": {"SS": [
    "Your receipt from Anthropic"
  ]},
  "parser_type": {"S": "html"},
  "active": {"BOOL": true},
  "supports_pdf": {"BOOL": false},
  "supports_html": {"BOOL": true},
  "logo_url": {"S": ""},
  "created_at": {"S": "2026-02-17T10:30:00Z"},
  "updated_at": {"S": "2026-02-17T10:30:00Z"}
}

Mevlana Moské (subscriptions)

{
  "vendor_id": {"S": "mevlana"},
  "vendor_name": {"S": "Mevlana Moské"},
  "invoice_category": {"S": "retail"},
  "invoice_sub_type": {"S": "subscriptions"},
  "default_email_patterns": {"SS": [
    "[e-mail]"
  ]},
  "default_subject_keywords": {"SS": [
    "Ditt kvitto från Mevlana Moské Göteborg"
  ]},
  "parser_type": {"S": "html"},
  "active": {"BOOL": true},
  "supports_pdf": {"BOOL": false},
  "supports_html": {"BOOL": true},
  "logo_url": {"S": ""},
  "created_at": {"S": "2026-02-08T10:30:00Z"},
  "updated_at": {"S": "2026-02-08T10:30:00Z"}
}

Notes and warnings

Set active to false to disable a vendor without deleting its record. The get_active_vendors() function filters on active = true, so the vendor will be excluded from all fetch runs until you re-enable it.

default_email_patterns and default_subject_keywords must be specific. A broad sender address like [e-mail] or a generic subject keyword like Order will match promotional and transactional emails that are not invoices, causing the parser to process irrelevant content and producing noise in the invoice store.