> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/azfar-imtiaz/PayPulse-Cloud/llms.txt
> Use this file to discover all available pages before exploring further.
# Vendor configuration overview
> How vendor-driven retail invoice fetching works in PayPulse Cloud.
Retail invoice fetching in PayPulse Cloud is driven entirely by configuration stored in the `VendorConfig` DynamoDB table. Adding a new vendor — or disabling an existing one — requires no Lambda code changes.
## The VendorConfig table
The `VendorConfig` table is defined in `aws-infra-terraform/dynamodb.tf` with `vendor_id` as its partition key and PAY\_PER\_REQUEST billing.
| Attribute | DynamoDB type | Description |
| -------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------- |
| `vendor_id` | String (PK) | Unique identifier for the vendor (e.g. `dominos`, `zalando`). Used as the S3 path segment and DynamoDB key. |
| `vendor_name` | String | Human-readable display name (e.g. `Dominos`, `Jack & Jones`). |
| `invoice_category` | String | Always `retail` for vendors in this table. |
| `invoice_sub_type` | String | One of the 8 retail categories (see below). Determines the S3 sub-path. |
| `default_email_patterns` | String Set | One or more sender email addresses to match (e.g. `domino@dominos.se`). |
| `default_subject_keywords` | String Set | One or more subject-line keywords to filter emails by (e.g. `Tack for din beställning`). |
| `parser_type` | String | Format of the invoice content — currently `html` for all vendors. |
| `active` | Boolean | When `true`, this vendor is included in every fetch run. Set to `false` to disable without deleting the record. |
| `supports_pdf` | Boolean | Whether this vendor sends PDF attachments. |
| `supports_html` | Boolean | Whether this vendor sends HTML email bodies. |
| `logo_url` | String | URL to the vendor logo asset in S3. May be empty. |
| `created_at` | String | ISO 8601 creation timestamp. |
| `updated_at` | String | ISO 8601 last-update timestamp. |
### Example record
The Zalando vendor config from `vendor_configs/clothing/zalando.json`:
```json theme={null}
{
"vendor_id": {"S": "zalando"},
"vendor_name": {"S": "Zalando"},
"invoice_category": {"S": "retail"},
"invoice_sub_type": {"S": "clothing"},
"default_email_patterns": {"SS": [
"info@service-mail.zalando.se"
]},
"default_subject_keywords": {"SS": [
"Thanks for your order"
]},
"parser_type": {"S": "html"},
"active": {"BOOL": true},
"supports_pdf": {"BOOL": false},
"supports_html": {"BOOL": true},
"logo_url": {"S": "https://paypulse-vendor-logos.s3.eu-west-1.amazonaws.com/svgs/zalando.svg"},
"created_at": {"S": "2025-10-06T10:30:00Z"},
"updated_at": {"S": "2025-10-06T10:30:00Z"}
}
```
## The 8 retail categories
Every vendor maps to one `invoice_sub_type`. This value determines the S3 path segment under `invoices/{user_id}/retail/` where the HTML email is stored, and which DynamoDB detail table the parsed invoice is written to.
Restaurant and food delivery orders (e.g. Dominos, Foodora).
Fashion and apparel purchases (e.g. Zalando, Jack & Jones, Fotproffsen).
Electronics and tech products.
Streaming services, SaaS receipts, memberships (e.g. Anthropic, Mevlana Moské).
Supermarket and grocery store purchases.
Electricity, water, internet, and other utility bills.
Transportation invoices — flights, trains, buses.
Other retail purchases that don't fit a named category.
Each sub-type has a dedicated DynamoDB detail table (`FoodDeliveryInvoices`, `ClothingInvoices`, `TechnologyInvoices`, etc.) in addition to the shared `RetailInvoices` base table.
## How fetch\_retail\_invoices reads vendor configs
The `fetch_retail_invoices` Lambda (`lambdas/invoices/fetch_retail_invoices/lambda_function.py`) runs the following flow on every `POST /v1/invoices/retail/ingest` call:
The Lambda calls `get_active_vendors(vendor_config_table)` to scan the `VendorConfig` table and return only records where `active = true`. An optional `vendor_category` filter in the request body narrows this to a single sub-type.
`determine_search_date_range()` checks the `last_retail_invoice_fetch` field in the `Users` table for the authenticated user.
* **First fetch**: searches the last 30 days.
* **Subsequent fetches**: searches from `last_retail_invoice_fetch` to now.
* **Custom range**: if `start_date` and `end_date` are provided in the request body (format `YYYY-MM-DD`), those take precedence.
For each vendor, `build_gmail_query()` combines:
* **Sender filter**: `from:(domino@dominos.se)` — one `OR`-joined entry per address in `default_email_patterns`.
* **Subject filter**: `subject:(Tack for din beställning)` — one `OR`-joined entry per keyword in `default_subject_keywords`.
* **Date filter**: `after:YYYY/MM/DD before:YYYY/MM/DD`.
If `default_email_patterns` is empty for a vendor, that vendor is skipped with a warning.
The Lambda calls the Gmail API `messages.list` endpoint with the generated query, capped at 100 results per vendor to stay within the Lambda timeout. For each matching message, it fetches the full email content.
Before uploading, the Lambda generates a deterministic S3 key (`generate_retail_invoice_s3_key`) from `user_id`, `vendor_id`, `sub_type`, email date, and Gmail message ID. It then calls `s3_file_exists()` — if the object already exists, the email is skipped. This prevents re-processing the same invoice on repeated fetch calls.
New emails are extracted as HTML (`extract_html_from_email`) and uploaded to:
```
{bucket}/invoices/{user_id}/retail/{sub_type}/{vendor_id}_{date}_{message_id_hash}.html
```
After processing all vendors, `update_last_retail_invoice_fetch()` writes the current UTC timestamp back to the `Users` table so the next call fetches only new emails.
A 100 ms sleep between individual emails and a 200 ms sleep between vendors prevents Gmail API rate-limit errors.
## Incremental fetching
The `last_retail_invoice_fetch` field in the `Users` table is an ISO 8601 timestamp (e.g. `2025-10-06T10:30:00Z`). On each successful run it is updated to the current UTC time, so subsequent calls only search the window between the last fetch and now.
If the stored timestamp cannot be parsed, the Lambda falls back to searching the last 30 days and logs a warning.
## Duplicate detection
Duplicate detection happens at the S3 layer before any upload. The S3 key encodes the Gmail message ID, so the same email always produces the same key. If `s3:HeadObject` confirms the object exists, the email is skipped without triggering the downstream parser. This means calling `POST /v1/invoices/retail/ingest` multiple times is safe — only genuinely new emails are processed.
## Currently configured vendors
| Vendor | Sub-type | Sender |
| ------------- | ------------- | --------------------------------------- |
| Dominos | food-delivery | `domino@dominos.se` |
| Foodora | food-delivery | `info@mail.foodora.se` |
| Zalando | clothing | `info@service-mail.zalando.se` |
| Fotproffsen | clothing | `support@fotproffsen.se` |
| Jack & Jones | clothing | `noreply@jackjones.com` |
| Anthropic | subscriptions | `invoice+statements@mail.anthropic.com` |
| Mevlana Moské | subscriptions | `invoice+statements@mevlanagoteborg.se` |