Event flow

How events cascade through the PayPulse Cloud pipeline.

PayPulse Cloud has three distinct event pipelines. Two involve rental invoices and one handles retail invoices.

Automated rental invoice pipeline

This pipeline runs fully automatically every weekday morning without any user interaction.

1. EventBridge fires the scheduled rule

   An EventBridge rule runs on a cron schedule every weekday at 08:30 UTC. It invokes the fetch_latest_invoice Lambda function.

2. fetch_latest_invoice searches Gmail

   fetch_latest_invoice retrieves the user’s OAuth 2.0 tokens from AWS Secrets Manager, creates a Gmail API client, and searches the inbox for the current month’s rental invoice email. If a matching invoice is found and no record exists in the RentalInvoices DynamoDB table for that month, the PDF attachment is uploaded to S3.

   s3://rental-invoices-bucket/invoices/{user_id}/rental/{filename}.pdf
   
3. S3 triggers parse_invoice

   The s3:ObjectCreated:* event on the rental-invoices-bucket (prefix invoices/, suffix .pdf) automatically invokes the parse_invoice Lambda function with the bucket name and object key.

4. parse_invoice parses the PDF and writes to DynamoDB

   parse_invoice downloads the PDF from S3, extracts the user ID from the object key path (invoices/{user_id}/rental/...), and calls HyresaviParser to extract structured invoice data. The parsed record — including InvoiceID, UserID, total amount, and due date — is written to the RentalInvoices DynamoDB table.

   # Invoice ID is derived from the filename
   invoice_id = "Invoice_" + filename.split('/')[-1].split('.')[0].split('_')[-1]
   create_invoice_in_dynamodb(table, invoice_id, user_id, parsed_data)
   
5. DynamoDB stream triggers send_invoice_notification

   The RentalInvoices table has stream_view_type = NEW_AND_OLD_IMAGES. When a new record is inserted, the stream event source mapping delivers the record to send_invoice_notification with batch_size = 1 and starting_position = LATEST.

6. send_invoice_notification publishes to SNS

   send_invoice_notification reads the INSERT record from the stream, extracts Due Date and Total Amount from the new image, and publishes a message to the NewInvoiceTopicNotification SNS topic.

   if record['eventName'] == 'INSERT':
       message = f"New rental invoice of {amount} SEK with due date of {due_date} is now available!"
       sns_client.publish(TopicArn=sns_topic_arn, Message=message, Subject="New invoice available!")
   
7. SNS delivers to email and iOS

   SNS fans out the notification to its subscriptions: an email address and an iOS push notification endpoint (registered via Cognito).

Manual rental invoice ingestion

This flow is triggered by the iOS app when the user explicitly requests a full sync of all rental invoices.

1. iOS app calls the ingest endpoint

   The app sends a JWT-authenticated request:

   POST /v1/invoices/rental/ingest
   Authorization: Bearer <jwt_token>
   
2. API Gateway routes to fetch_invoices

   API Gateway forwards the request to the fetch_invoices Lambda function via AWS_PROXY integration.

3. fetch_invoices queries Gmail and uploads PDFs

   fetch_invoices authenticates with the Gmail API using stored OAuth tokens, searches for all rental invoice emails matching the configured sender and subject, and for each email whose invoice month is not yet in DynamoDB, downloads the PDF attachment and uploads it to S3.

   s3://rental-invoices-bucket/invoices/{user_id}/rental/{filename}.pdf
   

   Each S3 upload triggers the automated pipeline from step 3 onward (S3 → parse_invoice → DynamoDB → stream → notification).

4. Response returned to iOS app

   fetch_invoices returns the number of new invoices ingested, or a message indicating none were found.

   { "message": "Rental invoices ingested successfully!", "data": { "invoiceCount": 3 } }
   

The S3 upload in fetch_invoices chains into the same parse → notification pipeline as the automated flow. Each uploaded PDF fires a separate s3:ObjectCreated event.

Retail invoice ingestion

Retail invoice ingestion is triggered either by the iOS app on demand or by a weekly EventBridge schedule.

1. Trigger: API call or EventBridge schedule

   The iOS app can trigger ingestion on demand:

   POST /v1/invoices/retail/ingest
   Authorization: Bearer <jwt_token>
   

   An optional request body can specify a custom date range. If omitted, fetch_retail_invoices uses the last_retail_invoice_fetch timestamp stored in the Users table for incremental fetching.

   Separately, a weekly EventBridge rule invokes fetch_retail_invoices automatically.

2. fetch_retail_invoices reads VendorConfig

   The function queries the VendorConfig DynamoDB table for all active vendor entries. Each entry contains email_patterns and subject_keywords used to construct Gmail API search queries.

3. Gmail API search per vendor

   For each active vendor, fetch_retail_invoices authenticates with the Gmail API and runs a targeted search. Duplicate detection prevents re-processing emails that have already been ingested.

4. HTML attachments uploaded to S3

   Matching invoice HTML files are uploaded to S3 under the vendor’s category path:

   s3://rental-invoices-bucket/invoices/{user_id}/retail/{sub_type}/{vendor}_{date}_{hash}.html
   

   The S3 s3:ObjectCreated:* event on .html files triggers the parse_retail_invoice Lambda function.

5. last_retail_invoice_fetch updated

   After processing, fetch_retail_invoices updates the last_retail_invoice_fetch field in the Users table so the next run only fetches newer invoices.

Retail invoice parsing (parse_retail_invoice) is triggered by S3 HTML uploads in the same way that parse_invoice is triggered by PDF uploads. HTML parsers per sub-type are planned — see the project roadmap in the README.