Skip to main content

message

  • Progress: Done

Overview​

Categories​

  • Envelope (message_id, timestamp_utc, correlation_id): Written by MQ infrastructure layer. Mechanical fields ensuring safe, traceable delivery. Never modified after publishing
  • Payload (payment_reference, amount_cents, currency, validation_result): Core business data. Self-contained design (embedded validation) allows Service B to process without external calls to Service A
  • Control (retry_count, ttl_seconds): Broker mechanisms managing failure automatically without human intervention

Envelope Structure​

Components​

  • message_id: Unique UUID generated by Service A at creation. Primary field for broker tracking and Service B duplicate detection
    • If missing: Service B processes duplicates during network hiccups or Service A retries. Causes double-charges and compliance violations
  • timestamp_utc: Time Service A validated and published message. Always UTC. Serves as audit timestamp proving exact system acceptance time, independent of Service B processing time
    • If missing: Fails dispute resolution and regulatory audits regarding payment acceptance times
  • correlation_id: Links individual payment message to parent bulk file and batch (e.g., "ANZ batch 0042"). Enables querying, tracing, debugging, and partial reprocessing
    • If missing: Partial batch failures require blind reprocessing

Payload Structure​

Components​

  • payment_reference: Corporation internal transaction ID. Bridging identifier between internal system and corporation records for immediate lookup
    • If missing: Support queries cannot be correlated to corporation statements
  • amount_cents: Amount in smallest currency unit (integers, never floats). Eliminates IEEE 754 floating-point rounding bugs (e.g., $1,500.00 = 150000 cents). Expected core banking format
    • If missing/float: Rounding errors accumulate across thousands of payments. Reconciliation fails; triggers audit flags
  • currency: Explicit ISO 4217 currency code. Differentiates identical integer values across currencies (e.g., 150000 cents in AUD vs. USD)
    • If missing: Currency assumed from context. USD processed as AUD creates massive discrepancy and SWIFT violation
  • validation_result: Proof Service A validated payment. Establishes contract: Service A owns validation, Service B owns transfer
    • If missing: Service B duplicates validation (wasteful) or skips completely (dangerous)

Control Structure​

Components​

  • retry_count: Number of delivery retries. Starts at 0. Increments when Service B fails and broker re-delivers. Stops at threshold (e.g., 3) and routes to Dead Letter Queue (DLQ)
    • If missing: Permanently failing payments (e.g., closed accounts) retry infinitely, blocking queues and consuming CPU
  • ttl_seconds: Time-to-live validity window (e.g., 86400 = 24 hours). Unconsumed messages expire to DLQ. Critical for same-day payment rails where delayed processing is legally invalid
    • If missing: Expired payments resurface during batch replays months later, causing unexplained mystery transfers

Message Lifecycle​

Flow​

  • Published: Service A validates payment, builds JSON, generates UUID, sets retry_count=0 and ttl=86400. Publishes to broker
  • Queued: Broker writes to disk, sends ACK to Service A. Service A freed. Message sits in durable queue (safe even if core banking offline)
  • In-Flight: Service B comes online and pulls message. Broker marks message "in-flight" (not deleted yet, awaiting success confirmation)
  • Success → ACK: Service B sends to core banking. Core banking confirms transfer. Service B sends ACK to broker. Broker permanently deletes message
  • Failure → NACK → DLQ: Service B crashes or core banking rejects. No ACK sent. Broker re-delivers after timeout (retry_count increments). At retry limit, routes to DLQ for manual review

Idempotency​

Best practices​

  • Scenario (Crash): Service B processes payment and core banking succeeds, but Service B crashes before sending ACK. Broker re-delivers message_id. Without idempotency → Double transfer ($1,500 sent twice). Guaranteed to happen under banking load
  • Scenario (Race Condition): Network glitch causes broker to assume Service B timed out. Broker delivers to second Service B instance. Without idempotency → Concurrent double transfer
  • The Fix (Idempotent Consumer): Service B maintains processed_messages table keyed on message_id. Before processing, Service B checks if message_id exists
    • If YES: Send ACK and skip
    • Result: Exactly-once processing. Delivering same message multiple times yields exact same outcome as delivering once

Broker Payload​

Examples​

  • JSON represents literal data sitting on broker's disk waiting for reliable delivery and ACK
{
  "message_id": "pmsg_7f3a91bc-4e2d-4c8a-b1f0-9d3e2a1c5b7d",
  "timestamp_utc": "2025-05-01T02:14:33.421Z",
  "correlation_id": "bulk_ANZ_20250501_batch_0042",

  "payment_reference": "PAY-ANZ-2025-00198432",
  "amount_cents": 150000,
  "currency": "AUD",
  "debtor_bsb": "012-345",
  "debtor_account": "123456789",
  "creditor_bsb": "033-001",
  "creditor_account": "987654321",
  "validation_result": {
    "passed": true,
    "checks": ["customer_exists", "bsb_valid", "limit_ok"]
  },

  "retry_count": 0,
  "ttl_seconds": 86400
}
Last updated on