Decoupling and messaging

Key points

• Standard vs FIFO: standard gives nearly unlimited throughput, at-least-once delivery and best-effort ordering. FIFO keeps order inside each MessageGroupId and drops duplicates sent within a 5-minute deduplication interval, at 300 API calls per second per action (3,000 messages per second with batches of 10), more with high throughput mode.
• Visibility timeout: a received message stays hidden for 30 seconds by default (0 seconds to 12 hours). If the consumer does not delete it in time, it reappears and is processed again, so set it above the processing time.
• Retention and size: messages are kept 4 days by default, from 1 minute to 14 days. A message is at most 1 MiB, larger payloads go to S3 through the Extended Client Library.
• Long polling: WaitTimeSeconds up to 20 seconds cuts empty responses and cost. Delay queues postpone delivery by up to 15 minutes.
• Dead-letter queue: after maxReceiveCount receives (1 to 1,000) a message moves to a DLQ of the same type, in the same account and Region. Standard queues keep the original enqueue timestamp, so give the DLQ a longer retention.
• Scaling workers: scale an Auto Scaling group on backlog per instance (ApproximateNumberOfMessages divided by in-service instances). Messages held by a terminated instance return to the queue.

Key points

• Subscribers: SQS queues, Lambda functions, Amazon Data Firehose and HTTP/S endpoints, plus email, SMS and mobile push for people.
• Fan-out: one topic with several SQS queues subscribed, each consumed by its own service. A slow consumer never delays the others and its messages wait in its queue.
• Message filtering: a filter policy on the subscription, applied to message attributes or to the message body, delivers only matching messages to that subscriber.
• FIFO topics: strict ordering and deduplication, delivered only to SQS FIFO or standard queues. Email, SMS, mobile push and HTTP/S cannot subscribe to a FIFO topic.

Key points

• Event bus and rules: a rule matches an event pattern and routes to one or more targets, optionally transforming the event first. Built for many sources to many targets.
• Pipes: point-to-point integration from one source to one target, with optional filtering and enrichment, e.g. an SQS queue enriched through an API destination and sent to Step Functions. You pay only for events that pass the filter.
• Scheduler: serverless cron, rate and one-time schedules with flexible time windows and retry limits.
• Archive and replay: keep events (indefinitely by default) and replay them to the source bus to recover from errors or test new consumers.

Key points

• Standard: up to 1 year, exactly-once execution, billed per state transition, execution history kept 90 days. Fits long and non-idempotent work such as payments.
• Express: up to 5 minutes, billed by executions, duration and memory. At-least-once when asynchronous, at-most-once when synchronous. Fits high-volume event processing and IoT ingestion.
• Callbacks and jobs: only Standard supports the .sync job-run and .waitForTaskToken callback patterns, e.g. waiting for a human approval.
• Fixed type: the workflow type cannot be changed after the state machine is created.

Key points

• Throttling: token bucket of 10,000 requests per second with a burst of 5,000 per account per Region in most Regions. Above it clients get 429 Too Many Requests. Stage, method and per-client usage plan limits sit below it.
• Caching: REST APIs only, per stage, TTL 300 seconds by default and up to 3,600. Only GET methods are cached by default.
• Integration timeout: 50 milliseconds to 29 seconds for REST APIs. Regional and private APIs can raise it, possibly with a lower throttle quota. Long jobs go asynchronous through SQS or Step Functions.
• Endpoint types: edge-optimized, Regional and private. Edge-optimized and private endpoints exist only for REST APIs.

Key points

• Protocols: JMS, NMS, AMQP 0.9.1, AMQP 1.0, STOMP, MQTT and WebSocket.
• High availability: ActiveMQ active/standby brokers replicated across Availability Zones, RabbitMQ 3-node clusters with quorum queues spread across AZs.
• Cross-Region: ActiveMQ cross-Region data replication copies messages asynchronously to a replica broker that can be promoted.
• New designs: for brand new cloud applications AWS recommends SQS and SNS instead.

Key points

• Shards: each shard takes up to 1 MB per second or 1,000 records per second of writes and serves up to 2 MB per second of reads. On-demand mode manages shards for you, provisioned mode bills per shard hour.
• Ordering: the partition key picks the shard, and each record gets a sequence number within it, so related records stay in order.
• Retention and replay: 24 hours by default, up to 365 days. Each application reads independently and can re-read, unlike SQS where a deleted message is gone.
• Record size: a data blob of up to 1 MB.

Key points

• Externalize session state: keep sessions in Amazon ElastiCache or DynamoDB so any instance can serve any request and scale-in loses nothing.
• Sticky sessions: an ALB can pin a client to one target, but a session that lives only on that instance is lost when the instance is terminated.
• Shared files: uploads go to S3 or Amazon EFS, never to the local disk of one instance.
• Horizontal over vertical: adding instances behind a load balancer scales without the ceiling and the stop of resizing a single instance.