Storage performance
Choosing storage services and configurations that meet performance demands and scale with growth. This is task 3.1 of Design High-Performing Architectures, the domain worth 24% of the scored content.
The first decision is the storage type. It defines how data is addressed, who can share it and how far it scales.
Key points
- Block (Amazon EBS, instance store): fixed-size blocks attached to an instance, with low latency and high IOPS. Fits boot volumes, databases and transactional workloads.
- File (Amazon EFS, Amazon FSx): a hierarchical file system shared by many clients at once over NFS or SMB.
- Object (Amazon S3): objects with metadata in a flat namespace, accessed through HTTPS APIs, with near-unlimited scale.
- Hybrid (AWS Storage Gateway): on-premises applications reach cloud storage through file, volume or tape interfaces.
On the exam
"Shared file system for many Linux instances" -> EFS, not EBS. "Lowest latency disk for one database instance" -> EBS (or instance store if the data is temporary). "Millions of images or a data lake" -> S3.
S3 performance
Amazon S3 performance
Amazon S3 scales automatically, but request rates are counted per prefix and large objects move faster when split into parallel requests.
Key points
- Prefixes: at least 3,500
PUT/COPY/POST/DELETEand 5,500GET/HEADrequests per second per partitioned prefix, with no limit on the number of prefixes. 10 prefixes can reach 55,000 reads per second. While S3 scales up, it can answer503 Slow Down. - Multipart upload: recommended from 100 MB and required above 5 GB, the limit of a single
PUT. Parts of 5 MiB to 5 GiB, up to 10,000 parts, uploaded in parallel and retried one by one. Objects reach 48.8 TiB. - Transfer Acceleration: uploads enter the nearest CloudFront edge location and travel to the bucket over an optimized network path. Uses the
bucket.s3-accelerate.amazonaws.comendpoint, needs a DNS-compliant bucket name without periods and can add transfer charges. - Byte-range fetches:
GETrequests with theRangeheader read parts of one object over concurrent connections, which raises aggregate throughput and makes retries cheaper. - Locality and caching: read the bucket from EC2 in the same Region, and put CloudFront or ElastiCache in front of S3 for repeated reads that need lower latency.
On the exam
"Large uploads from offices on other continents are slow" -> Transfer Acceleration plus multipart upload. "503 errors on a hot prefix" -> spread keys across more prefixes. "Read only a part of a huge file" -> byte-range fetch.
Every recommended class is designed for 99.999999999% (11 nines) durability. They differ in availability, number of AZs, minimum storage duration and retrieval time.
Key points
- Minimum duration: 30 days for Standard-IA and One Zone-IA, 90 days for Glacier Instant Retrieval and Glacier Flexible Retrieval, 180 days for Glacier Deep Archive. Deleting earlier still bills the remaining days.
- Minimum billable size: 128 KB for Standard-IA, One Zone-IA and Glacier Instant Retrieval. Intelligent-Tiering does not monitor objects under 128 KB.
- Single AZ: One Zone-IA and Express One Zone are not resilient to the loss of the Availability Zone. Keep only data you can re-create there.
- Archive retrieval: Glacier Flexible Retrieval restores in 1 to 5 minutes (Expedited), 3 to 5 hours (Standard) or 5 to 12 hours (Bulk, free). Deep Archive restores within 12 hours (Standard) or 48 hours (Bulk).
- Lifecycle rules: transition objects to cheaper classes as they age, expire them, and abort incomplete multipart uploads.
On the exam
"Unknown or changing access pattern" -> Intelligent-Tiering. "Read once a quarter but needs milliseconds" -> Glacier Instant Retrieval. "Compliance archive, 48 hours to retrieve is fine, lowest cost" -> Glacier Deep Archive.
S3 storage classes
| Class | Profile |
|---|---|
| Standard | Frequent access in milliseconds. 99.99% availability, 3 or more AZs, no minimum duration. |
| Intelligent-Tiering | Unknown or changing patterns. Moves objects to Infrequent Access after 30 days and Archive Instant Access after 90 days without access. Monitoring fee, no retrieval fee. |
| Express One Zone | Single-digit millisecond access in one AZ you choose, in directory buckets. Up to 10x faster than Standard with request costs 50% lower. |
| Standard-IA | Accessed about once a month, in milliseconds. 99.9% availability, per-GB retrieval fee. |
| One Zone-IA | Infrequent, re-creatable data in 1 AZ. 99.5% availability. |
| Glacier Instant Retrieval | Archive read about once a quarter, still in milliseconds. |
| Glacier Flexible Retrieval | Archive read once or twice a year, restored in minutes to 12 hours. Fits backup and DR. |
| Glacier Deep Archive | Lowest cost in AWS, read less than once a year, restored in 12 to 48 hours. |
EBS volume types
Amazon EBS
SSD volumes are tuned for IOPS on small random I/O. HDD volumes are tuned for throughput on large sequential I/O.
Key points
- gp3: 3,000 IOPS and 125 MiB/s included at any size. IOPS (up to 500 per GiB) and throughput are provisioned independently of size, up to 80,000 IOPS and 2,000 MiB/s, at a price per GiB 20% lower than gp2.
- gp2: 3 IOPS per GiB, from 100 to 16,000. Volumes under 1 TiB burst to 3,000 IOPS with I/O credits. Performance only grows with size.
- io2 Block Express: up to 256,000 IOPS, 4,000 MiB/s and 64 TiB, with 99.999% durability and average latency under 500 microseconds. For critical databases that need sustained IOPS.
- HDD (st1, sc1): 125 GiB to 16 TiB and never a boot volume. st1 for big data, data warehouses and logs, sc1 for the cheapest storage of rarely read data.
- Elastic Volumes: change the type, IOPS and throughput of a volume in use without interrupting the instance, such as gp2 to gp3.
- Instance side: use EBS-optimized instances to reach provisioned IOPS. Volumes above 64,000 IOPS need Nitro-based instances.
On the exam
"More IOPS without paying for more GiB" -> gp3 (gp2 ties IOPS to size). "More than 80,000 IOPS or sub-millisecond latency" -> io2 Block Express. "Cheapest EBS for sequential logs" -> st1, or sc1 if rarely read.
EBS volume types
| Type | Limits and fit |
|---|---|
| gp3 | 1 GiB to 64 TiB, 3,000 to 80,000 IOPS, 125 to 2,000 MiB/s. Default for boot volumes and most workloads. |
| gp2 | 1 GiB to 16 TiB, up to 16,000 IOPS (3 per GiB), up to 250 MiB/s. Burst credits. |
| io2 Block Express | 4 GiB to 64 TiB, up to 256,000 IOPS and 4,000 MiB/s, 99.999% durability. Multi-Attach. |
| io1 | 4 GiB to 16 TiB, up to 64,000 IOPS and 1,000 MiB/s. Multi-Attach. |
| st1 | 125 GiB to 16 TiB, up to 500 MiB/s. Big data, warehouses, logs. Not bootable. |
| sc1 | 125 GiB to 16 TiB, up to 250 MiB/s. Lowest cost cold data. Not bootable. |
Two block options outside the usual one durable volume per instance: a shared Provisioned IOPS volume, and fast local disks that live and die with the instance.
Key points
- Multi-Attach: one io1 or io2 volume attached to up to 16 Nitro-based instances in the same Availability Zone, all with read and write access. No extra charge.
- Cluster-aware writes: XFS and ext4 are not built for simultaneous writers. Use a clustered file system or an application that orders writes. Multi-Attach volumes cannot be boot volumes.
- Instance store: disks physically attached to the host, included in the instance price. Ideal for buffers, caches, scratch data and data replicated across a fleet.
- Ephemeral data: it survives a reboot but is lost on stop, hibernation, termination or disk failure. It can only be added at launch and never detached.
On the exam
"Clustered application in one AZ needs a shared block device" -> EBS Multi-Attach with io2. "Highest disk I/O for temporary data, loss is acceptable" -> instance store. Data that must survive a stop belongs on EBS.
EFS
Amazon Elastic File System
Serverless NFS file system that grows to petabytes on its own and is shared by EC2, ECS, EKS, Fargate and Lambda. Windows instances are not supported.
Key points
- Protocol: NFSv4.0 and NFSv4.1 with POSIX permissions. For SMB shares on Windows, use FSx for Windows File Server.
- File system type: Regional stores data across 3 or more AZs (recommended). One Zone keeps it in 1 AZ, cheaper but exposed to the loss of that AZ.
- Performance mode: General Purpose (default, lowest latency, recommended). Max I/O is a previous generation with higher latency, unavailable with Elastic throughput or One Zone.
- Throughput mode: Elastic (default) scales automatically for spiky or unpredictable loads. Provisioned fixes throughput for known needs. Bursting scales with the data stored in Standard.
- Storage classes: Standard (SSD, sub-millisecond reads), Infrequent Access and Archive (tens of milliseconds, 90-day minimum for Archive). Lifecycle management moves files between them.
- Replication: keeps a copy in another Region with RPO and RTO measured in minutes.
On the exam
"Hundreds of Linux instances across AZs share the same files" -> EFS Regional. "File system throttled in Bursting mode" -> switch to Elastic or Provisioned throughput. "Windows file shares" is never EFS.
FSx
Amazon FSx
Fully managed file systems built on popular commercial and open source technologies. Pick the one that matches the protocol or the file system already in use.
Key points
- FSx for Windows File Server: native Windows file system over SMB, with Microsoft Active Directory authentication and Windows ACLs. Single-AZ or Multi-AZ with a standby file server, SSD or HDD storage.
- FSx for Lustre: POSIX parallel file system for HPC, machine learning, video processing and financial modeling. Links to an S3 bucket, presents its objects as files and writes results back. Scratch (not replicated) or persistent deployments, Linux clients.
- FSx for NetApp ONTAP: multiprotocol NFS, SMB, iSCSI and NVMe access from Linux, Windows and macOS. Snapshots, clones, SnapMirror replication, compression, deduplication and automatic tiering of cold data.
- FSx for OpenZFS: NFS access for moving ZFS or Linux file servers, with latencies of hundreds of microseconds, snapshots, clones and compression. Multi-AZ or Single-AZ deployments.
On the exam
"Windows shares joined to Active Directory" -> FSx for Windows File Server. "HPC job over a dataset stored in S3" -> FSx for Lustre. "Move a NetApp NAS without changing tools" -> FSx for NetApp ONTAP. "Lift a ZFS server" -> FSx for OpenZFS.
Storage Gateway
AWS Storage Gateway
A software or hardware appliance on premises that gives local applications standard storage interfaces backed by AWS, with local caching for low latency.
Key points
- S3 File Gateway: NFS (v3, v4.1) and SMB file shares whose files are stored as objects in S3, cached locally. The data is then available to S3 lifecycle, replication and analytics.
- Volume Gateway: iSCSI block volumes. Cached volumes keep data in S3 and only frequent data on site. Stored volumes keep the full dataset on site and back it up as asynchronous point-in-time snapshots.
- Tape Gateway: a virtual tape library for existing backup software, archiving tapes to S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive.
- FSx File Gateway: no longer available to new customers. New designs reach FSx for Windows File Server directly over Direct Connect or Site-to-Site VPN.
- Deployment: VM on VMware ESXi, Microsoft Hyper-V or Linux KVM, a hardware appliance, or an EC2 instance.
On the exam
"Replace physical tapes without changing the backup software" -> Tape Gateway. "On-premises app writes files over NFS or SMB that must land in S3" -> S3 File Gateway. "Whole dataset on site with low latency, backups in AWS" -> Volume Gateway stored volumes.