Q: Your team is running stateful database containers (PostgreSQL, MongoDB) on standalone AWS EC2 instances managed by Docker Compose. If an EC2 host suffers a hardware failure, an operator must provision a new instance, detach the AWS EBS volume from the dead host, reattach it to the new host, and mount it before starting the container. You need to implement an automated Docker storage plugin (using RexRay or Docker CSI) that dynamically attaches and mounts AWS EBS and EFS volumes on container startup.

Docker supports storage extensibility through managed volume plugins (V2 plugin architecture). When a container requests a named volume backed by a plugin, the plugin communicates with cloud provider APIs (e.g., AWS EC2/EBS APIs) to dynamically attach the cloud disk to the host and mount it to `/var/lib/docker/plugins/...`.

<!-- Volume Plugin Flow -->
docker run -v my-ebs-vol:/data db:latest
  └── Docker Volume Plugin (RexRay / AWS EBS Driver)
        ├── Calls AWS API: DescribeVolumes & AttachVolume (vol-0a1b2c -> i-09876)
        ├── Waits for /dev/xvdf to appear on Linux host
        ├── Formats (mkfs.ext4 if new) & mounts to host
        └── Passes mountpoint into container /data

Install the managed storage plugin using `docker plugin install`, granting necessary IAM permissions for AWS EBS and EC2 volume management.

# Install RexRay AWS EBS volume plugin
docker plugin install rexray/ebs \
  EBS_REGION=us-east-1 \
  EBS_MAXRETRIES=10 \
  --grant-all-permissions

# Verify active plugins
docker plugin ls

Create a persistent named volume backed by the cloud driver specifying volume size, IOPS, and volume type (`gp3`). Mount the volume into the database container.

# Create 100GB GP3 volume via Docker CLI
docker volume create -d rexray/ebs \
  --name prod-postgres-ebs \
  -o size=100 \
  -o volumeType=gp3 \
  -o iops=3000

# Run stateful container using the persistent cloud volume
docker run -d \
  --name db-primary \
  -v prod-postgres-ebs:/var/lib/postgresql/data \
  postgres:16

Simulate host failure. Terminate the primary EC2 host and launch the database container on a secondary EC2 host using the same volume name. The plugin preemptively detaches the volume from the dead node and attaches it to the new host.

# On Host B:
docker run -d \
  --name db-primary \
  -v prod-postgres-ebs:/var/lib/postgresql/data \
  postgres:16
# RexRay detects existing EBS volume, detaches from dead node, attaches to Host B!