Q: A production RAG platform needs to search over 200 million 1536-dimensional vectors with sub-25ms p99 latency. How do you architect a distributed Milvus or Qdrant cluster on Kubernetes, and how do you choose between HNSW and IVF-PQ indexing to balance RAM costs and recall accuracy?

Deploy Milvus in distributed microservices mode via Milvus Operator: separate QueryNodes (stateless vector search execution), DataNodes (raw data ingestion and segment writing), IndexNodes (background index computation), and Coordinator nodes (RootCoord, QueryCoord). Use Apache Pulsar / Kafka as the WAL log broker and Amazon S3 as the persistent object storage backend.

apiVersion: milvus.io/v1beta1
kind: Milvus
metadata:
  name: milvus-production
spec:
  mode: cluster
  dependencies:
    etcd: { endpoints: ["etcd-cluster:2379"] }
    storage: { type: "s3", bucket: "milvus-vectors" }
    pulsar: { endpoint: "pulsar-proxy:6650" }

HNSW (Hierarchical Navigable Small World) provides superior recall (>98%) and sub-10ms search latency, but constructs extensive memory-resident link graphs requiring 1.5x to 2x raw vector size in RAM (costing ~2.5TB RAM for 200M vectors). In contrast, IVF-PQ (Inverted File with Product Quantization) compresses vectors into quantized centroids and codes, reducing memory by 85-90% at the cost of 5-10% lower recall. For high-scale deployments, use Scalar Quantization (SQ8) or HNSW with on-disk vectors (DiskANN / Milvus Knowhere) backed by local NVMe SSDs.

# Index creation parameters for HNSW with SQ8 quantization
index_params = {
    "metric_type": "COSINE",
    "index_type": "HNSW",
    "params": {"M": 16, "efConstruction": 200}
}

Configure Milvus data segment size to 512MB to balance compaction overhead and search parallelism. Horizontally scale stateless QueryNodes using Kubernetes HPA based on CPU and query latency metrics. QueryCoord automatically distributes segment replicas across QueryNodes, enabling linear search throughput scaling.

# QueryNode autoscaling policy
spec:
  components:
    queryNode:
      replicas: 12
      resources:
        requests: { cpu: "16", memory: "64Gi" }