Q: Your engineering organization spends $180,000/month on Datadog, and the CFO demands an architectural review of migrating to an open-source observability stack (Prometheus, Grafana, OpenTelemetry, Thanos/Mimir). How do you evaluate the technical, operational, and financial trade-offs in a senior staff interview?

Datadog bills via complex multi-vector pricing: per-host agent licenses ($15-$23/host), ingested log volume ($0.10/GB), indexed log retention ($1.70-$2.50/million logs), and critically, custom metrics ($5 per 100 custom metrics). In microservice and Kubernetes environments, appending high-cardinality labels (e.g., `user_id`, `order_id`, `container_id`) can unintentionally generate hundreds of thousands of metric combinations, causing month-end bill shock. Self-hosted Prometheus/Mimir/Thanos shifts costs to cloud infrastructure (S3 object storage for block chunks, stateless query/ingester nodes, EBS volumes). Storage in S3 costs ~$0.023/GB/month, yielding 60% to 80% raw vendor savings at scale.

# Datadog bill shock scenario: High-cardinality metric explosion
# 100 endpoints * 5 status codes * 10,000 user IDs = 5,000,000 metric series!
# Cost: 50,000 * $5 = $250,000/month unexpected bill!

# OpenTelemetry Metric Filter: Drop high-cardinality attributes before export
processors:
  filter/drop_cardinality:
    metrics:
      exclude:
        match_type: strict
        metric_names:
          - http_requests_total
        attributes:
          - key: user_id

While open-source software licenses are free, running a petabyte-scale observability platform requires dedicated platform/SRE engineering time. Managing a highly available Thanos or Grafana Mimir cluster requires maintaining etcd/consul rings, compactor operations, memcached caching layers, and tuning query-frontend sharding. A team of 2-3 dedicated observability engineers costs $400k-$700k/year in salary and benefits. Datadog offloads operational maintenance entirely to the vendor: agent deployment is a single Helm chart or DaemonSet, upgrades are automated, and uptime SLAs are backed by contract.

# Datadog simple single-agent deployment via Helm
helm install datadog-agent datadog/datadog \
  --set datadog.apiKey=$DD_API_KEY \
  --set datadog.appKey=$DD_APP_KEY \
  --set datadog.logs.enabled=true \
  --set datadog.apm.portEnabled=true

Datadog provides seamless correlation out of the box: clicking a latency spike in an APM trace automatically correlates to surrounding pod logs, host CPU/memory graphs, and container network metrics without manual schema alignment. The open-source ecosystem achieves this via OpenTelemetry (OTel), Grafana Tempo (traces), Loki (logs), and Prometheus/Mimir (metrics). While Grafana Labs has unified these tools dramatically via exemplars and trace-to-log links, the engineering team must design, enforce, and maintain consistent semantic conventions across all backend services.

# OpenTelemetry SDK auto-injecting TraceID and SpanID into application logs
import logging
from opentelemetry import trace

logger = logging.getLogger(__name__)

def process_order(order_id):
    current_span = trace.get_current_span()
    trace_id = trace.format_trace_id(current_span.get_span_context().trace_id)
    logger.info("Processing order", extra={"trace_id": trace_id, "order_id": order_id})

Leading enterprise platform teams avoid vendor lock-in by standardizing application instrumentation exclusively on OpenTelemetry (OTel). The OpenTelemetry Collector runs as a cluster gateway, evaluating and filtering telemetry. High-volume, low-criticality debug logs and metric streams are directed to cheap S3-backed Prometheus/Loki, while high-value production error traces and executive SLO metrics are routed to Datadog, cutting SaaS costs by 50% without losing turnkey APM features.