⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Observability & Monitoring Interview Questions Scenario 97 of 116 in Observability & Monitoring
Mid-Level DevOps / SRE (2–4 Years) Observability Grafana & Observability Core Architecture
🎯 Target Role / Context: Observability Domain · Grafana Interview Questions (2–4 Years DevOps Engineer)

Q: How does Grafana architecture work in production? Walk through the core backend components, frontend rendering, SQL metadata storage, and plugin communication pipeline.

Grafana architecture separates presentation from telemetry storage, using a Go backend for auth and proxying, a React SPA frontend, and an external SQL database for metadata.

#Grafana #Observability #Architecture #DevOps #Monitoring #2-4 Years
🎙️ Candidate Opening & Architectural Context
"Grafana architecture operates as a decoupled, stateless visualization engine that does not store raw metric or log data. The platform functions as an intelligent query proxy and rendering pipeline between client browsers and backend telemetry databases. Understanding production Grafana architecture requires examining four distinct layers: a Go-based HTTP backend, an external SQL state repository, a TypeScript/React frontend, and a pluggable data source driver system."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? The Linux Foundation's Prometheus Certified Associate (PCA) & Monitoring Labs covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: Grafana architecture operates as a decoupled, stateless visualization engine that does not store raw metric or log data. The platform functions as an intelligent query proxy and rendering pipeline between client browsers and backend telemetry databases. Understanding production Grafana architecture requires examining four distinct layers: a Go-based HTTP backend, an external SQL state repository, a TypeScript/React frontend, and a pluggable data source driver system.

1️⃣

Core Grafana Architecture & Backend Pipeline

The server backend is written in Go (grafana-server) and handles user authentication, data source connection management, and Unified Alerting evaluation.

# Inspect running Grafana server process and loaded plugins
ps aux | grep grafana-server
grafana-cli plugins ls
  • HTTP Request Routing: The backend exposes a REST API that authenticates sessions via JWT, OAuth2, or API tokens before processing visualization requests.
  • Stateless Data Source Proxy: When a user opens a dashboard, queries flow through the backend proxy. This secures database credentials inside Grafana instead of exposing raw TSDB endpoints to client browsers.
  • Plugin Engine: Core and community plugins run as separate sub-processes communicating over gRPC, isolating plugin crashes from the core server process.
2️⃣

SQL Metadata Database Architecture

Grafana requires a relational database exclusively for internal state, dashboard JSON definitions, user permissions, and alert rules.

# Example production grafana.ini database configuration
[database]
type = postgres
host = postgres-ha.internal:5432
name = grafana_prod
user = grafana_admin
password = ${DB_PASSWORD}
max_open_conn = 30
max_idle_conn = 10
conn_max_lifetime = 14400
  • SQLite Default: SQLite is used by default in single-instance development setups, but it locks under concurrent writes.
  • PostgreSQL / MySQL for Production: High-availability production clusters connect multiple stateless Grafana replicas to a shared PostgreSQL or Amazon RDS instance.
  • State Isolation: Telemetry data (Prometheus time series, Loki log streams) is never stored in this database; only dashboard configurations and access control lists reside here.
Advertisement
3️⃣

Frontend Rendering & Unified Alerting Engine

The frontend is a single-page React application that converts JSON series from the backend into vector graphics using HTML5 Canvas.

  • Canvas Visualizations: Time series panels use HTML5 Canvas rather than SVG to render tens of thousands of data points smoothly without browser DOM bottlenecks.
  • Unified Alerting Engine: Evaluates PromQL and LogQL alert rules on scheduled intervals in the backend, independent of whether any user has the dashboard open in a browser.
Pro Tip: In an interview, emphasize that Grafana instances are completely stateless in production when backed by PostgreSQL and shared session storage (Redis).
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Grafana architecture relies on a stateless Go backend proxy storing metadata in PostgreSQL and rendering telemetry via HTML5 Canvas."
⚡ 60-Second Elevator Pitch Talking Points
  • Grafana architecture consists of a Go backend HTTP proxy, a React SPA frontend, and an external SQL metadata database.
  • It never stores metric or log data; it translates dashboard panels into queries for underlying engines like Prometheus and Loki.
  • For production high availability, run multiple stateless Grafana pods connected to a shared PostgreSQL database and Redis session store.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Running multi-pod Grafana deployments against the default SQLite database SQLite lacks distributed locking. Multiple concurrent pods cause immediate database lock contention and database corruption.
  2. Believing Grafana stores raw Prometheus metrics or Loki logs internally Grafana is purely a query proxy and presentation layer; storage occurs entirely in the underlying TSDB or object store.
  3. Allowing client browsers to query backend data sources directly in Direct access mode Direct mode exposes TSDB credentials and network endpoints to public user browsers; always use Server Proxy mode.
Advertisement
Want more Observability & Monitoring scenarios?
Explore our complete collection of scenario-based Observability & Monitoring interview runbooks.
Browse All Observability & Monitoring Questions →