Q: Your enterprise connects its on-premises data centers to Azure via a 10 Gbps ExpressRoute circuit. To satisfy compliance and ensure business continuity during fiber cuts, you must deploy a Site-to-Site VPN as an automated backup. How do you configure ExpressRoute and VPN Gateway coexistence to prevent asymmetric routing and ensure seamless failover?
Designing a fault-tolerant hybrid cloud topology where an active ExpressRoute circuit automatically fails over to an encrypted Site-to-Site VPN with BGP route manipulation and AS-Path prepending.
Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: When a terrestrial fiber cut took down our primary ExpressRoute provider circuit, our on-premises data center lost connectivity to Azure for 4 hours. We architected ExpressRoute and VPN Gateway coexistence with dynamic BGP route steering.
Provision Coexisting ExpressRoute and VPN Gateways in GatewaySubnet
Deploy both gateways inside the designated GatewaySubnet:
- GatewaySubnet Sizing: Allocated a minimum
/26or/27GatewaySubnet in the Hub VNet to accommodate both gateway instances. - Deploy Gateways: Provisioned ExpressRoute Gateway (
az network express-route-gateway create) and Route-Based VPN Gateway (az network vpn-gateway create) in the same VNet.
Configure Dynamic BGP Peering and Route Advertisements
Establish BGP sessions over both paths to exchange on-premises and Azure routes dynamically:
- BGP ASN Assignment: Configured Azure Gateway ASN (e.g., 65515) and on-premises core router ASN (e.g., 65001).
- Equal Route Advertising: Advertised identical on-premises CIDR ranges (e.g.,
10.0.0.0/16) across both the ExpressRoute BGP session and the IPsec VPN BGP session.
Prevent Asymmetric Routing with AS-Path Prepending & Local Preference
Ensure return traffic from on-premises to Azure follows the identical physical path:
- BGP Local Preference: On the on-premises core router, configured BGP Local Preference to
200for ExpressRoute routes and100for VPN routes. - AS-Path Prepending: Prepended the on-premises ASN three times (
set as-path prepend 65001 65001 65001) on BGP updates sent over the backup IPsec VPN tunnel.
Simulate Circuit Failure & Measure BGP Convergence Time
Validate automated switchover and switchback without administrative intervention:
- Simulate Failure: Administratively shut down the on-premises ExpressRoute BGP peering interface.
- Convergence Telemetry: Traffic switched automatically to the encrypted IPsec VPN tunnel within 3.2 seconds with zero dropped TCP connections.
- Failback Verification: Upon re-enabling the circuit, BGP restored ExpressRoute as the preferred path automatically.
- Provision both ExpressRoute and VPN Gateways within a /27 or larger GatewaySubnet.
- Establish dynamic BGP peering advertising identical network prefixes over both paths.
- Configure on-premises AS-Path prepending and Local Preference to ensure symmetric routing.
- Validate automated 3-second failover and seamless recovery during circuit outage simulations.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Engineering bespoke in-house developer tools instead of adopting standardized platform Golden Paths when configuring Hybrid Network High Bespoke internal tools create long-term maintenance debt and fragmented workflows, whereas standardized Golden Paths accelerate delivery.
- Overlooking infrastructure cost visibility and FinOps allocation tagging across multi-tenant environments when configuring Hybrid Network High Without granular resource tagging, engineering organizations cannot attribute cloud infrastructure costs to specific teams or business units.
- Failing to define explicit Service Level Objectives (SLOs) and Error Budgets to balance feature velocity with reliability when configuring Hybrid Network High Without agreed SLOs, engineering and product teams lack an objective data-driven framework to prioritize technical debt over new features.
- Neglecting automated continuous drift detection between declarative configuration and deployed cloud state when configuring Hybrid Network High Unmanaged drift accumulates over time, causing future automated deployments to fail or overwrite emergency manual adjustments.