Staff Senior Reliability Engineer - Observability & CUJ

Robinhood’s mission is to democratize finance for all. As an estimated $124 trillion in assets transfers to younger generations over the next two decades, we are building an elite engineering organization to solve the world's largest financial challenges at scale.

The Robinhood Command Center (RCC) is a newly formed, high-visibility reliability organization serving as the front line for detecting, coordinating, and mitigating production incidents across all Robinhood products. As a Staff Senior Engineer and founding member of the RCC team, you will not own specific product microservices or core cloud infrastructure, but you will own the overarching processes, failure mitigation strategies, and observability frameworks that enable rapid incident response across hundreds of distributed systems. Partnering with product, SRE, infrastructure, and business teams, you will drive measurable reductions in Mean Time to Detect (MTTD) and Mean Time to Resolve (MTTR), shaping how Robinhood learns from failures at enterprise scale.

Key Responsibilities

• Founding Incident Leadership: Serve as a primary technical Incident Commander (IMOC), driving real-time mitigation during high-severity production events by facilitating time-sensitive decisions like traffic shifts, feature rollbacks, and multi-service isolation.
• Observability Strategy for Critical User Journeys (CUJs): Own and evolve the observability roadmap for Robinhood's core product flows—defining global monitoring dashboards, anomaly alerting, and telemetry tracking across hundreds of microservices.
• Failure Mitigation & Resiliency Architecture: Design and implement next-generation failover strategies and progressive degradation frameworks to avoid full-region or full-datacenter outages.
• Incident Response Tooling & Automation: Build, maintain, and scale incident response tooling, automation playbooks, and standardized measurement systems to continuously lower MTTD and MTTR metrics.
• Post-Incident Governance & Postmortems: Drive company-wide standards for postmortems, SEV reviews, and follow-up remediation tracking to ensure durable, long-term reliability improvements.
• Executive Reporting & Insights: Synthesize incident telemetry and reliability trends into clear, executive-level reports to guide business and engineering decisions around service availability.
• Technical Mentorship & Hiring: Raise the bar for operational excellence across engineering teams through hands-on coaching, technical writing, and active participation in recruiting top-tier talent.

Requirements

Software Engineering & SRE Technical Depth

• 8+ years of total software engineering experience, including deep hands-on expertise operating high-concurrency production systems at scale.
• 3+ years of focused experience in site reliability engineering (SRE), distributed systems, cloud infrastructure, or production operations.
• Incident Commander Record: Demonstrated experience serving in hands-on incident leadership roles (Incident Commander, IMOC, or primary incident escalation lead) for critical production environments.
• Modern Observability Stacks: Deep technical familiarity with enterprise telemetry frameworks and monitoring stacks (OpenTelemetry, Prometheus, Grafana, Datadog).
• Fault-Tolerant Architectures: Solid understanding of multi-region/multi-cluster topologies, traffic routing, capacity planning, and automated failover mechanics.
• Measurable Impact: Proven track record of architecting and implementing frameworks that yield measurable improvements in system availability, MTTD, MTTR, and customer impact reduction.

Preferred Qualifications

• Prior experience building or scaling centralized command centers, SRE organizations, or production operations teams in high-growth fintech or high-volume SaaS environments.
• Hands-on experience with modern cloud infrastructure orchestration (Kubernetes, AWS, Terraform) and low-latency API architectures.

Total Rewards & Benefits

• Compensation & Equity: Zone 1 base salary range of $230,000 – $270,000 USD + performance-driven bonus multipliers + Robinhood equity grants + 401(k) match.
• 100% Paid Employee Healthcare: 100% employer-paid health insurance premiums for employees (90% coverage for eligible dependents), plus employer-paid life, disability, and fertility benefits.
• Lifestyle Wallet: Highly flexible monthly spending account for wellness, education, and personal development.
• Office Experience & Perks: Dedicated work-from-home options, flexible PTO, paid holidays, and an exceptional office environment with daily catered meals and events.