Principal Production Engineer

Responsibilities

• Own the reliability and scalability of cloud infrastructure including compute, storage, and networking
• Define SLOs, lead incident response, and drive systemic improvements to reduce toil
• Build and mature the observability and tooling layer for network fabric, storage, and control planes
• Partner with software, hardware, and network engineering teams to influence architecture decisions
• Set technical standards for production engineering, including on-call culture and incident frameworks
• Mentor senior and staff engineers to elevate collective technical depth

Requirements

• 15+ years of experience in infrastructure, networking, or production engineering at internet scale
• Strong systems fundamentals in Linux, distributed systems, storage, and compute scheduling
• Hands-on data center experience including physical infrastructure, power, and thermal constraints
• Ability to write code for automation and tooling development
• Excellent analytical and problem-solving skills for synthesizing ambiguous signals
• Strong incident command skills and ability to lead calmly under pressure

Preferred Qualifications

• Deep networking expertise in BGP, OSPF, ECMP, and load balancing
• Experience with HPC infrastructure, GPU clusters, and job schedulers like Slurm or Kubernetes
• Experience with high-bandwidth interconnects such as InfiniBand or RoCE
• Prior principal or staff IC role influencing organizational technical strategy
• Exposure to sustainability-focused or energy-constrained compute environments

Benefits

• Competitive pay and Restricted Stock Units
• Health, vision, and dental insurance options
• 401(k) with 100% match up to 4%
• Paid parental leave and generous PTO
• Tuition reimbursement and cell phone reimbursement
• $300 monthly commuter benefit