Inference Engineer

About the Company

We're building autonomous research agents for recursive self-improvement (multi-agent systems that propose, run, and analyze machine learning experiments). We're a small team based in San Francisco.

Responsibilities

• Build, operate, and harden production inference systems serving large models at high throughput
• Own performance characteristics including throughput, latency, cost-per-token, and reliability
• Profile real workloads to identify bottlenecks and ship performance fixes
• Implement and integrate inference optimizations like quantization, custom kernels, and memory management
• Design observability including metrics, tracing, and alerting
• Run capacity planning, autoscaling, and load testing for varied workload shapes
• Diagnose and resolve production incidents and write postmortems

Requirements

• 3+ years of experience building production-grade, large-scale serving infrastructure
• Strong distributed systems experience and on-call experience
• Fluency in performance profiling and optimization
• Experience with GPU-accelerated inference at scale (multi-GPU, multi-node)
• Proficiency in Python and systems-level code (C++, CUDA, ROCm, or Triton)
• Track record of shipping production infrastructure at scale
• Strong written communication skills

Preferred Qualifications

• Open-source contributions to inference or serving frameworks
• Experience with mixed cloud and on-premises deployments
• Familiarity with hardware-aware optimization (memory hierarchy, NCCL/RDMA, NUMA)
• Background in compilers, runtimes, or accelerator software stacks