Inference Software Engineer

About the Company

Etched is building hardware for frontier intelligence. We co-design chips, racks, software, and manufacturing to deliver best-in-class throughput and latency across both prefill and decode workloads. Our first products are heavily focused on inference.

Responsibilities

• Support porting state-of-the-art models to our architecture
• Build programming abstractions and testing capabilities to rapidly iterate on model porting
• Build, enhance, and scale our runtime, including multi-node inference, intra-node execution, state management, and robust error handling
• Optimize routing and communication layers using our collectives
• Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues

Requirements

• Proficiency in C++ or Rust
• Understanding of performance-sensitive or complex distributed software systems (Linux internals, accelerator architectures, Compilers, or high-speed interconnects)
• Familiarity with PyTorch or JAX
• Experience porting applications to non-standard accelerator hardware or hardware platforms

Preferred Qualifications

• Experience developing low-latency, high-performance applications using kernel-level and user-space networking stacks
• Deep understanding of distributed systems concepts, algorithms, and challenges
• Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE)
• Experience building applications with extensive SIMD optimizations

Benefits

• Medical, dental, and vision packages with generous premium coverage
• $500 per month credit for waiving medical benefits
• Housing subsidy of $2k per month for those living within walking distance of the office
• Relocation support for those moving to San Jose
• Various wellness benefits covering fitness and mental health
• Daily lunch and dinner in our office
• Unlimited compute budget subject to ROI justification