Inference Intern

About the Company

Etched is building hardware for frontier intelligence. We co-design chips, racks, software, and manufacturing to deliver best-in-class throughput and latency across both prefill and decode workloads. Our first products are heavily focused on inference.

Responsibilities

• Support porting state-of-the-art models to our architecture
• Help build programming abstractions and testing capabilities to rapidly iterate on model porting
• Assist in building, enhancing, and scaling our runtime, including multi-node inference and intra-node execution
• Contribute to optimizing routing and communication layers using our collectives
• Utilize performance profiling and debugging tools to identify bottlenecks
• Develop and leverage a deep understanding of our architecture to co-design HW instructions and model architecture operations
• Implement high-performance software components for the Model Toolkit

Requirements

• Progress towards a Bachelor’s, Master’s, or PhD in computer science, computer engineering, applied mathematics, or a related field
• Proficiency in Python and C++
• Understanding of performance-sensitive or complex distributed software systems (Linux internals, accelerator architectures, Compilers, or high-speed interconnects)
• Experience porting applications to non-standard accelerator hardware or hardware platforms
• Deep knowledge of transformer model architectures and/or inference serving stacks (vLLM, SGLang, etc.)

Preferred Qualifications

• Proficiency in Rust
• Experience with low-latency, high-performance applications using kernel-level and user-space networking stacks
• Deep understanding of distributed systems concepts, algorithms, and challenges
• Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE)
• Experience building applications with extensive SIMD optimizations
• Familiarity with PyTorch or JAX
• Success in math competitions (AIME, AMC, etc)

Benefits

• 12-week paid internship
• Generous housing support for those relocating
• Daily lunch and dinner in our office
• Direct mentorship from industry leaders and world-class engineers