Staff Software Engineer - GenAI inference

Responsibilities

• Own and drive the architecture, design, and implementation of the inference engine
• Partner with researchers to bring new model architectures like sparsity and MoE into the engine
• Lead end-to-end optimization for latency, throughput, and memory efficiency across GPUs
• Architect scalable routing, batching, scheduling, and memory management mechanisms
• Ensure reliability and fault tolerance in inference pipelines, including A/B launches and model versioning
• Collaborate cross-functionally with platform, cloud infrastructure, and security teams
• Represent the team through benchmarks, whitepapers, and open-source contributions

Requirements

• BS/MS/PhD in Computer Science or a related field
• 6+ years of software engineering experience in performance-critical systems
• Deep understanding of ML inference internals including attention, MLPs, and quantization
• Hands-on experience with CUDA, GPU programming, and libraries like cuBLAS, cuDNN, or NCCL
• Strong background in distributed systems design, RPC frameworks, and memory partitioning
• Ability to solve performance bottlenecks across kernel, memory, networking, and scheduler layers
• Experience building instrumentation, tracing, and profiling tools for ML models

Preferred Qualifications

• Published research or open-source contributions in ML systems or inference optimization

About the Company

Databricks is the data and AI company. More than 10,000 organizations worldwide rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics and AI. Founded by the original creators of Lakehouse, Apache Spark™, Delta Lake and MLflow.