Software Engineer, AI Infrastructure

About the Company

At Fireworks, we’re building the future of generative AI infrastructure. Our platform delivers the highest-quality models with the fastest and most scalable inference in the industry. Fireworks is a Series C company valued at $4 billion and backed by top investors including Benchmark, Sequoia, Lightspeed, Index, and Evantic.

Responsibilities

• Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
• Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
• Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
• Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
• Participate in code reviews, technical discussions, and continuous integration and deployment processes

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related technical field
• 3 years of experience in software engineering, focusing on infrastructure or machine learning systems
• Strong programming skills in Python, Go, or similar
• Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes)
• Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation)

Preferred Qualifications

• 5+ years of experience in software engineering
• Experience with open source inference engines like vLLM, Sglang, or TRT-LLM
• Contributions to open-source infrastructure or ML projects
• Experience in building large scale ML/MLOps infrastructure