Software Engineer

Responsibilities

• Design and implement large-scale deployment architectures for LLM and multimodal inference
• Deploy and manage containerized workloads across Kubernetes clusters
• Diagnose production issues and implement temporary fixes
• Collaborate with customers’ DevOps teams to integrate infrastructure into CI/CD workflows
• Develop scripts, Helm charts, and Terraform modules for deployments
• Contribute field insights to shape platform reliability, observability, and scaling strategies
• Lead workshops and technical sessions to help customers master infrastructure best practices

Requirements

• 3+ years of experience in cloud infrastructure, DevOps, or reliability engineering
• Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
• Proficiency with Kubernetes, Docker, Terraform, and Helm
• Strong foundation in distributed systems, networking, and performance tuning
• Experience with GPU-based computing and generative AI model serving workloads
• Strong technical background in backend systems or AI tooling
• Experience operating workloads on AWS, GCP, or OCI
• Excellent problem-solving and debugging skills

Preferred Qualifications

• Experience deploying large models (LLMs, diffusion models) on GPUs or clusters
• Familiarity with inference frameworks like Triton, vLLM, TensorRT, or DeepSpeed-Inference
• Familiarity with observability stacks such as Prometheus, Grafana, Loki, ELK, or OTEL
• Understanding of networking security and compliance frameworks like SOC 2
• Experience supporting on-prem or hybrid-cloud deployments

Benefits

• Competitive compensation and benefits package
• Daily lunch and dinner provided
• Unlimited snacks and beverages
• Health check-up and top-tier hardware support
• Flexible working hours and a highly collaborative environment

About the Company

FriendliAI is building the next-generation AI inference platform that accelerates the deployment of large language and multimodal models with unmatched performance and efficiency.