Inference Infrastructure Engineer

About the Company

Rhoda AI is building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots.

Responsibilities

• Design and operate large-scale infrastructure to run model workloads across cloud and on-prem environments
• Build and maintain Kubernetes-based deployment pipelines for managing distributed ML workloads
• Own resource scheduling and orchestration across GPU clusters to optimize utilization and cost-performance
• Integrate and manage ML frameworks and model serving systems like Triton, Ray Serve, or TorchServe
• Build tooling for model deployment, versioning, and observability
• Contribute to the reliability and scalability of the infrastructure stack

Requirements

• 3+ years of experience in ML infrastructure, MLOps, or distributed systems
• Strong proficiency with Kubernetes and containerized deployment pipelines
• Experience with GPU orchestration and resource scheduling across large distributed jobs
• Experience with cloud providers such as AWS or GCP and hybrid cloud/on-prem infrastructure
• Familiarity with ML frameworks like PyTorch or JAX and model serving tools
• Strong debugging instincts and ownership mentality

Preferred Qualifications

• Experience with streaming systems or high-throughput data transport (Kafka, gRPC, NATS)
• Background in networking, low-latency systems, or network-aware scheduling
• Experience with edge/cloud hybrid deployment patterns
• Familiarity with on-robot or embedded inference environments
• Experience with large-scale cluster topology and scheduling systems (SLURM, Ray, Volcano)