Senior Virtualization Validation Engineer

Responsibilities

• Design and execute large-scale validation tests across multi-node virtualized clusters to ensure linear scaling and stability of GPU workloads
• Validate high-speed interconnects including NVLink, Infinity Fabric, InfiniBand, and RoCE within virtualized environments
• Lead the validation of QEMU and Cloud Hypervisor with a focus on PCIe passthrough (VFIO), IOMMU, and direct device assignment
• Architect and run comprehensive test suites using nccl-tests and rccl-tests to verify performance across node boundaries
• Validate SR-IOV and RDMA configurations to ensure near-bare-metal networking performance
• Develop and maintain automation frameworks in Python or Go to provision and stress-test multi-node environments
• Perform deep-dive analysis of performance regressions in multi-node communication

Requirements

• 5+ years of experience in a relevant technical role
• Bachelor's or Master's degree in Computer Science, Electrical Engineering, or a related field
• Proven experience with QEMU/KVM and Cloud Hypervisor
• Deep familiarity with NVIDIA (CUDA/NCCL) and/or AMD (ROCm/RCCL) stacks
• Strong understanding of RDMA, RoCE, and InfiniBand protocols
• Expert-level knowledge of Linux kernel internals, PCIe topology, VFIO, and memory management
• Advanced proficiency in Python and/or Bash for automation

Preferred Qualifications

• Experience with MNNVL (Multi-Node NVLink) or specialized AI fabric architectures
• Familiarity with hardware-level debugging tools and performance profilers like NVIDIA Nsight or AMD Omniperf
• Knowledge of containerized orchestration for GPUs such as Kubernetes

Benefits

• Competitive compensation and equity packages including RSUs
• Comprehensive health, dental, and vision insurance
• 401(k) retirement plan with company match up to 4%
• Paid parental leave and volunteer time off
• Professional development and tuition reimbursement
• Daily meals allowance and commuter benefits

About the Company

Crusoe is a vertically integrated AI infrastructure company that owns and operates each layer of the stack to power the world's most ambitious AI workloads using an energy-first approach.