Software Engineer – GPU Kernel

Responsibilities

• Design, implement, and optimize high-performance GPU kernels for AI inference (e.g., GEMM, attention, routing)
• Develop and maintain GPU code in CUDA and C++, including low-level assembly when needed
• Implement reduced-precision and quantized kernels (FP8/FP4) for low-latency or high-throughput inference
• Benchmark and ensure cross-vendor performance parity between NVIDIA and AMD hardware
• Contribute to internal GPU libraries and tune performance of performance-critical components
• Accelerate multi-modal model pipelines
• Investigate and integrate next-generation GPU features

Requirements

• 3+ years of experience in GPU programming, HPC, or performance-critical systems
• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field
• Strong proficiency in CUDA for NVIDIA GPUs or ROCm/HIP for AMD GPUs
• Deep understanding of GPU architecture: warps, threads, memory hierarchy, synchronization, and latency-throughput trade-offs
• Proficiency in C++
• Experience with GPU profiling and performance tuning
• Strong numerical background with understanding of precision trade-offs and quantization techniques

Preferred Qualifications

• Experience optimizing transformer, multi-modal, or Mixture-of-Experts (MoE) architectures at the kernel level
• Familiarity with the latest GPU libraries and frameworks (CUTLASS, Triton, …)
• Inter-GPU communication programming experience
• Open-source contributions related to GPU performance or ML acceleration
• Research or conference presentations on GPU optimization, HPC, or numerical computing

Benefits

• Flexible working hours
• Daily lunch and dinner provided; unlimited snacks and beverages
• Supportive and highly collaborative work environment
• Health check-up support and top-tier equipment/hardware support
• Competitive compensation, startup equity, and health insurance

About the Company

FriendliAI is building the world’s best AI inference platform that makes large language and multi-modal models fast, efficient, and deployable at scale.