Senior AI R&D Engineer

Responsibilities

• Profile training loops using PyTorch Profiler, Nsight Systems and Nsight Compute to identify system- and kernel-level bottlenecks
• Configure and tune composite parallelism strategies (e.g. TP, DP, HSDP/FSDP, EP) to optimize load balance and minimize bottlenecks
• Partner with AI Researchers to define model architectures for hardware efficiency without compromising convergence
• Maximize hardware utilization and training throughput on large-scale GPU clusters

Requirements

• Proficiency in Python and PyTorch
• Strong engineering background in parallel and/or distributed systems
• Hands-on experience with modern machine learning techniques and LLM lifecycles
• Deep understanding of the CUDA programming model
• Experience in distributed programming with APIs like NCCL or MPI
• Experience analyzing profiling traces with PyTorch Profiler and Nvidia Nsight

Preferred Qualifications

• Contributions to distributed training frameworks like TorchTitan, Megatron-LM, or DeepSpeed
• Familiarity with low-precision training formats (MXFP4, MXFP8)
• Deep understanding of NCCL communication primitives, NVSHMEM, or CUDA IPC
• Proven track record of implementing and optimizing transformer-based model training
• Experience working on the NVIDIA Blackwell architecture

About the Company

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training, building reliable models for high-stakes settings in finance, manufacturing, and public administration.