Senior AI Researcher

Responsibilities

• Own core elements of the training recipe including optimizers, schedules, and initialization
• Design PyTorch-based architectural improvements to maximize convergence and efficiency
• Develop hyperparameter scaling laws and scale-up methodologies using proxy experiments
• Investigate complex convergence issues such as loss spikes and divergence
• Resolve distributed system failures including communication bottlenecks and synchronization errors
• Partner with Compute Performance and Data teams to align model lifecycle with hardware constraints

Requirements

• Proficiency in Python and deep familiarity with PyTorch-based training workflows
• Strong track record in machine learning research and software engineering
• Strong mathematical foundation in optimization and scaling behavior
• Deep understanding of transformer training dynamics and large distributed training jobs
• Hands-on experience pre-training models with 7B+ parameters on 100+ GPU clusters
• Ability to work in Germany and collaborate regularly on-site in Heidelberg

Preferred Qualifications

• Experience training LLMs or multimodal models using Megatron-LM, DeepSpeed, or torchtitan
• Familiarity with scaling laws and hyperparameter transfer
• Experience profiling distributed jobs and diagnosing training anomalies
• Exposure to sparse training approaches like Mixture-of-Experts
• Research excellence demonstrated through top-tier publications (NeurIPS, ICML, ICLR)

About the Company

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training, building reliable models for finance, manufacturing, and public administration.