Senior ML Data Platform Developer

Responsibilities

• Design and maintain a hierarchical storage architecture and collaborate with the research team for seamless integration with training pipelines
• Scale and automate data processing infrastructure to handle petabytes of data
• Ensure efficient use of compute resources, including GPU access for intensive data processing tasks
• Assist the Infrastructure team in provisioning compute and storage environments to support scaling
• Ensure all datasets are versioned, reproducible, and fully traceable with accompanying datasheets
• Collaborate with research and other teams to develop tools for self-service data exploration, sampling, and analysis

Requirements

• Bachelor's degree in Computer Science, Computer Engineering, Software Engineering, or a related field
• 5+ years of experience in designing and managing web-scale storage
• Experience with HPC or large-scale distributed ML data environments
• Proficiency with technologies such as Lustre, Ray, Apache Spark, Apache Arrow, or Parquet
• Experience with workload managers like Ray, Kubernetes, or Slurm
• Familiarity with containerization tools like Docker or Enroot
• Familiarity with data infrastructures and platforms such as vector databases

Benefits

• Comprehensive health benefits including mental health and wellness management
• 20 days of vacation per year from start date
• 4% employer contribution to retirement savings
• Additional 8% compensation for retirement savings or bonuses
• Collaborative and inclusive work environment

About the Company

LoiZéro is a non-profit organization dedicated to advancing research and developing technical solutions for safe AI systems. Based in Montreal, LoiZéro conducts research to design non-agentic AI systems that learn to understand the world through transparent probabilistic reasoning.