Software Engineer, Ray Data

Responsibilities

• Develop high-quality open-source software to simplify distributed programming via Ray
• Identify, implement, and evaluate architectural improvements to Ray core and Datasets
• Optimize the performance of Ray Datasets at large scale using Arrow primitives and Ray object manager optimizations
• Improve testing processes and stability/stress testing infrastructure to ensure smooth releases
• Integrate streaming workloads into Ray, such as Beam on Ray
• Communicate technical work to the broader community through talks, tutorials, and blog posts

Requirements

• 5+ years of relevant work experience
• Solid background in algorithms, data structures, and system design
• Experience building scalable and fault-tolerant distributed systems
• Experience with data processing or database internals, including Spark or Dask

Preferred Qualifications

• Experience with streaming data workloads
• Familiarity with Apache Arrow or Ray Core components

About the Company

Anyscale is on a mission to democratize distributed computing. We are commercializing Ray, a popular open-source project that creates an ecosystem of libraries for scalable machine learning. Our platform allows developers and data scientists to scale ML applications from a laptop to a cluster seamlessly. Companies like OpenAI, Uber, and Spotify use Ray to accelerate their AI applications.