Staff Data Platform Engineer

About the Company

Rivian and Volkswagen Group Technologies is a joint venture between two industry leaders focused on the next chapter of automotive technology. The company addresses the challenges of electric vehicles through operating systems, zonal controllers, cloud, and connectivity solutions to set standards for software-defined vehicles.

Responsibilities

• Design, deploy, and maintain data platform infrastructure using Terraform and IaC principles
• Manage and optimize the Databricks platform for security, performance, and cost-effectiveness
• Lead technical design of data systems focusing on scalability, reliability, and performance
• Collaborate with data producers, consumers, and business leaders to gather requirements
• Proactively identify and resolve issues within the Databricks platform
• Stay updated on advancements in big data, cloud computing, and DevOps

Requirements

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field
• 3+ years of experience building and maintaining large-scale data processing systems
• Solid understanding of Databricks (Delta, Spark, Unity Catalog, data governance)
• Experience using Terraform for large-scale infrastructure management
• Proficiency in building and maintaining CI/CD pipelines
• Experience with cloud-based technologies such as AWS, GCP, or Azure
• Strong analytical and problem-solving skills

Preferred Qualifications

• Expertise in PySpark and the Spark ecosystem (Spark SQL, Spark Streaming)
• Proficiency in AWS services including IAM, VPC, EC2, and S3