Senior Data Engineer

Responsibilities

• Design, develop, and enhance a petabyte-scale vehicle data processing framework using Go
• Ensure seamless integration of the framework with the Databricks environment
• Implement and manage robust CI/CD pipelines on Kubernetes
• Monitor framework performance and implement optimizations for growing data volumes
• Proactively identify and resolve framework issues to maintain system uptime
• Collaborate with data producers and consumers to deliver scalable solutions

Requirements

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field
• 5+ years of experience in data engineering or similar roles
• Deep proficiency in Go programming and its concurrency model
• Solid understanding of big data ecosystems like Spark and Databricks
• Exceptional analytical and problem-solving skills
• Excellent communication and collaboration skills

Preferred Qualifications

• Experience with Kubernetes container orchestration and cluster management
• Familiarity with Delta Lake and the Delta protocol

About the Company

Rivian and Volkswagen Group Technologies is a joint venture focused on automotive software-defined vehicles, addressing challenges in connectivity, AI, and cloud solutions for electric vehicles.