Data Platform Engineer

Responsibilities

• Design, develop, and maintain robust data pipelines for ingesting, transforming, and storing large volumes of structured and unstructured data.
• Implement data workflows and ETL processes using technologies such as Spark, Delta Lake, and cloud-native tools.
• Support the production use of big data platforms including Databricks, Snowflake, and Google Cloud Platform.
• Monitor, troubleshoot, and resolve issues in production data systems, including job failures and performance bottlenecks.
• Optimize SQL queries, Spark jobs, and data processing workloads for efficiency and cost-effectiveness.
• Integrate data workflows into CI/CD pipelines and automate deployment processes using Git and DevOps tools.
• Implement data governance, security, and compliance measures such as access controls and audit logging.

Requirements

• 7+ years of experience in data engineering or related roles with hands-on production experience.
• Strong proficiency with big data platforms (Databricks, Snowflake, or GCP) and the Spark ecosystem.
• Deep expertise in Retrieval-Augmented Generation (RAG) systems and LLM integration for enterprise knowledge retrieval.
• Advanced skills in PySpark, Python, and SQL.
• Experience with Lakehouse architectures, Delta Lake, and vector databases.
• Knowledge of cloud messaging and streaming technologies like Apache Kafka or cloud Pub/Sub.
• Experience working in an Agile environment and using CI/CD tools (Git, Jenkins, or Terraform).

About the Company

AutoZone is the nation's leading retailer and a leading distributor of automotive replacement parts and accessories, with more than 6,000 stores across the US, Mexico, and Brazil. We are committed to providing the best parts, prices, and customer service in the automotive aftermarket industry. At AutoZone, you play a crucial role in creating a better experience for our customers while driving your career forward in a fast-paced, innovative environment.