Senior Software Engineer

Responsibilities

• Design, build, and maintain data pipelines for processing content catalog data using Apache Spark
• Develop efficient data processing workflows for large-scale data
• Create robust APIs to securely expose data services
• Collaborate with data scientists to integrate models and algorithms into production
• Build scalable data integration solutions connecting various sources and destinations
• Implement monitoring and observability for data pipelines
• Optimize data storage and retrieval mechanisms
• Contribute to architectural decisions regarding data infrastructure
• Apply best practices for data engineering, including quality checks and error handling

Requirements

• 7+ years of software development experience
• 5+ years specializing in data engineering or pipeline development
• Extensive experience with data warehousing (Vertica, Snowflake, or Databricks)
• Hands-on experience with machine learning pipelines and MLOps
• Proficiency in Java, Scala, or Python
• Strong expertise in big data technologies like Apache Spark and Kafka
• Experience with cloud platforms (AWS, GCP, or Azure) and Kubernetes
• Solid understanding of data modeling and relational/NoSQL databases
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field

About the Company

The Trade Desk is a global technology company and the world’s leading independent platform for digital advertising. Their technology helps advertisers reach the right audiences across the open internet, handling over 1 trillion queries per day.