Senior Data Engineer

About the Company

Our client is a leading company in the digital advertising and media industry, specializing in OTT (Over-The-Top) and CTV (Connected TV) advertising solutions.

Responsibilities

• Design, build, and maintain efficient and scalable data pipelines for data integration and transformation
• Manage and optimize large datasets using relational and NoSQL databases
• Utilize Apache Spark for distributed data processing and real-time analytics
• Implement and manage Kafka for data streaming and real-time integration
• Collaborate with cross-functional teams to translate business requirements into technical solutions
• Monitor and optimize data pipeline performance, availability, and reliability
• Ensure data quality, consistency, and integrity across all systems

Requirements

• 6-9 years of experience in data engineering
• Proficiency in relational databases (MySQL, PostgreSQL) and NoSQL (MongoDB, Cassandra)
• Experience building pipelines with data warehouses like Snowflake or Redshift
• Experience processing unstructured data from S3 using Athena or Glue
• Hands-on experience with Kafka for real-time data streaming
• Solid understanding of ETL processes and pipeline optimization
• Proficiency in Python, Java, or Scala
• B.E/B.Tech degree