Senior Data Engineer

Responsibilities

• Design and build optimized, scalable, and resilient data pipelines in a cloud environment.
• Construct infrastructure for efficient ETL processes from diverse sources and storage systems.
• Collaborate with Product and Business Managers to align technical solutions with business requirements.
• Architect and maintain database pipeline architectures to ensure readiness for AI/ML transformations.
• Implement algorithms and prototypes to transform raw data into actionable information.
• Develop innovative data validation methods, analytical tools, and reporting mechanisms.
• Ensure strict compliance with data governance and security policies.
• Monitor data trends and patterns to establish operational alerts and improve service observability.

Requirements

• 10+ years of hands-on experience designing, building, and maintaining distributed data pipelines.
• High proficiency in Scala/Java and Spark for large-scale data processing.
• Expertise in Big Data technologies, including Spark, Data Lake, Delta Lake, and Hive.
• Strong understanding of both batch and streaming data processing techniques.
• Expert-level SQL skills for writing complex, optimized queries across extensive data volumes.
• Experience with RDBMS and OLAP databases such as MySQL and Redshift.
• Familiarity with Agile methodologies and the Data Lifecycle Management process.

Preferred Qualifications

• Experience working in distributed environments with global teams.
• A product/engineering mindset focused on driving impactful data solutions.
• Strong passion for testing strategies, problem-solving, and continuous learning.
• Knowledge of architectural best practices for building robust data lakes.

About the Company

Luxoft is a global provider of high-end software engineering services, helping leading enterprises accelerate their digital transformation through cutting-edge technology solutions.