Lead Data Engineer

Responsibilities

• Lead and manage data engineering projects from inception to completion
• Collaborate with stakeholders to identify opportunities to leverage data
• Design, develop, and operate modern data architecture and end-to-end solutions
• Create and develop intricate data processing pipelines
• Collaborate with data scientists to design scalable model implementations
• Write clean, iterative code based on TDD and continuous delivery practices
• Advise clients on distributed storage and computing technologies
• Develop data models using various modeling techniques
• Manage data governance, security, and privacy requirements
• Define and incorporate data quality strategies into daily work

Requirements

• Experience leading system design and technical solution implementation
• Expertise in Big Data architecture and building/operating data pipelines
• Deep understanding of data modeling and modern engineering tools
• Hands-on experience with Python (required)
• Experience with Spark and big data processing
• Experience with at least one cloud platform (AWS, Azure, or GCP)
• Experience with distributed storage and processing platforms in production
• Proficiency with various database types including SQL and NoSQL
• Strong stakeholder management and leadership skills
• Ability to coach, mentor, and motivate teammates

Preferred Qualifications

• Experience with Databricks
• Experience with data visualization techniques

About the Company

Thoughtworks is a leading technology consultancy that has spent over 30 years helping clients solve complex business problems with technology. We are a dynamic and inclusive community dedicated to revolutionizing tech through purposeful and impactful work.