Data Engineer II

Responsibilities

• Build efficient and scalable data pipelines using PySpark and Python
• Integrate data from multiple source systems into a unified target system
• Orchestrate and schedule complex data workflows using Apache Airflow
• Enhance and optimize existing pipelines to improve performance and maintainability
• Troubleshoot pipeline failures and conduct root cause analysis to implement corrective measures
• Collaborate with stakeholders to understand and address business needs related to data processing
• Provide weekend and shift support as necessary to ensure continuous business operations

Requirements

• 3-5 years of experience in a data engineering role
• Proficiency in SQL, Python, and PySpark
• Strong understanding of data warehousing techniques and dimensional modeling
• Hands-on experience with Apache Airflow for workflow orchestration
• Familiarity with containerization using Docker and version control systems
• Knowledge of DBT (Data Build Tool) for data transformation and modeling

Preferred Qualifications

• Experience with AWS cloud services and infrastructure

About the Company

Yubi is re-defining global debt markets by freeing the flow of finance between borrowers, lenders, and investors. As a leading fintech unicorn, Yubi provides a holistic product suite that facilitates the discovery, investment, fulfillment, and collection of debt solutions globally.