Senior Data Engineer

Responsibilities

• Design, develop, and optimize end-to-end data pipelines using Azure Databricks
• Build and maintain scalable ETL/ELT frameworks for structured and unstructured data
• Ingest data from multiple sources including APIs, databases, files, and streaming systems
• Work with Azure Data Lake Storage (ADLS Gen2) for efficient data storage and retrieval
• Develop data transformation logic using PySpark and Spark SQL
• Ensure data quality, reliability, and performance across pipelines
• Implement data modeling concepts to support analytics and reporting use cases
• Collaborate with Data Scientists, BI teams, and business stakeholders
• Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency
• Apply best practices around security, governance, and compliance within Azure

Requirements

• 5+ years of hands-on experience as a Data Engineer
• Strong experience with Azure Databricks
• Proficiency in PySpark, Spark SQL, and Python
• Experience working with Azure Data Lake Storage (ADLS Gen2)
• Solid understanding of data warehousing and data modeling concepts
• Experience building batch and/or streaming data pipelines
• Familiarity with Azure Data Factory or similar orchestration tools
• Strong SQL skills and experience with relational databases
• Experience using Git or similar version control systems

Preferred Qualifications

• Experience with Delta Lake
• Knowledge of streaming technologies like Kafka, Azure Event Hubs, or Spark Structured Streaming
• Exposure to CI/CD pipelines using Azure DevOps
• Experience working in Agile/Scrum environments
• Understanding of cloud cost optimization strategies

About the Company

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people.