Data Engineer

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines using Microsoft Fabric, Databricks, or Snowflake
• Implement lakehouse/medallion architecture (bronze, silver, gold layers)
• Prepare structured and unstructured datasets for AI, RAG pipelines, and LLM evaluation
• Implement data validation, schema enforcement, and profiling to maintain high-quality datasets
• Support customer deployments of Quantios Insights across enterprise data platforms
• Collaborate with architects, product owners, and engineering teams in an Agile environment

Requirements

• Bachelor’s degree in Computer Science, Data Engineering, Data Science, or related field
• 4+ years of experience in data engineering
• Hands-on experience with Microsoft Fabric, Azure Databricks, or Snowflake
• Strong skills in Python and SQL, with exposure to PySpark or Spark SQL
• Experience with Azure Data Lake Storage, Delta Lake, and medallion architecture
• Familiarity with Power BI or Fabric semantic models
• Practical experience with CI/CD tools, specifically Azure DevOps
• Understanding of data governance and metadata management tools like Purview or Unity Catalog

Preferred Qualifications

• Exposure to AI-related data preparation including RAG datasets, embeddings, and unstructured text processing