Data Engineer

Responsibilities

• Design, develop, and maintain scalable and efficient data pipelines from a wide variety of sources
• Use dbt to transform data through various layers including Cleansed, Conformed, and Presentation
• Write custom Python connectors and leverage data-loading tools
• Operationalize enterprise data models for reporting, analytics, and data science
• Embed automated data quality checks, validations, and alerting across pipelines
• Work within Dagster for orchestration and observability
• Manage CI/CD workflows using GitHub, GitHub Actions, and Kubernetes
• Implement and review RBAC policies within Snowflake
• Leverage AI-assisted development tools like Anthropic Claude and GitHub Copilot

Requirements

• Advanced SQL expertise (CTEs, stored procedures, query tuning)
• Mandatory experience with Snowflake
• Proficiency with dbt and Jinja templating
• Hands-on experience with Snowflake Snowpark for Python-based transformations
• Experience with ELT/ETL tools such as Fivetran or Airbyte
• Proficiency in Python with libraries like Pandas and GreatExpectations
• Experience with orchestration platforms like Dagster, Airflow, or Prefect
• Strong understanding of data modeling (3NF, Star Schema, Kimball/Inmon)
• Experience with Git version control and CI/CD principles
• Degree in a related technology field or 3+ years of relevant experience

Preferred Qualifications

• Kubernetes and container orchestration
• Infrastructure as Code (Terraform, Pulumi)
• Snowflake Cortex AI and native app development
• Event streaming technologies like Kafka or Azure Service Bus
• SnowPro Certification

About the Company

Fastmarkets is an industry-leading price-reporting agency and information provider for global commodities, providing price data, news, analytics, and events for agriculture, forest products, metals, mining, and new-generation energy markets.