Data Engineer

Responsibilities

• Collect and integrate data from structured and unstructured sources including APIs, RDBMS, and real-time streams
• Design and implement scalable ETL/ELT pipelines for data cleaning and normalization
• Build and deploy data pipelines on AWS or Azure using managed services
• Optimize database architecture across relational, columnar, and key-value stores
• Manage high-volume, low-latency data streams using Apache Kafka
• Orchestrate complex data workflows using Apache Airflow
• Integrate LLMs and ML models into pipelines for data enrichment and inference

Requirements

• Active TS/SCI security clearance
• Bachelor's or master's degree in computer science, engineering, or related field
• 10+ years of experience in data engineering or software development
• Proficiency in Python and libraries such as pandas, PySpark, or FastAPI
• Experience with cloud services (AWS or Azure) and cloud-native data tools
• Proven experience with Kafka and Airflow
• Strong understanding of database internals and storage technologies
• Familiarity with data governance, lineage, and metadata management

Preferred Qualifications

• Knowledge of data cataloging tools and semantic layer design
• Experience with Docker and Kubernetes
• Familiarity with MLOps tools like SageMaker or MLflow

About the Company

M9 Solutions provides IT services and solutions to the Federal Government, specializing in IT talent solutions, data analytics, cybersecurity, and cloud migration.