Data Developer

Responsibilities

• Design and build data pipelines for RAG systems, including document ingestion, chunking, embedding generation, and vector storage
• Build ingestion pipelines for structured and unstructured data sources into a centralized data lake
• Develop data processing workflows to prepare and optimize datasets for fine-tuning and inference workloads
• Build monitoring and evaluation frameworks to measure retrieval quality, latency, and system performance
• Collaborate with ML engineers to optimize data formats and storage patterns for GPU-accelerated inference
• Implement caching strategies and data versioning systems to support efficient model serving
• Deploy and manage vector databases, embedding services, and data processing pipelines
• Drive initiatives to improve data quality, reduce latency, and enhance the accuracy of retrieval systems

Requirements

• Bachelor's or Master's degree in Computer Science, Data Engineering, or related field
• 2-5 years building data systems and pipelines in production environments
• Strong experience with RAG architectures and vector databases (Milvus, ChromaDB, Pinecone, Weaviate, or Qdrant)
• Proficiency in Python with experience using DAG-based orchestration platforms (Airflow, Dagster, Prefect, or similar)
• Hands-on experience with embedding models and semantic search systems
• Experience with distributed data processing frameworks (Apache Spark, Ray, or Dask)
• Understanding of LLM inference optimization techniques and prompt engineering
• Familiarity with Docker and containerization
• Strong grasp of data engineering best practices including data modeling and ETL/ELT patterns

About the Company

DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world.