Data Domain Architect Lead - Data & Annotation

Responsibilities

• Lead data labeling initiatives to produce high-quality, actionable datasets for machine learning and LLM training.
• Design and own the annotation operating model, including workflow design, task routing, and delivery governance.
• Translate business and ML objectives into implementable requirements, schemas, guidelines, and quality metrics.
• Scale labeling capacity across multiple lines of business while maintaining consistency and throughput.
• Implement LLM-as-judge approaches and agentic workflows to automate quality evaluation at scale.
• Leverage prompt engineering to improve task instructions and support synthetic data generation.
• Collaborate with ML engineers and scientists to define evaluation requirements and target data volumes.
• Drive innovation by automating the labeling lifecycle, including ingestion, validation, and audit-ready lineage.

Requirements

• Master's or PhD in Computational Linguistics, Computer Science, Data Science, or a related field.
• 5+ years of experience delivering data or machine learning-enabled products across the full lifecycle.
• Hands-on experience with annotation metrics, taxonomy design, and performing annotation reviews.
• Proficiency in Python for automation, data analysis, and cleaning, and experience using Git.
• Practical experience with prompt engineering for LLM labeling workflows.
• Working knowledge of LLM-as-judge methods and integrating automated signals into human-in-the-loop reviews.
• Experience designing labeling quality frameworks, including gold datasets and inter-annotator agreement targets.

Preferred Qualifications

• Experience managing globally distributed annotation teams and third-party vendors.
• Familiarity with metadata management, data cataloging, and dataset lineage practices.
• Experience applying machine learning to data quality monitoring and anomaly detection.
• Proven track record of influencing senior stakeholders and aligning priorities through measurable OKRs.
• Experience working with privacy, data governance, or model risk controls related to training data.

About the Company

JPMorganChase is a leading global financial services firm with over 200 years of history. We provide innovative financial solutions to millions of consumers, small businesses, and prominent corporate and institutional clients. Our Consumer & Community Banking division leverages cutting-edge data and analytics to build competitive advantages and deliver industry-leading digital solutions.