Machine Learning Data Engineer

Responsibilities

• Develop and maintain scalable data pipelines using Spark, Python, and ETL tools to support machine learning models and LLMs.
• Architect and implement robust data warehousing solutions and data models to ensure data quality and performance.
• Collaborate with Data Scientists to assist in the development, testing, and productionization of innovative AI architectures.
• Transform approaches for storing, transporting, and securing large, complex, and unstructured datasets.
• Identify and resolve performance bottlenecks and data quality issues within the data and ML infrastructure.
• Create and maintain comprehensive technical documentation for data pipelines, models, and machine learning workflows.

Requirements

• 1-3 years of experience in data engineering or machine learning roles.
• Proficiency in Python, SQL, and Spark (PySpark and/or Scala).
• Strong understanding of data modelling techniques (e.g., star schema, dimensional modelling) and data warehousing.
• Knowledge of data governance, data quality principles, and data security best practices.
• Experience with data integration, cleansing, and transformation processes on large datasets.
• Excellent written and verbal communication skills to convey technical concepts to diverse audiences.

Preferred Qualifications

• Solid understanding of ML fundamentals and libraries such as scikit-learn, TensorFlow, or PyTorch.
• Experience with Natural Language Processing (NLP) and Large Language Models (LLMs).
• Experience using orchestration tools like Apache Airflow.
• Familiarity with cloud platforms (AWS, Azure, or GCP) and their respective ML services.
• Knowledge of CI/CD pipelines for data and machine learning deployments.
• Experience with prompt engineering and fine-tuning of LLMs.

Benefits

• Opportunity to work on cutting-edge projects involving Large Language Models and generative AI.
• A dynamic technology environment that actively integrates the latest AI advancements.
• A collaborative and supportive culture within an innovation hub.
• Professional development and growth opportunities within a rapidly expanding SaaS scale-up.

About the Company

PredictX is an Enterprise SaaS provider revolutionizing critical decision-making for some of the world’s largest businesses, including three FAANG companies. We have lived and breathed AI and Machine Learning for over a decade, integrating predictive analytics into every aspect of our product to empower our clients through advanced technology.