Data Platform Engineer

About the Company

Cogent Security is on a mission to stop breaches and prevent cybercrime by innovating at the frontier of generative AI systems. We are building the world’s first AI cyber taskforce, composed of AI agents capable of human-caliber reasoning and execution of cybersecurity tasks.

Responsibilities

• Build the new standard for ingesting and extracting enterprise data through data connectors and pipelines
• Design highly performant data pipelines and indexing strategies for semantic search and retrieval augmented generation (RAG)
• Work with Applied AI teams to taxonomize, correlate, and deduplicate data across enterprise sources
• Architect the modern data platform for Gen AI and make build vs buy decisions for the LLM stack
• Implement generalizable design patterns to scale from one integration to thousands

Requirements

• 5+ years of professional experience as a hands-on engineer and technical leader
• Experience building data pipelines for Information Retrieval Systems (knowledge graphs, search engines, etc.)
• Experience in ETL orchestration and workflow management (Temporal, Airflow)
• Experience building batch processing and streaming systems (Apache Kafka, Flink)
• Expertise in database fundamentals, SQL, data reliability, and distributed computing
• Proficiency with Kubernetes, Terraform, Docker, Databricks, and multiple cloud environments