AgentOps Engineer

Responsibilities

• Deploy and maintain LLM-powered and multi-agent systems across cloud and on-prem environments
• Implement agentic observability frameworks to track performance, latency, cost, and accuracy
• Monitor execution traces, context windows, and agent interactions for anomalies
• Fine-tune agent configurations for cost efficiency, scalability, and response quality
• Build automated pipelines to evaluate agent performance, safety, and compliance
• Implement fallbacks, guardrails, and redundancy mechanisms to ensure reliability
• Collaborate with AI engineers and DevOps to align operational reliability with business outcomes

Requirements

• 2–5 years of experience in DevOps, MLOps, or AI systems engineering
• Hands-on experience with LLM orchestration frameworks such as LangChain, LangGraph, or LlamaIndex
• Proficiency in Python and scripting for automation
• Experience with cloud platforms (AWS, GCP, or Azure) and containerization (Docker, Kubernetes)
• Understanding of RAG pipelines, vector databases, and context orchestration
• Familiarity with monitoring and observability tools like Prometheus, Grafana, or ELK

Preferred Qualifications

• Exposure to multi-agent orchestration (MCP, A2A messaging, AgentBridge)
• Experience with AgentOps tools such as LangSmith, PromptLayer, or Weights & Biases
• Experience in Responsible AI or model evaluation frameworks
• Familiarity with CI/CD pipelines specifically for AI models and agents

About the Company

Xenonstack is a fast-growing Data and AI Foundry for Agentic Systems. We focus on making AI agents enterprise-ready, reliable, and production-grade, helping organizations gain real-time and intelligent business insights through advanced AI infrastructure and vision platforms.