Quality Engineer 3

Responsibilities

• Create and extend software frameworks and tools to run, monitor, and report on automated tests
• Build and maintain automation frameworks and CI/CD pipelines
• Architect and design software to automate API validation, performance/load testing, and fuzzing
• Develop test frameworks for GenAI and AI/ML applications, including model validation, LLM prompt-response consistency, and hallucination detection
• Implement evaluation pipelines to test LLM accuracy, latency, bias, toxicity, and reliability
• Review testing strategies and provide technical insights to development teams

Requirements

• Experience building and maintaining Automation frameworks and CI/CD pipelines
• Experience with API validation and performance testing
• Experience developing test frameworks for GenAI and AI/ML applications
• Experience deploying services on cloud platforms such as AWS, Azure, or GCP
• Expertise in functional, load, and fuzz testing tools (e.g., Locust, CATS)
• Proficiency with Python and testing tools like Pytest or Playwright

Preferred Qualifications

• Knowledge of distributed systems and microservices architecture
• Experience with Docker, Kubernetes, and LangChain
• Familiarity with LLM evaluation frameworks such as Ragas

About the Company

Arctic Wolf is a leader in AI-driven security operations, processing hundreds of billions of security signals daily to help organizations detect and respond to threats. Our platform uses advanced automation and a scalable microservices architecture to turn complex data into actionable insights, providing proactive protection through our Concierge Security approach.