Quality Engineer 3

Responsibilities

• Create and extend software frameworks and tools to run, monitor, and report on automated tests.
• Build and maintain automation frameworks and CI/CD pipelines.
• Architect, design, and build software to automate testing methods including API validation, performance/load testing, and fuzzing.
• Develop test frameworks for GenAI and AI/ML applications, focusing on model validation, LLM prompt-response consistency, and hallucination detection.
• Implement evaluation pipelines to test LLM accuracy, latency, bias, toxicity, and reliability.
• Review testing strategies and provide technical insights to development teams.

Requirements

• Experience building and maintaining automation frameworks and CI/CD pipelines.
• Experience with API validation and performance/load testing.
• Experience deploying services on cloud computing platforms such as AWS, Azure, or GCP.
• Proficiency with common testing tools (e.g., Pytest, Robot, Playwright, or Locust).
• Strong interest in working at the intersection of AI/ML, Big Data, and Cybersecurity.

Preferred Qualifications

• Experience with LLM evaluation or AI/ML testing (e.g., Ragas, LangChain).
• Knowledge of distributed systems and microservices architecture.
• Experience with containerization and orchestration using Docker and Kubernetes.

Benefits

• Equity for all employees.
• Flexible annual leave, paid holidays, and volunteer days.
• Comprehensive private benefits plan including medical insurance for you and your family, life insurance, and personal accident insurance.
• Fertility support and paid parental leave.
• Employee Assistance Program.

About the Company

Arctic Wolf is a leader in AI-driven security operations, processing hundreds of billions of security signals daily to help organizations detect and respond to threats. We provide a Concierge Security® approach, combining advanced automation with expert human analysis to protect over 10,000 customers worldwide.