Senior AI Quality Tester

Scavenger Tech is seeking a Senior AI Quality Tester to validate AI-driven systems including ML models, LLM solutions, and Agentic AI workflows. The role focuses on ensuring outputs are accurate, reliable, secure, and compliant with governance standards across model behavior, prompt reliability, automation flows, and enterprise integration testing.

• Test ML models using metrics (accuracy, precision, recall, F1) and adversarial/edge-case testing
• Validate LLM outputs for correctness, relevance, and hallucination control
• Run prompt-variant testing for response consistency and detect bias/incorrect reasoning
• Test multi-agent workflows, orchestration, decision logic, fallback handling, and automation scenarios
• Perform end-to-end scenario testing across APIs/services and validate real-time inference pipelines
• Conduct governance/compliance testing including RBAC, data access controls, and sensitive data handling
• Execute performance/reliability testing for load, latency, and scalability; monitor drift/degradation over time
• Own AI testing standards, metrics, and acceptance criteria

• 5+ years QA/testing experience with exposure to AI systems and ML model testing
• LLM evaluation techniques including hallucination detection and prompt testing
• Dataset validation and test data generation skills
• Automation tools: Selenium, Cypress, Playwright
• API testing tools: Postman, REST Assured
• Automation scripting in Python or JavaScript
• Basic Python for AI pipeline testing
• Azure environment awareness and CI/CD pipeline experience (Azure DevOps)
• Familiarity with embeddings, vector databases, and ML/LLM workflow concepts
• Nice-to-have: ISTQB Advanced Test Analyst certification and AI-900 certification

Authority: Owns AI testing standards/metrics and acceptance criteria; cannot approve release without meeting quality thresholds. Environment: Azure-based infrastructure with CI/CD pipelines and monitoring/logging systems.