Software Engineer in Test (AI)

Aquent, a global leader in talent solutions, is partnering with an innovative and influential organization that is at the forefront of leveraging artificial intelligence to transform its industry. This company is dedicated to building robust, reliable, and cutting-edge AI-powered applications that drive significant impact for its users and stakeholders.

Join a team where your expertise will directly contribute to shaping the future of AI technology and ensuring its responsible and high-quality deployment.

Shape the Future of AI Quality as a Mid-Level AI Software Test Engineer!

Are you passionate about the intersection of software quality and groundbreaking artificial intelligence? We are seeking a dynamic and experienced engineer to play a critical role in ensuring the integrity, safety, and performance of our next-generation AI solutions.

In this pivotal role, you will be instrumental in designing, automating, and executing advanced testing strategies for AI-powered applications, machine learning systems, intelligent agents, copilots, and generative AI solutions.

You will combine your strong software quality engineering background with emerging AI validation techniques to ensure our AI systems are not only innovative but also reliable, accurate, performant, and production-ready. Your work will directly enhance user trust and enable the scalable delivery of AI-powered features, making a significant impact on our product quality and strategic initiatives.

What You Will Do:

• Develop and implement comprehensive testing strategies for a diverse portfolio of AI applications, platforms, and services.
• Validate AI model outputs for critical attributes such as accuracy, consistency, reliability, and safety, ensuring high-quality user experiences.
• Design and execute a full spectrum of tests including functional, integration, end-to-end, regression, and performance testing for AI solutions.
• Create sophisticated test cases tailored for prompt-driven, agentic, and retrieval-based AI workflows.
• Validate AI guardrails, business rules, permissions, and governance controls to ensure responsible AI deployment.
• Perform adversarial, negative, and edge-case testing to proactively identify potential model failures and hallucinations.
• Build and maintain robust automated test frameworks specifically designed for AI applications, streamlining the testing process.
• Develop automated evaluation pipelines for AI responses and workflows, facilitating continuous quality assessment.
• Integrate AI testing seamlessly into CI/CD pipelines, promoting a culture of continuous quality and rapid iteration.
• Implement automated quality scoring and regression detection mechanisms to maintain high standards across releases.
• Create reusable test data, mocks, simulators, and validation frameworks to enhance testing efficiency and coverage.
• Test AI agents, workflows, APIs, and tool-calling capabilities, ensuring smooth and effective operations.
• Validate integrations with external systems, diverse data sources, and enterprise services.
• Verify the performance, reliability, scalability, and resiliency of AI workloads through rigorous testing.
• Execute load and stress testing to ensure AI services can handle anticipated demands.
• Collaborate closely with software engineers, AI engineers, product owners, architects, and security teams throughout the development lifecycle.
• Actively participate in design reviews, providing crucial quality feedback from the early stages of development.
• Contribute to defining test strategy, establishing quality standards, and promoting best practices across the organization.
• Support production readiness reviews and lead defect triage activities to ensure timely resolution of issues.

Required Qualifications:

• Bachelor's degree in Computer Science, Software Engineering, Information Systems, or a related technical field.
• 3–6 years of progressive experience in software testing, QA automation, or quality engineering.
• Proven experience developing automated test solutions using one or more of the following languages: Python, Java, JavaScript/TypeScript, C#.
• Demonstrated experience with API testing and automation tools (e.g., Postman).
• Strong foundational understanding of test automation principles and best practices.
• Solid grasp of the Software Development Lifecycle (SDLC) and Agile methodologies.
• Experience integrating testing into CI/CD pipelines (e.g., GitHub Actions, Jenkins).
• Experience testing distributed systems, web applications, and APIs.
• Familiarity with containerization technologies like Docker and orchestration platforms such as Kubernetes.
• Experience with testing on cloud platforms (Azure, AWS, or GCP).
• Proficiency in quality engineering concepts including Defect Analysis and Root Cause Investigation.

Preferred Qualifications (Nice-to-Have):

• Experience testing generative AI applications, LLM-based systems, AI agents, or RAG applications.
• Familiarity with leading AI models and platforms such as OpenAI, Claude, Gemini, or Azure OpenAI.
• Experience building evaluation and benchmarking frameworks specifically for AI solutions.
• Knowledge of responsible AI principles, AI governance frameworks, and AI risk management practices.
• Experience with AI-specific testing techniques including Prompt Testing, Response Evaluation, Hallucination Detection, Agent Workflow Validation, RAG Validation, AI Safety Testing, AI Regression Testing, and AI Benchmarking.
• Familiarity with test automation frameworks like Selenium, Playwright, JUnit, or PyTest.
• Exposure to various quality engineering domains such as Performance Testing, Load Testing, and Security Testing.

Client Description:

About Aquent

Talent:

Aquent Talent connects the best talent in marketing, creative, and design with the world's biggest brands. Our talent get access to amazing benefits like subsidized health, vision, and dental plans, paid sick leave, and retirement plans with a match. We also offer free online training through

Aquent Gymnasium

. More information on our awesome

benefits

!

Aquent is an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, and other legally protected characteristics. We're about creating an inclusive environment—one where different backgrounds, experiences, and perspectives are valued, and everyone can contribute, grow their careers, and thrive.

You will want to read this fine print. Part of our mission is to take care of you, so we offer the best benefits. Unlike other staffing companies, we provide talent with the same benefits that our CEO gets.

This includes:

• A choice of three major medical plans (including vision coverage) plus dental insurance
• Paid sick leave for all hourly talent in the U.S. and Canada
• 401(k) with a match and immediate vesting
• Exclusive employee discounts through Promo Code: Aquent
• And much more