Agentic AI Quality Engineer

Zoek nieuws, onderwerpen en adviesbureaus…
Lead with expertise, drive innovation. Are you an experienced professional ready to take your career to the next level? At Accenture the Netherlands, we're seeking an Agentic AI Quality Engineer who brings deep expertise, strategic thinking, and a passion for innovation to help us solve complex challenges and deliver exceptional results.
Automate evaluation of multi-turn conversations and long-horizon tasks using metrics for reasoning accuracy, evidence grounding, plan execution, latency, token usage, and recovery.
Apply schema validation, tracing, and calibrated LLM-as-a-Judge frameworks to detect inaccurate tool calls, loops, and misuse.

Implement filters and golden datasets to test harmful content, policy violations, hallucinations, bias, and brand drift.
Use human-in-the-loop review for high-risk edge cases.

Adversarial Security & Red Teaming
Run red-team, prompt-injection, jailbreak, privacy, and PII-leak tests across agent workflows.
Analyze failures in model logic and API integrations to identify root causes and controls.

Embed test harnesses in CI/CD and monitor production telemetry for regressions, behavioral drift, and recovery.
Maintain evaluation data lineage, versioning, curation, and golden responses.

Translate business risks into measurable quality criteria, practical controls, and release recommendations.
Align stakeholders on user journeys, edge cases, thresholds, mitigations, testability, observability, governance, and evaluation strategy.
Communicate findings and residual risks for go/no-go decisions; coach teams and share reusable assets and standards.

What Success Looks Like (KPIs)
High coverage of non-deterministic agent paths.
Rapid regression detection and remediation after model updates.

Industry Context & Methodology Alignment
Treat non-determinism as a core characteristic. As the notes, robust testing verifies correct reasoning, grounded evidence, scoped actions, and safe, compliant decisions.
Drive continuous optimization by embedding evaluation in CI/CD, using tunable judges, aligned scorers, automated dataset updates, and dashboards, as outlined by .

At Accenture, our teams operate within one of our core service lines— Digital Core, Industry & Enterprise, Supply Chain & Engineering, Cybersecurity, Song, Finance, Talent, Client Groups, Global Operations, Reinventions Services —each designed to drive reinvention at scale. This role falls under Digital Core, where we help clients modernize their technology foundations and accelerate business transformation through cloud, data, software engineering, and AI.
You will join multidisciplinary teams consisting of quality engineers, software engineers, architects, AI specialists, platform engineers, and business consultants. Together, we help organizations reinvent how software is designed, built, tested, deployed, and operated through the power of AI.
We go to market by industry, leveraging our global delivery network and strategic partnerships with ecosystem leaders like Microsoft, AWS, and Google. Our culture of shared success and continuous learning ensures that every team member contributes to—and benefits from—our reinvention journey.
: Bachelor’s or Master’s in Computer Science, Data Science, or a related quantitative field.
: 3+ years in machine learning QA, AI evaluation engineering, or MLOps engineering.
Dutch – B2+ for collaboration with Dutch-speaking clients and stakeholders.
: Experience evaluating autonomous agents, multi-step workflows, or Retrieval-Augmented Generation (RAG) applications.
: Strong Python proficiency and experience with AI evaluation tools such as MLflow, Galileo, and PyRIT.
: Deep knowledge of prompt engineering, vector databases, tool schema design, and execution telemetry.
: Hands-on experience in adversarial testing, threat modeling, or model safety guardrails.
: Strong communication, technical leadership, a growth mindset, and commitment to continuous innovation.

Contributions to open-source projects or engineering communities

4x9 Workweek Option, Hybrid Working, FlexDay
Competitive Base Salary, MyBenefits Budget, Annual Bonus, Net Allowance, Employee Share Purchase Plan
Collective Health Insurance Scheme, Mental Health Support, Well-being Hub
26 Vacation Days, Special Leave Types (parental leave, partner leave, bereavement leave, and gender affirmation leave), Culture Days
NS Business Card, Mobility Budget (option to choose between electric bike, car lease, or public transport reimbursement)
Performance Achievement Program, Learning & Development Platform, Guidance from People Leads
Inclusion, Diversity & Social Impact: Do Good Volunteer Program, I&D Networks, Refugee Talent Track
Accentrics & Friday Drinks, Annual Accenture NL Party, Flat Organizational Hierarchy

Ready to lead and innovate? Apply today and become a Reinventor at Accenture the Netherlands.
All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law.
Job candidates will not be obligated to disclose sealed or expunged records of conviction or arrest as part of the hiring process.
Accenture is committed to providing veteran employment opportunities to our service men and women.
Please read Accenture’s for more information on how we process your data during the Recruiting and Hiring process.
We work with one shared purpose: to deliver on the promise of technology and human ingenuity. Every day, more than 775,000 of us help our stakeholders continuously reinvent. Together, we drive positive change and deliver value to our clients, partners, shareholders, communities, and each other.
We believe that delivering value requires innovation, and innovation thrives in an inclusive and diverse environment. We actively foster a workplace free from bias, where everyone feels a sense of belonging and is respected and empowered to do their best work.
At Accenture, we see well-being holistically, supporting our people’s physical, mental, and financial health. We also provide opportunities to keep skills relevant through certifications, learning, and diverse work experiences. We’re proud to be consistently recognized as one of the World’s Best Workplaces™.
Join Accenture to work at the heart of change. Visit us at .
Now available in your app store.
is hét online platform voor de adviesbranche, met 12 websites wereldwijd. Vanuit ons hoofdkantoor in Amsterdam werken we met 600+ adviesbureaus in 100+ landen.