Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process services, dedicated to helping the world's leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world.
Our Culture:
Your passion, integrity and experience are integral to Cognizant's success. You will join a dynamic and expanding global leader in IT and Business consultancy where you will be valued for who you are. We take pride in our partnership with our clients, so your ability to add value and provide exceptional service to our clients is fundamental to your success. In return, you will be presented with opportunities to develop your career and collaborate with talented colleagues in a supportive, diverse environment.
At Cognizant we recognize that companies that are open and welcoming to a multi-culturally diverse workforce will thrive with fresh perspectives and collaborative knowledge. Cognizant focuses on promoting & increasing gender diversity and providing a workplace which encourages great participation and an equal playing field, where merit and accomplishment are the only criteria for success.
Job Summary:
We are seeking an experienced Agentic AI Evaluation Data Scientist to design and implement evaluation frameworks for conversational and autonomous AI systems supporting commerce and customer-facing use cases. The role will be responsible for assessing agent performance across task completion, tool-use accuracy, multi-turn coherence, recommendation relevance, hallucination, and customer experience outcomes.
The successful candidate will build automated and human-in-the-loop evaluation pipelines using Azure AI Foundry, establish regression and red-teaming practices, and analyze production telemetry to identify quality drift, failure patterns, bias, and fairness concerns. Working closely with AI Engineers and Solution Architects, the role will embed evaluation gates into MLOps and CI/CD pipelines and provide clear risk assessments to support responsible AI governance and release decisions.
Responsibilities :
- Design and implement evaluation frameworks for agentic AI systems (conversational shopping assistants, autonomous task agents), covering task success rate, tool-call accuracy, and multi-turn coherence.
- Build offline and online evaluation pipelines using Azure AI Foundry evaluation tools, combining LLM-as-judge techniques, human-in-the-loop review, and statistical significance testing.
- Define and track quality metrics specific to commerce agents — recommendation relevance, hallucination rate, task completion, escalation/handoff accuracy, and customer satisfaction proxies (CSAT, resolution rate).
- Design red-teaming and adversarial testing protocols to probe agent robustness against prompt injection, jailbreaks, and edge-case commerce scenarios (returns, fraud, pricing errors).
- Develop regression testing suites for agent to detect quality drift across model versions, prompt changes, and RAG knowledge base updates.
- Analyze production agent logs/telemetry to identify failure patterns, bias, and fairness issues, feeding insights back into prompt engineering, fine-tuning, and RAG retrieval improvements.
- Collaborate with AI Engineers and Solution Architects to instrument agents for observability (tracing, logging via Azure Monitor/Application Insights) and define evaluation gates within MLOps/CI-CD pipelines.
- Communicate evaluation findings and risk assessments to technical and business stakeholders, informing go/no-go decisions for agent releases and supporting responsible AI governance/compliance reporting.
Preferred Certifications:
- Microsoft Certified: Azure AI Engineer Associate (AI-102)
- Microsoft Certified: Azure Data Scientist Associate (DP-100)
- Microsoft Certified: Azure Fundamentals / AI Fundamentals (AZ-900 / AI-900) — baseline
- Relevant Responsible AI / LLM evaluation credentials (e.g., DeepLearning.AI courses on LLM evaluation, Azure AI Content Safety specialization)
- Statistics/ML academic background (MS/PhD in relevant field) — valued though not mandatory given strong practical evaluation experience
Skills Required :
- 5–8 years in data science/ML, with at least 2–3 years specifically focused on evaluating LLM-based or agentic AI systems in production, ideally within retail, e-commerce, or customer-facing conversational AI domains.
Benefits:
Joining Cognizant will give you the opportunity to learn and collaborate with some of the most talented people in the industry, while having your finger on the pulse of emerging industry trends and working on the cutting edge of technology in your field of expertise.
We recognize that our people perform at their best when they feel valued as significant contributors and that is why at Cognizant, taking care of our employees is a priority:
· You can pursue innovative career tracks and opportunities here
· You can enhance your professional development through education and dedicated training
· We’ll give you the skills you need to keep pace with the changing workplace while our compensation, benefits and wellness packages help you stay healthy and plan for the future.
Next Steps:
Please reach out to our friendly and welcoming team today to apply and register your interest in this full-time role. We'd love to help you get your next role and enable you to fulfil your professional ambitions.
Über Cognizant
Cognizant (NASDAQ: CTSH) i ist ein Technologiedienstleister und Entwickler von KI-Lösungen. Wir schlagen die Brücke zwischen KI-Investitionen und echtem unternehmerischem Mehrwert, indem wir ganzheitliche Full-Stack-KI-Lösungen für unsere Kunden entwickeln. Mit unserer fundierten Branchen-, Prozess- und Engineering-Expertise integrieren wir die spezifischen Anforderungen von Unternehmen passgenau in Technologiesysteme. So entfalten wir das menschliche Potenzial, erzielen greifbare Ergebnisse und sichern globalen Unternehmen in einer sich rasant wandelnden Welt den entscheidenden Vorsprung. Erfahren Sie mehr unter cognizant.ai oder @cognizant.
Zusätzliche Informationen zur Beschäftigung
Die Vergütungsinformationen sind zum Zeitpunkt der Veröffentlichung dieser Stellenausschreibung korrekt. Cognizant behält sich das Recht vor, diese Informationen jederzeit unter Beachtung der geltenden gesetzlichen Bestimmungen zu ändern.
Bewerberinnen und Bewerber können verpflichtet sein, an Vorstellungsgesprächen persönlich oder per Videokonferenz teilzunehmen. Darüber hinaus kann es erforderlich sein, bei jedem Gespräch einen gültigen staatlichen Lichtbildausweis vorzulegen.
Cognizant ist ein Arbeitgeber mit Chancengleichheit. Ihre Bewerbung und Kandidatur werden nicht aufgrund von Rasse, Hautfarbe, Geschlecht, Religion, Glaubensbekenntnis, sexueller Orientierung, Geschlechtsidentität, nationaler Herkunft, Behinderung, genetischen Informationen, Schwangerschaft, Veteranenstatus oder sonstiger durch bundes‑, landes‑ oder kommunalrechtliche Vorschriften geschützter Merkmale berücksichtigt oder abgelehnt.
