Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process services, dedicated to helping the world's leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world.
Our Culture:
Your passion, integrity and experience are integral to Cognizant's success. You will join a dynamic and expanding global leader in IT and Business consultancy where you will be valued for who you are. We take pride in our partnership with our clients, so your ability to add value and provide exceptional service to our clients is fundamental to your success. In return, you will be presented with opportunities to develop your career and collaborate with talented colleagues in a supportive, diverse environment.
At Cognizant we recognize that companies that are open and welcoming to a multi-culturally diverse workforce will thrive with fresh perspectives and collaborative knowledge. Cognizant focuses on promoting & increasing gender diversity and providing a workplace which encourages great participation and an equal playing field, where merit and accomplishment are the only criteria for success.
Job Summary:
We are seeking an experienced Agentic AI Evaluation Data Scientist to design and implement evaluation frameworks for conversational and autonomous AI systems supporting commerce and customer-facing use cases. The role will be responsible for assessing agent performance across task completion, tool-use accuracy, multi-turn coherence, recommendation relevance, hallucination, and customer experience outcomes.
The successful candidate will build automated and human-in-the-loop evaluation pipelines using Azure AI Foundry, establish regression and red-teaming practices, and analyze production telemetry to identify quality drift, failure patterns, bias, and fairness concerns. Working closely with AI Engineers and Solution Architects, the role will embed evaluation gates into MLOps and CI/CD pipelines and provide clear risk assessments to support responsible AI governance and release decisions.
Responsibilities :
- Design and implement evaluation frameworks for agentic AI systems (conversational shopping assistants, autonomous task agents), covering task success rate, tool-call accuracy, and multi-turn coherence.
- Build offline and online evaluation pipelines using Azure AI Foundry evaluation tools, combining LLM-as-judge techniques, human-in-the-loop review, and statistical significance testing.
- Define and track quality metrics specific to commerce agents — recommendation relevance, hallucination rate, task completion, escalation/handoff accuracy, and customer satisfaction proxies (CSAT, resolution rate).
- Design red-teaming and adversarial testing protocols to probe agent robustness against prompt injection, jailbreaks, and edge-case commerce scenarios (returns, fraud, pricing errors).
- Develop regression testing suites for agent to detect quality drift across model versions, prompt changes, and RAG knowledge base updates.
- Analyze production agent logs/telemetry to identify failure patterns, bias, and fairness issues, feeding insights back into prompt engineering, fine-tuning, and RAG retrieval improvements.
- Collaborate with AI Engineers and Solution Architects to instrument agents for observability (tracing, logging via Azure Monitor/Application Insights) and define evaluation gates within MLOps/CI-CD pipelines.
- Communicate evaluation findings and risk assessments to technical and business stakeholders, informing go/no-go decisions for agent releases and supporting responsible AI governance/compliance reporting.
Preferred Certifications:
- Microsoft Certified: Azure AI Engineer Associate (AI-102)
- Microsoft Certified: Azure Data Scientist Associate (DP-100)
- Microsoft Certified: Azure Fundamentals / AI Fundamentals (AZ-900 / AI-900) — baseline
- Relevant Responsible AI / LLM evaluation credentials (e.g., DeepLearning.AI courses on LLM evaluation, Azure AI Content Safety specialization)
- Statistics/ML academic background (MS/PhD in relevant field) — valued though not mandatory given strong practical evaluation experience
Skills Required :
- 5–8 years in data science/ML, with at least 2–3 years specifically focused on evaluating LLM-based or agentic AI systems in production, ideally within retail, e-commerce, or customer-facing conversational AI domains.
Benefits:
Joining Cognizant will give you the opportunity to learn and collaborate with some of the most talented people in the industry, while having your finger on the pulse of emerging industry trends and working on the cutting edge of technology in your field of expertise.
We recognize that our people perform at their best when they feel valued as significant contributors and that is why at Cognizant, taking care of our employees is a priority:
· You can pursue innovative career tracks and opportunities here
· You can enhance your professional development through education and dedicated training
· We’ll give you the skills you need to keep pace with the changing workplace while our compensation, benefits and wellness packages help you stay healthy and plan for the future.
Next Steps:
Please reach out to our friendly and welcoming team today to apply and register your interest in this full-time role. We'd love to help you get your next role and enable you to fulfil your professional ambitions.
À propos de Cognizant
Cognizant (NASDAQ : CTSH) est un AI Builder et une entreprise de services numériques (ESN) élaborant des solutions complètes d’IA maximisant les investissements pour des résultats concrets. Sa profonde expertise des métiers, des processus et des technologies lui permet d’intégrer dans les systèmes technologiques le contexte unique de chaque organisation de l’ingénierie à la production à l’échelle. Son objectif : améliorer l’efficacité des équipes, créer de la valeur et permettre aux grandes entreprises de rester performantes dans un monde qui évolue rapidement. Pour en savoir plus : cognizant.ai ou @cognizant.
Renseignments suppplémentaires sur l'emploi
Les informations relatives à la rémunération du poste à pourvoir dépendent de la date de publication de l’offre de poste. Cognizant se réserve le droit de modifier ces informations à tout moment, sous réserve des lois applicables.
Cognizant est un employeur soucieux de l'égalité des chances entre candidats. Votre candidature sera étudiée indépendamment de votre race, couleur, sexe, religion, croyances, orientation sexuelle, identité de genre, origine, handicap, informations génétiques, grossesse, statut d'ancien militaire ou de toute autre critère jugé discriminant par les lois européennes ou françaises.
Vous êtes porteur d'un handicap, vous pouvez-nous contacter par courriel [email protected] si vous souhaitez préciser les aménagements nécessaires pour le poste ou les entretiens à venir.
Les candidats peuvent être invités à participer à des entretiens en face à face ou par vidéoconférence. En outre, les candidats peuvent être amenés à présenter une carte d'identité valide lors de chaque entretien.
