Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process services, dedicated to helping the world's leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world.
Our Culture:
Your passion, integrity and experience are integral to Cognizant's success. You will join a dynamic and expanding global leader in IT and Business consultancy where you will be valued for who you are. We take pride in our partnership with our clients, so your ability to add value and provide exceptional service to our clients is fundamental to your success. In return, you will be presented with opportunities to develop your career and collaborate with talented colleagues in a supportive, diverse environment.
At Cognizant we recognize that companies that are open and welcoming to a multi-culturally diverse workforce will thrive with fresh perspectives and collaborative knowledge. Cognizant focuses on promoting & increasing gender diversity and providing a workplace which encourages great participation and an equal playing field, where merit and accomplishment are the only criteria for success.
Job Summary:
We are seeking an experienced Agentic AI Evaluation Data Scientist to design and implement evaluation frameworks for conversational and autonomous AI systems supporting commerce and customer-facing use cases. The role will be responsible for assessing agent performance across task completion, tool-use accuracy, multi-turn coherence, recommendation relevance, hallucination, and customer experience outcomes.
The successful candidate will build automated and human-in-the-loop evaluation pipelines using Azure AI Foundry, establish regression and red-teaming practices, and analyze production telemetry to identify quality drift, failure patterns, bias, and fairness concerns. Working closely with AI Engineers and Solution Architects, the role will embed evaluation gates into MLOps and CI/CD pipelines and provide clear risk assessments to support responsible AI governance and release decisions.
Responsibilities :
- Design and implement evaluation frameworks for agentic AI systems (conversational shopping assistants, autonomous task agents), covering task success rate, tool-call accuracy, and multi-turn coherence.
- Build offline and online evaluation pipelines using Azure AI Foundry evaluation tools, combining LLM-as-judge techniques, human-in-the-loop review, and statistical significance testing.
- Define and track quality metrics specific to commerce agents — recommendation relevance, hallucination rate, task completion, escalation/handoff accuracy, and customer satisfaction proxies (CSAT, resolution rate).
- Design red-teaming and adversarial testing protocols to probe agent robustness against prompt injection, jailbreaks, and edge-case commerce scenarios (returns, fraud, pricing errors).
- Develop regression testing suites for agent to detect quality drift across model versions, prompt changes, and RAG knowledge base updates.
- Analyze production agent logs/telemetry to identify failure patterns, bias, and fairness issues, feeding insights back into prompt engineering, fine-tuning, and RAG retrieval improvements.
- Collaborate with AI Engineers and Solution Architects to instrument agents for observability (tracing, logging via Azure Monitor/Application Insights) and define evaluation gates within MLOps/CI-CD pipelines.
- Communicate evaluation findings and risk assessments to technical and business stakeholders, informing go/no-go decisions for agent releases and supporting responsible AI governance/compliance reporting.
Preferred Certifications:
- Microsoft Certified: Azure AI Engineer Associate (AI-102)
- Microsoft Certified: Azure Data Scientist Associate (DP-100)
- Microsoft Certified: Azure Fundamentals / AI Fundamentals (AZ-900 / AI-900) — baseline
- Relevant Responsible AI / LLM evaluation credentials (e.g., DeepLearning.AI courses on LLM evaluation, Azure AI Content Safety specialization)
- Statistics/ML academic background (MS/PhD in relevant field) — valued though not mandatory given strong practical evaluation experience
Skills Required :
- 5–8 years in data science/ML, with at least 2–3 years specifically focused on evaluating LLM-based or agentic AI systems in production, ideally within retail, e-commerce, or customer-facing conversational AI domains.
Benefits:
Joining Cognizant will give you the opportunity to learn and collaborate with some of the most talented people in the industry, while having your finger on the pulse of emerging industry trends and working on the cutting edge of technology in your field of expertise.
We recognize that our people perform at their best when they feel valued as significant contributors and that is why at Cognizant, taking care of our employees is a priority:
· You can pursue innovative career tracks and opportunities here
· You can enhance your professional development through education and dedicated training
· We’ll give you the skills you need to keep pace with the changing workplace while our compensation, benefits and wellness packages help you stay healthy and plan for the future.
Next Steps:
Please reach out to our friendly and welcoming team today to apply and register your interest in this full-time role. We'd love to help you get your next role and enable you to fulfil your professional ambitions.
コグニザントについて
コグニザント（NASDAQ: CTSH）は、AI Builderおよびテクノロジーサービスプロバイダーとして、お客様にフルスタックのAIソリューションを構築することで、AI投資と企業価値を結ぶ架け橋となっています。業界、ビジネスプロセス、エンジニアリングに関する当社の深い専門知識を活かし、組織固有のビジネス環境をテクノロジー・システムに組み込みます。これにより、人間の可能性を最大限に引き出し、確かな成果を実現するとともに、急速に変化する世界においてグローバル企業が常に一歩先を行くための支援を行っています。 詳細については、cognizant.ai をご覧ください。
雇用に関する追加情報
本募集に記載されている報酬情報は、掲載日時点で正確なものです。Cognizantは、適用される法令に従い、いつでも本情報を変更する権利を留保します。
応募者は、対面またはビデオ会議による面接への参加を求められる場合があります。また、各面接の際に、現在有効な州政府または政府発行の身分証明書の提示を求められる場合があります。
Cognizantは機会均等雇用主です。応募および選考において、人種、肌の色、性別、宗教、信条、性的指向、性自認、国籍、障がい、遺伝情報、妊娠、退役軍人の地位、その他連邦法・州法・地方自治体の法律により保護されるいかなる特性に基づく差別も行いません。
