As a GenAI Test Automation Lead, you will design scalable, Python-driven automation frameworks to validate LLMs, prompt frameworks, and RAG systems across UI and API layers. You will leverage modern CI/CD pipelines and write complex SQL queries to perform deep backend and database data validation.
In this role, you will
- You will Lead the design, implementation, and maintenance of scalable end-to-end test automation frameworks tailored for GenAI applications and Large Language Models (LLMs).
- Build automation solutions that go beyond standard UI tools (like Selenium), integrating multiple testing tools, API layers, and backend systems into a cohesive automation suite.
- Actively test and evaluate LLMs, prompt engineering frameworks, and Retrieval-Augmented Generation (RAG) systems using specialized AI evaluation tools like LangChain.
- Embed automated GenAI evaluation and regression suites into modern CI/CD pipelines (e.g., GitHub Actions, GitLab CI/CD, or Jenkins) to achieve continuous testing and rapid feedback loops.
- Perform deep data validation and backend testing by writing complex SQL queries and managing test data across cloud data platforms like Snowflake.
- Guide and upskill traditional QA automation engineers in Python, AI evaluation techniques, and the nuances of testing cognitive applications.
Required skills: Overall 10+ years QA experience
- End to end test architecture / test integration for GenAI development project.
- Must have GenAI Experience: Strong experience in testing and evaluating GenAI models, prompt engineering frameworks, and RAG systems.
- Technical Proficiency: Proficiency in Python and AI evaluation tools like LangChain.
- Hands-on Automation Experience in python (Must Have).
- Hands-on experience with LLMs (Must Have).
- Experience with Integration of multiple tools with Automation (not just selenium).
- Hands-on DB experience - SQL/Snowflake (Must have).
- CI/CD with test automation - (Must have).
Preferred skills
- Ability to design scalable testing frameworks within cloud environments (AWS, Azure, GCP).
- Deep knowledge of Software Testing Life Cycle (STLC), CI/CD pipelines, and API testing.
We're eager to meet people who share our mission and can make an impact in various ways. Don't hesitate to apply, even if you only meet the required skills listed. Your transferable skills and experiences matter—help us see how you the right person for this role.
Total compensation:
We regularly assess market data to ensure we offer a competitive compensation package for our associates. The base salary for this position ranges between CAD 90,000 – 110,000 per year. Where the successful candidate may fall within the range depends on relevant education, work and/or management experience and other business-related and job-necessary qualifications. This position is also eligible for Cognizant’s discretionary annual performance-based bonus, as well as benefits that support your physical, mental and financial wellbeing.
Working arrangements
We believe hybrid work is the way forward as we strive to provide flexibility wherever possible. Based on this role’s business requirements, this is a hybrid position requiring 4 days a week in a Cognizant or client office in Toronto. Regardless of your working arrangement, we are here to support a healthy work-life balance through our various well-being programs.
Note: The working arrangements for this role are accurate as of the date of posting. This may change based on the project you’re engaged in, as well as business and client requirements. Rest assured; we will always be clear about role expectations.
Cognizant will only consider applicants for this position who are legally authorized to work in Canada without requiring employer sponsorship, now or at any time in the future.
Applications for this position are reviewed by our recruitment team without the use of artificial intelligence screening tools.
Sobre a Cognizant
Cognizant (NASDAQ: CTSH) é uma construtora de IA e fornecedora de serviços de tecnologia, criando a ponte entre o investimento em IA e o valor para as empresas por meio do desenvolvimento de soluções de IA completas para nossos clientes. Nossa profunda experiência em setores, processos e engenharia nos permite incorporar o contexto único de cada organização em sistemas tecnológicos que potencializam a capacidade humana, geram resultados tangíveis e mantêm empresas globais à frente em um mundo em rápida transformação. Saiba mais em cognizant.ai ou @cognizant.
A Cognizant é uma empregadora que investe em equidade. Sua candidatura não será pautada em raça, cor, gênero, sexualidade, credo, origem, deficiência, gravidez ou qualquer outra característica protegida pelas leis brasileiras.
Informações adicionais de emprego
Os candidatos podem ser obrigados a comparecer a entrevistas presenciais ou por videoconferência. Além disso, os candidatos podem ser obrigados a apresentar seu documento de identificação emitido pelo estado ou governo atual durante cada entrevista.
Embora nosso sistema permita a candidatura em todos os idiomas, o(s) idioma(s) e o(s) nível(is) de proficiência exigidos para o trabalho variam. No entanto, é necessário um nível básico de inglês para fins de comunicação em toda a empresa.
A Cognizant é uma empregadora que investe em equidade. Sua candidatura não será pautada em raça, cor, gênero, sexualidade, credo, origem, deficiência, gravidez ou qualquer outra característica protegida pelas leis brasileiras.
Se você tem uma deficiência que requer adaptações razoáveis para procurar uma vaga de emprego ou enviar uma candidatura, envie um e-mail para [email protected] com sua solicitação e informações de contato.
