Data Platform Engineering Lead
Level: M / SA
Role Overview
A hands-on technical lead responsible for building AI-ready enterprise data platforms, including scalable pipelines, lakehouse architectures, and governed data foundations that enable model training, evaluation, and inference. The role extends to backend services and integration layers that expose curated datasets, feature stores, and data products for AI/ML and LLM-driven applications.
Key Responsibilities
1. Build & Deploy Data Platform Solutions
· Design and develop data pipelines for batch and streaming workloads using Python and Spark/PySpark
· Build and manage AI-ready datasets, feature pipelines, and training data foundations
· Develop backend services and APIs (Python – FastAPI or equivalent) to expose data products
· Implement microservices and event-driven architectures for data ingestion, processing, and serving
· Ensure the platform is scalable, performant, and maintainable
· Mentor engineers on data engineering patterns and best practices.
2. Enable AI/ML & LLM Integration
· Develop AI-ready pipelines for model training, evaluation, and inference
· Enable feature store capabilities and reproducible datasets
· Support integration with LLM/AI services (RAG, embeddings, inference APIs)
· Enable data-to-AI pipelines including vectorization and retrieval workflows.
3. Oversee Implementation on Cloud Infrastructure
· Collaborate with infra teams on Azure, AWS, or GCP
· Build and integrate data lakes, lake-houses, SQL/NoSQL systems
· Enable integration between data platforms and AI/ML systems
· Implement containerized and serverless architectures.
4. Implement Modern Software Engineering Practices
· Implement CI/CD pipelines and observability
· Define and enforce data quality frameworks
· Support metadata, lineage, and governance
· Optimize platform performance, reliability, and scalability.
Required Capabilities / Skills / Experience
· 8+ years in data engineering and backend development
· Strong Python, APIs, and microservices experience
· Deep experience in Spark/PySpark or equivalent
· Expertise in ETL/ELT pipelines and lakehouse architectures
· Experience with feature stores and AI-ready datasets
· Experience with Azure/AWS/GCP
· Familiarity with Docker, Kubernetes, CI/CD
· Familiarity with LLM/GenAI integration patterns
· Familiarity with metadata and governance tooling
· Strong system design and problem-solving skills.
The Cognizant community:
We are a high caliber team who appreciate and support one another. Our people uphold an energetic, collaborative and inclusive workplace where everyone can thrive.
- Cognizant is a global community with more than 300,000 associates around the world.
- We don’t just dream of a better way – we make it happen.
- We take care of our people, clients, company, communities and climate by doing what’s right.
- We foster an innovative environment where you can build the career path that’s right for you.
About us:
Cognizant (Nasdaq: CTSH) is an AI Builder and technology services provider, building the bridge between AI investment and enterprise value by building full-stack AI solutions for our clients. Our deep industry, process and engineering expertise enables us to build an organization’s unique context into technology systems that amplify human potential, realize tangible returns and keep global enterprises ahead in a fast-changing world. See how at www.cognizant.com or @cognizant.
Cognizant is an equal opportunity employer. Your application and candidacy will not be considered based on race, color, sex, religion, creed, sexual orientation, gender identity, national origin, disability, genetic information, pregnancy, veteran status or any other characteristic protected by federal, state or local laws.
Disclaimer:
Compensation information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.
Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview.