Role Overview
As a Senior Data Engineer / Tech Lead, you will design, develop, and optimize scalable, enterprise-grade cloud-native data platforms and pipelines on the Microsoft Azure ecosystem . You will possess deep technical ownership and be responsible for implementing modern data architectures (such as Lakehouse and Medallion architecture) . This role requires close collaboration with architects, business analysts, and downstream data consumers to translate complex business requirements into high-performing and secure data solutions.
Key Responsibilities
1.Data Pipeline Design & Orchestration (ADF)
• Design, build, and maintain complex, scalable ETL/ELT pipelines using Azure Data Factory (ADF) to ingest heterogeneous data from diverse sources (APIs, SFTP, Oracle Fusion, SQL Server, Event Hubs, etc.)
• Implement parameter-driven dynamic pipelines, custom activities, and robust scheduling triggers
• Configure and manage Integration Runtimes (Self-Hosted and Azure IR) for secure and efficient on-premise to cloud data movement
2.Advanced Big Data Processing (Azure Databricks)
• Develop high-performance batch and real-time streaming data pipelines using Azure Databricks, PySpark, Spark SQL, or Scala
• Implement Medallion Architecture (Bronze, Silver, and Gold layers) and maintain modern Lakehouse architectures utilizing Delta Lake
• Optimize Spark compute usage through parameter-driven jobs, proper cluster scaling, and Databricks serverless compute configurations
• Perform deep performance tuning, query optimization, and memory management of heavy Spark workloads
3.Data Warehousing & Modeling
• Design and refine data models including dimensional modeling, star schemas, and Slowly Changing Dimensions (SCDs)
• Optimize databases by writing complex, performant T-SQL queries, stored procedures, and schemas on Azure SQL Database or Azure Synapse Analytics
4.Data Governance, Security & DevOps
• Enforce data governance, lineage, and access controls across the platform utilizing Unity Catalog .
• Secure data architectures utilizing Azure Key Vault, Managed Identities, Service Principals, and VNET-integrated environments
• Implement CI/CD pipelines using Git and Azure DevOps to automate database and pipeline deployments
5.Leadership & Collaboration
• Provide technical mentorship and guidance to junior developers and QA teams
• Enforce engineering discipline through standard code reviews, structured pre/post-deployment validations, and thorough documentation
Required Skills & Qualifications
• Overall Experience: Minimum of 8+ years of professional experience in Data Engineering or Data Warehousing
• Azure Experience: 4+ years of dedicated, hands-on experience designing and operating modern data platforms on Microsoft Azure
• Programming Languages: Proficient in Python/PySpark, SQL (advanced query optimization, indexing, stored procedures), and optionally Scala
• Core Azure Stack: Extensive knowledge of Azure Data Factory, ADLS Gen2, Azure Databricks (Delta Lake), Azure SQL Database, and Azure Synapse Analytics
• Big Data Concepts: Robust understanding of Apache Spark architecture, distributed computing, and the Hadoop ecosystem
• Data Integration: Strong background in CDC (Change Data Capture) mechanisms, API integrations, and event-driven data streaming (e.g., Event Hubs/Kafka)
• Soft Skills: Excellent analytical capabilities, strong documentation skills, and the ability to articulate technical strategies to non-technical business partners
Preferred / "Good-to-Have" Qualifications
• Certifications: Microsoft Certified: Azure Data Engineer Associate (DP-203), Azure Solutions Architect Expert, or Databricks Certified Associate/Professional
• Cloud Platforms: Exposure to multi-cloud environments (e.g., Google Cloud Platform or AWS)
• IaC: Experience using Infrastructure as Code (IaC) tools like Terraform or ARM Templates to deploy Azure resources
About Cognizant:
Cognizant (Nasdaq: CTSH) is an AI Builder and technology services provider, bridging the gap between AI investment and enterprise value by building full-stack AI solutions for our clients. Our deep industry, process and engineering expertise enables us to build an organization’s unique context into technology systems that amplify human potential, drive tangible outcomes and keep global enterprises ahead in a fast-changing world. See how at cognizant.ai or @cognizant.
Additional employment information
Compensation information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.
Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview.
Cognizant is an equal opportunity employer. Your application and candidacy will not be considered based on race, color, sex, religion, creed, sexual orientation, gender identity, national origin, disability, genetic information, pregnancy, veteran status or any other characteristic protected by federal, state or local laws.
If you have a disability that requires reasonable accommodation to search for a job opening or submit an application, please email [email protected] for roles based in the Americas or [email protected] for roles based in India.
