Skip to main content

Crawler Engineer

JD 9

We’re hiring!

At Cognizant we have an ideal opportunity for you to be part of one of the largest companies in the digital sector worldwide. A Great Place To Work where we look for people who contribute new ideas, experiencing a dynamic and growing environment. At Cognizant we promote an inclusive culture, where we value different perspectives providing career growth and development opportunities. #WelcomeToCognizant!

We have an exciting opportunity for an exceptional individual to work supporting one of our clients as a Crawler Engineer

Job summary:

This role focuses on enhancing and mastering Lex Machina's crawling infrastructure to support our customer-facing products. You will be responsible for designing and maintaining robust automated data collection systems in a challenging and large-scale environment. Your work will play a critical role in driving the success of our products, making this an impactful and rewarding opportunity.

Responsibilities:

  • Build, maintain, and optimize automated data collection and storage systems, ensuring reliability and scalability.

  • Master and enhance Lex Machina's crawling infrastructure to support customer-facing applications.

  • Extract and process large datasets from industries such as retail pricing, neobanks, and airlines or OTAs.

  • Debug, troubleshoot, and optimize crawlers for maximum performance and accuracy.

  • Collaborate with cross-functional teams to align data collection processes with business goals.

  • Create and maintain comprehensive documentation of crawling infrastructure and processes.

  • Communicate effectively with stakeholders by asking and answering questions, addressing issues and providing detailed updates.

Qualifications:

  • Bachelor's degree in Computer Science, Software Engineering, or a related field (or equivalent experience).

  • 3-5+ years of experience in backend software engineering, with a focus on Python in challenging environments.

  • Proven expertise in large-scale data collection, processing, and storage systems.

  • Strong problem-solving skills with an enthusiasm for detail and mastery of technical systems.

  • Excellent communication skills, including thorough documentation and the ability to ask/answer questions diligently.

  • Self-motivated, with a passion for delivering high-quality work that supports customerfacing products.

Preferred Qualifications:

  • Experience with distributed systems and scalable architecture.

  • Knowledge of modern software development practices, such as CI/CD and Agile methodologies.

  • Familiarity with cloud environments such as AWS or Azure.

Why Cognizant?

Improve your career in one of the largest and fastest growing IT services providers worldwide
Receive ongoing support and funding with training and development plans
Have a highly competitive benefits and salary package
Get the opportunity to work for leading global companies
We are committed to respecting human rights and build a better future by helping your minds and the environment
We invest in people and their wellbeing.

We create conditions for everyone to thrive. We do not discriminate based on race, religion, color, sex, age, disability, nationality, sexual orientation, gender identity or expression, or for any other reason covered.

At Cognizant we believe than our culture make us stronger!
Join us now!

#BeCognizant #IntuitionEngineered

The Cognizant community:
We are a high caliber team who appreciate and support one another. Our people uphold an energetic, collaborative and inclusive workplace where everyone can thrive.

  • Cognizant is a global community with more than 300,000 associates around the world.
  • We don’t just dream of a better way – we make it happen.
  • We take care of our people, clients, company, communities and climate by doing what’s right.
  • We foster an innovative environment where you can build the career path that’s right for you.

About us:
Cognizant is one of the world's leading professional services companies, transforming clients' business, operating, and technology models for the digital era. Our unique industry-based, consultative approach helps clients envision, build, and run more innovative and efficient businesses. Headquartered in the U.S., Cognizant (a member of the NASDAQ-100 and one of Forbes World’s Best Employers 2024) is consistently listed among the most admired companies in the world. Learn how Cognizant helps clients lead with digital at www.cognizant.com

Our commitment to diversity and inclusion:
Cognizant is an equal opportunity employer that embraces diversity, champions equity and values inclusion. We are dedicated to nurturing a community where everyone feels heard, accepted and welcome. Your application and candidacy will not be considered based on race, color, sex, religion, creed, sexual orientation, gender identity, national origin, disability, genetic information, pregnancy, veteran status or any other protected characteristic as outlined by federal, state or local laws.

Disclaimer: 
Compensation information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.

Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview.

Join our talent community

Haven’t found the right opportunity yet? Receive the latest updates on job opportunities, recruitment events and company news tailored just for you.

Sign up