India / India / TN / Chennai /

#: 744000087077460 / R25_0012454|IN

Mid-Senior Level

Full-Time

Job Description

Data Engineer, Chennai

We’re seeking a highly motivated Data Engineer to join our agile, cross-functional team and drive end-to-end data pipeline development in a cloud-native, big data ecosystem. You’ll leverage ETL/ELT best practices and data lakehouse paradigms to deliver scalable solutions. Proficiency in SQL, Python, Spark, and modern data orchestration tools (e.g. Airflow) is essential, along with experience in CI/CD, DevOps, and containerized environments like Docker and Kubernetes. This is your opportunity to make an impact in a fast-paced, data-driven culture.

Responsibilities

• Responsible for data pipeline development and maintenance.
• Contribute to development, maintenance, testing strategy, design discussions, and operations of the team.
• Participate in all aspects of agile software development including design, implementation, and deployment.
• Responsible for the end-to-end lifecycle of new product features / components.
• Ensuring application performance, uptime, and scale, maintaining high standards of code quality and thoughtful application design.
• Work with a small, cross-functional team on products and features to drive growth.
• Learning new tools, languages, workflows, and philosophies to grow.
• Research and suggest new technologies for boosting the product.
• Have an impact on product development by making important technical decisions, influencing the system architecture, development practices and more.

Qualifications

• Excellent team player with strong communication skills.
• B.Sc. in Computer Sciences or similar.
• 3-5 years of experience in Data Pipeline development.
• 3-5 years of experience in PySpark / Databricks.
• 3-5 years of experience in Python / Airflow.
• Knowledge of OOP and design patterns.
• 3-5 years of server-side technologies such as Java, Spring
• Experience with Docker containers, Kubernetes and Cloud environments
• Expertise in testing methodologies (Unit-testing, TDD, mocking).
• Fluent with large scale SQL databases.
• Good problem-solving and analysis abilities.

Requirements – Advantage

• Experience with Azure cloud services.
• Experience with Agile Development methodologies.
• Experience with Git.

https://jobs.smartrecruiters.com/NielsenIQ/744000087077460-data-engineer-airflow-python-pyspark-databricks