Software Development Engineer II

Design, develop, deploy & maintain robust, scalable data pipelines using Apache Spark and Scala with Hive and Hadoop, OR Azure Databricks ecosystem while adhering to modern security and coding best practices.

· Apply strong knowledge of design patterns, data structures, algorithms, and SOLID principles to deliver high-quality, maintainable code.

· Work closely with DevOps, platform and product teams to ensure that solutions meet business requirements and are reliable in production.

· Participate in product demos, design discussions, and release reviews, bringing your technical expertise to shape our products.

· Continuously improve development processes and engineering practices, sharing knowledge and mentoring others.

· Stay up to date with the latest tools, technologies, and frameworks in the Data Engineering ecosystem.

· Deploy, monitor, and maintain workflows on Oozie/Airflow/Databricks Jobs/Databricks Lakeflow schedulers.

· Design and work with HIVE, S3, Hadoop/HDFS, different file formats, to build efficient, performant data pipelines.

· Follow Agile best practices using tools like GitHub Actions, Sonar, Snyk to ensure we follow CICD and apply automation to what we do.

· Adoption of AI tools in code and test development phases to reduce build time & deliver change faster.

· Understanding of Kubernetes (AKS) and leverage containerization best practices with Docker, will be an added advantage.

· Provide Support to TESCO business for the system that you, along with your team, owns.

Experience:

6-7 years overall in Data Engineering, at least 2-3 years in Azure Databricks ecosystem. Excellent communication skills.

Skills needed:

· Expert-level:

o Strong experience with Databricks (Delta Lake, notebooks, jobs/lakeflow, clusters) & building data pipelines using that

o Hands-on experience with Azure cloud data stack

o Strong expertise on Apache Spark v3/4, Scala, Scalatest, Shell scripting, Hadoop ecosystem

o Proficient in performance optimisation with Spark

o Proficient in source control like Github & its usage

o Have proficient understanding of OOP, Data structures, SOLID principles, Clean Code, Data Design Patterns & their application in Data Engineering

o Expertise in Hive

· Intermediate-level:

o Experience of Orchestration tools like Oozie, Airflow

o Experience on any other distributed SQL engines like Dremio SQL

o Experience working with containers & container orchestration like Docker, Kubernetes and distributed object storage like S3

o Code quality, code security, application monitoring leveraging tools like SonarQube, Snyk, Splunk, New Relic

o Able to work with CICD pipelines using Github Actions, Jenkins

o Artefact repository like Nexus, ACR

o Any reporting tool preferably Tableau

o Data governance using Unity Catalog

At Tesco, inclusion is at the heart of everything we do. We believe in treating everyone fairly and with respect, valuing individuality and uniqueness to create a true sense of belonging. Diversity and inclusion are deeply embedded in our values—we treat people how they want to be treated. Our goal is for all colleagues to feel they can be themselves at work, and we are committed to helping them thrive. Across the Tesco group, we are building an inclusive workplace that actively celebrates the cultures, personalities, and preferences of our colleagues, who in turn contribute to the success of our business and reflect the diversity of the communities we serve. 

 

 At Tesco Bengaluru, we are proud to be a Disability Confident Committed Employer, highlighting our commitment to creating a supportive environment for individuals with disabilities. We are dedicated to offering equal opportunities for all candidates and encourage applicants with disabilities to apply. Our recruitment process is fully accessible, and we are happy to provide reasonable adjustments during interviews. If you need any accommodations to participate in the recruitment process, please let us know. We are here to ensure that everyone has the chance to succeed. 

 

 We also believe in fostering a work environment where you can excel both professionally and personally. Our hybrid model allows you to work flexibly—spend 60% of your week collaborating in person with colleagues at our office locations or local sites, and the rest of the time working remotely. We understand that everyone’s life journey is unique, whether you are starting your career, pursuing passions, or navigating life changes, and we are here to support you. Flexibility is a core part of our culture, and we encourage open conversations about how we can best accommodate your needs, so talk to us throughout your application process on the support required.