Experience: 4-7 Years
Location: Bangalore
No. of Positions: 01
Role Description
As a Data Engineer, you will be responsible for end-to-end design and operation of enterprise-scale data pipelines and infrastructure using Azure Databricks, Azure Service Fabric, and related Azure services. You will ensure data availability, reliability, and quality across HR, Procurement, and Finance functions while applying software engineering best practices such as modular design, TDD, and CI/CD automation.
You will collaborate with business stakeholders, architects, ML/AI engineers, Fabric service teams, and Power BI developers to align delivery with business outcomes, operating within an agile team structure.
Key Responsibilities
Data Pipeline & Microservices Engineering: Design and operate scalable, high-performance data pipelines and service-oriented components using Azure Databricks, Azure Service Fabric, and other Azure ecosystem services.
Code as Assets: Develop all artifacts as version-controlled code (Git) with CI/CD automation.
Data Reliability & Quality: Ensure availability, integrity, and quality of data supporting analytics in HR, Procurement, and Finance.
Analytical Assets: Build data products and assets leveraging software engineering best practices.
Engineering Practices: Apply modular design, TDD, reusable code patterns, and performance optimization.
Fabric Service Monitoring & Management: Deploy, monitor, and maintain microservices on Azure Service Fabric, ensuring high availability, resiliency, and scalability.
Collaboration: Work closely with architects, ML/AI engineers, functional IT, Power BI developers, Fabric service teams, and business stakeholders to align solutions with requirements.
Agile Contribution: Actively participate in sprint planning, retrospectives, and demos within an enterprise agile product team.
Monitoring & Optimization: Monitor and improve data & service infrastructure performance, proactively identifying bottlenecks, failure points, and optimization opportunities.
Required Skills & Qualifications
Strong experience with Azure Databricks, Azure Service Fabric, and the broader Azure Data ecosystem (ADF, Synapse, Data Lake, Event Hub).
Proficiency in Python / Scala / SQL for data engineering.
Hands-on experience in CI/CD automation, Git version control, and test-driven development (TDD).
Deep understanding of data modeling, ETL/ELT pipelines, modular design patterns.
Experience in building, deploying, and maintaining microservices in Azure Service Fabric (or equivalent orchestrated service mesh / containerized services).
Ability to build analytical data products using engineering-first principles.
Experience working in agile, enterprise-scale environments.
Strong problem-solving and debugging skills.
Preferred Qualifications
Exposure to AI/ML pipelines integration in data workflows.
Experience with Power BI or equivalent BI platforms.
Knowledge of DevOps practices in a cloud-first environment.
Microsoft Azure Data Engineer Associate certification (or equivalent).
Familiarity with containerization technologies (e.g. Docker), orchestration (e.g. Kubernetes), in addition to Service Fabric.
Desired Profile
A pragmatic engineer passionate about building reliable, reusable, scalable data and microservice solutions.
Collaborative and able to work seamlessly with business, IT, analytics, and infrastructure (service fabric) stakeholders.
Self-driven with strong ownership, accountability, and problem-solving
