Data Engineer at Accenture

Mumbai, Maharashtra, India

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

TechnologyIndustries

Requirements

Minimum 7.5 years of experience in data engineering or software development
15 years full-time education (Bachelor’s or Master’s degree in Computer Science, Engineering, or related field)
3+ years hands-on experience with Databricks and PySpark
Strong Python programming skills, including writing reusable libraries and frameworks
Experience designing and implementing test frameworks for ETL/ELT pipelines
Experience building data quality frameworks for validation, monitoring, and anomaly detection
Proficiency in SQL and experience with cloud data warehouses (Snowflake, Redshift, BigQuery)
Familiarity with Datadog or similar monitoring tools for metrics, dashboards, and alerts
Experience integrating Databricks with AWS, Azure, or GCP services
Working knowledge of CI/CD, Git, Docker/Kubernetes, and automated testing
Strong understanding of data architecture patterns (medallion/lakehouse architectures preferred)
Must have skills: Databricks Unified Data Analytics Platform

Responsibilities

Develop and maintain ETL/ELT pipelines in Databricks using PySpark and Python
Build reusable, modular frameworks to accelerate development and enforce standards across pipelines
Implement test frameworks for automated unit, integration, and regression testing of pipelines
Design and maintain data quality frameworks to validate ingestion, transformation, and output
Optimize Spark jobs for performance, scalability, and cost-efficiency
Collaborate with data architects to define robust data models and design patterns
Integrate Databricks pipelines with cloud-native storage services (e.g., S3, ADLS, Snowflake)
Implement CI/CD pipelines for Databricks notebooks and jobs using Git, Jenkins, or Azure DevOps
Ensure pipelines follow best practices for modularity, reusability, and maintainability
Use Datadog to monitor pipeline performance, resource utilization, and system health
Build dashboards and alerts for proactive monitoring and troubleshooting
Analyze metrics and logs to identify bottlenecks and improve reliability
Partner with data scientists, analysts, and business stakeholders to translate requirements into scalable solutions
Conduct code reviews, enforce best practices, and mentor junior engineers
Promote knowledge-sharing of reusable frameworks, testing practices, and data quality approaches

Skills

Databricks

PySpark

Python

ETL

ELT

Spark

AWS

Azure

GCP

Datadog

Accenture

Global professional services for digital transformation

About Accenture

Accenture provides a wide range of professional services, including strategy and consulting, technology, and operations, to help organizations improve their performance. Their services assist clients in navigating digital transformation, enhancing operational efficiency, and achieving sustainable growth. Accenture's offerings include cloud migration, cybersecurity, artificial intelligence, and data analytics, which are tailored to meet the needs of various industries such as financial services, healthcare, and retail. What sets Accenture apart from its competitors is its extensive industry knowledge and ability to deliver comprehensive solutions that address both immediate challenges and long-term goals. The company's aim is to support clients in reducing their environmental impact while driving innovation and growth.

Dublin, IrelandHeadquarters

1989Year Founded

$8.5MTotal Funding

IPOCompany Stage

Consulting, Enterprise Software, CybersecurityIndustries

10,001+Employees

Risks

Rapid AI advancements may outpace Accenture's current capabilities, risking competitive disadvantages.

Integration challenges from multiple acquisitions could affect Accenture's operational efficiency.

The rise of AI-driven startups may disrupt Accenture's market share in customer service solutions.

Differentiation

Accenture's acquisitions enhance its capabilities in digital twin technology for financial services.

The company is expanding its expertise in net-zero infrastructure through strategic acquisitions.

Accenture's focus on software-defined vehicles positions it as a leader in automotive innovation.

Upsides

Accenture's investment in EMTECH supports central bank modernization amid digital currency evolution.

The acquisition of Award Solutions boosts Accenture's presence in the growing 5G and IoT markets.

Accenture's strategic acquisitions align with high-growth markets like digital twins and net-zero projects.

Land your dream remote job 3x faster with AI

Try Jobo Free