Data Engineer at WPP

Gurugram, Haryana, India

Apply Now

Not SpecifiedCompensation

Mid-level (3 to 4 years)Experience Level

Full TimeJob Type

UnknownVisa

Marketing, AdvertisingIndustries

Requirements

Minimum of a Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field preferred
4-6+ years of relevant experience in data engineering, with a strong focus on data ingestion and integration
4+ years of strong, hands-on experience in Python, with an emphasis on PySpark and libraries for API interaction (e.g., requests)
Strong hands-on experience with Spark architecture, writing and optimizing PySpark and Spark SQL jobs for ingestion and basic transformation
Deep, practical experience with Databricks Auto Loader, COPY INTO, and Structured Streaming
Solid understanding of Delta Lake for creating reliable landing zones for raw data, proficient in writing data to Delta tables and understanding core concepts like ACID transactions and schema enforcement
Proficiency in SQL

Responsibilities

Design, build, and maintain robust data ingestion pipelines to collect data from diverse sources such as APIs, streaming sources (e.g., Kafka, Event Hubs), relational databases (via JDBC), and cloud storage
Heavily utilize Databricks Auto Loader and COPY INTO for the efficient, incremental, and scalable ingestion of files into Delta Lake
Develop and manage Databricks Structured Streaming jobs to process near-real-time data feeds
Ensure the reliability, integrity, and freshness of the Bronze layer in the Medallion Architecture, which serves as the single source of truth for all raw data
Perform initial data cleansing, validation, and structuring to prepare data for further transformation in the Silver layer
Monitor, troubleshoot, and optimize ingestion pipelines for performance, cost, and stability
Develop Python scripts and applications to automate data extraction and integration processes
Work closely with platform architects and other data engineers to implement best practices for data ingestion and management
Document data sources, ingestion patterns, and pipeline configurations
Conform to agile development practices, including version control (Git), CI/CD, and automated testing

Skills

PySpark

Python

SQL

Databricks

Auto Loader

Structured Streaming

Kafka

Event Hubs

JDBC

Google Analytics 4

WPP

Global marketing and communications services provider

About WPP

WPP operates in the marketing and communications industry, providing a variety of services such as branding, digital marketing, media planning, market research, public relations, and business transformation. Their approach integrates creativity with data and technology, helping clients build strong brands and engage effectively with their audiences. WPP stands out from competitors by focusing on innovation and sustainability, participating in global initiatives like the United Nations Climate Change Conference, and producing thought leadership content like the Atticus Journal. The company's goal is to be a strategic partner for businesses, aiding them in navigating the complexities of modern marketing and achieving their objectives.

London, United KingdomHeadquarters

2015Year Founded

IPOCompany Stage

Consulting, Social ImpactIndustries

10,001+Employees

Risks

Increased competition from Publicis, gaining market share and outperforming WPP in certain areas.

Potential over-reliance on strategic partnerships may limit WPP's flexibility and independence.

Challenges in effectively integrating AI into operations indicate potential gaps in current capabilities.

Differentiation

WPP excels in creative transformation, combining creativity with data and technology.

The company is a leader in sustainability, participating in global initiatives like the UN Climate Change Conference.

WPP's strategic partnerships, such as with Universal Music Group, enhance its unique market offerings.

Upsides

Increased focus on AI-driven marketing solutions enhances WPP's service offerings and client engagement.

Strategic partnerships with music industry leaders provide unique marketing opportunities and vast music access.

Winning the 'Omnichannel Excellence Award' highlights GroupM's leadership in integrated marketing solutions.

Land your dream remote job 3x faster with AI

Try Jobo Free