Senior Data Engineer (Pipeline)
Demandbase$190,000 - $284,000/year
Full Time
Senior (5 to 8 years)
Not SpecifiedCompensation
Mid-level (3 to 4 years)Experience Level
Full TimeJob Type
UnknownVisa
Consumer Intelligence, Data ProcessingIndustries
Requirements
- Significant experience in writing large-scale data pipelines using Python and Polars
- Mastery of engineering best practices, including writing code with unit and integration tests
- Experience with CI/CD pipelines (e.g., GitHub Actions, GitLab CI, Jenkins)
- Proficiency in mandatory technologies: Python (data engineering and automation), Polars (high-performance dataframe processing), PostgreSQL (data modeling, queries, and optimization), unit testing frameworks (e.g., pytest)
- Good-to-have: DuckDB, Kubernetes, cloud data environments (GCP, AWS, Azure), image processing or computer vision tools
Responsibilities
- Use modern Python libraries to create and maintain data pipelines for image comparison
- Design, program, deploy, and maintain large-scale data processing workflows
- Focus on reliability, performance, and maintainability in data pipelines
- Design complex pipelines involving decision-making
Skills
Python
Polars
PostgreSQL
pytest
CI/CD
GitHub Actions
GitLab CI
Jenkins
DuckDB
Kubernetes
GCP
AWS
Azure
Nielsen
Global measurement and data analytics provider
About Nielsen
Nielsen provides measurement and data analytics services to help businesses understand consumers and markets globally. The company operates through two main divisions: Nielsen Global Media, which offers reliable metrics for the media and advertising industries, and Nielsen Global Connect, which supplies consumer packaged goods manufacturers and retailers with actionable insights about the marketplace. Nielsen combines its proprietary data with other sources to give clients a comprehensive view of current trends and future opportunities. With a presence in over 100 countries, Nielsen aims to support companies in making informed decisions to drive innovation and growth.
Town of Clarkstown, New YorkHeadquarters
1963Year Founded
$45.1MTotal Funding
ACQUISITIONCompany Stage
Data & Analytics, ConsultingIndustries
10,001+Employees
Benefits
Health Insurance
401(k) Retirement Plan
401(k) Company Match
Unlimited Paid Time Off
Risks
Competition from digital-native analytics firms threatens Nielsen's market share.
Privacy regulations may impact Nielsen's data collection capabilities.
Private equity acquisition could lead to strategic shifts misaligned with client needs.
Differentiation
Nielsen offers comprehensive cross-platform measurement solutions, including mobile media markets.
The Diverse Media Equity program elevates diverse-owned media companies' visibility.
Nielsen Impact Score evaluates marketing value of university athletic programs.
Upsides
Acquisition of Telephia enhances Nielsen's mobile media measurement capabilities.
Partnerships with P&G and NMSDC support diverse media representation.
Collaboration with TikTok improves digital audience measurement services.
Related Jobs
RemoteRemoteSenior Analytics Engineer II
AlmaSalary not specified
- Full Time
- Senior (5 to 8 years)
RemoteStaff Integration Engineer
Chainalysis$110,000 - $150,000/year
- Full Time
- Senior (5 to 8 years)
RemoteStaff Data Engineer
Life360$166,500 - $245,000/year
- Full Time
- Expert & Leadership (9+ years)
RemoteSoftware Engineer, Data Systems (Python)
NorthbeamSalary not specified
- Full Time
- Junior (1 to 2 years)
RemoteSenior Cloud Engineer - Product Metrics
ClickhouseSalary not specified
- Full Time
- Senior (5 to 8 years)
RemotePrincipal Data Architect
Access SystemsSalary not specified
- Full Time
- Expert & Leadership (9+ years)
RemoteAnalytics Engineer
Machinify$180,000 - $200,000/year
- Full Time
- Mid-level (3 to 4 years)