Senior Data Engineer
DemandbaseSalary not specified
- Full Time
- Senior (5 to 8 years)
Not SpecifiedCompensation
Junior (1 to 2 years)Experience Level
Full TimeJob Type
UnknownVisa
Information Technology & ServicesIndustries
Requirements
Candidates should possess a Bachelor's degree in Computer Science, BIS, CIS, Electrical Engineering, or Operations Research, with coursework or experience in Numerical Analysis, Mathematics, or Statistics being a plus. They must have at least 5-8 years of experience as a data engineer, proficiency in using the Spark framework (Python or Scala), extensive knowledge of Data Warehousing concepts, and programming experience in Python, SQL, and Scala. Hands-on experience with building data pipelines using Apache Spark (preferably in Databricks) and Airflow, designing and delivering solutions using Azure, including Azure Storage, Azure SQL Data Warehouse, and Azure Data Lake, is required. Experience with big data technologies (Hadoop) and Databricks & Azure Big Data Architecture Certification is preferred.
Responsibilities
The Data Engineer will be responsible for assembling large, complex sets of data to meet business requirements, identifying and implementing internal process improvements, including infrastructure re-design and automation. They will build required infrastructure for data extraction, transformation, and loading from various sources using Azure, Databricks, and SQL technologies, transform R&D algorithms into production-ready code, integrate finished models into larger data processes using UNIX scripting languages, produce and maintain documentation for released data sets and programs, and ensure quality deliverables to clients by following quality processes and visually inspecting data for reasonableness.
Skills
Scala
Databricks
Azure
SQL
Python
Spark
UNIX scripting
Mathematics
Data Warehousing
Nielsen
Global measurement and data analytics provider
About Nielsen
Nielsen provides measurement and data analytics services to help businesses understand consumers and markets globally. The company operates through two main divisions: Nielsen Global Media, which offers reliable metrics for the media and advertising industries, and Nielsen Global Connect, which supplies consumer packaged goods manufacturers and retailers with actionable insights about the marketplace. Nielsen combines its proprietary data with other sources to give clients a comprehensive view of current trends and future opportunities. With a presence in over 100 countries, Nielsen aims to support companies in making informed decisions to drive innovation and growth.
Town of Clarkstown, New YorkHeadquarters
1963Year Founded
$45.1MTotal Funding
ACQUISITIONCompany Stage
Data & Analytics, ConsultingIndustries
10,001+Employees