Data Analytics Engineer
dv01Salary not specified
- Full Time
- Junior (1 to 2 years)
$140,000 – $180,000Compensation
Mid-level (3 to 4 years), Senior (5 to 8 years)Experience Level
InternshipJob Type
UnknownVisa
Data Engineering, Cloud ComputingIndustries
Requirements
Candidates should possess at least 6 years of experience in Data Engineering, with a focus on building data pipelines, data platforms, and ETL processes on Hadoop and Databricks. They should have strong expertise in Databricks (Spark on Databricks, Delta Lake, etc.) preferably on GCP, and demonstrate expertise in the Hadoop ecosystem (Spark, MapReduce, HDFS) with solid foundations of Spark and its internals. Proficiency in Scala and Java, along with strong SQL knowledge, are also required.
Responsibilities
The Data Engineer III will assess and plan the migration of LivePerson’s existing data processing ecosystem from Hadoop to Databricks on GCP, including reviewing the existing Hadoop infrastructure, analyzing Java and Scala codebases, identifying dependencies, and proposing a migration plan. They will refactor Spark jobs to run efficiently on Databricks, migrate MapReduce jobs, update Java and Scala code, develop unit and integration tests, compare performance metrics, implement error handling, and fine-tune Spark configurations for performance improvements. Furthermore, they will deploy migrated jobs to production, document changes, and provide knowledge transfer to internal teams.
Skills
Spark
Scala
Java
SQL
Databricks
Delta Lake
Hadoop
MapReduce
HDFS
GCP
Spark on Databricks
ETL
Unit Testing
Integration Testing
Performance Tuning
LivePerson
Digital customer engagement and communication tools
About LivePerson
LivePerson provides businesses with tools to engage with their customers through digital channels. Their main product, LiveEngage, facilitates real-time interactions via live chat, messaging, and AI-driven bots across various platforms like websites, mobile apps, and social media. LivePerson's subscription-based model allows clients to access a suite of tools, including APIs and SDKs, to manage customer interactions and analyze performance. Unlike competitors, LivePerson focuses on enhancing customer experience with features like "click to message" and "click to chat," which enable direct communication from digital platforms. The company's goal is to improve customer service and engagement for businesses, ultimately leading to increased sales and satisfaction.
Key Metrics
New York City, New YorkHeadquarters
1995Year Founded
$40.5MTotal Funding
IPOCompany Stage
Data & Analytics, Consumer Software, Enterprise SoftwareIndustries
1,001-5,000Employees