LivePerson

Data Engineer III (Databricks)

Hyderabad, Telangana, India

$140,000 – $180,000Compensation
Mid-level (3 to 4 years), Senior (5 to 8 years)Experience Level
InternshipJob Type
UnknownVisa
Data Engineering, Cloud ComputingIndustries

Position Overview

  • Location Type: Remote
  • Employment Type: Full-time
  • Salary: Not specified

LivePerson is the global leader in enterprise conversations. Hundreds of the world’s leading brands — including HSBC, Chipotle, and Virgin Media — use our award-winning Conversational Cloud platform to connect with millions of consumers. We power nearly a billion conversational interactions every month, providing a uniquely rich data set and safety tools to unlock the power of Conversational AI for better customer experiences.

At LivePerson, we foster an inclusive workplace culture that encourages meaningful connection, collaboration, and innovation. Everyone is invited to ask questions, actively seek new ways to achieve success, and reach their full potential. We are continually looking for ways to improve our products and make things better. This means spotting opportunities, solving ambiguities, and seeking effective solutions to the problems our customers care about.

We are looking for an experienced Data Engineer to provide data engineering expertise and support to various analytical products of LivePerson, and assist in migrating our existing data processing ecosystem from Hadoop (Spark, MapReduce, Java, and Scala) to Databricks on GCP. The goal is to leverage Databricks’ scalability, performance, and ease of use to enhance our current workflows.

Requirements

  • 6+ years of experience in Data Engineering with focus on building data pipelines, data platforms and ETL (Extract, transform, Load) processes on Hadoop and Databricks.
  • Strong Expertise in Databricks (Spark on Databricks, Delta Lake, etc.) preferably on GCP.
  • Strong expertise in the Hadoop ecosystem (Spark, MapReduce, HDFS) with solid foundations of Spark and its internals.
  • Proficiency in Scala and Java.
  • Strong SQL knowledge.

Responsibilities

  • Assessment and Planning:
    • Review the existing Hadoop infrastructure, including Spark and MapReduce jobs.
    • Analyze Java and Scala codebases for compatibility with Databricks.
    • Identify dependencies, libraries, and configurations that may require modification.
    • Propose a migration plan with clear timelines and milestones.
  • Code Migration:
    • Refactor Spark jobs to run efficiently on Databricks.
    • Migrate MapReduce jobs where applicable or rewrite them using Spark DataFrame/Dataset API.
    • Update Java and Scala code to comply with Databricks' runtime environment.
  • Testing and Validation:
    • Develop unit and integration tests to ensure parity between the existing and new systems.
    • Compare performance metrics before and after migration.
    • Implement error handling and logging consistent with best practices in Databricks.
  • Optimization and Performance Tuning:
    • Fine-tune Spark configurations for performance improvements on Databricks.
    • Optimize data ingestion and transformation processes.
  • Deployment and Documentation:
    • Deploy migrated jobs to production in Databricks.
    • Document changes, configurations, and processes thoroughly.
    • Provide knowledge transfer to internal teams if required.

Application Instructions

  • Not specified. (Contact LivePerson for application details.)

Skills

Spark
Scala
Java
SQL
Databricks
Delta Lake
Hadoop
MapReduce
HDFS
GCP
Spark on Databricks
ETL
Unit Testing
Integration Testing
Performance Tuning

LivePerson

Digital customer engagement and communication tools

About LivePerson

LivePerson provides businesses with tools to engage with their customers through digital channels. Their main product, LiveEngage, facilitates real-time interactions via live chat, messaging, and AI-driven bots across various platforms like websites, mobile apps, and social media. LivePerson's subscription-based model allows clients to access a suite of tools, including APIs and SDKs, to manage customer interactions and analyze performance. Unlike competitors, LivePerson focuses on enhancing customer experience with features like "click to message" and "click to chat," which enable direct communication from digital platforms. The company's goal is to improve customer service and engagement for businesses, ultimately leading to increased sales and satisfaction.

New York City, New YorkHeadquarters
1995Year Founded
$40.5MTotal Funding
IPOCompany Stage
Data & Analytics, Consumer Software, Enterprise SoftwareIndustries
1,001-5,000Employees

Benefits

Health Insurance
Dental Insurance
Vision Insurance
Life Insurance
Disability Insurance
401(k) Company Match
401(k) Retirement Plan
Paid Vacation
Paid Sick Leave
Paid Holidays
Parental Leave
Family Planning Benefits
Fertility Treatment Support
Professional Development Budget
Tuition Reimbursement
Wellness Program
Mental Health Support
Flexible Work Hours
Remote Work Options
Employee Stock Purchase Plan

Risks

Emerging AI platforms could erode LivePerson's market share with competitive pricing.
Rapid AI advancements may require significant R&D investment to maintain competitiveness.
Data privacy concerns and regulatory changes could impact LivePerson's operations.

Differentiation

LivePerson's Conversational Cloud platform offers seamless AI-driven customer engagement solutions.
The company partners with industry leaders like Telnyx to enhance real-time communication.
LivePerson's subscription-based model ensures a steady revenue stream from diverse industries.

Upsides

Increased adoption of AI-driven solutions boosts LivePerson's market presence.
Recognition in Inc.'s 2024 Power Partner Awards enhances LivePerson's B2B reputation.
Partnerships with companies like Capitec expand LivePerson's customer service capabilities.

Land your dream remote job 3x faster with AI