Job Description: Data Engineer

Position Overview

• Location Type: Remote
• Employment Type: Full-time
• Salary: Not specified

PayPay is a FinTech company with over 65M users as of August 2024. Our team is diverse, representing members from over 50 countries. Our vision is unlimited, and we encourage taking risks and challenging ourselves to realize a future beyond our imagination. We are looking for individuals who can embrace this challenge, refresh the product at breakneck speed, and promote PayPay with professionalism and passion.

Please note that you cannot apply or be selected in parallel with PayPay Corporation, PayPay Card Corporation, and PayPay Securities Corporation.

Job Description

PayPay’s growth is driving a rapid expansion of PayPay product teams, and a robust Data Engineering Platform is crucial to support our business needs. We are seeking a Data Engineer for the Data Insights department, whose mission is to drive product improvements by engineering systems based on a scientific understanding of user and merchant behavior.

Main Responsibilities

• Design, develop, and maintain scalable data ingestion pipelines using AWS Glue, Step Functions, Lambda, and Terraform.
• Design, build, and maintain infrastructure to continuously support the improvement and deployment of ML models.
• Optimize and manage large-scale data pipelines to ensure high performance, reliability, and efficiency.
• Implement data processing workflows using Hudi, Delta Lake, Spark, and Scala.
• Maintain and enhance Lakeformation and Glue Data Catalog for effective data management and discovery.
• Collaborate with cross-functional teams to ensure seamless data flow and integration across the organization.
• Implement best practices for observability, data governance, security, and compliance.

Qualifications

• 5+ years of experience as a Data Engineer or in a similar role.
• Some familiarity with building machine learning systems or data infrastructures supporting machine learning development and deployment is preferable.
• Hands-on experience with Apache Hudi, Delta Lake, Spark, and Scala.
• Experience designing, building, and operating a DataLake or Data Warehouse.
• Knowledge of Data Orchestration tools such as Airflow, Dagster, or Prefect.
• Strong expertise in AWS services, including Glue, Step Functions, Lambda, and EMR.
• Familiarity with change data capture tools like Canal, Debezium, and Maxwell.
• Experience with data warehousing tools like AWS Athena, BigQuery, or Databricks.
• Proficiency in Python and SQL (any variant), preferably experience in Scala and/or Java.
• Experience with data cataloging and metadata management.

Company Information

PayPay is a FinTech company.