Data Engineer

Position Overview

• Location Type: Remote
• Employment Type: Full-time
• Salary: Not specified

The Commodities Market Data team at Kpler is responsible for the ingestion, filtering, enrichment, and distribution of externally sourced data related to maritime trade. This data is critical for Kpler’s business processing entities to provide a live and accurate picture of the global trade to their customers. Kpler is dedicated to simplifying global trade information and providing valuable insights to organizations in commodities, energy, and maritime sectors.

Mission/Responsibilities

• Work alongside other engineers to develop and implement core data ingestion and distribution pipelines and associated back-end systems based on project requirements and design specifications.
• Help troubleshoot and optimize existing workflows to improve performance and efficiency.
• Ensure data integrity and accuracy across various pipelines.
• Demonstrate strong analytical and debugging skills with a proactive approach to learning.

Requirements

• Proficient in Python: With experience in data manipulation and transformation (such as with Pandas).
• Data Integration Knowledge: Understanding of data integration, ETL processes, and batch/streaming data processing.
• Containerization & Orchestration: Familiarity with containerisation and orchestration tools (e.g., Docker, Kubernetes, Airflow).
• SQL/RDBMS/Big Data Technologies: Experience with SQL, RDBMS or Big Data technologies.
• Git & Agile: Comfortable working with Git, code reviews, and Agile methodologies.
• Cloud Experience: Experience working with AWS (or another cloud provider), using Terraform (Nice to have).

Company Information

Kpler is a dynamic company dedicated to nurturing connections and innovating solutions to tackle market challenges head-on. They foster relationships and develop creative solutions to address market challenges. Kpler is committed to providing a fair, inclusive and diverse work-environment. They believe that different perspectives lead to better ideas, and better ideas allow them to better understand the needs and interests of their diverse, global community. They welcome people of different backgrounds, experiences, abilities and perspectives and are an equal opportunity employer.

Application Instructions

• Don’t meet every single requirement? Research shows that women and people of color are less likely than others to apply if they feel like they don’t match 100% of the job requirements. Don’t let the confidence gap stand in your way, we’d love to hear from you! We understand that experience comes in many different forms and are dedicated to adding new perspectives to the team.
• Application instructions are not provided in the original text.