Job Description

Location Type: Remote Employment Type: Full-time

Position Overview

Do you want to boost your career and collaborate with expert, talented colleagues to solve and deliver against our clients' most important challenges? We are growing and are looking for people to join our team. You'll be part of an entrepreneurial, high-growth environment of 300,000 employees. Our dynamic organization allows you to work across functional business pillars, contributing your ideas, experiences, diverse thinking, and a strong mindset. Are you ready?

Job Overview

We are looking for a skilled and experienced Data Engineer to join our data team. The Data Engineer will be responsible for designing, building, and maintaining scalable and efficient data pipelines and architectures that support various data-driven initiatives. The ideal candidate will have strong technical expertise in data processing, data storage, and data integration, and will work closely with data scientists, analysts, and other stakeholders to ensure the integrity, availability, and reliability of the data.

Key Responsibilities

Design and Build Data Pipelines

• Develop, construct, test, and maintain scalable data pipelines for ingestion, processing, and storage of large data sets from multiple sources.
• Optimize data architectures for both structured and unstructured data to support batch and real-time data processing.

Data Integration and Management

• Integrate data from different source systems (APIs, databases, cloud services) into the data lake or data warehouse.
• Implement and manage ETL (Extract, Transform, Load) processes for efficient data movement, cleaning, and transformation.
• Ensure data quality, consistency, and integrity through proper validation and testing.

Database and Data Warehousing Solutions

• Design and implement database solutions that support data storage, transformation, and querying.
• Build and maintain data warehouse solutions to facilitate business intelligence and analytics needs.

Cloud Infrastructure

• Work with cloud platforms such as AWS, Azure, or Google Cloud to implement data storage, processing, and streaming architectures.
• Set up and manage cloud-based data infrastructure, ensuring scalability, security, and performance optimization.

Collaboration and Stakeholder Management

• Collaborate with data scientists, analysts, software engineers, and business teams to understand data requirements and deliver solutions that meet their needs.
• Document data flows, schemas, and pipeline designs to ensure transparency and understanding across the team.

Data Governance and Security

• Implement data governance standards, data security, and compliance measures to protect sensitive data.
• Ensure compliance with data privacy regulations such as GDPR, CCPA, etc.
• Define and maintain data retention policies and ensure adherence to best practices for data management.

Performance Monitoring and Optimisation

• Monitor data pipeline performance, identify bottlenecks, and troubleshoot issues as they arise.
• Continuously improve data processes to ensure they are efficient and scalable for growing data sets.

Automation and Tools Development

• Automate repetitive tasks and processes to streamline data engineering workflows.
• Develop tools and frameworks for data access, management, and analysis, enabling the business to easily work with data.

Requirements

Skills and Qualifications

• Essential Skills:
  • Bachelor’s degree in Computer Science, Engineering, or a related field.
  • Proven experience as a Data Engineer, Software Engineer, or similar role in large-scale data platform implementation (project team size of >= 10 FTEs and/or project duration 6 months or more).
  • Strong experience in SQL and relational database systems (e.g., MySQL, PostgreSQL, Oracle).
  • Proficiency in programming languages such as Python, Java, or Scala for data processing.