Data Engineer

Position Overview

Anew is seeking a passionate and creative Data Engineer to play a crucial role in evolving the organization's data-driven initiatives. As an architect and builder of data infrastructure, you will ensure that data is collected, stored, processed, and made accessible for analysis, reporting, and machine learning.

Company Information

At Anew, we are passionate about delivering meaningful, measurable climate impact in a decarbonizing economy. Inspired by our values of integrity, trust, creativity, and hope, and grounded by decades of experience and unsurpassed knowledge in environmental markets, we make it possible for organizations and communities to thrive while building a sustainable future. We’re seeking high-energy, creative team players eager to apply their talents to the defining challenge of our time. If you’re committed to excellence and excited by the prospect of doing well by doing good, we look forward to hearing from you.

Responsibilities and Competencies

Design and Implement ETL/ELT Processes

• Build robust Extract, Transform, Load (ETL) or Extract, Load, Transform (ELT) pipelines to move data from various sources (databases, APIs, streaming sources, external data providers) into data warehouses, data lakes, or other data repositories.
• Create efficient and scalable data pipelines to handle large volumes of structured and unstructured data.
• Write code and scripts to automate repetitive tasks in data collection, processing, and delivery.
• Continuously monitor pipeline performance, identify and resolve data-related issues, errors, and performance bottlenecks.

Data Architecture and Storage

• Select appropriate technologies for data storage (e.g., relational databases, NoSQL databases, data lakes, data warehouses, cloud storage services) and design efficient data models and schemas.
• Optimize data storage for high-performance queries and efficient analytical and operational use cases.
• Research and implement new data technologies, tools, and frameworks to improve data infrastructure and processes.
• Deploy and manage data solutions in the cloud using Azure.

Data Quality, Governance, and Security

• Implement validation rules, cleansing procedures, and monitoring systems to detect and rectify anomalies, ensuring data accuracy.
• Define standards and policies for data usage, ensuring consistency, reliability, and compliance (e.g., GDPR, HIPAA).
• Implement security controls and access management policies to protect sensitive information from unauthorized access or theft.

Collaboration and Support

• Set up dashboards, analytics tools, and API endpoints to make processed data accessible to end-users and applications.
• Create comprehensive documentation for data pipelines, architecture, and processes to facilitate system transparency.
• Enable organizations to make informed business decisions by providing clean, reliable, and accessible data.

Skills

• Programming Languages: Python, Java, SQL and related technologies
• Database Systems: Strong knowledge of relational databases (e.g., PostgreSQL, MySQL, SQL Server) and NoSQL databases (e.g., MongoDB, Cassandra).
• Data Warehousing: Experience with data warehousing and platforms (e.g., Snowflake).
• ETL Tools: Proficiency with various ETL tools such as Snowflake and Databricks.
• Cloud Platforms: Experience with Azure Suite.
• Big Data Technologies: Hadoop, Spark, Kafka, Hive.
• Data Modeling and Schema Design: Ability to design efficient data models.
• Scripting