Data Engineer

Our Story

Able has experienced significant growth and evolution since its founding in 2013.

• Chapter 1 (2013): Founded as a product and engineering hub for early-stage startups, operating as a hybrid in-house/external shared services model. This phase established our operational and cultural foundation.
• Chapter 2 (2019): Expanded vision to grow beyond our initial partner base, achieving success in new relationships and delivering high-value work.
• Chapter 3 (2023): Entered a new phase with expanded ambition, focusing on two key audiences:
  • Venture Capital: Providing trusted product and technology solutions for VC firms to distribute across their portfolios at scale.
  • Private Equity: Offering trusted solutions to catalyze growth for PE portfolio companies at scale.
• Chapter 3a (Current Phase): Accelerated by applied AI, focusing on creating practices that, combined with world-class talent, can deliver software significantly faster than legacy techniques. This aims to increase value for partners by enhancing their product organizations' capacity.

About the Role

This Data Engineer role will support the Director of Engineering and the broader Engineering team, working cross-functionally with teams across Able and closely aligning with the Engineering discipline. The role involves direct partnership with a specific client, collaborating across Product, Design, and Engineering to deliver robust, scalable data solutions that address critical business needs.

Day-to-Day Responsibilities

Strategic Architecture Leadership

• Shape large-scale data architecture vision and roadmaps for client engagements.
• Establish governance, security frameworks, and regulatory compliance standards.
• Lead strategy for platform selection, integration, and scaling.
• Guide organizations in adopting data lakehouse and federated data models.

Client/Partner Value Creation

• Lead technical discovery sessions to understand client needs.
• Translate complex architectures into clear, actionable value for stakeholders.
• Build trusted advisor relationships and guide strategic decisions.
• Align architecture recommendations with business growth and goals.

Technical Architecture & Implementation

• Design and implement modern data lakehouse architectures using Delta Lake and Databricks.
• Build and manage ETL/ELT pipelines at scale using Spark (PySpark preferred).
• Leverage Delta Live Tables, Unity Catalog, and schema evolution features.
• Optimize storage and queries on cloud object storage (e.g., AWS S3, Azure Data Lake).
• Integrate with cloud-native services such as AWS Glue, GCP Dataflow, and Azure Synapse Analytics.
• Implement data quality monitoring, lineage tracking, and schema versioning.
• Build scalable pipelines using tools like Apache Airflow, Step Functions, and Cloud Composer.

Business Impact & Solution Design

• Develop cost-optimized, scalable, and compliant data solutions.
• Design Proofs of Concept (POCs) and pilots to validate technical approaches.
• Translate business requirements into production-ready data systems.
• Define and track success metrics for platform and pipeline initiatives.

What We’re Looking For

The ideal candidate will possess:

• 10+ years of data engineering experience with enterprise-scale systems.
• Expertise in Apache Spark and Delta Lake, including ACID transactions, time travel, Z-ordering, and compaction.
• Deep knowledge of Databricks (Jobs, Clusters, Workspaces, Delta Live Tables, Unity Catalog).
• Experience building scalable ETL/ELT pipelines using [Information truncated]