Lead - Data Platform Engineering at FreshBooks

Chennai, Tamil Nadu, India

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

Technology, SoftwareIndustries

Requirements

Strong programming expertise in one or more of the following: Scala, Spark, Java, or Python
6 - 10 Years of experience
Proven experience working with Kafka (Confluent or Apache) for building event-driven or CDC-based pipelines
Hands-on experience with distributed data processing frameworks (Apache Spark, Databricks, or Flink) for large-scale data handling
Solid understanding of Kubernetes for deploying and managing scalable, resilient data workloads (EKS experience preferred)
Practical experience with AWS Cloud Services such as S3, Lambda, EMR, Glue, IAM, and CloudWatch
Strong expertise in designing and implementing MapReduce jobs to process and transform large-scale datasets efficiently across distributed systems
Capable of optimizing MapReduce workflows for performance, fault tolerance, and scalability in big data environments

Responsibilities

Own and manage the enterprise Data Lake infrastructure on AWS and Databricks, ensuring reliability, scalability, and governance
Design, develop, and optimize data ingestion and transformation pipelines from MySQL to Kafka (CDC pipelines) and from Kafka to Databricks using Spark Structured Streaming
Build robust batch and near real-time data pipelines capable of handling high-volume, high-velocity data efficiently
Develop and maintain metadata-driven data processing frameworks, ensuring consistency, lineage, and traceability
Implement and maintain strong observability and monitoring systems (logging, metrics, alerting) using Prometheus, Grafana, or equivalent tools
Work closely with Product, Regulatory, and Security teams to ensure compliance, privacy, and data quality across the data lifecycle
Collaborate with cross-functional teams to build end-to-end data lakehouse solutions integrating multiple systems and data sources
Apply best practices in code quality, CI/CD automation (Jenkins, GitHub Actions), and infrastructure as code (IaC) for deployment consistency
Ensure system reliability and scalability through proactive monitoring, performance tuning, and fault-tolerant design
Stay up to date with the latest technologies in data engineering, streaming, and distributed systems, and drive continuous improvements

Skills

Data Engineering

Data Lake

Databricks

AWS

MySQL

Kafka

Spark

CDC Pipelines

Spark Structured Streaming

Data Pipelines

Data Governance

FreshBooks

Cloud-based accounting software for businesses

About FreshBooks

FreshBooks provides cloud-based accounting software aimed at simplifying financial management for small to medium-sized businesses and self-employed professionals. The software allows users to efficiently track expenses, time, and invoices, making it accessible even for those without accounting experience. Users subscribe to the service with various pricing tiers, and they can take advantage of a 30-day money-back guarantee and an 80% discount for the first four months. FreshBooks stands out from competitors by offering a user-friendly interface and mobile accessibility, enabling users to manage their finances on the go. The company's goal is to help business owners and accountants streamline their financial processes, improve organization, and enhance client relationships.

Toronto, CanadaHeadquarters

2003Year Founded

$149.6MTotal Funding

DEBTCompany Stage

Consumer Software, FintechIndustries

201-500Employees

Benefits

Health Insurance

Dental Insurance

401(k) Company Match

Stock Options

Unlimited Paid Time Off

Parental Leave

Home Office Stipend

Professional Development Budget

Risks

Increased competition from market leaders may impact FreshBooks' market share.

Dependency on Stripe's technology could pose risks if Stripe faces issues.

Economic downturns may threaten FreshBooks' subscription-based revenue model.

Differentiation

FreshBooks offers a user-friendly platform for non-accountants, simplifying financial management.

The company provides mobile-friendly solutions, catering to the rise of remote work.

FreshBooks' integration with Shopify enhances e-commerce capabilities for online businesses.

Upsides

FreshBooks' subscription model aligns with industry trends, boosting customer acquisition and retention.

The expansion of Stripe Connect features enhances FreshBooks Payments, improving payment processing.

AI-driven financial analytics tools can provide predictive insights, enhancing decision-making.

Land your dream remote job 3x faster with AI

Try Jobo Free