Staff Data Engineer

Toronto, Ontario, Canada

Apply with AI Apply

Not SpecifiedCompensation

Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

Data Products, BiotechnologyIndustries

Requirements

Candidates should possess a Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field, along with 6+ years of experience in enterprise data engineering, specifically with Fortune 500 companies. They must have expert-level experience with the Databricks platform, including job scheduling and cluster management, and hands-on experience with Active Batch or similar enterprise job orchestration platforms. Strong knowledge of AWS data services such as DMS, S3, and database connectivity is required, along with proven experience with CDC (Change Data Capture) implementations at an enterprise scale, and a deep understanding of Delta Live Tables (DLT) and streaming data processing.

Responsibilities

The Staff Data Engineer will implement and optimize enterprise-scale data pipeline architecture processing 1500+ daily jobs across multiple business domains, build and maintain Databricks job orchestration solutions, and implement data ingestion frameworks supporting CDC via AWS DMS and S3-based data lake patterns. They will troubleshoot and resolve pipeline reliability issues, build observability and monitoring solutions, implement near real-time ingestion patterns using DLT and streaming architectures, and build and maintain data quality frameworks integrated with data governance processes. Additionally, they will implement serverless migration strategies, optimize cloud resource utilization, lead technical workshops, and provide hands-on guidance to client engineering teams, and contribute to enterprise pipeline expertise, operational excellence, performance optimization, and technical consulting skills.

Skills

Data Pipelines

Data Ingestion

Databricks

AWS DMS

Data Lake

CDC

DLT

Streaming Architectures

Data Quality

Data Governance

Serverless

Cloud Resource Optimization

Batch Processing

Active Batch

Observability

Monitoring

Failure Recovery

Compute Optimization

ShyftLabs

Data-driven decision-making solutions for organizations

About ShyftLabs

ShyftLabs helps organizations adopt a data-first approach to their decision-making processes. Their services focus on establishing systems that enable companies to make quicker and more informed decisions based on data analysis. This approach allows businesses to gain insights that can keep them ahead of their competitors. Unlike other companies that may offer generic consulting services, ShyftLabs emphasizes the importance of data in driving decisions, ensuring that organizations can leverage their data effectively to enhance their strategic planning and operational efficiency.

None, CanadaHeadquarters

2018Year Founded

VENTURE_UNKNOWNCompany Stage

Data & Analytics, ConsultingIndustries

11-50Employees

Staff Data Engineer

Requirements

Responsibilities

Skills

ShyftLabs

About ShyftLabs

Benefits

Health Insurance

Hybrid Work Options

Professional Development Budget

Risks

Increased competition from startups offering innovative, cost-effective solutions.

Growing demand for in-house analytics teams reducing reliance on consultants.

Rapid AI advancements may outpace ShyftLabs' current technology offerings.

Differentiation

ShyftLabs specializes in data governance, warehousing, and predictive analysis services.

The firm empowers organizations with a data-first approach for decision-making.

ShyftLabs establishes processes for faster, insightful decisions to outpace competition.

Upsides

Increased demand for data governance due to stricter privacy regulations.

Growing interest in predictive analytics in retail for inventory optimization.

Rising adoption of cloud-based BI tools among SMEs for cost-effectiveness.