Senior Data Engineer (Pipeline)
DemandbaseSalary not specified
Not SpecifiedCompensation
Senior (5 to 8 years)Experience Level
Full TimeJob Type
UnknownVisa
TechnologyIndustries
Requirements
- 5+ years of professional experience in software or data engineering using Python
- Strong understanding of software engineering best practices (testing, version control, CI/CD)
- Proven experience building and optimizing ETL/ELT pipelines and data workflows
- Proficiency in SQL and database concepts
- Experience with data processing frameworks (e.g., Pandas)
- Understanding of software design patterns and scalable architecture principles
- Experience with cloud platforms (GCP preferred)
- Knowledge of CI/CD pipelines and Infrastructure as Code tools
- Familiarity with containerization (Docker, Kubernetes)
- Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience)
- Excellent problem-solving, analytical, and communication skills
- Preferred Qualifications
- Experience with GCP services such as Cloud Run and Dataflow
- Experience with stream processing technologies (e.g., Pub/Sub)
- Familiarity with workflow orchestration tools (e.g., Airflow)
- Exposure to data visualization tools or libraries
- Knowledge of GitLab CI/CD and Terraform
- Experience with Snowflake, BigQuery, or Databricks
- GCP Data Engineer Certification
Responsibilities
- Design, develop, and maintain data pipelines and ETL/ELT processes using Python
- Build and optimize scalable, high-performance data applications
- Collaborate with cross-functional teams to define requirements and deliver reliable solutions
- Develop and manage real-time streaming pipelines using Pub/Sub or Apache Beam
- Participate in code reviews, architecture discussions, and continuous improvement initiatives
- Monitor, troubleshoot, and optimize production data systems for reliability and performance
Skills
Key technologies and capabilities for this role
Questions & Answers
Common questions about this position
What compensation and benefits does ShyftLabs offer?
ShyftLabs offers a competitive salary alongside a strong insurance package.
Is this Data Engineer position remote or office-based?
This information is not specified in the job description.
What are the key skills required for the Data Engineer role?
The role requires 5+ years of experience in software or data engineering using Python, proficiency in SQL and database concepts, proven experience with ETL/ELT pipelines, and familiarity with cloud platforms like GCP.
What is the company culture like at ShyftLabs?
ShyftLabs emphasizes collaboration with cross-functional teams, employee growth through extensive learning and development resources, and participation in code reviews and continuous improvement.
What makes a strong candidate for this Data Engineer position?
Strong candidates will have 5+ years of Python-based data engineering experience, expertise in ETL/ELT pipelines and SQL, plus preferred skills like GCP services, stream processing with Pub/Sub, and a GCP Data Engineer Certification.
ShyftLabs
Data-driven decision-making solutions for organizations
About ShyftLabs
ShyftLabs helps organizations adopt a data-first approach to their decision-making processes. Their services focus on establishing systems that enable companies to make quicker and more informed decisions based on data analysis. This approach allows businesses to gain insights that can keep them ahead of their competitors. Unlike other companies that may offer generic consulting services, ShyftLabs emphasizes the importance of data in driving decisions, ensuring that organizations can leverage their data effectively to enhance their strategic planning and operational efficiency.
None, CanadaHeadquarters
2018Year Founded
VENTURE_UNKNOWNCompany Stage
Data & Analytics, ConsultingIndustries
11-50Employees
Benefits
Health Insurance
Hybrid Work Options
Professional Development Budget
Risks
Increased competition from startups offering innovative, cost-effective solutions.
Growing demand for in-house analytics teams reducing reliance on consultants.
Rapid AI advancements may outpace ShyftLabs' current technology offerings.
Differentiation
ShyftLabs specializes in data governance, warehousing, and predictive analysis services.
The firm empowers organizations with a data-first approach for decision-making.
ShyftLabs establishes processes for faster, insightful decisions to outpace competition.
Upsides
Increased demand for data governance due to stricter privacy regulations.
Growing interest in predictive analytics in retail for inventory optimization.
Rising adoption of cloud-based BI tools among SMEs for cost-effectiveness.
Related Jobs
RemoteRemoteSenior Data Automation Professional
HumanaSalary not specified
RemoteStaff Data Engineer
Life360$166,500 - $245,000/year
Full Time
Expert & Leadership (9+ years)
RemoteStaff Integration Engineer
Chainalysis$110,000 - $150,000/year
- Full Time
- Senior (5 to 8 years)
RemoteSenior Cloud Engineer - Product Metrics
ClickhouseSalary not specified
- Full Time
- Senior (5 to 8 years)
RemoteAnalytics Engineer
Machinify$180,000 - $200,000/year
- Full Time
- Mid-level (3 to 4 years)
RemoteStaff Database Administrator
Wikimedia FoundationSalary not specified
- Full Time
- Mid-level (3 to 4 years)
RemoteData Platform Software Engineer
Muon SpaceSalary not specified
- Full Time
- Senior (5 to 8 years)