Python Data Engineer at ShyftLabs

Noida, Uttar Pradesh, India

ShyftLabs Logo
Not SpecifiedCompensation
Mid-level (3 to 4 years)Experience Level
Full TimeJob Type
UnknownVisa
Data ProductIndustries

Requirements

Candidates must have over 5 years of professional software development experience with Python and SQL, along with a strong understanding of software engineering best practices including testing, version control, and CI/CD. Proficiency in building and optimizing ETL/ELT processes and data pipelines, along with strong SQL and database concepts, is required. Experience with data processing frameworks like Pandas, understanding of software design patterns and architectural principles, and the ability to write clean, maintainable code are essential. Experience with unit testing, test automation, any cloud provider (GCP preferred), CI/CD pipelines, Infrastructure as Code, and containerization technologies like Docker or Kubernetes is also necessary.

Responsibilities

The Python Data Engineer will develop and maintain data pipelines and ETL/ELT processes using Python, and design and implement scalable, high-performance applications. They will collaborate with cross-functional teams to define requirements and deliver solutions, and develop and manage near real-time data streaming solutions using PubSub or Beam. Responsibilities also include contributing to code reviews, architecture discussions, and continuous improvement initiatives, as well as monitoring and troubleshooting production systems to ensure reliability and performance.

Skills

Python
SQL
ETL
ELT
Data Streaming
Pandas
Software Design Patterns
CI/CD
Infrastructure as Code

ShyftLabs

Data-driven decision-making solutions for organizations

About ShyftLabs

ShyftLabs helps organizations adopt a data-first approach to their decision-making processes. Their services focus on establishing systems that enable companies to make quicker and more informed decisions based on data analysis. This approach allows businesses to gain insights that can keep them ahead of their competitors. Unlike other companies that may offer generic consulting services, ShyftLabs emphasizes the importance of data in driving decisions, ensuring that organizations can leverage their data effectively to enhance their strategic planning and operational efficiency.

None, CanadaHeadquarters
2018Year Founded
VENTURE_UNKNOWNCompany Stage
Data & Analytics, ConsultingIndustries
11-50Employees

Benefits

Health Insurance
Hybrid Work Options
Professional Development Budget

Risks

Increased competition from startups offering innovative, cost-effective solutions.
Growing demand for in-house analytics teams reducing reliance on consultants.
Rapid AI advancements may outpace ShyftLabs' current technology offerings.

Differentiation

ShyftLabs specializes in data governance, warehousing, and predictive analysis services.
The firm empowers organizations with a data-first approach for decision-making.
ShyftLabs establishes processes for faster, insightful decisions to outpace competition.

Upsides

Increased demand for data governance due to stricter privacy regulations.
Growing interest in predictive analytics in retail for inventory optimization.
Rising adoption of cloud-based BI tools among SMEs for cost-effectiveness.

Land your dream remote job 3x faster with AI