Senior Data Pipeline Engineer - Global Team (India)

India

Apply with AI Apply

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Market Research, Analytics, Financial ServicesIndustries

Requirements

Candidates should possess a Bachelor's or Master's degree in Computer Science, STEM, or a related technical discipline, coupled with 6+ years of experience as a Data Engineer or in similar technical roles. A strong understanding of Spark and SQL, along with experience in data pipeline development, is essential. Proficiency in PySpark, Delta, and Databricks for handling large-scale datasets is required, as is the ability to understand business needs and align data transformations with organizational goals. Excellent verbal and written communication skills and a proactive, collaborative approach are also necessary. Experience with Airflow, dbt, or Snowflake is considered a plus.

Responsibilities

The Senior Data Engineer will be responsible for building and maintaining end-to-end data pipelines, contributing to the establishment of best practices for data modeling and pipeline construction, and creating documentation and architecture diagrams. This role involves becoming proficient in PySpark and SQL to resolve complex data pipeline issues, collaborating with stakeholders to integrate business logic into central pipelines, and gaining expertise in Databricks, Spark, and internal ETL tools.

Skills

Spark

SQL

Data Pipelines

Big Data

YipitData

Data analysis and research services provider

About YipitData

YipitData provides data analysis and research services by converting large amounts of raw data from various sources into clear and actionable insights. These insights help clients, primarily investors and corporate entities, understand market trends and company performance. YipitData operates on a subscription model, offering clients regular datasets and reports that include updates on various sectors such as autos, marketplaces, and logistics. Each subscription provides tailored information with three layers of coverage to meet specific client needs. The company differentiates itself by offering deep dives into topics, continuous tracking of metrics, and data integration best practices, ensuring clients have access to relevant and detailed information. The goal of YipitData is to empower clients to make informed decisions based on comprehensive data analysis.

New York City, New YorkHeadquarters

2013Year Founded

$561.3MTotal Funding

SERIES_ECompany Stage

Data & Analytics, Enterprise Software, Financial ServicesIndustries

501-1,000Employees

Benefits

401(k) Company Match

Flexible Work Hours

Unlimited Paid Time Off

Parental Leave

Wellness Program

Professional Development Budget

Risks

Increased competition from AI-driven analytics platforms like ChatGPT.

Shift towards e-commerce may reduce demand for traditional market research services.

Inflation volatility could impact accuracy and demand for predictive analytics products.

Differentiation

YipitData specializes in alternative data, offering unique insights beyond traditional data sources.

The company provides custom reports tailored to specific business needs and sectors.

YipitData's subscription model ensures clients receive regular, detailed datasets and reports.

Upsides

Expansion into Europe and China opens new markets for localized data insights.

Carlyle Group's investment provides capital for technology and market expansion.

Focus on tracking inflation through alternative data offers a unique client value proposition.

Land your dream remote job 3x faster with AI

Try Jobo Free