Lead Data Engineer - R01553054 at Brillio

Bengaluru, Karnataka, India

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

N/AIndustries

Requirements

5+ years in Data Engineering with a strong focus on cloud and big data technologies
Minimum 2+ years of hands-on experience with GCP, specifically BigQuery
Proven experience migrating on-premise data systems to the cloud
Strong development experience with Apache Airflow, Python, and Apache Spark
Expertise in streaming data ingestion, particularly in IoT or sensor data environments
Strong SQL development skills; experience with BigQuery performance tuning
Solid understanding of cloud architecture, data modeling, and data warehouse design
Familiarity with Git and CI/CD practices for managing data pipelines
Primary Skills: IICS, Alation, Data Modelling Fundamentals, Data Warehousing, ETL Fundamentals, Modern Data Platform Fundamentals, PLSQL, T-SQL, Stored Procedures, Python, SQL (Basic + Advanced), Talend

Responsibilities

Analyze legacy on-premises and hybrid cloud data warehouse environments (e.g., SQL Server)
Lead the migration of large-scale datasets to Google BigQuery
Design and implement data migration strategies ensuring data quality, integrity, and performance
Integrate data from various structured and unstructured sources, including APIs, relational databases, and IoT devices
Build real-time streaming pipelines for large-scale ingestion and processing of IoT and telemetry data
Modernize and refactor legacy SSIS packages into cloud-native ETL pipelines
Develop scalable, reliable workflows using Apache Airflow, Python, Spark, and GCP-native tools
Ensure high-performance data transformation and loading into BigQuery for analytical use cases
Write and optimize complex SQL queries, stored procedures, and scheduled jobs within BigQuery
Develop modular, reusable transformation scripts using Python, Java, Spark, and SQL
Continuously monitor and optimize query performance and cost efficiency in the cloud data environment

Skills

Key technologies and capabilities for this role

GCPBigQueryPythonSQLApache AirflowSparkETLSSISPLSQLT-SQLTalendIICSAlation

Questions & Answers

Common questions about this position

What are the key required skills for this Lead Data Engineer role?

The role requires 5+ years in Data Engineering with focus on cloud and big data, minimum 2+ years hands-on with GCP and BigQuery, experience migrating on-premise systems to cloud, strong skills in Apache Airflow, Python, Apache Spark, SQL with BigQuery tuning, plus data modeling and warehouse design.

What is the salary for this position?

This information is not specified in the job description.

Is this a remote position or what is the location requirement?

This information is not specified in the job description.

What is the company culture like for this role?

The role offers a fast-paced, collaborative environment where you work with cutting-edge technologies and lead large-scale cloud transformation initiatives.

What makes a strong candidate for this Lead Data Engineer position?

Strong candidates have 5+ years in data engineering focused on cloud/big data, 2+ years with GCP/BigQuery, proven migration experience, expertise in Airflow/Python/Spark/SQL, plus preferred GCP certification, dbt/Kafka/Terraform, or ML/DataOps exposure.

Brillio

Technology consulting for digital transformation

About Brillio

Brillio helps businesses with their digital transformation by providing technology consulting services. The company focuses on creating customized digital solutions that utilize artificial intelligence and cloud technologies to meet the specific needs of each client. This includes improving operational efficiency and enhancing user experiences through real-time responses and better workflows. Brillio serves a variety of clients, including large Fortune 500 companies, with a strong emphasis on the healthcare sector. Unlike many competitors, Brillio's approach is centered on developing bespoke solutions that give clients a competitive advantage, aiming to help them evolve into 'generative enterprises.' The company's goal is to drive significant improvements in business operations and outcomes through its consulting services and technology implementations.

Edison, New JerseyHeadquarters

2014Year Founded

LATE_VCCompany Stage

Consulting, AI & Machine Learning, HealthcareIndustries

1,001-5,000Employees

Benefits

Remote Work Options

Risks

High competition in tech may impact Brillio's talent acquisition and service scaling.

Over-reliance on healthcare exposes Brillio to sector-specific regulatory and funding risks.

Expansion into new markets may stretch resources, leading to potential operational inefficiencies.

Differentiation

Brillio specializes in bespoke digital solutions for diverse industries, focusing on AI and cloud.

The company is recognized as an Enterprise Innovator in healthcare payer services.

Brillio's Cloud and AI Studio aims to modernize enterprise IT systems and optimize workflows.

Upsides

Increased demand for AI-driven healthcare solutions boosts Brillio's market potential.

Expansion in India enhances service delivery and innovation through a growing tech talent pool.

Recognition in Open Banking IT Services highlights Brillio's role in financial digital transformation.

Land your dream remote job 3x faster with AI

Try Jobo Free