Lead Data Engineer - R01553055 at Brillio

Bengaluru, Karnataka, India

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

TechnologyIndustries

Requirements

5+ years in Data Engineering with a strong focus on cloud and big data technologies
Minimum 2+ years of hands-on experience with GCP, specifically BigQuery
Proven experience migrating on-premise data systems to the cloud
Strong development experience with Apache Airflow, Python, and Apache Spark
Expertise in streaming data ingestion, particularly in IoT or sensor data environments
Strong SQL development skills; experience with BigQuery performance tuning
Solid understanding of cloud architecture, data modeling, and data warehouse design
Familiarity with Git and CI/CD practices for managing data pipelines
Preferred Qualifications
GCP Professional Data Engineer certification
Experience with modern data stack tools like dbt, Kafka, or Terraform
Exposure to ML pipelines, analytics engineering, or DataOps/DevOps methodologies

Responsibilities

Analyze legacy on-premises and hybrid cloud data warehouse environments (e.g., SQL Server)
Lead the migration of large-scale datasets to Google BigQuery
Design and implement data migration strategies ensuring data quality, integrity, and performance
Integrate data from various structured and unstructured sources, including APIs, relational databases, and IoT devices
Build real-time streaming pipelines for large-scale ingestion and processing of IoT and telemetry data
Modernize and refactor legacy SSIS packages into cloud-native ETL pipelines
Develop scalable, reliable workflows using Apache Airflow, Python, Spark, and GCP-native tools
Ensure high-performance data transformation and loading into BigQuery for analytical use cases
Write and optimize complex SQL queries, stored procedures, and scheduled jobs within BigQuery
Develop modular, reusable transformation scripts using Python, Java, Spark, and SQL
Continuously monitor and optimize query performance and cost efficiency in the cloud data environment

Skills

Key technologies and capabilities for this role

GCPBigQueryPythonAirflowSQLDBTSparkJava

Questions & Answers

Common questions about this position

What are the primary skills required for the Lead Data Engineer role?

The primary skills are GCP, Big Query, Python, Airflow, SQL, and DBT.

What experience is required for this position?

Candidates need 5+ years in Data Engineering with a focus on cloud and big data technologies, minimum 2+ years hands-on with GCP and BigQuery, proven experience migrating on-premise data systems to the cloud, and strong experience with Apache Airflow, Python, and Apache Spark.

Is remote work available for this role?

Yes, the role offers remote flexibility.

What is the salary for this position?

This information is not specified in the job description.

What makes a strong candidate for this Lead Data Engineer role?

A strong candidate has 5+ years of data engineering experience focused on cloud technologies, hands-on GCP and BigQuery expertise, experience with data migration from on-premise systems, proficiency in Airflow, Python, Spark, and SQL, plus preferred qualifications like GCP certification or tools like dbt and Kafka.

Brillio

Technology consulting for digital transformation

About Brillio

Brillio helps businesses with their digital transformation by providing technology consulting services. The company focuses on creating customized digital solutions that utilize artificial intelligence and cloud technologies to meet the specific needs of each client. This includes improving operational efficiency and enhancing user experiences through real-time responses and better workflows. Brillio serves a variety of clients, including large Fortune 500 companies, with a strong emphasis on the healthcare sector. Unlike many competitors, Brillio's approach is centered on developing bespoke solutions that give clients a competitive advantage, aiming to help them evolve into 'generative enterprises.' The company's goal is to drive significant improvements in business operations and outcomes through its consulting services and technology implementations.

Edison, New JerseyHeadquarters

2014Year Founded

LATE_VCCompany Stage

Consulting, AI & Machine Learning, HealthcareIndustries

1,001-5,000Employees

Benefits

Remote Work Options

Risks

High competition in tech may impact Brillio's talent acquisition and service scaling.

Over-reliance on healthcare exposes Brillio to sector-specific regulatory and funding risks.

Expansion into new markets may stretch resources, leading to potential operational inefficiencies.

Differentiation

Brillio specializes in bespoke digital solutions for diverse industries, focusing on AI and cloud.

The company is recognized as an Enterprise Innovator in healthcare payer services.

Brillio's Cloud and AI Studio aims to modernize enterprise IT systems and optimize workflows.

Upsides

Increased demand for AI-driven healthcare solutions boosts Brillio's market potential.

Expansion in India enhances service delivery and innovation through a growing tech talent pool.

Recognition in Open Banking IT Services highlights Brillio's role in financial digital transformation.

Land your dream remote job 3x faster with AI

Try Jobo Free