Senior Lead Data Engineer - R01557349 at Brillio

San Francisco, California, United States

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

TechnologyIndustries

Requirements

High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing
Good knowledge in Python for data manipulation and scripting
Mastery of advanced SQL and deep experience working with relational databases and datawarehouses
Hands-on experience with workflow management tools like Apache Airflow
Familiarity with Unix shell scripting for automation and system management

Responsibilities

Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL
Manage and optimize the infrastructure ensuring reliability and performance
Design and implement data models and schemas for enterprise data warehouse and data marts
Write and maintain scripts in Python for various applications
Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases
Work closely with analysts and product managers to understand data requirements and deliver high-quality data
Provide technical guidance and mentorship to other team members fostering best practices in data engineering
Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability

Skills

AKS

Event Hub

Azure DevOps

Cosmos DB

Azure Functions

Hadoop

Spark SQL

Python

SQL

HDFS

MapReduce

Yarn

Scala

Apache Airflow

Unix shell scripting

Brillio

Technology consulting for digital transformation

About Brillio

Brillio helps businesses with their digital transformation by providing technology consulting services. The company focuses on creating customized digital solutions that utilize artificial intelligence and cloud technologies to meet the specific needs of each client. This includes improving operational efficiency and enhancing user experiences through real-time responses and better workflows. Brillio serves a variety of clients, including large Fortune 500 companies, with a strong emphasis on the healthcare sector. Unlike many competitors, Brillio's approach is centered on developing bespoke solutions that give clients a competitive advantage, aiming to help them evolve into 'generative enterprises.' The company's goal is to drive significant improvements in business operations and outcomes through its consulting services and technology implementations.

Edison, New JerseyHeadquarters

2014Year Founded

LATE_VCCompany Stage

Consulting, AI & Machine Learning, HealthcareIndustries

1,001-5,000Employees

Benefits

Remote Work Options

Risks

High competition in tech may impact Brillio's talent acquisition and service scaling.

Over-reliance on healthcare exposes Brillio to sector-specific regulatory and funding risks.

Expansion into new markets may stretch resources, leading to potential operational inefficiencies.

Differentiation

Brillio specializes in bespoke digital solutions for diverse industries, focusing on AI and cloud.

The company is recognized as an Enterprise Innovator in healthcare payer services.

Brillio's Cloud and AI Studio aims to modernize enterprise IT systems and optimize workflows.

Upsides

Increased demand for AI-driven healthcare solutions boosts Brillio's market potential.

Expansion in India enhances service delivery and innovation through a growing tech talent pool.

Recognition in Open Banking IT Services highlights Brillio's role in financial digital transformation.

Land your dream remote job 3x faster with AI

Try Jobo Free