ShyftLabs

Databricks Architect

Noida, Uttar Pradesh, India

Not SpecifiedCompensation
Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level
Full TimeJob Type
UnknownVisa
Data & Analytics, AI & Machine LearningIndustries

Requirements

Candidates should possess a Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field, along with 10+ years of hands-on experience in data engineering, including at least 5+ years as a Databricks Architect and with Apache Spark. They must be proficient in SQL, Python, or Scala for data processing and analytics, and have extensive experience with cloud platforms such as AWS, Azure, or GCP for data engineering. Strong knowledge of ETL frameworks, data lakes, and Delta Lake architecture is required, alongside familiarity with data security, compliance, and governance best practices.

Responsibilities

The Databricks Architect will be responsible for architecting, designing, and optimizing big data and AI/ML solutions on the Databricks platform, developing and implementing scalable ETL pipelines, leading the adoption of Apache Spark, defining and enforcing data governance and security policies, optimizing data lakehouse architectures, collaborating with data scientists and engineers, overseeing Databricks clusters and jobs, automating data workflows using CI/CD pipelines and infrastructure-as-code, ensuring data integrity and reliability, and troubleshooting performance bottlenecks.

Skills

Apache Spark
SQL
Python
Scala
AWS
Azure
GCP
ETL
Data Governance
CI/CD
Infrastructure-as-Code
Data Lakehouse
Data Engineering
AI/ML

ShyftLabs

Data-driven decision-making solutions for organizations

About ShyftLabs

ShyftLabs helps organizations adopt a data-first approach to their decision-making processes. Their services focus on establishing systems that enable companies to make quicker and more informed decisions based on data analysis. This approach allows businesses to gain insights that can keep them ahead of their competitors. Unlike other companies that may offer generic consulting services, ShyftLabs emphasizes the importance of data in driving decisions, ensuring that organizations can leverage their data effectively to enhance their strategic planning and operational efficiency.

Key Metrics

None, CanadaHeadquarters
2018Year Founded
VENTURE_UNKNOWNCompany Stage
Data & Analytics, ConsultingIndustries
11-50Employees

Benefits

Health Insurance
Hybrid Work Options
Professional Development Budget

Risks

Increased competition from startups offering innovative, cost-effective solutions.
Growing demand for in-house analytics teams reducing reliance on consultants.
Rapid AI advancements may outpace ShyftLabs' current technology offerings.

Differentiation

ShyftLabs specializes in data governance, warehousing, and predictive analysis services.
The firm empowers organizations with a data-first approach for decision-making.
ShyftLabs establishes processes for faster, insightful decisions to outpace competition.

Upsides

Increased demand for data governance due to stricter privacy regulations.
Growing interest in predictive analytics in retail for inventory optimization.
Rising adoption of cloud-based BI tools among SMEs for cost-effectiveness.

Land your dream remote job 3x faster with AI