BI/ Reporting Engineer at Arrow Electronics

Penang, Malaysia

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

ElectronicsIndustries

Requirements

Overall 10 years of experience in Industry including 6+ years as developer using Databricks/Spark Ecosystems
Hands-on experience on Unified Data Analytics with Databricks, Databricks Workspace User Interface, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL
Good understanding of Spark Architecture with Databricks, Structured Streaming
Experience setting up Microsoft Azure with Databricks, Databricks Workspace for Business Analytics, Manage Clusters in Databricks, Managing the Machine Learning Lifecycle
Hands-on experience in Data extraction (extract, Schemas, corrupt record handling and parallelized code), transformations and loads (user-defined functions, join optimizations) and Production (optimize and automate Extract, Transform and Load)
Technical skills: Spark Data Frame API, Python for Data Science, Spark Programming, SQL for Data Analysis, Simplify Data analysis with Python, Manage Clusters Databricks, Databricks Administration, Data Extraction and Transformation and Load, Implementing Partitioning and Programming with MapReduce, Setting up Azure Databricks Account, Linux Command
Bachelor’s Degree in IT relevant field
Minimum of 6-8 years of related experience
Excellent verbal and written communication and negotiations skills
Ability to build good relationships internally with cross function teams
Proactive and strategic mindset
Resilience, energetic and positive attitude
Problem-solving skills along with good interpersonal & communication skills
Self-starter who can pick up any other relevant Azure Services in the Analytics space

Responsibilities

Develop Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into customer usage patterns
Extract, Transform and Load data from source systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics
Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks
Develop Spark applications using PySpark and Spark SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing and transforming the data to uncover insights into customer usage patterns
Develop SQL Scripts for automation
Estimate the cluster size, monitor, and troubleshoot Spark Databricks clusters
Apply Spark DataFrame API to complete Data manipulation within Spark session
Collaborate with delivery leadership to deliver projects on time adhering to quality standards
Contribute to the growth of the Microsoft Azure practice by helping with solutioning for prospects

Skills

Key technologies and capabilities for this role

DatabricksSparkPythonDelta LakeSpark SQLPySparkAzure DatabricksAzure Data FactoryT-SQLU-SQLAzure Data LakeAzure SQLAzure SynapseETLStructured StreamingLinux

Questions & Answers

Common questions about this position

What experience level is required for this BI/Reporting Engineer role?

Overall 10 years of experience in the industry is required, including 6+ years as a developer using Databricks/Spark ecosystems. A Bachelor’s Degree in a relevant IT field is also required, with a minimum of 6-8 years of related experience.

What are the key technical skills needed for this position?

Key skills include hands-on experience with Databricks, Spark Ecosystems, Delta Lake with Python and Spark SQL, Spark DataFrame API, Python for Data Science, SQL for Data Analysis, Azure Data Factory, and Databricks cluster management.

Is this a remote position or does it require office work?

This information is not specified in the job description.

What is the salary or compensation for this role?

This information is not specified in the job description.

What soft skills or qualities make a strong candidate for this role?

Strong candidates have problem-solving skills, good interpersonal and communication skills, are self-starters able to pick up new Azure services, and can collaborate with leadership to deliver projects on time.

Arrow Electronics

Global provider of technology solutions and services

About Arrow Electronics

Arrow Electronics provides technology solutions that support innovation for technology manufacturers and service providers. The company develops and delivers high-tech solutions that improve business operations and everyday life. Its offerings include computing, power management, and Internet of Things (IoT) applications. Arrow collaborates with clients to create complex technology systems, such as smart battery ecosystems for electric motorbikes and wearable health monitoring devices for firefighters. Unlike many competitors, Arrow not only sells technology products but also offers consulting and engineering services to help clients design and implement customized systems. The goal of Arrow Electronics is to drive innovation and enhance efficiency and quality of life through its comprehensive technology solutions.

Englewood, ColoradoHeadquarters

1935Year Founded

IPOCompany Stage

Hardware, Enterprise Software, AI & Machine LearningIndustries

10,001+Employees

Risks

Increased competition in cloud services could impact Arrow's growth.

Rapid technological changes in IoT may challenge Arrow's innovation pace.

Supply chain vulnerabilities, especially in semiconductors, could disrupt product delivery.

Differentiation

Arrow Electronics offers comprehensive technology solutions across multiple industries.

The company collaborates with clients to engineer complex technology systems.

Arrow's business model includes consulting and engineering services for customized solutions.

Upsides

Growing demand for IoT security solutions presents expansion opportunities for Arrow.

Increased investment in smart city projects boosts demand for Arrow's integrated systems.

Rising interest in sustainable tech aligns with Arrow's electric motorbike technologies.

Land your dream remote job 3x faster with AI

Try Jobo Free