Senior Data Engineer

Remote

Apply with AI Apply

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Data ServicesIndustries

Requirements

Candidates must possess specialized expertise in data streaming technologies, with deep experience in Apache Kafka, AWS Managed Streaming for Apache Kafka (MSK), and Amazon Kinesis. Proficiency in cloud environments and experience with event-driven architectures, microservices patterns, and stream processing applications like Apache Spark Streaming, Kafka Streams, or AWS Lambda are essential. Familiarity with Infrastructure as Code (IaC) tools such as Terraform or CloudFormation and CI/CD pipelines is also required.

Responsibilities

The Senior Data Engineer will be responsible for building and maintaining high-performance data streaming architectures and real-time data pipelines. This includes designing, developing, and optimizing streaming systems using Kafka, MSK, and Kinesis, managing Kafka clusters and connectors, and implementing Kinesis solutions. Responsibilities also involve developing stream processing applications, integrating streaming systems with data lakes and warehouses, managing data serialization and schema evolution, and implementing IaC for streaming platforms and automating deployments.

Skills

Apache Kafka

AWS Managed Streaming for Apache Kafka (MSK)

Amazon Kinesis

Data Streaming

Real-time Data Processing

Data Pipelines

Event-driven Architectures

Microservices

Kafka Connect

Data Architecture

Fault Tolerance

Data Recovery

Effectual

Cloud innovation and IT modernization services

About Effectual

Effectual specializes in modernizing IT infrastructure for businesses, focusing on cloud innovation. The company helps both commercial enterprises and public sector customers enhance their IT systems by using established methods and cloud solutions to accelerate digital transformation. Effectual's services include modern application development, cloud strategy, data analytics, and generative AI services, all aimed at improving data-driven decision-making. A significant part of their approach is ensuring data security and compliance, which helps businesses mitigate risks and protect their reputations. Additionally, Effectual assists clients in optimizing their cloud costs by reducing technical debt and the total cost of ownership. The goal of Effectual is to turn business challenges into successful outcomes through effective IT modernization.

Jersey City, New JerseyHeadquarters

2018Year Founded

GROWTH_EQUITY_VCCompany Stage

Data & Analytics, Enterprise Software, Cybersecurity, AI & Machine LearningIndustries

51-200Employees

Benefits

Health Insurance

Dental Insurance

Vision Insurance

Life Insurance

Disability Insurance

401(k) Retirement Plan

401(k) Company Match

Paid Vacation

Paid Holidays

Hybrid Work Options

Risks

Increased competition from new Cloud Native Computing Foundation members.

Potential over-reliance on AWS as a primary partner.

Investment pressure may prioritize growth over service quality.

Differentiation

Effectual is an AWS Premier Partner with seven AWS competencies.

The company specializes in IT modernization for both commercial and public sectors.

Effectual emphasizes data security, achieving SOC 2 Type 1 and Type 2 compliance.

Upsides

Effectual ranked No. 44 on CRN's 2024 Fast Growth 150 list.

Increased demand for cloud cost optimization boosts Effectual's service relevance.

Rising interest in generative AI aligns with Effectual's service offerings.

Land your dream remote job 3x faster with AI

Try Jobo Free