Staff Data Engineer (Spark)

United States

Apply with AI Apply

Not SpecifiedCompensation

Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

E-commerce, AI, BiotechnologyIndustries

Requirements

Candidates should have 5+ years of software development and data engineering experience with demonstrated ownership of production-grade data infrastructure. A Bachelor's degree in Computer Science or a related field, or equivalent practical experience, is required. Deep expertise in scaling Spark in production (Databricks, EMR, etc.) is essential, along with a strong understanding of distributed computing and modern data modeling for scalable systems. Proficiency in Python with experience implementing software engineering best practices and hands-on experience with both relational (MySQL / PostgreSQL) and NoSQL (MongoDB, DynamoDB, Cassandra) databases are necessary. Strong communication skills with experience influencing cross-functional stakeholders are also required. Experience working in early-stage, high-growth environments, familiarity with MLOps pipelines, and a proactive approach to problem-solving are considered nice to have.

Responsibilities

The Staff Data Engineer will design and evolve scalable, distributed data infrastructure across cloud platforms. They will build and maintain real-time and batch data processing pipelines supporting analytics and AI/ML workloads. Responsibilities include developing and managing integrations with third-party e-commerce platforms, ensuring data availability, reliability, and quality through monitoring and automated auditing, and partnering with engineering, AI, and product teams on data solutions for business-critical needs. Additionally, they will mentor and support data engineers, establishing best practices and code quality standards.

Skills

Spark

Databricks

EMR

Python

SQL

NoSQL

MongoDB

DynamoDB

Cassandra

MySQL

PostgreSQL

Distributed Computing

Data Modeling

Data Infrastructure

Data Pipelines

Real-time Data Processing

Batch Data Processing

AI/ML

MLOps

Software Engineering Best Practices

Wizard

AI-powered SMS platform for businesses

About Wizard

Wizard.com offers an SMS platform that enables businesses to facilitate product discovery and purchasing through text messages. Customers can browse and pay for products or services in a single conversation, making the buying process simpler and more enjoyable. The platform is designed for various businesses, including e-commerce, beauty brands, and pet product providers, aiming to improve customer engagement and loyalty. Wizard.com differentiates itself by using AI technology to create personalized interactions based on customer conversations, which enhances customer retention compared to traditional methods that rely on basic segmentation. The company's goal is to help businesses build stronger relationships with their customers, focusing on engagement rather than just transactions.

New York City, New YorkHeadquarters

2021Year Founded

$48.6MTotal Funding

SERIES_ACompany Stage

Consumer Software, AI & Machine LearningIndustries

51-200Employees

Benefits

Health Insurance

Dental Insurance

Vision Insurance

Life Insurance

Disability Insurance

401(k) Retirement Plan

Flexible Work Hours

Unlimited Paid Time Off

Paid Vacation

Paid Sick Leave

Paid Holidays

Parental Leave

Mental Health Support

Risks

Competition from established platforms with AI-driven features poses a threat to Wizard.

Privacy concerns and regulatory scrutiny could impact Wizard's AI-driven SMS solutions.

Dependence on SMS may face challenges from alternative messaging platforms.

Differentiation

Wizard offers a unique SMS solution for seamless e-commerce transactions.

The platform provides hyper-personalized interactions, enhancing customer engagement and loyalty.

Wizard's AI learns from conversations, surpassing traditional segmentation methods.

Upsides

Rising SMS marketing trends present significant opportunities for Wizard's AI-powered solutions.

Increased adoption of conversational AI boosts demand for Wizard's platform.

Hyper-personalization in customer service drives consumer preference for Wizard's offerings.

Land your dream remote job 3x faster with AI

Try Jobo Free