[Remote] Senior Software Engineer, Dataplane Management at Astronomer

United States

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Technology, Data OrchestrationIndustries

Requirements

Experience building and operating SaaS infrastructure, or experience managing a large scale internal compute platform
Software engineering expertise with Golang, or similar languages with a desire to learn Golang
A passion for reliability and operational excellence. A low tolerance for toil, alert fatigue, and other nonsense
Production experience with a container based orchestration system (Kubernetes preferred, but not essential)
Understanding of how to build with security and isolation in mind, so that Astronomer’s managed platform can securely integrate with any customer environment and ensure strict isolation between customer workloads
Strong communication skills, both written and verbal, with experience in working with a globally distributed team in delivery
Proactive approach to identifying and addressing issues, with a focus on ownership and accountability
Experience as part of an on-call rotation - this role involves periodic on-call for the services and system we own

Responsibilities

Own key endpoints and features of our flagship product, Astro, to extend our offering to more complex customer networking options
Work across domains to develop diverse features for our core infrastructure (ie: workload identity, multitenancy, cross region disaster recovery, cloud resource management, private network access)
Evolve our fleet orchestration system to enable us to safely make (and roll back) changes across our infrastructure, and to enable us to scale from the 500+ clusters we have now to thousands in the future
Gain experience working in a multi-cloud environment by working with managed Kubernetes offerings and network/auth primitives from AWS, Azure, and GCP
Deepen your operational knowledge of Kubernetes-based workloads by managing the data pipelines of many of the largest companies in the world

Skills

Key technologies and capabilities for this role

KubernetesApache AirflowAWSAPIsVPCTransit GatewayVNetWorkload IdentityMultitenancyDisaster RecoveryCloud InfrastructureFleet Orchestration

Questions & Answers

Common questions about this position

Is this position remote?

Yes, this is a remote position.

What programming languages or technical expertise is required?

Software engineering expertise with Golang or similar languages is required, with a desire to learn Golang. Production experience with a container-based orchestration system like Kubernetes is also needed.

What salary or compensation is offered for this role?

This information is not specified in the job description.

What experience makes a strong candidate for this position?

Candidates with experience building and operating SaaS infrastructure or managing large-scale internal compute platforms, a passion for reliability and operational excellence, and understanding of security and isolation will stand out.

What is the team and work environment like at Astronomer?

The R&D organization features innovative minds in cloud infrastructure and open-source software. The Data Plane Management team manages a large-scale fleet of 500+ Kubernetes clusters and develops features for the Astro platform in a multi-cloud environment.

Astronomer

Data orchestration platform for pipeline management

About Astronomer

Astronomer.io provides a data orchestration platform that utilizes Apache Airflow to simplify the deployment of data pipelines. Its main product, Astro, helps businesses manage and monitor their data flows, allowing them to focus on delivering essential data pipelines. The platform supports data unification across various clouds and offers over 1500 integrations, making it suitable for data and machine learning teams in industries like finance and e-commerce. Astronomer.io distinguishes itself from competitors by offering enterprise-grade security, zero-downtime upgrades for Airflow, and tools for monitoring pipeline health, which enhance compute efficiency and reduce delays in task scheduling. The company's goal is to empower organizations to optimize their data strategies and achieve a significant return on investment by ensuring their applications operate with maximum reliability.

Cincinnati, OhioHeadquarters

2018Year Founded

$272.1MTotal Funding

SERIES_CCompany Stage

Data & Analytics, Enterprise Software, CybersecurityIndustries

201-500Employees

Benefits

Remote Work Options

Company Equity

Risks

Increased competition from platforms like Prefect and Dagster poses a market challenge.

Over-reliance on Apache Airflow could be risky if community focus shifts.

Rapid AI advancements may outpace Astronomer's feature integration, risking client dissatisfaction.

Differentiation

Astronomer offers Astro, a modern data orchestration platform powered by Apache Airflow.

Astro enables data teams to build, run, and observe data pipelines-as-code efficiently.

Astronomer provides enterprise-grade security and over 1500 integrations for data unification.

Upsides

Astronomer's revenue grew over 200% in H1 2023, indicating strong market demand.

Collaboration with Microsoft Azure enhances Astronomer's cloud-native data orchestration capabilities.

Integration with Anyscale boosts Astronomer's offering for scalable machine learning workflows.

Land your dream remote job 3x faster with AI

Try Jobo Free