Senior Software Engineer, Applied AI at Astronomer

New York, New York, United States

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Technology, AIIndustries

Requirements

Empathy for users, and a deep interest in improving the workflows of data professionals
Familiarity with early-stage product development; comfortable working with ambiguity in a fast-changing field
Experience with LLMs, vector databases, embeddings, or other applied AI areas—or a strong desire to dive in
A creative, experimental mindset: you enjoy exploring uncharted areas, validating hypotheses, and learning through iteration
Strong collaboration and communication skills—you can explain complex systems clearly to both technical and non-technical audiences
A collaborative approach and comfort working in an evolving, research-driven environment where ideas move quickly
Bonus: A passion for AI systems for data, developer tools

Responsibilities

Shape the future of AI for data engineering - build intelligent systems that understand, reason about, and optimize the flow of data across entire organizations
Design and engineer the brain of Astronomer’s context layer, crafting components that power data modeling, semantic search, retrieval, and code generation
Push the boundaries of applied AI - experiment with LLMs, embeddings, and cutting-edge retrieval techniques to create developer tools that deliver insights to you and our customers
Turn research into reality - work side by side with R&D and product teams to bring early AI concepts to life in the product experience
Solve high-impact information retrieval and search challenges at a global scale, leveraging Astronomer’s unparalleled visibility into data pipelines across industries
Influence the technical vision and architecture for the next generation of AI-driven data products
Represent Astronomer in the community - through open-source contributions, technical talks, and publications that showcase our leadership in AI and data innovation

Skills

Apache Airflow

Python

SQL

LLMs

DataOps

machine learning

information retrieval

code generation

root cause analysis

Astronomer

Data orchestration platform for pipeline management

About Astronomer

Astronomer.io provides a data orchestration platform that utilizes Apache Airflow to simplify the deployment of data pipelines. Its main product, Astro, helps businesses manage and monitor their data flows, allowing them to focus on delivering essential data pipelines. The platform supports data unification across various clouds and offers over 1500 integrations, making it suitable for data and machine learning teams in industries like finance and e-commerce. Astronomer.io distinguishes itself from competitors by offering enterprise-grade security, zero-downtime upgrades for Airflow, and tools for monitoring pipeline health, which enhance compute efficiency and reduce delays in task scheduling. The company's goal is to empower organizations to optimize their data strategies and achieve a significant return on investment by ensuring their applications operate with maximum reliability.

Cincinnati, OhioHeadquarters

2018Year Founded

$272.1MTotal Funding

SERIES_CCompany Stage

Data & Analytics, Enterprise Software, CybersecurityIndustries

201-500Employees

Benefits

Remote Work Options

Company Equity

Risks

Increased competition from platforms like Prefect and Dagster poses a market challenge.

Over-reliance on Apache Airflow could be risky if community focus shifts.

Rapid AI advancements may outpace Astronomer's feature integration, risking client dissatisfaction.

Differentiation

Astronomer offers Astro, a modern data orchestration platform powered by Apache Airflow.

Astro enables data teams to build, run, and observe data pipelines-as-code efficiently.

Astronomer provides enterprise-grade security and over 1500 integrations for data unification.

Upsides

Astronomer's revenue grew over 200% in H1 2023, indicating strong market demand.

Collaboration with Microsoft Azure enhances Astronomer's cloud-native data orchestration capabilities.

Integration with Anyscale boosts Astronomer's offering for scalable machine learning workflows.

Land your dream remote job 3x faster with AI

Try Jobo Free