Data Engineer - AI (REMOTE) at UpBound

San Francisco, California, United States

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

Technology, Artificial Intelligence, Cloud ComputingIndustries

Requirements

10+ years of software/data engineering experience with at least 4 years in technical leadership roles
Proven track record building data platforms that support production systems at scale
Deep expertise in both traditional data engineering (Spark, Airflow, data lakes) and ML-specific infrastructure (feature stores, model serving)
Experience with vector databases (Pinecone, Weaviate, Qdrant, Milvus, pgvector, Opensearch, ElasticSearch)
Demonstrated experience with LLM applications, including RAG architectures and semantic search implementations
Understanding of Kubernetes, cloud-native architectures, and infrastructure-as-code principles
Strong understanding of data requirements for AI/ML systems: training pipelines, feature stores, and inference infrastructure
Hands-on experience building knowledge bases and semantic search systems for technical documentation and code
Experience with embedding models for code and technical documentation
Knowledge of time-series data processing for infrastructure metrics and events
Understanding of graph databases and their application to infrastructure dependency modeling
Exceptional technical judgment with the ability to navigate both the AI and cloud-native landscapes

Responsibilities

Define and drive the technical vision for data platforms that support AI-powered features in Crossplane and Upbound Spaces
Lead the design of data pipelines that transform infrastructure and data into training datasets for ML models
Architect vector search and RAG systems that leverage Crossplane Control Planes & Upbound Marketplace as a knowledge store
Build data infrastructure that processes resources, extensions, and compositions for semantic search
Establish frameworks for collecting, processing, and analyzing infrastructure configuration data
Design data pipelines that handle Crossplane-specific data
Create infrastructure for indexing and searching Upbound Marketplace content, documentation, and community patterns
Develop metrics and monitoring for AI features integrated with Upbound's control plane architecture
Design data systems that power AI agents for infrastructure provisioning & operations, helping users generate and optimize Crossplane compositions
Create feature engineering platforms that extract signals from control plane operations, resource status, and reconciliation patterns
Implement data infrastructure for training models that predict infrastructure failures, optimize resource allocation, and suggest configuration improvements
Drive the development of knowledge graph representations of infrastructure dependencies and relationships

Skills

Key technologies and capabilities for this role

CrossplaneRAGVector SearchSemantic SearchData PipelinesML ModelsCloud Native Infrastructure

Questions & Answers

Common questions about this position

Is this Data Engineer position remote?

Yes, this is a remote position as indicated by the job title.

What experience level is required for this role?

The role requires 10+ years of software/data engineering experience with at least 4 years in technical leadership roles.

What technical expertise is needed for this Data Engineer position?

Candidates need proven track record building data platforms that support production systems at scale, plus deep expertise in traditional data engineering tools like Spark, Airflow, and data lakes.

What is the salary or compensation for this position?

This information is not specified in the job description.

What makes a strong candidate for this Principal Data Engineer role?

A strong candidate will have 10+ years of software/data engineering experience including 4+ years in technical leadership, proven experience building scalable data platforms, and deep expertise in tools like Spark, Airflow, and data lakes.

UpBound

Cloud-based solutions for infrastructure management

About UpBound

Upbound provides cloud-based solutions designed to help businesses streamline their operations and manage their cloud infrastructure more effectively. Their services include managed control planes that allow platform teams to scale resources as needed, ensuring optimal performance. A key feature is the ability to auto-scale control planes to support platforms with over 1,000 Custom Resource Definitions (CRDs), which means resources can adjust automatically based on demand. Upbound also offers Upbound Spaces, enabling organizations to deploy managed control planes in their own environments, which is beneficial for compliance with data privacy regulations. Unlike many competitors, Upbound focuses on simplifying the management of various cloud service providers and tools through centralized control. The company's goal is to empower businesses to innovate rapidly while maintaining efficient infrastructure management, with a subscription-based model that generates recurring revenue.

Seattle, WashingtonHeadquarters

2017Year Founded

$67.1MTotal Funding

SERIES_BCompany Stage

Enterprise Software, CybersecurityIndustries

51-200Employees

Benefits

Equity

Health care benefits

401k plan

Work from anywhere

Flexible hours & PTO

Home office stipend

Risks

Name confusion with rent-to-own company involved in a lawsuit poses reputational risk.

Increased competition in cloud-native platforms may dilute Upbound's market share.

Rapid technological advancements require continuous innovation, straining Upbound's resources.

Differentiation

Upbound offers managed control planes powered by Crossplane, a unique cloud-native solution.

The company provides auto-scaling for platforms with over 1,000 Custom Resource Definitions.

Upbound Spaces allows deployment of managed control planes in private environments for data compliance.

Upsides

Increased adoption of Kubernetes boosts demand for Upbound's Crossplane solutions.

Multi-cloud strategies create opportunities for Upbound's centralized control solutions.

Rise of platform engineering increases need for Upbound's managed control planes.

Land your dream remote job 3x faster with AI

Try Jobo Free