Senior Data Engineer
Built TechnologiesSalary not specified
- Full Time
- Senior (5 to 8 years)
Not SpecifiedCompensation
Senior (5 to 8 years)Experience Level
Full TimeJob Type
UnknownVisa
Data & AnalyticsIndustries
Requirements
Candidates should possess at least 5 years of hands-on experience in a data engineering role, ideally within a SaaS environment. They must be proficient in SQL, dbt, and Python, along with advanced knowledge of data engineering tools and frameworks such as Airflow, Airbyte, and Kafka. Experience working with services in a cloud environment, specifically AWS or GCP, is required. A strong commitment to data quality, integrity, and security is essential, as is the ability to thrive in ambiguity and create clear pathways from complex challenges.
Responsibilities
The Data Engineer will be responsible for owning, building, and maintaining data pipelines, ETL/reverse-ETL processes, and the BigQuery environment. They will implement an orchestration tool, design a suite of tests and validations to ensure data consistency and accuracy, develop reliable data models while optimizing for scale, catalog and document all aspects of data pipelines, and create the foundation for a data governance program. Furthermore, they will partner with product and engineering teams to deliver data solutions, establish data engineering best practices, and serve as a subject matter expert on the data architecture, models, and systems. Finally, they will proactively manage and build relationships across multiple departments and stakeholder levels.
Skills
SQL
dbt
Python
Airflow
Airbyte
Kafka
GitHub workflows
AWS
GCP
Data quality
Data security
Data modeling
Writer
AI platform for business process optimization
About Writer
Writer.com provides a platform that uses artificial intelligence to enhance business processes for various clients, including major companies. The platform integrates Large Language Models (LLMs), Natural Language Processing (NLP), and Machine Learning (ML) to tailor AI solutions to a client's specific brand and knowledge. LLMs generate human-like text, NLP helps computers understand human language, and ML enables learning from data. Writer's platform is built on secure, enterprise-grade LLMs called Palmyra, which are designed to be open and transparent, ensuring that client data is never used for training. This allows clients to self-host the models, giving them control and visibility over their data. The platform assists businesses in speeding up processes and generating customized outputs, such as summaries and insights, while adhering to legal and brand guidelines. Writer.com operates on a subscription-based model, where clients pay a recurring fee for access to its features.
San Francisco, CaliforniaHeadquarters
2020Year Founded
$317.1MTotal Funding
SERIES_CCompany Stage
Data & Analytics, Enterprise Software, AI & Machine LearningIndustries
1,001-5,000Employees