Lead Data Engineer
Access SystemsInternship
Mid-level (3 to 4 years), Senior (5 to 8 years)
Candidates should possess over 5 years of experience in data engineering, big data, and distributed systems for SaaS products. Proficiency in Python, Django, and SQL is required, along with hands-on experience with Snowflake for ETL/ELT pipelines, data warehousing, and analysis. Experience with Great Expectations or similar data quality testing tools, data modeling, data warehousing, and distributed systems is necessary. Familiarity with privacy-compliant data processing (GDPR, CCPA) for advertising/retail media use-cases and experience managing real-time streaming data using tools like Kafka, Kinesis, or Pub/Sub are also required. A track record of successful collaboration with engineering and product teams is essential. Bonus points for experience with data lakes and ML pipelines.
The Senior Data Engineer will lead architectural discussions and guide the development of data infrastructure to ensure high-quality, trustworthy data. They will oversee data processing and transformation, maintaining data hygiene for downstream applications. The role involves writing maintainable Python code for scalable data management solutions, documenting new and existing features clearly, and coaching the Measurement team to enhance collective capabilities. Communication with stakeholders across the organization to align expectations and ensure transparency regarding data infrastructure and management is also a key responsibility.
Cloud cost management and optimization platform
Vantage.sh is a platform designed to help businesses manage and optimize their cloud costs. It provides tools for creating detailed reports on cloud expenditure, allowing users to filter and group costs by various dimensions, set monthly budgets, and receive alerts when spending exceeds those budgets. The platform supports multiple cloud providers and offers in-depth resource-level analytics, enabling users to track costs across different subscriptions and projects. A standout feature is its Kubernetes cost optimization, which helps users allocate costs by service and identify areas for efficiency. Vantage.sh operates on a self-serve model, making it easy for businesses to start using the service and save on costs, charging a low fee of 5% of the savings generated. The goal of Vantage.sh is to provide businesses with a clear understanding of their cloud spending and help them find opportunities for cost reduction.