Senior Data Engineer
Rad AI- Full Time
- Senior (5 to 8 years)
Candidates should have 5+ years of experience in data engineering, big data, or distributed systems. Strong expertise in Python and SQL or equivalent big data processing frameworks is required. Proficiency in ETL/ELT pipelines using Apache Airflow or similar orchestration tools is necessary. Experience working with real-time streaming data such as Kafka, Kinesis, or Pub/Sub is essential. A strong understanding of data modeling, data warehousing, and distributed systems is needed, along with familiarity with privacy-compliant data processing for advertising and retail media use cases.
The Senior Data Engineer will design, develop, and optimize data pipelines, ETL/ELT workflows, and data warehouses to support large-scale retail media analytics. They will manage big data and streaming platforms, handling real-time and batch processing at scale. The role involves collaborating cross-functionally with data scientists, analysts, software engineers, and product teams to ensure seamless data integration and access. Additionally, the Senior Data Engineer will implement robust monitoring, validation, and security controls to maintain high data reliability.
Cloud cost management and optimization platform
Vantage.sh is a platform designed to help businesses manage and optimize their cloud costs. It provides tools for creating detailed reports on cloud expenditure, allowing users to filter and group costs by various dimensions, set monthly budgets, and receive alerts when spending exceeds those budgets. The platform supports multiple cloud providers and offers in-depth resource-level analytics, enabling users to track costs across different subscriptions and projects. A standout feature is its Kubernetes cost optimization, which helps users allocate costs by service and identify areas for efficiency. Vantage.sh operates on a self-serve model, making it easy for businesses to start using the service and save on costs, charging a low fee of 5% of the savings generated. The goal of Vantage.sh is to provide businesses with a clear understanding of their cloud spending and help them find opportunities for cost reduction.