Staff Data Engineer (Spark)
WizardFull Time
Expert & Leadership (9+ years)
Candidates must have 7+ years of development experience in data engineering, with at least 5 years specifically writing PySpark. Experience building large-scale data processing pipelines (batch and stream), ETL/ELT processes, and handling massive datasets (100s of Terabytes and Petabytes) is essential. Strong proficiency in PySpark and Python is required, along with expertise in database systems, data modeling, relational databases, and NoSQL databases like MongoDB. Familiarity with big data technologies such as Kafka, Spark, Iceberg, Datalake, and the AWS stack (EKS, EMR, Serverless, Glue, Athena, S3, etc.) is necessary. Knowledge of security best practices and data privacy concerns, along with strong problem-solving skills and attention to detail, are also key requirements. Experience with data processing platforms like Databricks or Snowflake, and an understanding of Graph and Vector data stores are considered advantageous.
The Senior Data Engineer will create and maintain data pipelines and foundational datasets to support product and business needs. They will design and build database architectures for massive and complex data, balancing computational load and cost. Responsibilities include developing audits for data quality at scale with necessary alerting, and creating scalable dashboards and reports to support business objectives and enable data-driven decision-making. The role also involves troubleshooting and resolving complex issues in production environments, and working closely with product managers and other stakeholders to define and implement new features.
Provides background check services for employers
Checkr provides background check services primarily for businesses, staffing agencies, and recruiters. Their platform allows clients to conduct thorough background screenings to verify potential employees' histories, which is especially important in high-risk industries. The service operates on a software-as-a-service (SaaS) model, where clients subscribe to access a user-friendly dashboard and various features that help manage the background check process efficiently. Checkr also offers educational resources to promote fair hiring practices. Unlike many competitors, Checkr's platform includes innovative solutions like Checkr Assess, which can significantly reduce talent acquisition costs. The company's goal is to help businesses hire trustworthy employees while ensuring compliance and promoting fair hiring practices.