Senior Data Engineer – Python & AWS at Egen

Hyderabad, Telangana, India

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

TechnologyIndustries

Requirements

Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field
5–10 years of hands-on experience in data engineering or data platform development
Expert-level proficiency in Python (pandas, PySpark, boto3, SQLAlchemy)
Advanced experience with AWS Data Services, including: AWS Glue, Lambda, EMR, Step Functions, DynamoDB, EDW Redshift, Athena, S3, Kinesis, Amazon Quicksight, IAM, CloudWatch, CloudFormation / Terraform (for infrastructure automation)
Strong experience in SQL, data modeling, and performance tuning
Proven ability to design and deploy data lakes, data warehouses, and streaming solutions
Solid understanding of ETL best practices, partitioning, error handling, and data validation
Hands-on experience in version control (Git) and CI/CD for data pipelines
Knowledge of containerization (Docker/Kubernetes) and DevOps concepts
Excellent analytical, debugging, and communication skills

Responsibilities

Architect and implement scalable, fault-tolerant data pipelines using AWS Glue, Lambda, EMR, Step Functions, and Redshift
Build and optimize data lakes and data warehouses on Amazon S3, Redshift, and Athena
Develop Python-based ETL/ELT frameworks and reusable data transformation modules
Integrate multiple data sources (RDBMS, APIs, Kafka/Kinesis, SaaS systems) into unified data models
Lead efforts in data modeling, schema design, and partitioning strategies for performance and cost optimization
Drive data quality, observability, and lineage using AWS Data Catalog, Glue Data Quality, or third-party tools
Define and enforce data governance, security, and compliance best practices (IAM policies, encryption, access control)
Collaborate with cross-functional teams (Data Science, Analytics, Product, DevOps) to support analytical and ML workloads
Implement CI/CD pipelines for data workflows using AWS CodePipeline, GitHub Actions, or Cloud Build
Provide technical leadership, code reviews, and mentoring to junior engineers
Monitor data infrastructure performance, troubleshoot issues, and lead capacity planning

Skills

Key technologies and capabilities for this role

PythonpandasPySparkboto3SQLAlchemyAWSAWS GlueAWS LambdaEMRStep FunctionsRedshiftS3AthenaETLELTKafkaKinesisData ModelingCI/CDCodePipelineGitHub Actions

Questions & Answers

Common questions about this position

What are the required skills for the Senior Data Engineer role?

Required skills include expert-level proficiency in Python (pandas, PySpark, boto3, SQLAlchemy), advanced experience with AWS Data Services like Glue, Lambda, EMR, Step Functions, Redshift, Athena, S3, Kinesis, strong SQL and data modeling skills, plus experience with Git, CI/CD, Docker/Kubernetes, and excellent analytical and communication skills.

What is the salary or compensation for this position?

This information is not specified in the job description.

Is this a remote position or does it require office work?

This information is not specified in the job description.

What education and experience are required for this role?

A Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field is required, along with 5–10 years of hands-on experience in data engineering or data platform development.

What are the key responsibilities of the Senior Data Engineer?

Key responsibilities include architecting scalable data pipelines using AWS services like Glue, Lambda, EMR, building data lakes and warehouses on S3, Redshift, Athena, developing Python ETL frameworks, ensuring data quality and governance, collaborating with cross-functional teams, implementing CI/CD, and providing technical leadership.

Egen

Provides API strategies and cloud solutions

About Egen

Egen Solutions specializes in API strategies and cloud solutions to help businesses modernize and scale. They offer two main paths for application development: Rebuild, which improves existing functionalities, and Replace, which creates new applications in the cloud. Egen stands out by providing tailored digital transformation services that focus on delivering measurable ROI and quick results. Their goal is to enhance productivity and reduce operational costs for clients across various sectors, including retail and public health.

Naperville, IllinoisHeadquarters

2000Year Founded

$1.3MTotal Funding

PRE_SEEDCompany Stage

Consulting, Consumer Software, Enterprise SoftwareIndustries

1,001-5,000Employees

Benefits

Fun team outings

Health insurance

401(K)

Dental & vision insurance

Paid time-off

Food & drinks

Parental leave

Risks

Emerging tech startups offering similar solutions at lower costs threaten Egen's market share.

Rapid technological changes may outpace Egen's ability to update its solutions.

Economic downturns in retail and e-commerce could impact Egen's revenue.

Differentiation

Egen offers tailored digital transformation services with a focus on ROI and innovation.

The company specializes in secure, dynamic API strategies for seamless business growth.

Egen provides two application development paths: Rebuild and Replace, ensuring flexibility.

Upsides

Rising demand for cloud-native applications boosts Egen's cloud solutions offerings.

Growing interest in API-first development enhances Egen's API strategies and integration services.

Increased focus on cybersecurity in API development aligns with Egen's secure solutions.

Land your dream remote job 3x faster with AI

Try Jobo Free