Data Engineer – Python & AWS at Egen

Hyderabad, Telangana, India

Apply Now

Not SpecifiedCompensation

Mid-level (3 to 4 years), Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

TechnologyIndustries

Requirements

Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field
Minimum 5 to 8 years of experience as a Data Engineer or similar role
Strong programming experience in Python (pandas, boto3, PySpark, SQLAlchemy, etc.)
Deep hands-on experience with AWS services, including: AWS Glue, Lambda, EMR, Redshift, Athena, S3, Step Functions, IAM, CloudWatch, Kinesis (for streaming), and ECS/EKS (for containerized workloads)
Experience with SQL and NoSQL databases (e.g., PostgreSQL, DynamoDB, MongoDB)
Strong knowledge of data modeling, schema design, and ETL orchestration
Familiarity with version control (Git) and CI/CD pipelines for data projects
Understanding of data governance, lineage, and cataloging principles
Excellent problem-solving, debugging, and performance-tuning skills
Strong analytical and communication skills
Ability to work independently and in cross-functional teams
Passion for automation and continuous improvement
Adaptability in fast-paced, evolving cloud environments
Preferred Skills
Experience with Apache Spark or PySpark on AWS EMR
Exposure to Airflow, dbt, or similar workflow orchestration tools
Knowledge of containerization (Docker, Kubernetes) and DevOps practices
Experience with machine learning data pipelines or real-time streaming (Kafka, Kinesis)
Familiarity with AWS Glue Studio, AWS DataBrew, or AWS Lake Formation

Responsibilities

Design, develop, and maintain ETL/ELT data pipelines using Python and AWS native services (Glue, Lambda, EMR, Step Functions, etc.)
Build and manage data lakes and data warehouses using Amazon S3, Redshift, Athena, and Lake Formation
Implement data ingestion from diverse sources (RDBMS, APIs, streaming data, on-premise systems)
Optimize data workflows for performance, cost, and reliability using AWS tools like Glue Jobs, Athena, and Redshift Spectrum
Develop reusable, modular Python-based frameworks for data ingestion, transformation, and validation
Work with stakeholders to understand data requirements, model data structures, and ensure data consistency and governance
Deploy and manage data infrastructure using Infrastructure as Code (IaC) tools such as Terraform or AWS CloudFormation
Implement data quality, monitoring, and alerting using CloudWatch, Glue Data Catalog, or third-party tools
Support data security and compliance (IAM roles, encryption, data masking, GDPR policies, etc.)
Collaborate with DevOps and ML teams to integrate data pipelines into analytics and AI workflows

Skills

Key technologies and capabilities for this role

Pythonpandasboto3PySparkSQLAlchemyAWSGlueLambdaEMRRedshiftAthenaS3Step FunctionsTerraformCloudFormationCloudWatchETLELTSQL

Questions & Answers

Common questions about this position

What experience level is required for this Data Engineer role?

Minimum 5 to 8 years of experience as a Data Engineer or similar role is required, along with a Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.

What are the key technical skills needed for this position?

Strong programming experience in Python (pandas, boto3, PySpark, SQLAlchemy, etc.), deep hands-on experience with AWS services like Glue, Lambda, EMR, Redshift, Athena, S3, and experience with SQL/NoSQL databases are required.

Is this a remote position or does it require office work?

This information is not specified in the job description.

What is the salary or compensation for this role?

This information is not specified in the job description.

What soft skills are important for this Data Engineer job?

Strong analytical and communication skills, ability to work independently and in cross-functional teams, passion for automation and continuous improvement, and adaptability in fast-paced environments are key.

Egen

Provides API strategies and cloud solutions

About Egen

Egen Solutions specializes in API strategies and cloud solutions to help businesses modernize and scale. They offer two main paths for application development: Rebuild, which improves existing functionalities, and Replace, which creates new applications in the cloud. Egen stands out by providing tailored digital transformation services that focus on delivering measurable ROI and quick results. Their goal is to enhance productivity and reduce operational costs for clients across various sectors, including retail and public health.

Naperville, IllinoisHeadquarters

2000Year Founded

$1.3MTotal Funding

PRE_SEEDCompany Stage

Consulting, Consumer Software, Enterprise SoftwareIndustries

1,001-5,000Employees

Benefits

Fun team outings

Health insurance

401(K)

Dental & vision insurance

Paid time-off

Food & drinks

Parental leave

Risks

Emerging tech startups offering similar solutions at lower costs threaten Egen's market share.

Rapid technological changes may outpace Egen's ability to update its solutions.

Economic downturns in retail and e-commerce could impact Egen's revenue.

Differentiation

Egen offers tailored digital transformation services with a focus on ROI and innovation.

The company specializes in secure, dynamic API strategies for seamless business growth.

Egen provides two application development paths: Rebuild and Replace, ensuring flexibility.

Upsides

Rising demand for cloud-native applications boosts Egen's cloud solutions offerings.

Growing interest in API-first development enhances Egen's API strategies and integration services.

Increased focus on cybersecurity in API development aligns with Egen's secure solutions.

Land your dream remote job 3x faster with AI

Try Jobo Free