Data Engineer - AWS/PySpark
TekIT Software Solutions Pvt. Ltd. (India & USA)
Date: 10 hours ago
City: Kolkata, West Bengal
Contract type: Full time
Job Title : Data Engineer (AWS & PySpark)
Experience : 8 YEARS (Minimum 5 years in Apache PySpark with AWS, GLU)
Location : Hyd/Pune/ Kolkatta
Time Zone : Normal IST
Mandatory Skills
Experience : 8 YEARS (Minimum 5 years in Apache PySpark with AWS, GLU)
Location : Hyd/Pune/ Kolkatta
Time Zone : Normal IST
Mandatory Skills
- AWS Glue
- AWS Step Function
- PySpark
- Python
- Apache PySpark with Python and AWS (Minimum 5 years of experience)
- Develop and optimize ETL pipelines using AWS Glue and PySpark
- Implement AWS Step Functions for workflow automation
- Work with SparkSQL, SQL, and AWS cloud environments to manage and process large datasets
- Ensure robust data engineering practices with DevOps, CloudOps, and DataOps methodologies
- Follow CI/CD best practices with a Site Reliability Engineering (SRE) mindset
- Design and implement Lakehouse and Data Warehouse (DW) architectures
- Develop APIs and Microservices for data integration and transformation
- Apply version control using Git for efficient code management
- Strong written and verbal communication skills
- Ability to translate business needs into system requirements
- Analytical mindset with strong problem-solving skills
- Ability to collaborate effectively in a team-oriented environment
- Hands-on experience with cloud-based data processing and analytics
- Strong understanding of dimensional modeling and database optimization
- Exposure to modern data engineering best practices and architectures
See more jobs in Kolkata, West Bengal