AWS Glue Data Pipeline Tester
ProArch
The candidate should be able to define and validate data quality rules covering:
- Completeness
- Accuracy
- Consistency
- Validity
- Uniqueness
- Timeliness
- Referential integrity
- Conformity to business rules
- Schema and metadata consistency
Testing Areas
The role will include the following types of testing:
- Functional testing
- End-to-end integration testing
- Data validation testing
- Source-to-target reconciliation
- Parallel Testing with legacy interfaces testing.
- Regression testing
- System integration testing
- User acceptance testing support
- Negative testing
- Exception and error-path testing
- Performance and volume testing
- Scalability testing
- Restart and recovery testing
- Failover and resiliency testing
- Security and access testing
- Production smoke testing
- Operational and monitoring validation
Requirements
- AWS Glue Jobs
- AWS Glue Workflows
- Amazon S3
- Amazon CloudWatch
- AWS IAM
- Amazon EventBridge
- AWS Lambda
- Amazon SNS
- Amazon SQS
- AWS Secrets Manager
- AWS Step Functions
- AWS KMS
- Ability to review AWS Console configurations, job parameters, logs, metrics, and execution history.
- Data Pipeline and File Processing Interface Testing
- Strong understanding of Inbound and Outbound Data File Processing / Data Pipeline.
- Experience testing inbound/outbound file transfers across S3, SFTP/FTPS, AWS Transfer Family, APIs, and external partner endpoints, including authentication, connectivity, retry, timeout, and delivery validation.