ML Data Associate III, AWS Quick

Amazon Web Services (AWS)

Description

Design and execute linguistic test cases to validate Quick's natural language features across diverse inputs, edge cases, and user scenarios.

Evaluate model outputs for accuracy, fluency, and relevance by applying linguistic expertise to identify regressions, hallucinations, and quality gaps.

Develop and maintain testing guidelines and rubrics that ensure consistent, repeatable evaluation standards across annotators and test cycles.

Analyze test results and report findings using Python and SQL to surface patterns, quantify defect rates, and prioritize issues for engineering teams.

Collaborate with engineers, scientists, and PMs to define acceptance criteria, triage language-related bugs, and validate fixes before releases

Work on high complex annotations and audits as needed

Work effectively both independently and as part of a team

Uphold a high bar for quality while delivering accurate work at pace

Apply strong attention to detail and Critically assess data quality

Manage competing tasks and adapt quickly to shifting priorities

Take ownership and Exercise sound judgment in escalating

Handle highly complex tasks and Scale with growing workloads

Basic Qualifications

3+ years of working with Data & AI related technologies, including, but not limited to, AI/ML, GenAI, Analytics, Database, and/or Storage experience
Experience with Machine Learning and Large Language Model fundamentals, including architecture, training/inference lifecycles, and optimization of model execution
Completed a bachelor’s degree in engineering, Computer Science, Machine Learning, Operations Research, Data Science/ Linguistics
Knowledge of linguistic fundamentals, including handling ambiguity, natural language processing concepts, and query (Q&A) interpretation

Preferred Qualifications

Experience programming in Python or a related language
Proficient in SQL

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Company - ADCI MAA 15 SEZ

Job ID: A10510427