Evaluation sets and rubrics
Curated prompts and expected behaviours, scored by trained raters and by automated checks.
AI / ML · 1047 OPEN ROLES
QA for AI and ML
Testing an AI product means testing something that can be wrong in new ways: biased, unsafe, inconsistent or confidently incorrect. QA here is evaluation design, rubric-based scoring, red-teaming and data-quality checks that run for every model change.
Reply within one working day. NDA before we see your backlog. Shortlist in about 4 working days.
What we test
What we test in AI and ML.
The risks that matter most in this industry, and the checks we put in place for each. Engineers are matched on these, not only on tool names.
Safety and red-teaming
Structured attempts to elicit harmful, private or off-policy output, with severity ratings.
Regression across model versions
Score sheets that show what got better and what got worse when a model or prompt changes.
Data quality
Labels, deduplication, drift and leakage checks on training and evaluation data.
Typical roles
All QA roles
The roles AI and ML teams hire.
Each role links to the hiring page and to the role guide. Pay shown is the US median for mid-level, gross per year.
AI / LLM Evaluation QA
Designs and runs the evaluation.
Data / ETL QA Engineer
Verifies data pipelines and labels.
QA Automation Engineer
Tests the product around the model.
Security / Penetration Tester
Prompt injection and abuse testing.
Localization QA Tester
Quality in non-English languages.
Another role?
We staff 15 QA roles in every industry. Tell us the one you need and the stack it runs on.
All QA roles
Tooling
The stack our AI and ML engineers work in.
Tools common in AI and ML QA. We staff against your stack, so this list is a starting point and not a limit.
PythonLanguagePostmanAPI testingJiraManagementDockerInfrastructureGitHub ActionsCI/CDPlaywrightWeb automation
Each tool page shows demand, pay and a learning path. Licences stay in your name.
Compliance considerations
What shapes AI and ML testing.
These standards describe obligations that usually apply to the product or the client. PassRidge does not hold them on your behalf and nothing here is legal advice. Your compliance owner sets the scope.
01
EU AI Act (Regulation 2024/1689)
Risk-based obligations for AI systems, phasing in from 2025 to 2027, including testing and documentation for high-risk systems.
02
NIST AI Risk Management Framework 1.0
A voluntary US framework for mapping, measuring and managing AI risk.
03
ISO/IEC 42001
A management-system standard for organisations that develop or use AI.
04
OWASP Top 10 for LLM applications
A list of common weaknesses in LLM applications, such as prompt injection, used to structure security tests.
Our engineers work inside your controls, on masked or synthetic data, and produce test evidence in the format your reviewers ask for. How security works in every engagement.
Engagement model
Compare all five
Why Managed QA (QAaaS) fits AI and ML.
Evaluation is a repeatable service with a rubric, raters and a score sheet each release. It is a starting recommendation: all five models are available for AI and ML work.
Managed QA (QAaaS)
We run QA end to end and report against agreed SLAs.
QA staff augmentation
Add vetted QA engineers to your team, under your management.
Dedicated QA team
A stable QA squad that works only on your product.
Project-based testing
Fixed scope, fixed price: a release tested, an app audited.
Open roles
View all 1047
Recent AI and ML QA roles.
Real listings with pay shown. Looking for a job in this industry? Browse them all.
Permanenttoday
Quality Assurance Analyst
$97k / year
AI company · Atlanta · On-site
SeleniumPostmanREST Assured
Permanenttoday
Software Tester (Manual) - WFH 90%
฿540k–780k / year
AI company · Bangkok · Hybrid
PlaywrightSeleniumCypress
Contracttoday
SDET - Automation Testing
₹1,500–1,800 / hour · est.
AI company · Bengaluru · Hybrid
PlaywrightSeleniumCucumber / BDD
FAQ
QA for AI and ML: your questions.
How many raters do we need?
It depends on the evaluation set size and the agreement you need. We start with a pilot of one to three raters, measure agreement and size the team from that.
Can raters work in several languages?
Yes. We recruit native-level raters for the languages you ship, in many regions.
Do you handle sensitive content?
Safety evaluation can expose raters to harmful material. We brief them in advance, limit exposure, let them opt out and provide support. Tell us in the brief what the content involves.
Which engagement model do you recommend for AI and ML?
Usually managed qa (qaaas). Evaluation is a repeatable service with a rubric, raters and a score sheet each release. Managed QA (QAaaS) starts in about 21 days, mid-level rates begin at $48 an hour and the minimum term is 6 months. The five models are compared at /outsourcing/.
Hire for AI and ML
Tell us what you test. We reply in one working day.
Send the product, the stack and the release cadence. An engagement manager answers with a team outline, a recommended model and a price range.
Other industries
All ten
305 open roles
Cybersecurity
Security products and teams that need testers who think like attackers.
QA for cybersecurity 478 open rolesFintech
Payments, lending and trading platforms where a wrong decimal is an incident.
QA for fintech 46 open rolesBanking & Insurance
Core-banking migrations, regulated releases and long regression cycles.
QA for banking and insurance