Passing QA and being right are two different things. A survey can have working routing, correct quotas and text that matches the questionnaire word for word, and still answer the wrong question.
As AI takes on more of research operations, that gap gets easier to miss.
AI is moving deeper into research operations quickly. According to Qualtrics’ 2026 Market Research Trends report, 78% of researchers predict AI agents will run more than half of all research projects by 2028. That makes one question more urgent: when the checking is automated, who is still deciding?
We build AI tools for survey QA. We also think the answer has to be: a person.
The Short Answer
Do you still need human QA when you use AI tools? Yes. AI and automation are very good at checking whether a survey matches its specification, consistently and at speed. They can’t decide whether that specification is right for the research question, the market, the client or the regulator. That call needs a specialist, which is why human oversight in survey QA matters more as automation grows, not less. At CodexMR, the tools check and people decide.
AI Is Moving Faster Than the Rules Around It
Adoption is no longer the open question. Governance is. Greenbook’s 2026 GRIT Insights Practice Report finds the industry has converged on three tasks where agentic AI is already embedded: analyzing data, updating reports, and preparing and integrating data. It also finds that AI governance remains a critical gap, and that confidence in AI risk management correlates directly with exceeding business goals.
Quality is part of the same picture. In Greenbook’s 2025 GRIT Business Outlook, 80% of organizations endorsed AI, yet nearly half were still struggling with questions of quality and impact. Leading firms now audit AI outputs as rigorously as they once audited human respondents.
The industry’s own standards point the same way. ESOMAR published its 20 Questions to Help Buyers of AI-Based Services. One of its five groups of questions is dedicated to human oversight, covering how much a person reviews AI output. Buyers are now expected to ask that question, and vendors are expected to have a good answer.
What QA Tools Do Well (And Why We Built Them)
This isn’t an argument against QA tools. We built two of them:
- QAReady checks a survey link against its source questionnaire. It confirms that every question is there with the right question type, in the right order, with text and answer options that match the document. It checks randomization and exclusivity settings, filtering, routing, terminations and quotas, and whether the survey code runs. You activate it when you need it, and it runs the same checks the same way every time. It doesn’t get tired late in the day, and it doesn’t skip a step unless you tell it to. It also compares wave to wave and copy to copy, so on a tracker you can see every change between versions before processing starts.
- ResearchReady works one step earlier, on the questionnaire itself. It reviews the questionnaire through seven areas: Survey Statistics, Research Design, Survey Logic, Data Compliance, Data Quality, Language & Localization and Respondent Experience. A full review takes about 15 minutes, depending on how complex the questionnaire is. That’s seven expert perspectives before programming begins, in less time than most status meetings.
Both work inside the CodexMR platform or on their own. Both handle the part of QA that people find hardest: checking hundreds of details, perfectly, again. That frees your specialists for the part they’re best at.
A Survey Can Match the Spec and Still Be Wrong-ish
Automated QA has one limit it can’t get around. It checks a survey against a reference, whether that’s the questionnaire, the last wave or a rule. If the reference is wrong, a perfect match is a perfect copy of the mistake.
Programming can follow the specification exactly while the logic is still wrong. That’s why final responsibility stays with the person who signs off the questionnaire.
Most of the industry’s quality investment sits later in the process. Fraud detection has become standard practice, with 70 to 88% of users across every segment using these tools regularly, according to the 2026 GRIT report. That protects the data during and after fieldwork. It can’t fix a question that was wrong before the first respondent arrived. Cleaning can find problems in the data, but it can’t go back and change the question.
ResearchReady helps because it questions the questionnaire, not just the code. Even so, its job is to raise the flag, not to make the decision. We designed it that way on purpose.
The Automation Trap: Easier Checking, Less Looking
Adding a tool does not automatically make a team more careful. Sometimes it has the opposite effect. Researchers call this automation bias: people rely on the tool and look less closely themselves.
The effect has been measured. In a study of electronic prescribing, incorrect decision support raised omission errors (problems users failed to notice) by between 24.5% and 33.3% across three experiments. That’s a clinical setting, but the pattern is familiar to anyone who has watched a green QA report end a conversation early. As one review of human oversight research puts it, automated systems may simply produce different kinds of errors rather than fewer errors overall.
The fix isn’t to use fewer tools. It’s to be clear about who owns the decision. A tool that flags an issue and explains its reasoning keeps the specialist engaged.
A tool that simply shows “passed” invites them to stop looking. That’s why our tools explain, not just approve.
Five Calls a Tool Can’t Make for You
1. Is This the Right Question?
A tool can spot a question with no clear link to the stated objective. Only someone who knows the client can say whether the business decision needs it. ResearchReady’s Research Design lens asks this directly: would anyone miss this question if it disappeared? The answer comes from a conversation with the client, not from the document.
2. Was This Change Supposed to Happen?
On a tracker, QAReady will show you that Wave 7’s answer list differs from Wave 6. It can’t know whether the client requested that change months ago or someone copied the wrong version. The difference is a fact. Whether it’s an error is a judgment.
3. Is This Risk Acceptable?
ResearchReady flags compliance items and cites the source, such as UK GDPR, CCPA, PIPEDA or ESOMAR guidance. It doesn’t decide whether an open question that might collect personal data is acceptable for this client in this market. The compliance owner does, and the test is a human one: would we be comfortable explaining this to a regulator?
4. Will This Work With This Sample, in This Market?
ResearchReady builds an incidence rate estimate criterion by criterion, using public sources. It’s a country-level benchmark for the general population, and your panel may land somewhere very different. A PM who has run this audience before knows whether a lower IR is a risk or a certainty. Translation works the same way: a phrase can be technically correct and still land badly with respondents.
5. What Did the Tool Miss?
We tested this ourselves. We planted eight deliberate errors in a questionnaire and ran ResearchReady on it. It caught seven. It also found issues nobody had planted, including logic that sent questions to the wrong group of respondents.
The one it missed was a removed randomization and anchor instruction.
We’re still annoyed about the eighth. We’re not surprised by it, though. A tool narrows the search, and a specialist closes it.
How We Split the Work: Tools Check, People Decide
QAReady and ResearchReady are two tools inside a bigger system. The CodexMR platform runs the whole quant workflow, from uploading specs and generating survey code to automated QA, data processing and delivery. The same split applies at every step: AI and automation handle the repetitive work, and people make the calls.
It’s also how teams work with the platform. There are three ways to use it, and each one is a model of human oversight across the whole project, not just QA:
- DIY: your own team runs the platform end to end, including QAReady and ResearchReady, on its own schedule. Your specialists make the calls and sign off.
- DIT: our experts work alongside your team on complex projects, adding experienced eyes wherever the judgment calls sit, from questionnaire review to final data.
- DIFM: our 24/7 global team runs the project on the platform from start to finish, and our specialists make and own those calls.
In every mode, a named person signs off before a respondent sees the survey. The tools make sure that person spends their time on judgment, not on checking whether Q37 still has five answer options.
ResearchReady and QAReady are also available as standalone tools.
What This Means When You’re Buying
If you run a research team, it’s tempting to see a QA tool as a way to need fewer specialists. We’d put it the other way round: the tool removes the repetitive work that slows specialists down and keeps them focused on the judgment calls that matter.
Listen to the people closest to the work. The same Qualtrics research found a perception gap: 83% of research leaders say AI has improved efficiency, but only 65% of individual contributors agree. That gap usually means the tool made some tasks faster while the judgment work stayed where it was. Your programmers and PMs know exactly which parts those are.
Skipping that judgment doesn’t cost you anything in QA. The bill arrives later, as a re-field, an unfillable quota, or a client asking why the numbers moved.
Keep the specialist. Give them better tools. Let them go home on time.
Where to Go Next
- Read Survey QA Has a Scaling Problem for a closer look at why build QA keeps stretching timelines.
- See how QAReady checks a survey script against its questionnaire, and try it on your next survey: QAReady
- Explore ResearchReady’s seven review areas: ResearchReady.
- Want to see the full workflow on your own project? Book a call.
