Quality Can Be Assigned. The Skills Behind It Can’t.
On agent-driven teams, responsibility for quality can be assigned to other roles, but the judgment behind it is a discipline of its own, and it doesn't transfer with the assignment. That’s the central question for QA in agent-driven teams right now: with the shift toward more generalist roles, not every project will have a dedicated QA, and quality may end up assigned to the developer, supported by agents that write and run tests. The part of QA that agents automated was the most visible part, and the part they can't automate gets more exposed.
Who owns QA in agent-driven teams today?
On our projects at Rootstrap, quality today is the whole team's responsibility. An agent-driven team is one where AI agents write code and tests, while people decide what gets built and what ships. Developers test their own code, and agents generate tests and test cases. Product makes sure what gets built serves the business. And most projects still have a dedicated QA, a person whose only role on the team is to look at quality end to end and ask the questions the rest of the team doesn’t have the time, or the incentive, to ask.
Depending on a project's size, risk level or budget, that QA may not be there anymore. If the agent writes the code and the tests too, why have a separate role? That logic rests on the most common belief I hear about QA on AI-driven teams, that QA means running tests. Running tests was never the core of the work. It was just the part people could see.
Which QA skills don’t transfer to developers and AI agents?
The QA skills don’t transfer when quality moves to developers and AI agents, and no job description lists them: knowing where a system tends to break, finding gaps in the definition, walking the full flow like a user, distrusting a green result, separating the agent’s theories from the evidence, and evaluating the testing agent itself. None can be learned from a checklist, because they're built through exposure to failure, through distance from whoever built the solution, and through incentives that are different from shipping.
Knowing where a system tends to break
In my experience, a senior QA doesn’t test everything equally. They go straight to role permissions, state transitions, payments and integrations, because that's where things fail again and again. On one project, QA directed the agent to test every action with one user per role, through the UI and against the API, and that uncovered a permission that let a user do more than their role allowed. I described that case in why AI agents make mistakes and how QA catches them.
Why it's hard to hand off: this isn't knowledge of the code. It's a memory of failures built up across many projects. A developer knows how they built the system, while QA knows how systems like it tend to break.
Finding gaps in the definition before testing
Before writing a single test case, QA cross-checks the spec, the acceptance criteria and what's actually deployed. That's how you find criteria other tickets have already replaced, flags that no longer exist and scenarios nobody defined.
Why it's hard to hand off: it requires distance from the solution. Whoever built the feature has already filled those gaps in their head, and the agent, unless pushed, tests exactly what the acceptance criteria say and nothing more.
Walking the full flow the way a user would
One of the most serious bugs we found was a charge on a project that had already been deleted. It left no errors in the console or the API, and no monitoring would have caught it. It surfaced because QA had the agent walk through the full purchase flow end to end, and it was fixed before release.
Why it's hard to hand off: development work is organized around tickets and components, and agents validate whatever they're pointed at. Thinking about the whole journey, including its detours, and deciding it needs to be walked, is a different way of looking at the product.
Distrusting a green result
On agent-driven teams, a “partial pass” can hide real bugs. A green result proves nothing if the fix isn't deployed, if the test was rerun until it passed, or if "partial" was counted as passed.
Why it's hard to hand off: the developer's incentive is to close, and green closes. This distrust is a practice that gets trained, not a personality trait.
Separating the agent's theories from the evidence
I’ve seen agents present wrong root causes with complete confidence, and one reported an element as missing when it was on screen with different text. QA asks for evidence, checks preconditions and confirms with a screenshot or a person before filing a bug.
Why it's hard to hand off: agent output sounds certain, and without the habit of demanding proof, a confident tone gets mistaken for certainty.
Evaluating the testing agent itself
Agent-driven teams add a new task: measuring which duplicates the agent missed, which severities it got wrong, how many false positives it produces and which tests fail intermittently.
Why it's hard to hand off: it's a QA skill applied to a tool, and without that judgment, no one is checking the checker.
Running tests, writing test cases and closing tickets transfer with the role, while six QA judgment skills stay behind, from knowing where systems break to evaluating the testing agent.
What can AI agents handle in QA, and where do they stop?
AI agents are very good at the part of QA work that can be described, and they fall short on everything that isn't written down. If someone tells the agent what to look at and how, it does so with a breadth and consistency no person can sustain. Once QA decides permissions are critical, the agent tests every role and action without skipping combinations. It also cross-checks documents fast and walks defined flows in every variant.
What it still can't do is choose where to look, propose scenarios nobody wrote down, or see what its tools can't see. Playwright doesn't perceive zoom or real screen size, and it can skip input masks, so what you observe on a real device still counts for more.
The line is that the agent applies judgment but doesn't generate it. It can carry out a strategy someone already thought through, extremely well, but it can't decide which strategy was needed or notice that the one it has isn't enough. On our teams, QA with AI agents works best when QA defines where to look and what counts as evidence, and the agent takes care of looking everywhere.
AI agents execute defined risk strategies, cross-check documents and walk defined flows, while QA chooses where to look, finds undefined scenarios, sees past its tools and questions the results.
What do developers miss when they review agent output?
When a developer reviews agent output, they review it as its author, so they tend to miss what sits outside the change. QA reviews the system: what the change affects, and what nobody asked for. Three things slip through most often.
The first is AI-generated tests that confirm the code instead of the requirement. When the agent writes both the code and its tests, the tests tend to verify what the code does, not what it should do, so they pass by design. The second is what works technically but is wrong for the business. On a payments project, the system was saving cards that had failed CVC validation, without throwing a single error. The code did exactly what was written, and the problem was what wasn't. The third is what isn't in the diff. On that same payments project, 2 of its 15 pricing tiers had no price configured, and users got stuck at checkout. It was configuration, so no code review was ever going to catch it.
On another project, several blocking bugs had been moved to "Done" and the automated checks showed nothing wrong. QA reverified them in the real environment, with each fix confirmed as deployed. Of 7 tickets marked "Done", 3 were still broken. Nobody was negligent. A ticket in "Done" says someone made a change, and that's different from the problem being gone for the user.
When does a project need a dedicated QA?
A project needs a dedicated QA when its risk outgrows what developers and agents can judge on their own, and the first sign is usually subtle: issues start appearing in the client’s testing before they appear in yours. One of our projects started without a dedicated QA, which made sense at the time: limited scope, a small team, and developers with agents covering day-to-day testing. As the product grew, that’s exactly what happened. Nothing catastrophic, but enough for the client to start testing with suspicion. We brought in a senior QA to set up the strategy, not to run more tests: where the risk was, what "done" meant, and evidence the client could see.
On a different Rootstrap project, a dedicated QA working inside our delivery system caught 70 of the 76 bugs found over roughly two months, about 92%, before they reached users. Of those 70, sixteen had been introduced before QA joined and were caught in staging. The six that reached production were all low severity, and none of the 172 tickets QA verified had to be reopened after verification. On another project, preparing test plans, test cases and automation scripts went from two or three days to about one. The time saved didn't reduce the need for QA judgment, because it expanded what one QA per pod could reach.
These are measurements from specific projects, not averages. The escaped count only covers what got through after the QA joined. Bugs that were already in production when the QA arrived aren't part of the 76. The comparison that matters most, the same numbers with and without a dedicated QA, is something we don't have yet. Measuring it is our next priority.
What should engineering leaders do before removing QA from a team?
Before deciding whether an agent-driven team needs QA, a leader should ask who is going to hold the judgment. Assigning quality to another role is easy, and keeping that judgment on the team is the hard part. The industry data points the same way: Google's 2024 DORA report found that delivery stability went down as AI adoption went up. I'd recommend three things:
- Decide on QA presence by risk, not by size. A small project with payments, permissions or sensitive data may need more quality judgment than a large one with little risk.
- If there's no dedicated QA, name who defines what "done" means and who signs off on what ships. Without a name attached, no one handles it.
- Treat evaluating your testing agents as real work. An agent that tests and isn't evaluated is just another source of confident errors.
The same goes for evidence. An agent can produce a polished test report in minutes, but a client needs someone who decides what to test, can explain why a risk was left open, and answers for it. Evidence becomes trustworthy when there's a name behind it.
The QA role has its own part to play too, by talking less about test cases and more about risk. If quality is measured by how many tests get run, agents already look like they're doing QA's job. Measured by how many problems reach the user, it's a different conversation. I wrote about where the role was heading in the future of the QA role in the age of AI, and if you're building an agent-driven team, you can see how Rootstrap's delivery works.
