From Peer Review to Self-Review: The Epistemic Risk of Agentic Development

Modern code review has traditionally been a team practice. One developer produces a change, while one or more peers inspect it before it becomes part of the shared codebase. Although defect detection is an important part of this process, empirical research has repeatedly shown that code review serves broader purposes, including knowledge transfer, team awareness, finding better solutions, and maintaining shared understanding of the system (Bacchelli & Bird, 2013).

Agentic development is beginning to change this arrangement. When a developer delegates implementation work to an agent, much of the first review naturally moves into the interaction between that developer and the agent. The developer gives the agent a task, inspects what it produces, asks questions, runs tests, requests changes, reviews the revised implementation, and eventually decides whether the result is acceptable. Recent observations of AI-generated pull requests already show review activity shifting toward automation-mediated interactions in which humans often participate by steering agents rather than performing standalone reviews (Duma et al., 2026).

I use self-review here to describe this first detailed human review performed by the developer who directed the agent. This does not necessarily mean that the developer wrote the code themselves. In fact, the shift becomes particularly interesting when they did not. The developer increasingly moves from being the direct producer of every implementation detail towards steering the work and subsequently verifying whether the implementation that emerged is actually the one they intended to create.

There are good reasons why this shift is happening. Self-review removes substantial coordination and social overhead, keeps review close to the context in which the work was produced, and allows verification to become much more iterative. At the same time, moving review into the developer-agent loop also removes some of the independence that peer review traditionally provides. This creates an epistemic problem: the person who framed the task, directed the agent, and participated in important decisions is also the person who must determine whether the resulting implementation and the assumptions behind it can be trusted.

Why Self-Review Is Attractive

A peer who receives a pull request after the implementation has been completed first needs to understand what changed and why. Recent observational research into code-review comprehension shows that reviewers construct a mental model of the change and relate it to their understanding of the surrounding system before they can meaningfully evaluate the implementation (Gonçalves et al., 2025). The developer who directed the agent, in contrast, has usually followed the work and should already have substantial context about the task, the intended solution, and at least some of the decisions that were made along the way.

Review can therefore happen immediately instead of waiting for another developer to become available and reconstruct that context. The steering developer can inspect part of the implementation, ask the agent why it chose a particular solution, request additional tests, run another agent review, revise the specification, and continue the implementation. Review becomes tightly interwoven with development rather than a separate activity that begins when the change is considered finished.

This also removes some of the social costs of traditional code review. Code review is not merely a technical exchange. Developers disagree, criticism has to be formulated and interpreted, and interpersonal conflicts sometimes emerge. Qualitative research has found that conflicts during code review are commonplace and strongly intertwined with the developer, team, organizational, and technical context (Gonçalves, Çalıklı & Bacchelli, 2022). A subsequent study with 154 developers found that 77% reported experiencing interpersonal conflicts during code review at least sometimes (Gonçalves et al., 2024).

An agent does not introduce the same interpersonal relationship into the interaction. I can ask it to review the same implementation repeatedly, reject its suggestions, ask basic questions without worrying about appearing uninformed, or explicitly tell it to be highly critical. There is no colleague whose time I am consuming and no interpersonal relationship I need to protect while doing so.

Early research comparing peer and LLM-assisted review interactions provides some support for this difference. In a qualitative study with 20 software engineers, Alami and colleagues found that peer review involved emotional self-regulation and social calibration, while participants experienced lower emotional costs when interacting with an LLM reviewer. The authors also caution that reduced emotional friction can affect how readily developers engage with and adopt AI feedback (Alami et al., 2025). As this is a small qualitative preprint, these findings should be interpreted as an emerging observation rather than a general result.

Self-review therefore has properties that are genuinely useful: high context, little coordination overhead, almost unlimited opportunities for iteration, and the possibility of deliberately requesting criticism without many of the social costs associated with human disagreement.

The problem is that asking for criticism and obtaining an independent perspective are not necessarily the same thing.

The Developer Has Context, but Limited Independence

The steering developer is well positioned to understand the implementation because they participated in creating it. For exactly the same reason, they are not an independent reviewer.

During agentic development, the developer determines the initial problem framing, chooses what information to provide, answers questions from the agent, accepts or rejects intermediate decisions, and decides when the implementation has progressed far enough. Even when the agent makes substantial autonomous decisions, its work takes place within a development process that the steering developer has shaped.

The resulting implementation and the developer’s understanding of it may therefore share the same assumptions.

Suppose, for example, that the developer misunderstands an important requirement and describes that requirement incorrectly to the agent. The agent may implement the incorrect requirement consistently, generate tests for that implementation, explain why the implementation satisfies the requirement, and subsequently review the code against the same understanding. The code, tests, explanation, and review can all agree with each other while the underlying premise remains wrong.

This problem does not disappear simply by adding more automation. A separate review agent can provide additional challenge and may reduce some coupling between implementation and review, but it can still operate within the same incomplete specification, missing organizational knowledge, or incorrect framing.

Current AI reviewers also remain imperfect. For example, the 2026 SWE-PRBench preprint evaluated eight frontier models against issues identified in human-reviewed pull requests and found that the models detected only 15–31% of the human-flagged issues in its diff-only configuration. The results varied with the supplied context, illustrating both the current limitations of automated review and the importance of what information the reviewer receives (Kumar, 2026).

The important question is therefore not only whether sufficient review has happened, but whether the evidence produced during that review is sufficiently diverse and independent to expose the assumptions on which the work is based.

The Problem of Challenging Your Own Assumptions

This problem is not unique to AI. Psychology has studied for decades how difficult it is for people to evaluate evidence independently of beliefs they already hold.

Confirmation bias broadly describes the tendency to search for, interpret, or give greater weight to information that supports an existing belief or hypothesis (Nickerson, 1998). One consequence is that people are often better at collecting evidence that is consistent with their current explanation than deliberately searching for evidence that would show the explanation to be wrong.

This becomes relevant during self-review because the steering developer rarely approaches the implementation as a neutral observer. They already have a theory about what the change should accomplish and, after working with the agent for some time, probably also have a theory about how it accomplishes it.

Imagine that I believe the agent has implemented a cache correctly. I run the expected scenario and it works. The tests pass. I inspect the central implementation and it looks plausible. I ask the agent whether invalidation has been handled correctly, and it provides a convincing explanation of the invalidation mechanism.

I have now collected several pieces of evidence supporting my current understanding, but I may still not have seriously investigated the conditions under which that understanding would be wrong.

Research on debiasing provides a useful idea here. In experiments on biased reasoning, Lord, Lepper, and Preston found that instructing participants to consider the opposite, meaning to actively consider how their initial judgment might be wrong, reduced bias more effectively than simply asking them to be fair or objective (Lord, Lepper & Preston, 1984).

For code review, this suggests a subtle but important shift in how we use agents. Instead of primarily asking an agent to confirm whether an implementation satisfies our expectations, we can ask it to help construct plausible explanations for why it might not.

There is an additional complication: the agent itself may reinforce our framing. Research on LLM sycophancy has shown that models can adjust their answers towards views expressed by the user rather than consistently maintaining an independent position. Sharma and colleagues demonstrated this behavior across five AI assistants and found that responses matching users’ stated views were also more likely to be preferred in human preference data (Sharma et al., 2024).

This does not mean that an agent will simply agree with every developer during code review. It does mean that developers should not treat a convincing agent response as evidence of independence merely because it was produced by another model.

The way we formulate review questions therefore matters. Asking, “Does this implementation correctly handle concurrent updates?” already provides both a hypothesis and the desired property. Asking instead, “What assumptions does this implementation make about concurrent updates, and under which execution sequences could those assumptions fail?” opens a wider search space.

The difference is small in terms of prompting, but substantial in terms of review intent. One approach primarily seeks support for an existing conclusion, while the other deliberately searches for reasons to revise it.

Automation Can Also Make Weak Evidence Feel Strong

Agentic review introduces another well-studied human factors problem: automation bias.

Research on automated decision-support systems has shown that people can rely excessively on automated recommendations and may fail to notice contradictory information or problems the automation fails to flag. This effect has been demonstrated in controlled experiments involving simulated aviation decision tasks (Skitka, Mosier, Burdick & Rosenblatt, 2000). These studies are not about software development, so their findings should not simply be transferred directly to code review. However, they demonstrate a general problem in human-automation interaction that becomes increasingly relevant when developers use automation not only to produce code but also to evaluate it.

Early evidence from agentic code review provides additional reasons to study this carefully. A 2026 preprint analyzed 11,429 reviews of agent-generated pull requests performed by 400 repeat reviewers over seven months. Across increasing reviewer experience with agent PRs, approval rates increased while inline review comments decreased by 22%. The authors describe the combined pattern as being consistent with habituation under growing workload, while explicitly noting that the observational design cannot establish causality and that improving agent quality may partly explain the results (Yu et al., 2026).

Another 2026 preprint analyzed 1.02 million reviewed pull requests across 207 GitHub projects and compared human-centric, LLM-assisted, and agentic review patterns. Some forms of agent involvement were associated with faster review decisions, but the authors did not find corresponding improvements in review quality (Zhong et al., 2026). Again, these are observational results and should not be interpreted as evidence that AI involvement itself causes lower review quality.

This distinction becomes important during self-review. A strong test suite, successful static analysis, several agent reviews, and a plausible explanation of the implementation can substantially increase confidence. However, that confidence should depend on what those safeguards were actually capable of establishing.

Tests may encode the same incorrect requirement. Agent reviews may lack context that exists only in another team member’s head. A developer may interpret an agent’s confident explanation as stronger evidence than it is. The more automation we introduce, the more important it becomes to assess the reliability and limits of that automation rather than merely counting how many checks have passed.

Self-Review Therefore Requires Different Skills

Traditional code review skills do not disappear in an agentic environment. Developers still need to read code, reason about behavior, understand architecture, assess tests, and recognize problematic implementation decisions.

However, self-review adds another skill: developers need to become good at challenging work whose assumptions they helped create.

This is partly a technical skill, but it is also an epistemic one. The objective is to identify where our confidence comes from, which assumptions different pieces of evidence share, what we may not know, and which questions would be most likely to reveal that our current understanding is incomplete.

In practice, I would include the following activities in a serious self-review of agent-produced work:

1. Re-establish the intended behavior independently of the implementation. Before relying on the agent’s summary, restate what the change should accomplish, which important constraints it must preserve, and what must not change. Otherwise, the implementation can gradually redefine the requirement it was supposed to satisfy.
2. Identify assumptions explicitly. Ask which assumptions the implementation makes about data, callers, concurrency, ordering, failure handling, system state, external services, security boundaries, and surrounding architecture. Then determine which of these assumptions were actually established and which were merely inferred.
3. Actively search for evidence that the implementation is wrong. Instead of only demonstrating successful behavior, construct counterexamples, boundary conditions, failure scenarios, unexpected sequences, and inputs that could distinguish a correct implementation from a plausible but incorrect one. Asking the agent to “consider the opposite” can help generate these cases, but the developer should evaluate them rather than delegate the judgment back to the agent. This approach is consistent with experimental research showing that actively considering how one’s current belief could be wrong can reduce biased reasoning (Lord, Lepper & Preston, 1984).
4. Challenge the tests and automation as part of the review. Passing tests are evidence only for the behaviors those tests meaningfully exercise. Review which important defects could still survive the existing test suite, what the static checks do not cover, and whether agent reviewers had sufficient context to assess the concerns that matter for this particular change.
5. Use agents to create alternative perspectives rather than repeated confirmation. A fresh context, a separate review agent, different instructions, or an explicitly adversarial role can reduce some shared framing. These techniques should not be treated as equivalent to an independent human review, but they can make self-review less circular. Research on LLM agents also suggests that explicitly prompting systems to anticipate failures and alternative paths can improve their problem-solving behavior, although this evidence comes from agent-task settings rather than code review (Wang et al., 2024).
6. Explore alternative designs. Ask what other reasonable solutions exist and compare their assumptions, system impact, complexity, and failure modes. An implementation can look convincing when considered in isolation while being clearly inferior once alternatives are made explicit.
7. Check your own mental model. The steering developer should be able to explain how the important parts of the change work, why significant decisions were made, where the implementation can fail, and why the available evidence supports the current level of confidence. Research on code-review comprehension reinforces the importance of reviewers developing such a mental representation of both the expected and actual implementation (Gonçalves et al., 2025). If this is no longer possible, the developer-agent loop has become too large or moved beyond the developer’s understanding.
8. Make uncertainty visible. Areas that were difficult to understand, decisions largely made by the agent, assumptions that could not be verified, unfamiliar technologies, weak tests, and concerns that remain unresolved are precisely the information another reviewer needs. Hiding uncertainty makes the subsequent review appear stronger while making it less useful.

None of these techniques guarantees an unbiased review. The point is to structure self-review so that it does not simply become a process of collecting increasingly sophisticated confirmation for a solution that has already been accepted mentally.

There Is Also Something We Lose When Review Becomes Individual

The epistemic problem of self-review is only one side of the shift. Traditional code review is also one of the mechanisms through which knowledge moves across a team.

Bacchelli and Bird’s study of modern code review identified knowledge transfer and team awareness among its important outcomes (Bacchelli & Bird, 2013). Subsequent work has investigated this knowledge-transfer function directly. Caulo and colleagues analyzed 32,062 peer-reviewed pull requests across 4,981 GitHub repositories and found evidence consistent with developers learning from previous review activity (Caulo et al., 2020).

If detailed review increasingly happens between one developer and an agent, this transfer no longer happens automatically.

This matters even if the implementation is technically correct. A developer-agent pair may introduce a new abstraction that other developers will later need to maintain. They may make an architectural decision that affects how others should extend the system. They may solve a problem in a way that conflicts with plans or assumptions elsewhere in the organization. Another developer may know about a constraint that was never documented and therefore never available to the agent.

Software development is ultimately a team activity, and the implementation needs to make sense not only to the developer who produced it with an agent, but also within the team’s shared understanding of the system.

This is why I do not believe the natural move towards self-review means that Peer Review should disappear. Instead, the roles of Developer Review and Peer Review become more clearly differentiated.

The steering developer is particularly well positioned to perform the first detailed human verification because they have context. Peers are particularly valuable because they introduce knowledge, assumptions, perspectives, and judgment that do not originate from the same developer-agent loop.

Combining Self-Review and Peer Review

This distinction is one of the ideas behind SCOPE: Staged Code Oversight with Proportional Escalation, the code review operating model I am currently developing for agentic software development. In SCOPE, the developer who directs the agent explicitly takes over a substantial part of the detailed code review and verification work during Developer Review. The developer inspects enough of the implementation and tests to maintain an accurate mental model, verify that the agent produced what was intended, challenge important agent decisions, and eventually accept responsibility for the work.

The change subsequently reaches Peer Review, but peers are not expected to blindly repeat all the detailed inspection the steering developer has already performed. Their role is to provide an independent perspective, challenge important assumptions and decisions, and develop the shared understanding that cannot emerge entirely inside the developer-agent loop. Depending on the change, this may involve examining the rationale and evidence, discussing architectural or cross-system consequences, inspecting critical implementation paths, or performing a detailed code review.

How much peer involvement is needed depends on the required oversight. In SCOPE, higher risk increases the need for verification and independent challenge, while higher understanding needs increase the need for collaboration and knowledge sharing. The assurance already established through automation and Developer Review influences how that oversight can be carried out, but strong automation does not remove the need for independent challenge when the consequences of being wrong are substantial or when the team needs to develop a shared mental model of the change.

This also provides an escalation path when self-review reaches its limits. If the steering developer no longer understands the implementation sufficiently, cannot explain important decisions, has weak evidence for critical behavior, or is working in an area where another developer has substantially more expertise, another person can be involved earlier and more deeply. Peer involvement then becomes a response to an identifiable need rather than an obligation for every developer to inspect every line with the same depth.

Agentic development therefore does not simply automate traditional code review. It changes where different parts of review can happen effectively.

Some of the detailed verification work that previously depended on peers can move into the developer-agent loop, where the steering developer has more context, can iterate quickly, and can use agents extensively to inspect and challenge the implementation. This has the potential to reduce coordination overhead and make review more closely integrated with development.

At the same time, moving review into this loop concentrates implementation, context, and judgment around one developer. The resulting challenge may be extensive without being independent, and knowledge that previously spread through peer review can remain confined to the developer and their agents.

The skill we therefore need to develop is not simply how to use agents to review more code. Developers need to learn how to systematically challenge the assumptions behind their own work, understand what their automated evidence can and cannot establish, and recognize when their own perspective is no longer sufficient. Teams, in turn, need review models that preserve independent judgment, alignment, and shared understanding without requiring peers to reconstruct and re-review every implementation detail produced by increasingly capable agents.

That is the balance I am trying to establish with SCOPE.

Updated on August 11th, 2026