At SignatureAPI, AI coding agents write and review most of our code. A year ago, between 17 and 29 pull requests merged into our development branch each month. In September 2026 it was 103, and in October 68. Agents changed how much our team can ship in a month.
| Month | Merged | Month | Merged |
|---|---|---|---|
| Oct 2025 | 17 | May 2026 | 31 |
| Nov 2025 | 29 | Jun 2026 | 74 |
| Dec 2025 | 25 | Jul 2026 | 35 |
| Jan 2026 | 25 | Aug 2026 | 31 |
| Feb 2026 | 24 | Sep 2026 | 103 |
| Mar 2026 | 28 | Oct 2026 | 68 |
| Apr 2026 | 9 |
At that pace, the question is how to keep every change reviewed, traceable and auditable. We hold a SOC 2 Type II report, and SOC 2 expects every change to be approved. For any pull request, we want to be able to show what was checked, by whom, and what happened to the problems the check found. That answer has to be the same for the hundredth pull request of a month as for the first, and it can’t depend on a person reading every diff.
On October 27, 2026 we turned on the control we built for this. Every pull request carries a written review record for its exact head commit, the depth of review follows the size of the change, people make the release and sign-off decisions, and each month all of it becomes one signed report. This post explains the ideas behind it and the rules that implement them, for other teams that run development through agents and need the same guarantees.
A second person approving every pull request
The standard SOC 2 policy template handles change approval with one rule, that the reviewer of a change must be a different person from its author. It is the obvious choice, and GitHub can enforce it with a required approval on every pull request. On a team where people write the code, it works well. Someone other than the author has read the change and agreed to it, and the approval on the pull request is the evidence.
When agents produce the work, that approval measures the wrong thing. It records that a person clicked approve. It doesn’t record what was checked, which findings came up, or what was done about each of them, and those are the questions we want our evidence to answer. The approval carries a name without describing the review that took place. We wanted the evidence to describe the review itself, so our system description says what happens. Changes go through automated gates, AI-assisted review, a staging environment, and a separate authorization before production.
Reviewing every push in full, or only what each push adds
Once review is something an agent runs on demand, a second question follows. A pull request rarely lands in one push. It picks up commits that address findings or fix a failing test, and each push needs some review before it can merge.
One answer is to repeat a full review of the whole pull request with our most capable models after every push. It is the simplest rule to state and the hardest to argue with. It is also slow and costly for a follow-up of a few lines written in response to a finding, and most of that review would cover code that was already reviewed. The opposite answer is to review only what each push adds. That keeps follow-ups cheap, but it judges each push on its own, so enough small pushes can add up to a large change that no review ever looked at as a whole. The rules we shipped take the cheap path for small follow-ups and add a running total that forces a full review when the small ones add up.
What we shipped
The control rests on seven ideas. Review is an artifact attached to an exact commit. Every blocking finding is resolved in writing. Review depth is proportional to the size of the change. Every pull request is judged by the rules already on the base branch. People make the decisions that need a person. An emergency path exists, and it is reviewed afterwards. All of it closes into one signed document per month.
A review record attached to an exact commit
A pull request into our development branch can merge only when all of the following hold for its head commit:
- The required CI checks passed. These are lint, build, unit tests, and a contract check of the API against our OpenAPI spec.
- The title or branch name carries a ticket id.
- A secret scan of the pull request’s commits finds no secret.
- A valid review record exists for that exact commit, posted by someone with write access to the repository.
- An unbroken chain of review records covers everything from the point where the branch left the development branch up to the head.
A review record is a comment on the pull request. It names the model that ran the review and the procedure that ran. That can be a review command at a stated depth, a single reviewer agent, or an orchestrator that splits the change among several sub-reviewers, each listed with its model and what it focused on. Then come the findings and a verdict of pass or fail. Our tooling adds the head commit and the range of commits the review covered, which is what ties the record to one exact state of the code.
When all five conditions hold, an automated reviewer account approves the pull request. That approval belongs to the commit. Any new push makes it stale, and the check goes back to “Waiting for review record”. The check reports that state as pending, not failed, since a push with no record yet is an ordinary step in the work, and it fails only when something is actually wrong.
Every blocking finding resolved in writing
Each finding in a record has a number, a severity, a summary, a resolution and a reason. The severity decides what the record has to show before the gate accepts it.
| Severity | What the record has to show |
|---|---|
| Critical | Fixed, or dismissed with a written reason |
| Important | Fixed, or dismissed with a written reason |
| Minor | Fixed, dismissed, or left open |
A dismissal is a normal outcome. Reviewers raise findings that turn out not to apply, and the record keeps the reason next to the finding instead of leaving it to memory. The record also has to state exactly what ran. The gate treats the record as evidence and checks that it exists, belongs to the current commit, and is consistent with these rules. Its truthfulness rests on two things the process makes visible, the statement of what ran and the reason on every dismissal, and every dismissed blocking finding reaches the monthly report that management signs.
Review depth proportional to the change
The first record on a pull request is always a full review of the whole change from its merge base. After that, the size of each new push decides what kind of review it needs.
| Small follow-up | Large change | |
|---|---|---|
| Size since the previous record | At most 200 changed lines in at most 8 files | Anything bigger, or a size that can’t be computed |
| What gets reviewed | Only the new commits | The whole pull request from its merge base |
| Who can review | Any allowed model, including smaller, faster ones, at any depth | One of our two most capable models, at medium depth or deeper |
Lockfiles and generated clients don’t count toward the size. A reviewer agent, a review skill or an orchestrator all count as medium depth, and under an orchestrator the sub-reviewers may use any allowed model. This puts our most capable models on the reviews where they matter and lets a quick fix get a quick review.
The cumulative rule closes the gap that per-push sizing leaves open. On a large pull request, the gate adds up everything pushed since its last large-tier full review. Once that total is over the small tier, every follow-up review is rejected, however small its own push was, until a new full review starts the count again. Splitting the remaining fixes into more pushes doesn’t change the total.
Merging the development branch into a pull request never counts toward a delta, since those commits came through their own pull requests. Conflict resolutions in that merge do count, because they are new code written on the branch.
Rules from the base branch
The gate always runs the base branch’s version of its own code. It never checks out or runs the pull request’s code, so a pull request is judged by the rules already in place and can’t change them for itself. The secret scan works the same way. Its allowlist comes only from the base branch, so a pull request can’t add its own exceptions.
This matters more when agents write the changes, since an agent asked to make a check pass may change the check. Here that edit has no effect on its own pull request. It takes effect only after it has merged under the old rules, and changes to the gate’s own files are listed in the monthly report. One practical consequence is that a pull request changing the format of the record has to post a record that the old format still accepts.
People for release authorization
Production is a separate step with a person in it. A release is a pull request from the development branch to the production branch, and merging it deploys production. The gate passes a release only if our automation opened it and a full end-to-end run passed on staging on that exact commit. That suite runs against staging after every staging deploy and validates every API response against our OpenAPI spec.
The gate never approves a release. A person does, acting as release manager. That approval is a release authorization, a decision that this set of commits goes to production, made after the review records, the CI checks and the staging run are already in place. The monthly report records who prepared, approved and merged each release.
An emergency path reviewed afterwards
The gate can be bypassed in two situations only, an outage of the CI system or a false block on a production fix. An ordinary urgent fix goes through the normal path with a label.
To use break-glass, someone comments BREAK-GLASS on the pull request with a reason and a ticket. A repository admin then adds a temporary bypass, merges, and removes the bypass right away. Within one business day, a review record is posted for the merged commit and the gate is run again on it, so the change ends up with the same evidence as every other one. Every bypass appears in the monthly report, and a bypass still in place at the end of the month is reported as an exception.
One signed document a month
On the 1st of each month a change report is generated automatically, and it is due within 10 business days. It lists:
- Pull requests into the development branch that merged without an approval
- Releases, with who prepared, approved and merged each one, and the result of its staging end-to-end run
- Blocking findings that were dismissed, with the reason given for each
- BREAK-GLASS comments
- Changes to the gate’s own files and to the repository rules
- Drift from the approved baseline of repository settings
The report is written for senior management, who sign it without reading code. Each release appears with the names of the people who prepared, approved and merged it, and each dismissed finding with its reason, so the person signing reads decisions and who made them rather than diffs. The signed report goes to our compliance platform as SOC 2 evidence for the approval of changes control. An auditor starts from one signed document per month and goes to the repository for any line they want to trace back to its pull request and review record.
What we don’t know yet
The gate has been live since October 27, and so far 35 pull requests and 6 releases have gone through it. That is too few to tune the numbers with confidence. We chose 200 changed lines in 8 files as the line between a small follow-up and a large change, and we don’t yet know whether that line is in the right place for the way our pull requests grow. We also don’t know how often the cumulative rule fires in a typical month.
Both questions need data the gate doesn’t keep yet. We will start counting rejected records by the rule that rejected them, and look at those counts across the first full months before moving either threshold.
If you need our SOC 2 report for your own vendor review, the SOC 2 page explains how to request it.
