check_diff

allow(gateway)
  .to receive(:charge)
  .and_return(receipt)

def charge(amount)
  Result.new(…)
end

The problem

A stub can go on returning a value that the real method no longer returns. The example app’s checkout spec, spec/billing/checkout_spec.rb:11, stubs the gateway:

gateway = instance_double(Billing::Gateway)
allow(gateway).to receive(:charge).and_return(:ok)

The real Billing::Gateway#charge returns a Billing::Receipt. The gateway’s own spec passes, and the checkout spec passes too, because a double returns what it was told to return. The build and the review both pass.

A design problem can pass review in the same way. A method that makes Reporting call back into Enrollment closes a dependency cycle, and the diff does not show it, because the cycle is spread across two files and a method call.

Running every design check on the whole app gives a reviewer a long list, most of it older than the change.

The fix

ra check --since reports what the changed methods introduced. This is the example app, checked against the commit before the smells were added:

$ ra check --since a66f55e~1
check --since a66f55e~1
· 15 findings introduced · 185 methods changed
✗ stub Billing::Gateway#charge is a false positive
  spec/billing/checkout_spec.rb:11
  your stub returns :ok Symbol
  the real one returns Billing::Receipt, which receives #id, #paid?
  from 3 spec examples and 58 app runs
  → return Billing::Receipt, as the real method did
  … two more :ok stubs in the same file
✗ stub Curriculum#unlocked_for is a false positive
  spec/models/enrollment_spec.rb:9
  your stub returns [:geometry] Array<Symbol>
  the real one returns an Array whose elements receive #unlocked?, #title
  from 9 spec examples and 42 app runs
  a Symbol was never seen answering any of them
  → return an Array of Book, as the real method did
· stub Account#find is unverified: the real method never ran
  spec/models/report_spec.rb:20
⚠ Enrollment#eligible_books
  app/models/enrollment.rb:24
  4 calls to Curriculum for every 1 to itself (10 recorded calls)
  → move it onto Curriculum   SRP
  … one more envious method and six cycles
· findings present before the change are not subtracted yet: no base recording is diffed

Update the stub to the recorded shape:

receipt = instance_double(Billing::Receipt, id: "rc_1", paid?: true)
allow(gateway).to receive(:charge).and_return(receipt)

To run it on every pull request, add it to the CI job:

- run: bundle exec rspec
- run: bundle exec ra check --since origin/${{ github.base_ref }}

A stub that is a false positive makes the command exit non-zero, which fails the job. Design findings are warnings and do not fail the build.

How it works

1. It runs git diff against the base ref and maps the hunks onto the methods whose lines they touch. It also collects the spec files the diff touched.
2. verify_mock runs on the changed spec files and on the specs whose stubs name a changed class. break_cycle and feature_envy run on the recording, and the tool keeps the findings that mention a changed method.
3. It prints the findings, and exits non-zero when any stub is a false positive.

Over MCP an agent can run the same check on its own work before it opens the PR.

A second recording is planned: a base recording kept from the main branch’s last CI run, so that findings already present on main are subtracted and the report ends with “N already on main, not shown”. Today the report shows every finding the changed methods are part of, and its last line says so. Posting the result to the pull request as a review comment is also planned.

Run it on your app

The output on this page comes from our example Rails app, and it is regenerated whenever a tool changes. These are the commands behind it. On your own app, run ra record first, then put your own class, method or file in their place.

ra check --since a66f55e~1

a66f55e is the commit that added the smells. The stubs are spec/billing/checkout_spec.rb:11 and spec/models/enrollment_spec.rb:9.

Reading the result

• There is no base recording yet, so a finding on a changed method is reported whether or not it predates the change.
• By default the build fails for stubs the change made wrong. Design findings do not fail it, and each one is a hint you can dismiss.

Every tool reads the same recording. What gets recorded and how far to trust an answer are on the questions page.

Related tools