verify_mock

def unlock_next
  curriculum
    .unlocked_for(student)
    .select(&:unlocked?)
end

allow(curriculum)
  .to receive(:unlocked_for)
  .and_return([:geometry])

def unlocked_for(student)
  eligible(student)
end

The problem

A double answers for a collaborator the way you told it to, and your suite does not check that the real collaborator answers the same way. instance_double verifies that the method exists and that its arity matches, then returns whatever the spec told it to return.

So this example, from the example app’s spec/models/enrollment_spec.rb, passes:

it "unlocks the next book" do
  curriculum = instance_double(Curriculum)
  allow(curriculum).to receive(:unlocked_for).and_return([:geometry])
  enrollment = Enrollment.new(student: Student.new)
  allow(enrollment).to receive(:curriculum).and_return(curriculum)

  expect(enrollment.unlock_next).to eq([:geometry])
end

The real Curriculum#unlocked_for returns Books, and the code that consumes them, ProgressController#update and Enrollment#progress, calls #unlocked? and #title on each one. The spec passes while asserting against an Array of Symbols, which the real method does not return.

Mutation testing passes this spec too. Mutant changes the code under test, flipping a condition or dropping a line, and checks that some test fails. This spec does fail when unlock_next is mutated, so every mutant is killed and the score is perfect. Mutant measures whether your tests notice a change to the code, and it leaves the double as written, so its score is the same whether or not the double matches the real object.

The fix

Run it against the spec:

$ ra verify_mock spec/models/enrollment_spec.rb
verify_mock spec/models/enrollment_spec.rb
· 1 stub · 1 false positive · 0 unverified
✗ stub Curriculum#unlocked_for is a false positive
  spec/models/enrollment_spec.rb:9
  your stub returns [:geometry] Array<Symbol>
  the real one returns an Array whose elements receive #unlocked?, #title
  from 9 spec examples and 42 app runs
  a Symbol was never seen answering any of them
  → return an Array of Book, as the real method did

The fix is a double that matches the recorded shape:

book = instance_double(Book, unlocked?: true, title: "Geometry")
allow(curriculum).to receive(:unlocked_for).and_return([book])

expect(enrollment.unlock_next).to eq([book])

The spec now passes against a double that returns what the real method returns. The same check on spec/billing/checkout_spec.rb finds three stubs of Billing::Gateway#charge that return :ok, where the real gateway returns a Billing::Receipt and the controller calls #id and #paid? on it.

Where it shows up

• In your editor (LSP), opening the spec publishes a diagnostic on the stub line (severity error), with the same text as the CLI finding. A quick-fix that rewrites the double is planned. See the roadmap.
• Over MCP, the verify_mock tool returns the rendered text plus the findings as structured blocks, with the stub’s file and line, so your agent can rebuild the double it has just written. A hook that runs it on every spec the agent writes is planned.
• On the command line and in CI, ra verify_mock on a spec file or directory exits non-zero on a false positive. ra check --since origin/main runs it over the spec files a change touched.

How it works

1. Each allow(...).to receive(...), expect(...).to receive(...) and instance_double(K, m: v) in the spec gives a collaborator’s class, a message and what the stub returns. A double bound to a let or a local is followed back to its class.
2. The recording has every real call to that method, from the collaborator’s own spec, from integration specs and from any app runs you recorded. Each call includes the class of what it returned.
3. The recording keeps object identity, so a returned object can be followed to the calls made on it afterwards, and for an Array the elements are followed too. Those messages are the shape the code depends on, which here is elements that receive #unlocked? and #title.
4. A stub is reported as a false positive when the class it returns, or the class of its elements, was not observed in any return from the real method.

A recording of the test suite is enough for this example, because the real Curriculum ran in its own spec.

Run it on your app

The output on this page comes from our example Rails app, and it is regenerated whenever a tool changes. These are the commands behind it. On your own app, run ra record first, then put your own class, method or file in their place.

ra verify_mock spec/models/enrollment_spec.rb
ra verify_mock spec/billing/checkout_spec.rb

The false-positive stubs are at spec/models/enrollment_spec.rb:9 and spec/billing/checkout_spec.rb:11, :19 and :26.

Reading the result

• It needs a recorded call to the real method. If the real method did not run anywhere you recorded, the stub has nothing to be compared against and is reported as unverified.
• It compares classes. It finds that :geometry is a Symbol and that no Symbol appeared among the real elements. It has no way to check whether title: "Geometry" is the right book.
• The shape of a return value comes from the calls made on it. A returned value that receives no calls has no observed shape, so a stub for it is judged on its class alone.

Every tool reads the same recording. What gets recorded and how far to trust an answer are on the questions page.

Related tools