testing_pyramid

def call
  gateway.charge(cart.total)
end

allow(gateway)
  .to receive(:charge)

def charge(amount)
  …
end

The problem

Coverage counts lines, so a suite can cover every line of two classes without once running them together.

Billing::Checkout’s spec stubs the payment gateway, which is correct for a unit spec about Checkout. Billing::Gateway’s spec tests the gateway on its own. Both files are at 100% coverage, and no example runs the real Checkout calling the real Gateway#charge. If the two disagree about what #charge takes or returns, every test still passes. In the example app they do disagree: the stubs return :ok and the real gateway returns a Billing::Receipt.

This happens whether you write tests first or after, and most often in a suite that uses many mocks, where every unit is isolated on purpose and no test checks that the units work together.

The fix

$ ra testing_pyramid --stubbed-only
testing_pyramid
· 3 boundaries only crossed via doubles
  Billing::Checkout#complete → Billing::Gateway#charge  5  0  58
  (no recorded caller) → Billing::Gateway.new           2  0   0
  (no recorded caller) → Account.find                   1  0   0
  models     37
  billing     9
  requests    6
  finance     4
  services    4
  jobs        2
  reporting   2

The columns are stubbed, real and app-run crossings. Five specs stub Checkout → Gateway#charge. Three are unit specs whose stub returns :ok, which the real gateway does not return, and the other two are the request and job specs, whose stubs return a receipt as the real gateway does. No spec crosses the boundary with the real gateway, and the recorded traffic crossed it 58 times. The next two rows are stubs with no recorded call to compare them against: Billing::Gateway.new is stubbed wherever a spec uses it, and Account.find in spec/models/report_spec.rb is ActiveRecord’s method, so the recording holds no real call to it. Below the boundaries is the pyramid, which is the number of examples in each spec directory.

--scaffold drafts the test that crosses a boundary, with setup taken from a recorded run:

$ ra testing_pyramid --stubbed-only --scaffold 'Billing::Checkout#complete'

# drafted by ra scaffold from the recorded setup in spec/billing/checkout_spec.rb:25
RSpec.describe Billing::Checkout do
  it "complete" do
    checkout = Billing::Checkout.new(gateway: instance_double(…), notifier: instance_double(…))
    expect(checkout.complete(build(:order))).to be_a(Billing::Receipt)
  end
end

The draft takes the classes and the call order from the recording. The instance_double(…) placeholders mark where the recorded setup used doubles. To make this test cross the boundary, replace the gateway placeholder with Billing::Gateway.new.

How it works

1. Every stub in a spec identifies an edge. A spec that stubs #charge on a Billing::Gateway double identifies Billing::Gateway#charge, and the app method that called the double in that example is the caller side of the edge.
2. Each call tree is stamped with the spec example that produced it, so every real call into Gateway#charge from app code is attributed to the spec that made it. Calls from app traffic you recorded are counted separately.
3. An edge that has stubs and no real crossing from a spec is reported as crossed through doubles alone.
4. The gaps are ordered by their app-run count, so the untested boundary your app crosses most often comes first.

Run it on your app

The output on this page comes from our example Rails app, and it is regenerated whenever a tool changes. These are the commands behind it. On your own app, run ra record first, then put your own class, method or file in their place.

ra testing_pyramid --stubbed-only
ra testing_pyramid --stubbed-only --scaffold 'Billing::Checkout#complete'

The stubs are in spec/billing/checkout_spec.rb, the real crossing is app/billing/billing/checkout.rb:11, and the traffic that crosses it comes from CheckoutsController#create and RenewalJob#perform.

Reading the result

• A boundary is reported when a spec stubs it. A collaboration that no spec stubs is not listed, even when no test covers it.
• A stub on a library method has no real call to compare against. Account.find is ActiveRecord’s, so the recording holds no real call to it and the row shows no caller.
• The layers of the pyramid are your directories. It counts examples per directory under spec/.
• The scaffold is a draft. Its setup comes from a recorded run, and the assertion is left for you to write.

Every tool reads the same recording. What gets recorded and how far to trust an answer are on the questions page.

Related tools