Runtime Analysis in a test-first workflow.
Runtime Analysis records your test suite as it runs on your laptop, and it needs no production traffic. When you write a test first, the class under test does not exist yet. Its collaborators do, and the tests from earlier cycles already ran their real code.
On a new project, the first red starts with nothing recorded. Every later cycle adds to code that earlier cycles produced, so the collaborators you are about to stub already exist.
Record rspec on your laptop. Each call tree is tagged with the example that
produced it, so every observation is reported with its spec. Staging and production are not
involved.
When Billing::Checkout's spec mocks Billing::Gateway, the gateway's
own spec still runs the real class. Each class runs for real in its own spec, so each double
has a recording of the real object to be checked against.
Through the cycle
What it reports at red, green and refactor.
When you stub a collaborator, the stub is checked against what the real collaborator did in its own spec: the messages it received, the classes it was passed and the shape it returned.
Run only the specs that reach the code you changed. The recording already contains the example that produced each call tree, so the map from method to example comes from the same recording.
See which part of an interface each caller uses, and extract the role while few callers depend on it. Your editor puts a hint on the public methods that only their own spec calls.
At red
Every stub is checked against what the real method returned.
In a new spec, the double describes code you did not write in this cycle. The real
Billing::Gateway#charge ran in spec/billing/gateway_spec.rb, so the
stub in spec/billing/checkout_spec.rb is compared with what it did there.
8
9
10
11
12
13
14
15
16
it "charges the order total" do gateway = instance_double(Billing::Gateway) allow(gateway).to receive(:charge).and_return(:ok) checkout = Billing::Checkout.new(gateway: gateway, notifier: NullNotifier.new) expect(checkout.complete(order)).to eq(:ok) expect(gateway).to have_received(:charge).with(order.total) end
stub Billing::Gateway#charge is a false positive
your stub returns :ok Symbol
the real
one returns Billing::Receipt, which receives #id, #paid?
from 3 spec examples and 58 app runs
$ ra verify_mock spec/billing/checkout_spec.rb verify_mock spec/billing/checkout_spec.rb · 4 stubs · 3 false positives · 0 unverified ✗ stub Billing::Gateway#charge is a false positive spec/billing/checkout_spec.rb:11 your stub returns :ok Symbol the real one returns Billing::Receipt, which receives #id, #paid? from 3 spec examples and 58 app runs → return Billing::Receipt, as the real method did … 2 more
instance_double confirms that #charge exists with the right arity, and
it leaves the return value unchecked. The real method returns a Billing::Receipt,
and CheckoutsController sends #id and #paid? to the value
it gets. A spec that passes with the double should also pass with the real object. Your editor
shows the same finding as a diagnostic on the stub line. A quick-fix that rewrites the double is
planned.
Across the suite
Matching collaboration tests to contract tests.
A stub in one spec can be trusted when a test elsewhere shows the real object doing the same thing. Keeping the two in step by hand is easy to forget, and a recording of your suite contains both.
verify_mock
Stubs that disagree with the real object. Compares what each stub in a spec returns with what the real method returned in the recording, and reports the stubs that return a class the real method did not.
See more →check_diff
Check the methods a diff changed. Given a base ref, it reports the findings the changed methods introduced: a stub that no longer matches the real method, a dependency cycle, feature envy.
See more →dead_api
Find dead public API. The public methods that no caller reached in any recorded run, including those whose only caller is their own spec. Shown as a hint in your editor.
See more →how_far_it_reached
How far each spec example reached. For each spec example, the number of classes its recorded call tree touched and how deep the calls went. A unit spec that reaches fourteen classes is an integration test.
See more →Each class still runs for real in its own spec, so every double has a recording of the real object to be checked against.
verify_mock has nothing to check. The spec map, the reach per example, the
interface splits and the list of code that only its own spec calls still apply.
On a new project nothing is recorded yet. Runtime Analysis is useful from the second cycle, when there is a real collaborator to stub.