unit_test_candidates

module Finance
  class Accounts
    def debit(account, amount) = …
  end
end

The problem

You’ve inherited an app with patchy tests, or you’ve decided the suite needs to catch more. There is more untested code than time to test it, so you need an order to work in.

A coverage report lists the uncovered methods sorted by file name. It gives Enrollment#withdraw the same weight as Finance::Accounts#debit, which checkout, refunds, fees and transfers all go through. If debit is wrong, everything that moves money is wrong, and the coverage report shows it as one uncovered line among the others.

Mutation testing has a similar problem. Mutant is slow, so running it evenly across the app spends most of its time on code that little else depends on.

The fix

Rank every recorded method by how much depends on it and how close a spec gets:

$ ra unit_test_candidates app/

test these first
· 165 methods
  fan-in  distance  method
      12         1  Money#cents
       7         0  Money#initialize
       6         0  Invoice#number
       6         0  Invoice#total
       6         1  Money#to_s
       5         0  Finance::Accounts#initialize
       5         0  Finance::Accounts#debit
  … 158 more
       1  ∞         Enrollment#withdraw
       1  ∞         Cancellation#call
       0  ∞         DashboardController#show

Fan-in is the number of distinct callers across your specs and app runs. Distance is the fewest calls from a spec example’s body to the method, so 0 means a spec calls it directly and ∞ means no spec reaches it. Finance::Accounts#debit has five callers and spec/finance/accounts_spec.rb calls it directly. Enrollment#withdraw has one caller in the recorded traffic, Cancellation#call, and no spec reaches it, so it goes first. Ask how_to_reach for the setup and write the spec, then run mutant on the methods at the top of the same list instead of on the whole app.

How it works

1. Every recorded call edge includes its caller’s method. A method’s fan-in is the number of distinct callers it has across your app runs and your specs.
2. Each call tree is stamped with the entry point that started it, which is a spec example or an app run. A method’s distance is the shallowest depth at which a tree started by a spec reaches it. A method that specs reach five calls deep and no shallower is barely tested, and a method no spec reaches is untested.
3. Methods are ranked by higher fan-in first. Within one fan-in, methods a spec reaches come first, nearest first, and methods no spec reaches come last.

The ranking needs a recording of the code being used. If the app has no tests, record the app running in development, staging or production.

Run it on your app

The output on this page comes from our example Rails app, and it is regenerated whenever a tool changes. These are the commands behind it. On your own app, run ra record first, then put your own class, method or file in their place.

ra unit_test_candidates app/

The untested method is app/models/enrollment.rb:47, and its one caller is in app/services/cancellation.rb.

Reading the result

• Distance measures how close a spec gets to a method. A spec that calls debit and checks nothing still counts as reaching it. The report has no information about what a test asserts.
• Fan-in counts callers. A method with one caller can still be the most important one to test, like the one place you charge a card, so treat the ranking as an order to look in.
• The report lists every recorded method. There is no filter for the ∞ rows yet, so you find the untested methods by reading the distance column.

Every tool reads the same recording. What gets recorded and how far to trust an answer are on the questions page.

Related tools