How to customize test run execution to make your cfix test runs more effective
One of the features introduced back in cfix 1.2 was the ability to customize test execution with the command line parameters -fsr and -fsf. Using these switches can make your test runs more effective and can help simplify debugging – so it is worth spending a minute on this topic.
Assume our test run comprises two fixtures, Fixture A and Fixture B. As fixtures are always run in alphabetic order, and tests run in the order they are defined, the first test to be executed is Test 1 of Fixture A, followed by Test 2 of Fixture A, and so on. Assuming all tests of Fixture A succeed, execution then proceeds with the first test of Fixture B. The following figure illustrates this:
[caption id=“attachment_370” align=“alignnone” width=“214” ]
[/caption]
Things get interesting when one of the tests fails: Let us assume Test 2 of Fixture A leads to a failed assertion. The default behavior of cfix, which equals the behavior of JUnit and NUnit, is to report the failure and proceed with Test 3 of the same fixture:
[caption id=“attachment_371” align=“alignnone” width=“220”]
[/caption]
In many cases, this is the appropriate thing to do – letting the test run to completion and figure out the reasons for the failure afterwards. But in certain scenarios, resuming with Test 3 might not be the smartest thing to do – rather, as soon as a test fails, execution should proceed with the next fixture. This is what -fsf (“short-circuit fixture”) does:
[caption id=“attachment_375” align=“alignnone” width=“216”]
[/caption]
Once a test case has failed, all remaining tests of the same fixture are skipped.
When debugging, even resuming at the next fixture is often not desired. Rather, as soon as one test fails, you may immediately want to take a closer a look at the failure, so resuming the run is mute. -fsr (“short-circuit run”) does right that: It aborts the run immediately.
[caption id=“attachment_374” align=“alignnone” width=“218”]
[/caption]
By the way – in Visual Assert, you can quickly access these options in the Options menu in the Test Explorer window:
