I picked up this piece of advice somewhere a year or so
ago, and it's had a big impact on my testing.
There are two situations in which I use multiple
asserts. The first is when additional information would
be noise. For example:
Assert.IsNotNull(actualValue);
Assert.AreEqual(expectedValue, actualValue);
If I put these into separate methods, both tests fail
(one assertion failure, one null reference exception).
The null reference exception doesn't give me useful
information beyond what I already know from the
IsNotNull test. In fact, it's distracting noise. In this
situation, I prefer to put the assertions in the same
test method.
The other situation is when I'm testing a sequence of
operations:
testSubject.doStep1();
Assert something about step 1
testSubject.doStep2();
Assert something about step 2
testSubject.doStep3();
Assert something about step 3
...
Even in this case, I prefer to test each step
separately. But I'll test them together if, for example,
I don't have a way to prep for doStep3() other than
executing doStep1() and doStep2(). In other words, if I
don't have enough control over the test subject that I
can directly establish the entry conditions for
doStep3().
I've started to consider that situation to be a design
smell. Rather than splitting the tests, I now try to
change the test subject to give me the direct control I
want.
But even so, something tells me that I may want to test
the sequence as above. The jury is still out on this
one.
But in the vast majority of situations, my strong
preference is to write one assertion per test.
In this case:
testSubject.doStep1();
Assert something about step 1
testSubject.doStep2();
Assert something about step 2
testSubject.doStep3();
Assert something about step 3
I'll usually do something like this:
public void testStep1()
{
testSubject.doStep1();
Assert something about step 1
}
public void testStep2()
{
testSubject.doStep1();
testSubject.doStep2();
Assert something about step 2
}
public void testStep3()
{
testSubject.doStep1();
testSubject.doStep2();
testSubject.doStep3();
Assert something about step 3
}
Not to say that there's not a design smell in there to
uncover, but this gives you the separation of assertions
you're looking for. In fact, it gives you very useful
information about the sequence depending on what fails.
I've even seen cases where testStep1 fails but the other
2 pass because something in the implementation of
doStep2 righted the wrong imposed by doStep1.
Cheers,
David
I think you've mischaracterized the problem.
In
Assert.AreEqual(3, Sum(1001,1,2);
Assert.AreEqual(3, Sum(1,1001,2);
Assert.AreEqual(3, Sum(1,2,1001);
you don't just have multiple asserts. You also have
multiple invocations on the function under test - and
that's the flaw.
This has multiple asserts, all of which are really
necessary for this particular case, but only one call on
the function under test. Doing this as three separate
test methods would just make it harder to read the tests
and figure out what was being tested. If, for example,
you left out the middlename assertion, it would be
easier to spot this here than if the assertions were
scattered. It's arguable whether separating them out
would make it easier to diagnose any problems.
Also, there are times when it's justified to have
multiple calls to the method under test, if it's
stateful. But in such a case, it would be suspicious if
the assertions verified anything other than the final
state. In other words, I agree with David's example
above, except that I would permit multiple assertions at
the end, after all calls on testSubject.doStepX were
done.
Finally, just because NUnit today stops at the first
assertion failure doesn't mean that that's cast in
stone. I see no reason that there couldn't, in theory,
be AssertAndContinue.xxx, AssertAndAbort.xxx, and even
AssertAndAbortFixture.xxx. In other words, the core
problem isn't really putting multiple asserts into a
test method; it's a limitation of the particular test
framework.
Gary
I agree with this, you might consider changing
"multiple asserts" to
"multiple tests".
One other point, in my opinion this is OK:
testSubject.doStep1();
Assert something about step 1
testSubject.doStep2();
Assert something about step 2
testSubject.doStep3();
Assert something about step 3
However, your assertion should include a *good* message
that describes what went wrong. That will direct you to
the problem spot.