How to unit test method that returns a collection while avoiding logic in the test

Question

I am test-driving a method that is to generate a collection of data objects. I want to verify that the properties of the objects are being set correctly. Some of the properties will be set to the same thing; others will be set to a value which is dependent on their position in the collection. The natural way to do this seems to be with a loop. However, Roy Osherove strongly recommends against using logic in unit tests (Art of Unit Testing, 178). He says:

A test that contains logic is usually testing more than one thing at a time, which isn't recommended, because the test is less readable and more fragile. But test logic also adds complexity that may contain a hidden bug.

Tests should, as a general rule, be a series of method calls with no control flows, not even try-catch, and with assert calls.

However, I can't see anything wrong with my design (how else do you generate a list of data objects, some of whose values are depended on where in the sequence they are?—can't exactly generate and test them separately). Is there something non-test-friendly with my design? Or am being too rigidly devoted to Osherove's teaching? Or is there some secret unit test magic I don't know about that circumvents this problem? (I'm writing in C#/VS2010/NUnit, but looking for language-agnostic answers if possible.)

Answer 1

TL;DR:

• Write the test
• If the test does too much, the code may do too much too.
• It may not be a unit test (but not a bad test).

---

The first thing for testing is about dogma being unhelpful. I enjoy reading The Way of Testivus which points out some issues with dogma in a lighthearted way.

Write the test that needs to be written.

If the the test needs to be written some way, write it that way. Attempting to force the test into some idealized test layout or not have it at all is not a good thing. Having a test today that tests it is better than having a "perfect" test some later day.

I will also point to the bit on the ugly test:

When the code is ugly, the tests may be ugly.

You don’t like to write ugly tests, but ugly code needs testing the most.

Don’t let ugly code stop you from writing tests, but let ugly code stop you from writing more of it.

These can be considered truisms to those who have been following for a long time... and they just become ingrained in the way of thinking and writing tests. For people that haven't been and are trying to get to that point, reminders can be helpful (I even find re-reading them helps me avoid getting locked into some dogma).

---

Do consider that when writing an ugly test, if the code it may be an indication that the code is trying to do too much too. If the code that you are testing is too complex to be properly exercised by writing a simple test, you might want to consider breaking the code into smaller parts that can be tested with the simpler tests. One should not write a unit test that does everything (it might not be a unit test then). Just as 'god objects' are bad, 'god unit tests' are bad too and should be indications to go back and look at the code again.

You should be able to exercise all the code with reasonable coverage through such simple tests. Tests that do more end to end testing that deal with larger questions ("I have this object, marshalled into xml, sent to the web service, through the rules, back out, and unmarshalled") is an excellent test - but it certainly isn't a unit test (and falls into the integration testing realm - even if it has mocked services that it calls and custom in memory databases to do the testing). It may still use the XUnit framework for testing, but the testing framework doesn't make it a unit test.

Answer 2

I'm adding a new answer because my perspective is different from when I wrote the original question and answer; it doesn't make sense to mesh them together into one.

I said in the original question

However, I can't see anything wrong with my design (how else do you generate a list of data objects, some of whose values are depended on where in the sequence they are?—can't exactly generate and test them separately)

This is where I went wrong. After doing functional programming for the last year, I now realize that I just needed a collection operation with an accumulator. Then I could write my function as a pure function that operated on one thing and use some standard library function to apply it to the collection.

So my new answer is: use functional programming techniques and you will avoid this problem entirely most of the time. You can write your functions to operate on single things and only apply them to collections of things at the last moment. But if they are pure you can test them without reference to collections.

For more complex logic, lean on property-based tests. When they do have logic, it should be less than and inverse to the logic of the code under test, and each test verifies so much more than a case-based unit test that the small amount of logic is worth it.

Above all always lean on your types. Get the strongest types you can and use them to your advantage. This will reduce the number of tests you have to write in the first place.

Answer 3

Don't try to test too many things at once. Each of the properties of each data object in the collection is too much for one test. Instead, I recommend:

1. If the collection is fixed-length, write a unit test to validate length. If it is variable length, write several tests for lengths that will characterize its behavior (e.g. 0, 1, 3, 10). Either way, do not validate properties in these tests.
2. Write a unit test to validate each of the properties. If the collection is fixed-length and short, just assert against one property of each of the elements for each test. If it is fixed-length but long, choose a representative but small sample of the elements to assert against one property each. If it is variable-length, generate a relatively short but representative collection (i.e. maybe three elements) and assert against one property of each.

Doing it this way makes the tests small enough that leaving out loops doesn't seem painful. C#/Unit example, given method under test ICollection<Foo> generateFoos(uint numberOfFoos):

[Test]
void generate_zero_foos_returns_empty_list() { ... }
void generate_one_foo_returns_list_of_one_foo() { ... }
void generate_three_foos_returns_list_of_three_foos() { ... }
void generated_foos_have_sequential_ID()
{
    var foos = generateFoos(3).GetEnumerable();
    foos.MoveNext();
    Assert.AreEqual("ID1", foos.Current.id);
    foos.MoveNext();
    Assert.AreEqual("ID2", foos.Current.id);
    foos.MoveNext();
    Assert.AreEqual("ID3", foos.Current.id);
}
void generated_foos_have_bar()
{
    var foos = generateFoos(3).GetEnumerable();
    foos.MoveNext();
    Assert.AreEqual("baz", foos.Current.bar);
    foos.MoveNext();
    Assert.AreEqual("baz", foos.Current.bar);
    foos.MoveNext();
    Assert.AreEqual("baz", foos.Current.bar);
}

If you are used to the "flat unit test" paradigm (no nested structures/logic), these tests seem pretty clean. Thus logic is avoided in the tests by identifying the original problem as trying to test too many properties at once, rather than lacking loops.