28

Say you have tests for two methods of a class. The first method gathers data from another tier and puts it into some sort of storage independent of the runtime (such as a SQL table), so all of the data handled by this test is hardcoded into the test. The second method is responsible for taking data from where the first method left it and transforming it in some way (calculation, moving certain parts elsewhere, etc.).

Now this second method could have hardcoded inputs like the first one, or it could be assumed that the two tests would be run sequentially and it could pick up where the 1st test left off, taking the data that was really stored by the first test.

If you went with the second option, you would really have a good idea that the two methods play well together, however, if the 1st test failed, all tests after it would fail, taking away testing benefit of helping to isolate bugs more quickly.

If you went with the first option, each method would be isolated and tested independently, but you would never really know that they can really work together properly.

Which is the better option here? Is there some sort of alternative like having a single test for each isolated method with hardcoding, and then bigger tests that contain both methods in one?

Improve this question
1
  • 3
    I actually wish I could easily randomize the order of the unit tests every time they run. Right now they run in some unknown, albeit relatively fixed order.
    – Job
    Commented Jul 13, 2011 at 18:20

4 Answers 4

Reset to default
12

If you went with the first option, each method would be isolated and tested independently, but you would never really know that they can really work together properly.

If your methods are truly independent this shouldn't matter. Your second method should:

a) Work correctly when presented with valid data.

b) Fail sensibly and consistently when presented with invalid data.

Equally your first method should do the same. So as long as you handle the error cases they will work together properly.

If you want to test that the methods work correctly together then that's integration testing, not unit testing.

Improve this answer
31

If the tests can not run independently then they are not unit tests.

A unit test should not rely on any external state, such as contents of a database table. It should purely test one unit of code in isolation.

Tests that alter or require a certain state are valid, they may form some part of integration testing for example, and in such cases it is important to ensure that appropriate set up is done, but these would not be unit tests. In this case I would still not advise that one test requires another to be run. If you are in this case you should probably factor out the code that is required into a seperate set-up method. You may well have one test that then just calls the set-up code and verifies no exception is thrown, for instance, and then another test that activly uses the data set up in the set-up method.

Improve this answer
6
  • @Steve, so in this example, would you say: one test for method 1, one test for method 2, and one test that runs 1 and 2 in the same test?
    – user3792
    Commented Apr 1, 2011 at 13:23
  • 2
    Yes. the first two would be unit tests, the third sounds like an integration test.
    – Steve
    Commented Apr 1, 2011 at 13:25
  • if you have customer module and orders module and order can't be created without relation to customer. How will you test it independent from customer module: create customer record in database with sql (insert into customer) or use Customer.createCustomer(). And IMHO use second one is better, as you don't need impelement any logic in test, but it works only if your test on creating customers pass.
    – Dainius
    Commented Apr 1, 2011 at 13:28
  • @Dainius. In a unit test scenario you would typically use mock objects, so you would pass a mock customer to your order module. You are right in that you would not want to use sql in this case.
    – Steve
    Commented Apr 1, 2011 at 13:30
  • it seems that in any scenario where method B depends on method A, there will almost always be a method C that calls A then calls B. Since this is the case, you could test A, B, and C independently.
    – user3792
    Commented Apr 1, 2011 at 16:57
9

I'm sure it seems OK right now to have unit test B that depends on the state left by unit test B. But consider a year from now when you have a thousand unit tests. Do you really want to wait for ten minutes for your entire test suite to complete every time you need to make a change?

It depends on your development style of course, but if you want any hope of decent test driven development, in which you might run an individual test many times when developing a feature, I suggest you give each test the ability to stand alone.

Improve this answer
1
  • 1
    +1 Raskolnikov, I didn't consider the fact that this would be huge time sink later on when I have "run all tests" later on down the line.
    – user3792
    Commented Apr 1, 2011 at 13:30
3

Sounds like you're talking about test setup, which can be performed in several ways. You want a clean copy of the test data (called a fixture) for each test, so each of them should not depend on each other.

There are several frameworks that allow this type of testing, and tools like DBUnit that allow you to build up & tear down data structures quickly at the start and end of tests and test suites.

Improve this answer