Jorg's answer is correct, avoiding complex methods is the way to go. But I wanted to add a practical example of how to segregate these (while trying to minimize the change required).
for example a method that returns all possible moves of a knight in a given chess position
When you think about it, what you're really trying to test is much simpler: checking if two given fields are a possible start/end position for a knight move.
The rest of the logic (returning all valid fields on the board tht you can move to) is merely a for wrapped around the "is valid start/end field" logic.
The second test can arguably be omitted (IMHO) since it's really just testing a trivial for loop over the board, and you shouldn't test trivial code.
Martin, despite being one of the fiercest champions of unit testing and TDD, contributed to the debate with his blog entry "The pragmatics of TDD." Martin doesn't test-drive everything. Here's his list of scenarios where he doesn't use TDD:
- Getters and setters
- Member variables
- Functions that are obviously trivial
- GUIs
- Code written by trial and error
- Code written by third parties
A simple for loop wrapped around a method (which is unit tested already) fits the definition of "obviously trivial".
I'd rather generate 1000 positions at random, and check the result for each one - but how?
Just a short mention, but don't do random. Random is inherently unpredictable, and is bound to lead to inconsistent behavior. E.g. if you run your test twice using random values, and your code happens to have a bug in the white fields but not the black, you're going to get two false positives 1/4 of the time.
The more you test, the smaller that chance becomes to never stumble on an actual error, but the issue remains the same that your unit tests can come up with different results without ever having changed the codebase, which is nonsensical and bound to send you on a quixotical quest to solve a heisenbug (or mistakenly ignore a real bug).
If anything, random dependencies need to be mocked to be not-random for testing purposes, e.g. you test a dice roll with a MockedDice whose outcome you've fixed (e.g. always rolls 6).
Just to be clear, you are allowed to randomly pick any arbitrary amount of fields and then hardcode them (i.e. you pick them at random, but the runtime doesn't).
You can reuse the same testing method by abstracting and parametrizing it (e.g. void TestForField(Field field, Field[] expectedResults) and then using it in all tests where you simply call TestForField(fieldA1, new Field[] { fieldB3, fieldC2 }). This is just a simple example to point out there's nothing wrong with DRY in unit testing - up to a point of simple data changes and not behavioral changes.
However, if you're thinking of writing your tests for all 64 fields, then you need to reconsider (see next paragraph).
I would have to basically re-code the whole method again in the test, to be able to verify the result!?
This is where testing goes off the deep end and becomes a futile exercise in writing the same thing twice and hoping you wrote it right at least one of the times. This is the outer edge of the usefulness of testing, where you realize you've gone too far.
Don't worry, every developer encounters this. When you are overly diligent in testing, you end up realizing that this is exactly where your zeal has led you.
If you are able to list all fields and their expected outcomes, and are sure of its correctness (since you want to use it as the measure of correctness for your unit tests), then you've proven that the code you're testing is pointless. You can delete the code and instead just read data from the mapping you just created (your test data) without needing to calculate anything. The data is correct and complete, so there's no point to calculating anything on the fly anymore!
If you are not able (or willing) to list everything, that means your code has a purpose (it does what you're unable/unwilling to do manually). And to verify that your code does its job well, you perform some sampling, i.e. testing with a subset of possible values so you have a rudimentary understanding of whether the logic works or not. You do this by testing the basics and any easy errors you can think of. For example:
- Basic test to see if the logic does return the expected outcomes when the input field is in the middle of the board
- Test to see if the logic doesn't return "off the board" results when the input field is on the board's edge
- Test to see if the logic doesn't return any fields where there is already a piece of the same color
- Test to see if the logic does return any fields where there is already a piece of the opposing color
This catches most errors you're likely to make.
Does this catch every possible error? No. There are four things missing (that I can think of - there are probably even more):
- It's not adequately testing for pawns where normal moves and attack moves are fundamentally different
- It's not testing for en passant movement
- It's not testing for castling
- It's not testing for cases where moving the current piece would render your king check (making it impossible to move this piece).
Note: the sheer variety of situations suggests that your complex method needs to be separated into multiple behaviors, which should all be tested separately. But that's not what I want to focus on here.
But you can't be expected to know every possible error and fringe case in advance, nor can you be expected to never forget something. If you were able to do that, you'd never need to write tests to begin with because you'd be writing perfect code!
Unit tests should be refined iteratively. You write the tests that seem valid, and then you rely on them. If you find another bug in the logic, you revisit the tests and wonder why the tests didn't catch it. When you figure that out, you add another test which now catches the thing you forgot to write a test for the first time. And then you rely on your tests until you find another bug, and you repeat the process.
I've written a chess engine, and the latter two unit tests I mentioned are tests I had to add at a later stage because I hadn't yet thought about it back when I was making my basic movement logic and their unit tests. It's absolutely normal to not be able to catch everything beforehand and have to expand your tests later.
Never succumb to the arrogance of thinking you will get it right the first time. The real development skill is in correcting with mistakes gracefully, as opposed to the pipedream of never making mistakes in the first place.
Unit tests are there to make sure you don't have to solve the same thing twice. Unit tests do not prevent you having to solve an issue once.
