PHPUnit - Mark test as failed when two related functions fail

Question

I started using PHPUnit for unit testing recently and I structured my tests to check correct behavior when a function succeeds and fails.

For example I could test function connectToDevice() expecting that when connection succeeds it returns true, otherwise it returns false. Then I would have:

public function testConnectToDeviceSuccess()
{
    $this->assertTrue($this->myObject->connectToDevice());
}

public function testConnectToDeviceFailure()
{
    $this->assertFalse($this->myObject->connectToDevice());
}

At the moment one of those function will necessarily fail, then PHPUnit will report me 1 fail. Instead I would set it to mark test failed only when both functions fail. Therefore when 1 succeeds the other will be skipped.

Is this possible in PHPUnit maintaining those tests in two different functions?

Answer 1

I think the problem you are having is that you are missing a setup that ensures the right test outcome. That may sound a bit odd, but consider this scenario

class ConnectedResource
{
    private $connector;

    public function __construct($connector)
    {
        $this->connector = $connector;
    }

    public function connectToDevice()
    {
        try {
            $this->connector->connect();
        } catch (\Exception $e) {
            return false;
        }

        return true;
    }
}

That is really simplified, but the essential bit is that you want to test both paths: connect() success and the Exception.

What you would do in your test now is either

public function testSuccessfulConnectReturnsTrue()
{
    $connector = new Connector(
        // config to make a successful connection
    );
    $myObject = new ConnectedResource($connector);

    $this->assertTrue($myObject->connectToDevice());
}

public function testFailedConnectReturnsFalse()
{
    $connector = new Connector(
        // invalid config that will raise an exception
    );
    $myObject = new ConnectedResource($connector);

    $this->assertFalse($myObject->connectToDevice());
}

Now both tests will work, because they use different connections (one that works and one that doesn't).

The other possibility is to pass in a Mock-Connector. In other words instead of a real connector you create a dummy object where you can decide yourself what $this->connector->connect() would return. Think of it as something along the lines of "assuming I have a connector that returns true then ConnectedResource should behave like this". The code for this could look something like this:

public function testSuccessfulConnectReturnsTrue()
{
    $connector = $this->createMock(Connector::class);
    $connector->expect($this->any())
        ->method('connect')
        ->willReturn(true)
    ;
    $myObject = new ConnectedResource($connector);

    $this->assertTrue($myObject->connectToDevice());
}

and for the failure scenario it would look something like this:

public function testFailedConnectReturnsFalse()
{
    $connector = $this->createMock(Connector::class);
    $connector->expect($this->any())
        ->method('connect')
        ->willReturn($this->throwException(new Exception))
    ;
    $myObject = new ConnectedResource($connector);

    $this->assertTrue($myObject->connectToDevice());
}

Now you control everything outside the scope of the test (the connection) and only test the behavior defined inside connectToDevice(). This will make your test safe against changes in the other class, but could cause problems when for example in a future update the connect() function changes, e.g. arguments are added or changed.

The important bit is, that you have to ensure the requirements for runnning your test are fulfilled.

Answer 2

It is possible (you can make one test depend on the outcome of another test, see Test Dependencies), however consider what you would test effectively in your questions here, wrapping that in one test:

public function testConnectToDeviceBehavior()
{
    $actual = $this->myObject->connectToDevice() === $this->myObject->connectToDevice();
    $this->assertTrue($actual);
}

This will make the test fail if, and only if, connecting to the device does not twice give the same result.

You could write that with test dependencies over multiple tests, however this is most likely not what you want to test in the end.

However if, check the linked section of the Phpunit docs.

Otherwise consider what is outlined in the other answer, using dependency injection and test your object with a working and a failing connect to the device.