Testing exception PHPUnit

Question

So I playing around with PHPUnit and would like to get some insight to the output that PHPUnit generates when I try to test for an Exception. I am confused as to why I am getting a failed test. Here is my test:

class ConfigTest extends PHPUnit_Framework_Testcase
{
    public function testTrueIfJobGivenExists()
    {
       $conf = Config::getInstance('test1.php', new Database());
       $setup = $conf->getConfig();
       $this->assertTrue($setup);
    }

    /**
     * @expectedException   Exception
     */
    public function testExceptionIfJobGivenNotExists()
    {
        $conf = Config::getInstance('test.php', new Database());
        $setup = $conf->getConfig();
    }
}

In here I am not mocking the Database class (I have not learned how to do that yet) but basically the code looks for and entry for a job called test.php and pulls the config col for that. If the job does not exists it throws a new Exception. Here is my output:

PHPUnit 4.1.0 by Sebastian Bergmann.

.F

Time: 26 ms, Memory: 3.50Mb

There was 1 failure:

1) ConfigTest::testExceptionIfJobGivenNotExists
Failed asserting that exception of type "Exception" is thrown.

FAILURES!
Tests: 2, Assertions: 2, Failures: 1.

Here to me seems that the test is failing but looking at the PHPUnit documentation about testing exception the output looks similar. Is my test working?

---

EDIT: New test fail

Using Mockery I created my test like:

class ConfigTest extends PHPUnit_Framework_Testcase
{
    public function tearDown()
    {
        Mockery::close();
    }
    public function testTrueIfConfigForGivenJobExists()
    {
        $dbJSON = array( array(
                    'jobConfig' => '{
                        "config": {
                            "aquisition": {
                            "type": "xx",
                            "customerKey": "xxxxx",
                            "login":"xxxx",
                            "password":"xxxxx",
                            "host":"xxxxxx",
                            "account":"",
                            "email":""
                         }
                     }
                 }'
             ) );

        $database = Mockery::mock('Database');
        $database->shouldReceive('select->where->runQuery->fetch')->andReturn($dbJSON);
        $conf = Config::getInstance('getLoadsPE.php', $database);
        $setup = $conf->getConfig();
        $this->assertTrue($setup);
    }

    /**
     * @expectedException   Exception
     */
    public function testExceptionIfJobGivenNotExists()
    {
        $database = Mockery::mock('Database');
        $database->shouldReceive('select->where->runQuery->fetch')->andReturn(null);

        $conf = Config::getInstance('getLoadsPE.php', $database);
        $setup = $conf->getConfig();
        $this->assertTrue($setup);
    }
}

and I get

PHPUnit 4.1.0 by Sebastian Bergmann.

.F

Time: 39 ms, Memory: 4.75Mb

There was 1 failure:

1) ConfigTest::testExceptionIfJobGivenNotExists
Failed asserting that exception of type "Exception" is thrown.

FAILURES!
Tests: 2, Assertions: 3, Failures: 1

With this I dont know where the 3rd assertion is coming from. Also I dont get why Im getting the Fail test. If I comment the first test then the second passes. Any thoughts anyone?

FYI

This is what getConfig() looks like:

public function getConfig()
{
    if ($this->flag) {
        // Config has already been set
        return true;
    }

    $data = self::$database->select('configs', ['jobConfig'])
                            ->where('jobName', self::$jobName)
                            ->runQuery()
                            ->fetch();
    if (empty($data)) {
        throw new Exception("Config Exception: No config available for " . self::$jobName, 1);
    }
    if (count($data) > 1) {
        throw new Exception("Config Exception: More than one config for same job!!!", 1);
    }

    $arr = json_decode($data[0]['jobConfig'], true);
    // maybe threre is a better way of doing this
    if (array_key_exists('aquisition', $arr['config'])) {
        $this->aquisition = $arr['config']['aquisition'];
    }
    if (array_key_exists('ftpSetup', $arr['config'])) {
        $this->ftpSetup = $arr['config']['ftpSetup'];
    }
    if (array_key_exists('fileSetup', $arr['config'])) {
        $this->fileSetup = $arr['config']['fileSetup'];
    }
    if (array_key_exists('fileMaps', $arr['config'])) {
        $this->fileMaps = $arr['config']['fileMaps'];
    }
    if (array_key_exists('fileRows', $arr['config'])) {
        $this->fileRows = $arr['config']['fileRows'];
    }
    $this->flag = true;
    return true;
}

}

Answer 1

@expectedException Exception is not a good idea here. If an exception is thrown in your test setup (e.g. first line of you test) your test will still pass.

You could use:

//given
$conf = ...;

try {
    //when
    $conf->getConfig();

    $this->fail("YourException expected");
//then
} catch (YourException $e) {}

But it's messy and will not work with Exception (because phpunit fail also throws Exception). So you would have to use a custom exception.

You can try CatchException from ouzo goodies:

//given
$conf = ...;

//when
CatchException::when($conf)->getConfig();

//then
CatchException::assertThat()->isInstanceOf("Exception");

Answer 2

The obvious answer to your question is that there actually is a job in your database at the time the test is run, and therefore your code is not throwing the exception you expect.

This may or may not be correct, but from your code there is no way for us to know that; in fact, from your code, there is no way for the test to adequately test for that case because you don't know what's going to be in the database at the time the test is run. This is why mocking dependencies like the database is so important: you won't actually test your code the way you think you're testing it unless all the external stuff is set up exactly the way you want it to be.

I personally find it hard to wrap my head around testing and when I've been away from it for a few days (like on a Monday morning), I find it helpful to stick to a formula to get me back into the groove. I like to use the "given, when, then" pattern when writing my tests. I picked this up from Jeffrey Way's book "Laravel Testing Decoded". Here's what it looks like:

public function testSomething()
{
    // given
    // ... set up everything for the test here

    // when
    // ... execute the code that's being tested

    // then
    // ... assert
}

So in your case, it might look something like this:

/**
 * @expectedException   Exception
 */
public function testExceptionIfJobGivenNotExists()
{
    // given
    // I don't know what your database interface looks like, so I'm
    // making stuff up here
    $database = Mockery::mock("Database");
    $database->shouldReceive("query")->with("SELECT something")->andReturn(null);

    // set up the config object
    $conf = Config::getInstance('test.php', $database);

    // when
    // execute your code here
    $setup = $conf->getConfig();

    // then
    // normally you'd assert something here, but in this case, the exception
    // handler will take care of it
}

(I've assumed you're using Mockery for mocking. I've also assumed that your database class will return null when you query for a job and one doesn't exist.)

Don't sweat the mocking - it's really not that hard. All I've done here is replace your new Database() with a "fake" database object (ie. mocked). The shouldReceive method simply tells Mockery that the query method should be called with some arguments and return a value. Mocking can be much more complicated than this, but to start out, this is pretty simple.

Why is this important?

This test is supposed to tell you what happens when there's no job in the database. In your test, you have no way of knowing whether your test is actually testing that, because you really have no way of knowing what the database class is returning when you run the query. In my example, on the other hand, I've "faked" the database class and ensured that it's going to return exactly what I want it to return, every time the test is run.

Answer 3

You expect exception to be thrown but it is not thrown. That's why your test fails. Check if your code with dataset test.php should really throw an exception.