Difference between revisions of "Unit Tests"

From PKP Wiki
Jump to: navigation, search
(PHPUnit)
(Useful links: Added another link for php unit)
 
(3 intermediate revisions by one user not shown)
Line 4: Line 4:
  
 
Please install a [[Web Tests]] environment if you intend to create or execute automated web tests with PHPUnit.
 
Please install a [[Web Tests]] environment if you intend to create or execute automated web tests with PHPUnit.
 
== PEAR ==
 
 
You'll need PEAR installed and upgraded for the following steps to work. If you are on Linux simply install the corresponding package provided by your packaging system. To upgrade, execute:
 
<pre>
 
sudo pear upgrade PEAR
 
</pre>
 
 
If you are on Windows execute the "go-pear.bat" in the PHP installation directory and follow the on-screen instructions.
 
 
If you are on Linux you may also find some or all of the other required software pre-packaged for your distribution. Please check that before installing software manually.
 
  
 
== PHPUnit ==
 
== PHPUnit ==
Line 20: Line 9:
 
On the command line execute
 
On the command line execute
 
<pre>
 
<pre>
sudo pear channel-discover pear.phpunit.de
+
wget https://phar.phpunit.de/phpunit.phar
sudo pear install phpunit/PHPUnit-3.6.4
+
chmod +x phpunit.phar
 +
sudo mv phpunit.phar /home/YOUR-USER-FOLDER/bin/phpunit
 
</pre>
 
</pre>
  
'''Currently we need the 3.6.4 version of PHPUnit to run the existing tests'''
+
'''Currently we are using the 4.1.0 version of PHPUnit to run the existing tests'''
  
 
== PKP Unit Test Framework ==
 
== PKP Unit Test Framework ==
Line 40: Line 30:
 
= Developing Unit Tests =
 
= Developing Unit Tests =
  
This is a very terse documentation of the PHPUnit features we usually use in our unit tests. It is meant to document PKP-specific usage of PHPUnit and as a cheat sheet for the most important PHPUnit functionality. It does not replace the [http://www.phpunit.de/manual/3.4/en/index.html PHPUnit manual]. Please refer to the manual when you are using PHPUnit for the first time.
+
This is a very terse documentation of the PHPUnit features we usually use in our unit tests. It is meant to document PKP-specific usage of PHPUnit and as a cheat sheet for the most important PHPUnit functionality. It does not replace the [http://phpunit.de/manual/current PHPUnit manual]. Please refer to the manual when you are using PHPUnit for the first time.
  
 
== Nomenclature ==
 
== Nomenclature ==
Line 48: Line 38:
 
* We'll use the same folder structure inside the tests directory that we use for the classes we test.
 
* We'll use the same folder structure inside the tests directory that we use for the classes we test.
 
* Tests for PKP library classes will be in "lib/pkp/tests". Tests for PKP application classes will be in "/tests".
 
* Tests for PKP library classes will be in "lib/pkp/tests". Tests for PKP application classes will be in "/tests".
 +
 +
== What should be tested ==
 +
 +
* Generally you want to test only public interface and public object states. Also, test only methods with logic, including constructors.
  
 
== Which base class to extend ==
 
== Which base class to extend ==
Line 53: Line 47:
 
* Unit tests that require database access extend "DatabaseTestCase".
 
* Unit tests that require database access extend "DatabaseTestCase".
 
* All other test cases extend "PKPTestCase".
 
* All other test cases extend "PKPTestCase".
 +
 +
== What should be mocked ==
 +
 +
All tested code external dependencies. Doubles (or mocks, a type of double object) are meant to isolate the tested code, so we can unit test it without executing the other system parts.
 +
 +
You can also mock your class under test, to inject dependencies, for example. But be careful to not mock internal functionalities. That way you are not completely testing the class. Also, internal methods are implementation details and should be free to be changed. Mocking them would couple too much test and class under test.
 +
 +
== How should template methods of abstract classes be tested ==
 +
 +
Test [http://en.wikipedia.org/wiki/Template_method template methods] of abstract classes mocking only the abstract methods that the template method needs. You should not test the public interface of an abstract class using a subclass. The subclass might have its own test, but it should only test details that are implemented there.
 +
 +
== What to refactor to make it possible to test ==
 +
 +
* Add getters and setters if you need to inject external dependencies. Eg.: ScheduledTask uses Mail class, so it has a getMail() method that can be mocked to return the mail double created to avoid that external class dependency.
 +
* Consider that tests are also users of the application. So you might want to expose some interfaces to make it easier to test the class. This should be taken with care, you don't want to expose something that's clearly meant to be private to make the class easier to test.
 +
 +
== How to write maintainable tests ==
 +
 +
* Avoid much logic into the tests methods. Tests should be simple.
 +
* Each test should test only one thing, ideally. Sometimes is hard, but tests that needs lots of mock objects and logic are almost testing more than they should. That makes harder to read and understand the code.
 +
* Avoid tested code duplication in tests. A symptom might be the necessity of a lot of logic in your tests. Pay attention to one test template being used with a data provider to test a lot of cases. This could be a tested code logic duplication code smell.
 +
 +
=== When unit turns into integration ===
 +
 +
If your test depends on anything outside your tested class, mock it. If you don't do that, you are not unit testing, you are doing an integration test. When you allow external dependencies running their code, your test code is coupled with them, and if something changes there, it might brake your unit test. That makes tests harder to maintain.
 +
 +
Also, make sure you don't test how the external dependencies worked. You can test what parameters your testing class passed to the external dependencies, if it's needed. But you don't want to test if and how they were stored by the dependency entities.
  
 
== Test Documentation ==
 
== Test Documentation ==
Line 172: Line 193:
  
 
The @depends defines a logical dependency of one test case from another. You use the depends annotation if you want to document such dependency or if you want to use results from one test case as input to another test case. Test execution order is only dependent on the order of the methods in the test class, not on the @depends annotation.
 
The @depends defines a logical dependency of one test case from another. You use the depends annotation if you want to document such dependency or if you want to use results from one test case as input to another test case. Test execution order is only dependent on the order of the methods in the test class, not on the @depends annotation.
 +
 +
== Useful links ==
 +
 +
=== PHPUnit Docs ===
 +
 +
Both can be used in mocks:
 +
* [http://phpunit.de/manual/3.7/en/test-doubles.html#test-doubles.mock-objects.tables.matchers Matchers list]
 +
* [http://phpunit.de/manual/current/en/appendixes.assertions.html#appendixes.assertions.assertThat.tables.constraints Constraints list]
  
 
= Running Unit Tests =
 
= Running Unit Tests =
 +
 +
== Command line ==
 +
 +
You can run them by using the runAllTests tool, inside the lib/pkp/tools folder. In you application folder, execute this:
 +
<pre>
 +
.lib/pkp/tools/runAllTests.sh -cC
 +
</pre>
 +
 +
The lowercase optino 'c' will run application unit tests. The 'C' option will run library unit tests.
 +
 +
== Eclipse ==
  
 
If you have followed our tutorial to [[Configure Eclipse for PHPUnit]] then running tests will be very easy.
 
If you have followed our tutorial to [[Configure Eclipse for PHPUnit]] then running tests will be very easy.
Line 204: Line 244:
  
 
That's it. Now you're ready to develop and execute your tests. Enjoy!
 
That's it. Now you're ready to develop and execute your tests. Enjoy!
 +
 +
You can debug your test setting a break point and execute the debugger. See [[Configure Eclipse for PHPUnit]] for debug configuration instructions.
  
 
== What if my test fails with an error? ==
 
== What if my test fails with an error? ==
  
 
You've got various possibilities to debug your test. Usually the test will show a backtrace when there was an error. If this is not the case then have a look in "tests/results/error.log". Errors that occur within the PKP library will be logged there.
 
You've got various possibilities to debug your test. Usually the test will show a backtrace when there was an error. If this is not the case then have a look in "tests/results/error.log". Errors that occur within the PKP library will be logged there.
 
If this still doesn't help then try to debug your test. Set a break point and execute the debugger. See [[Configure Eclipse for PHPUnit]] for debug configuration instructions.
 
  
 
= Installing and Configuring the Build Environment =
 
= Installing and Configuring the Build Environment =

Latest revision as of 05:42, 20 June 2014

Installing and Configuring your Unit Test Environment

Web Test Environment (Selenium)

Please install a Web Tests environment if you intend to create or execute automated web tests with PHPUnit.

PHPUnit

On the command line execute

wget https://phar.phpunit.de/phpunit.phar
chmod +x phpunit.phar
sudo mv phpunit.phar /home/YOUR-USER-FOLDER/bin/phpunit

Currently we are using the 4.1.0 version of PHPUnit to run the existing tests

PKP Unit Test Framework

The PKP unit test framework can be checked out as part of the pkp library. You find it in pkp/tests.

You'll however have to use the unit test framework within an application to work properly.

Before starting your tests, please make copies of the files in "/lib/pkp/tests/config", remove the "TEMPLATES" placeholder and configure the pgsql and mysql database connection data.

Eclipse integration

Please see our Configure Eclipse for PHPUnit tutorial.

Developing Unit Tests

This is a very terse documentation of the PHPUnit features we usually use in our unit tests. It is meant to document PKP-specific usage of PHPUnit and as a cheat sheet for the most important PHPUnit functionality. It does not replace the PHPUnit manual. Please refer to the manual when you are using PHPUnit for the first time.

Nomenclature

  • Unit test classes are called after the class they are testing, e.g. the test to test the "Config" class would be "ConfigTest".
  • PHPUnit works well with our usual PKP file name nomenclature. Files are named after the test class name with a ".inc.php" postfix, e.g. the ConfigTest class would be found in the file "ConfigTest.inc.php".
  • We'll use the same folder structure inside the tests directory that we use for the classes we test.
  • Tests for PKP library classes will be in "lib/pkp/tests". Tests for PKP application classes will be in "/tests".

What should be tested

  • Generally you want to test only public interface and public object states. Also, test only methods with logic, including constructors.

Which base class to extend

  • Unit tests that require database access extend "DatabaseTestCase".
  • All other test cases extend "PKPTestCase".

What should be mocked

All tested code external dependencies. Doubles (or mocks, a type of double object) are meant to isolate the tested code, so we can unit test it without executing the other system parts.

You can also mock your class under test, to inject dependencies, for example. But be careful to not mock internal functionalities. That way you are not completely testing the class. Also, internal methods are implementation details and should be free to be changed. Mocking them would couple too much test and class under test.

How should template methods of abstract classes be tested

Test template methods of abstract classes mocking only the abstract methods that the template method needs. You should not test the public interface of an abstract class using a subclass. The subclass might have its own test, but it should only test details that are implemented there.

What to refactor to make it possible to test

  • Add getters and setters if you need to inject external dependencies. Eg.: ScheduledTask uses Mail class, so it has a getMail() method that can be mocked to return the mail double created to avoid that external class dependency.
  • Consider that tests are also users of the application. So you might want to expose some interfaces to make it easier to test the class. This should be taken with care, you don't want to expose something that's clearly meant to be private to make the class easier to test.

How to write maintainable tests

  • Avoid much logic into the tests methods. Tests should be simple.
  • Each test should test only one thing, ideally. Sometimes is hard, but tests that needs lots of mock objects and logic are almost testing more than they should. That makes harder to read and understand the code.
  • Avoid tested code duplication in tests. A symptom might be the necessity of a lot of logic in your tests. Pay attention to one test template being used with a data provider to test a lot of cases. This could be a tested code logic duplication code smell.

When unit turns into integration

If your test depends on anything outside your tested class, mock it. If you don't do that, you are not unit testing, you are doing an integration test. When you allow external dependencies running their code, your test code is coupled with them, and if something changes there, it might brake your unit test. That makes tests harder to maintain.

Also, make sure you don't test how the external dependencies worked. You can test what parameters your testing class passed to the external dependencies, if it's needed. But you don't want to test if and how they were stored by the dependency entities.

Test Documentation

  • One of the functions of a test case is documenting the class specification in a very detailed and precise way. Please make sure that your source code contains enough method and inline documentation so that an outsider can quickly understand how the class' API is to be used. This is not necessary if the test method name speaks for itself.
  • Use a @see annotation in the class comment that points to the tested class.

How to test exceptions

  • A test method that is annotated with "@expectedException ExceptionClass" will fail if it does not throw the declared exception.

What if my test class calls a DAO?

We always try to test classes with as little dependencies as possible. We want our unit tests to concentrate on one class without relying on the functionality of other classes.

An example: We do not want to rely on a functioning database infra-structure when testing application objects that draw data from the DAO application layer.

Luckily PHPUnit provides us with a very elegant and lean solution to this dependency problem. See the following test code for an example:

class SubmissionTest extends PKPTestCase {
  [...]
  public function testGetUser() {
    // Our system under test (SUT)
    $submission = new Submission();
    $submission->setUserId(5);

    // Mock the UserDAO
    $mockUserDAO =& $this->getMock('UserDAO', array('getUser'));
    DAORegistry::registerDAO('UserDAO', $mockUserDAO);
	
    // Set up the mock getUser() method
    $result = new PKPUser();
    $result->setId(5);
    $mockUserDAO->expects($this->once()) // getUser should be called once
                ->method('getUser')
                ->with($this->equalTo(5)) // expected input
                ->will($this->returnValue($result)); // mock output

    // The following assertion will pass if and only if the following
    // conditions are met:
    // * UserDAO->getUser() is called exactly once
    // * UserDAO->getUser() is called with one parameter equal to 5
    // * $submission->getUser() returns exactly the object we return from
    //   our mock object.
    self::assertEquals($result, $submission->getUser());
  }
  [...]
}

See the chapter about mock objects in the PHPUnit manual for further details.

What if my test class depends on calls to static class methods?

From a testing point of view, static method calls are evil. It is impossible to mock static method calls with the system under test being unaware of it. Static method calls are not so nice for other reasons as well: They make it difficult to "drop in" different implementations of a class at runtime. This restricts design flexibility and code maintainability. It is usually better to use the singleton pattern instead.

If you cannot avoid static method calls or if you have to rely on legacy classes then you can manually create a mock class that will be included rather than the standard PKP implementation:

  • create a file called "Mock<Classname>.inc.php" in the same directory as your unit test class.
  • in this file implement a mock version of <Classname>.
  • implement only the methods required either during PKP framework initialization or for your own test.
  • the test framework will automatically drop in the mock class instead of the original class

If you need the same mock class several times then you can implement it in the 'tests/mock' folder and drop a file with the same name into your current unit test directory that imports the central mock class from the 'tests/mock' folder.

Here is an example for a mock Locale class. The class is placed in the file MockLocale.inc.php in the '/lib/pkp/tests/mock' directory. It is being included from many different unit test directories. One example is '/lib/pkp/tests/classes/core'.

class Locale {
	/*
	 * method required during setup of
	 * the PKP application framework
	 */
	function initialize() {
		// do nothing
	}
	
	/*
	 * method required during setup of
	 * the PKP application framework
	 * @return string test locale
	 */
	function getLocale() {
		return 'de_DE';
	}
	
	/*
	 * method required during setup of
	 * the PKP application framework
	 */
	function registerLocaleFile($locale, $filename, $addToTop = false) {
		// do nothing
	}
	
	/**
	 * Mocked method
	 * @return array a test array of locales 
	 */
	function getLocalePrecedence() {
		return array('de_DE', 'en_US');
	}
}

PHPUnit annotations

Annotations have to be declared where they apply.

Please see the PHPUnit manual for details on each annotation.

Declaring Coverage

The @covers annotation indicates to the test coverage analyser the class or method to be covered by a given unit test. The @covers annotation is mandatory for all tests. The function format is always preferable to the class format. You'll very rarely write unit tests that cover a whole class.

  • method format: @covers ClassName::methodName
  • class format: @covers ClassName

Documenting Test Dependency

The @depends defines a logical dependency of one test case from another. You use the depends annotation if you want to document such dependency or if you want to use results from one test case as input to another test case. Test execution order is only dependent on the order of the methods in the test class, not on the @depends annotation.

Useful links

PHPUnit Docs

Both can be used in mocks:

Running Unit Tests

Command line

You can run them by using the runAllTests tool, inside the lib/pkp/tools folder. In you application folder, execute this:

.lib/pkp/tools/runAllTests.sh -cC

The lowercase optino 'c' will run application unit tests. The 'C' option will run library unit tests.

Eclipse

If you have followed our tutorial to Configure Eclipse for PHPUnit then running tests will be very easy.

Select the test file you want to execute. You can do this in two ways.

  • Select the editor tab that contains the test class:

PHPUnit RunTest SelectEditorTab.JPG

  • Select the file with the test class in the project explorer

PHPUnit RunTest SelectFile.JPG

Make sure that the file / edit tab is marked blue as in the screen shots. Otherwise you'll get an error when executing your test.

Next you have to decide whether you want to simply run or whether you want to debug your test.

  • To run your test select the phpunit-ext external tool configuration from the external tools' favorites drop-down:

PHPUnit RunTest RunExtTool.JPG

You can see the test's output in the console view.

  • To debug your test set breakpoints at appropriate places in the code, then select the phpunit debug configuration from your debug configurations' favorites drop-down:

PHPUnit RunTest RunDebugConfig.JPG

You can see test results in the "Debug Output" view. Sometimes the console output is not complete in the debug output. Use the external tool if you want the complete console output.

That's it. Now you're ready to develop and execute your tests. Enjoy!

You can debug your test setting a break point and execute the debugger. See Configure Eclipse for PHPUnit for debug configuration instructions.

What if my test fails with an error?

You've got various possibilities to debug your test. Usually the test will show a backtrace when there was an error. If this is not the case then have a look in "tests/results/error.log". Errors that occur within the PKP library will be logged there.

Installing and Configuring the Build Environment

Phing

On the command line execute

pear channel-discover pear.phing.info
pear install phing/phing

The section about the build environment is not complete yet. Please ignore it for the time being!