diff --git a/Development.md b/Development.md index ea27b67..a235a1d 100644 --- a/Development.md +++ b/Development.md @@ -29,7 +29,7 @@ More details can be found in the [API documentation](https://zerobin.dssr.ch/doc For unit tests of the php code the folder `tst/` can be used. Apart from helping to implement new features correctly it helps to keep regressions at bay. By default a code coverage report is generated after phpunit is run, the one of the latest release is [publicly](https://codeclimate.com/github/PrivateBin/PrivateBin/coverage) [available](https://www.codacy.com/app/PrivateBin/PrivateBin/files). In order to run these tests, you will need to install the following packages -and its dependencies: +and their dependencies: * phpunit * php-gd * php-sqlite3 @@ -40,17 +40,26 @@ Example for Debian and Ubuntu: $ sudo apt install phpunit php-gd php-sqlite php-xdebug ``` -To run the tests, just change into this directory and run phpunit: +To run the tests, change into the `tst` directory and run phpunit: ```console $ cd PrivateBin/tst $ phpunit ``` -Additionally there is the `ConfigurationTestGenerator`. Based on the configurations defined in its constructor, it generates the unit test file `tst/ConfigurationCombinationsTest.php`, containing all possible combinations of these configurations and test for (most) valid combinations. Some of combinations can't be tested with this method, i.e. a valid option combined with an invalid one. Other very specific test cases (i.e. to trigger multiple errors) are covered in `tst/PrivateBinTest.php`. Here is how to generate the configuration test and run it: +Additionally there is the `ConfigurationTestGenerator`. Based on the +configurations defined in its constructor, it generates the unit test file +`tst/ConfigurationCombinationsTest.php`, containing all possible combinations +of these configurations and tests for (most of the) valid combinations. Some of +combinations can't be tested with this method, i.e. a valid option combined with +an invalid one. Other very specific test cases (i.e. to trigger multiple errors) +are covered in `tst/PrivateBinTest.php`. Here is how to generate the +configuration test and run it: - $ cd PrivateBin/tst - $ php ConfigurationTestGenerator.php - $ phpunit ConfigurationCombinationsTest.php +```console +$ cd PrivateBin/tst +$ php ConfigurationTestGenerator.php +$ phpunit ConfigurationCombinationsTest.php +``` Note that it can take an hour or longer to run the several thousand tests. @@ -90,6 +99,48 @@ $ cd PrivateBin/js $ istanbul cover _mocha ``` +#### Property based unit testing + +In the JavaScript unit tests we use the JSVerify library to leverage property +based unit testing. Instead of artificially creating specific test cases to +cover all relevant paths of the tested code (with the generated coverage reports +providing means to check the tested paths), property based testing allows us to +describe the patterns of data that are valid input. + +With each run of the tests, for each `jsc.property` 100 random inputs are +generated and tested. For example we tell the test to generate random strings, +which will include empty strings, numeric strings, long strings, unicode +sequences, etc. This is great for finding corner cases that one might not think +of when explicitly writing one test case at a time. + +There is another benefit, too: When an error is found, JSVerify will try to find +the smallest, still failing test case for you and print this out including the +associated random number generator (RNG) state, so you can reproduce it easily: + +```console +[...] + + 30 passing (3s) + 1 failing + + 1) Helper getCookie returns the requested cookie: + Error: Failed after 30 tests and 11 shrinks. rngState: 88caf85079d32e416b; Counterexample: ["{", "9", "9", "YD8%fT"]; [" ", "_|K:"]; + +[...] +``` + +Of course it may just be that you need to adjust a test case if the random +pattern generated is ambiguous. In the above example the cookie string would +contain two identical keys "9", something that may not be valid, but that our +code could encounter and needs to be able to handle. + +After you adjusted the code of the library or the test you can rerun the test +with the same RNG state as follows: + +```console +$ istanbul cover _mocha -- test.js --jsverifyRngState 88caf85079d32e416b +``` + ## Data Model If you want to create your own data models, you might want to know how the arrays, that you have to store, look like: