arno/privatebin
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Added examples regarding property based testing

El RIDO 2017-03-26 06:45:07 +02:00
parent 056bf96d48
commit 94d34b2d92
1 changed files with 57 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

63
Development.md

@ -29,7 +29,7 @@ More details can be found in the [API documentation](https://zerobin.dssr.ch/doc
For unit tests of the php code the folder `tst/` can be used. Apart from helping to implement new features correctly it helps to keep regressions at bay. By default a code coverage report is generated after phpunit is run, the one of the latest release is [publicly](https://codeclimate.com/github/PrivateBin/PrivateBin/coverage) [available](https://www.codacy.com/app/PrivateBin/PrivateBin/files).
In order to run these tests, you will need to install the following packages
and its dependencies:
and their dependencies:
* phpunit
* php-gd
* php-sqlite3
@ -40,17 +40,26 @@ Example for Debian and Ubuntu:
$ sudo apt install phpunit php-gd php-sqlite php-xdebug
```
To run the tests, just change into this directory and run phpunit:
To run the tests, change into the `tst` directory and run phpunit:
```console
$ cd PrivateBin/tst
$ phpunit
```
Additionally there is the `ConfigurationTestGenerator`. Based on the configurations defined in its constructor, it generates the unit test file `tst/ConfigurationCombinationsTest.php`, containing all possible combinations of these configurations and test for (most) valid combinations. Some of combinations can't be tested with this method, i.e. a valid option combined with an invalid one. Other very specific test cases (i.e. to trigger multiple errors) are covered in `tst/PrivateBinTest.php`. Here is how to generate the configuration test and run it:
Additionally there is the `ConfigurationTestGenerator`. Based on the
configurations defined in its constructor, it generates the unit test file
`tst/ConfigurationCombinationsTest.php`, containing all possible combinations
of these configurations and tests for (most of the) valid combinations. Some of
combinations can't be tested with this method, i.e. a valid option combined with
an invalid one. Other very specific test cases (i.e. to trigger multiple errors)
are covered in `tst/PrivateBinTest.php`. Here is how to generate the
configuration test and run it:
$ cd PrivateBin/tst
$ php ConfigurationTestGenerator.php
$ phpunit ConfigurationCombinationsTest.php
```console
$ cd PrivateBin/tst
$ php ConfigurationTestGenerator.php
$ phpunit ConfigurationCombinationsTest.php
```
Note that it can take an hour or longer to run the several thousand tests.
@ -90,6 +99,48 @@ $ cd PrivateBin/js
$ istanbul cover _mocha
```
#### Property based unit testing
In the JavaScript unit tests we use the JSVerify library to leverage property
based unit testing. Instead of artificially creating specific test cases to
cover all relevant paths of the tested code (with the generated coverage reports
providing means to check the tested paths), property based testing allows us to
describe the patterns of data that are valid input.
With each run of the tests, for each `jsc.property` 100 random inputs are
generated and tested. For example we tell the test to generate random strings,
which will include empty strings, numeric strings, long strings, unicode
sequences, etc. This is great for finding corner cases that one might not think
of when explicitly writing one test case at a time.
There is another benefit, too: When an error is found, JSVerify will try to find
the smallest, still failing test case for you and print this out including the
associated random number generator (RNG) state, so you can reproduce it easily:
```console
[...]
30 passing (3s)
1 failing
1) Helper getCookie returns the requested cookie:
Error: Failed after 30 tests and 11 shrinks. rngState: 88caf85079d32e416b; Counterexample: ["{", "9", "9", "YD8%fT"]; [" ", "_|K:"];
[...]
```
Of course it may just be that you need to adjust a test case if the random
pattern generated is ambiguous. In the above example the cookie string would
contain two identical keys "9", something that may not be valid, but that our
code could encounter and needs to be able to handle.
After you adjusted the code of the library or the test you can rerun the test
with the same RNG state as follows:
```console
$ istanbul cover _mocha -- test.js --jsverifyRngState 88caf85079d32e416b
```
## Data Model
If you want to create your own data models, you might want to know how the arrays, that you have to store, look like: