Added examples regarding property based testing
parent
056bf96d48
commit
94d34b2d92
@ -29,7 +29,7 @@ More details can be found in the [API documentation](https://zerobin.dssr.ch/doc
|
|||||||
For unit tests of the php code the folder `tst/` can be used. Apart from helping to implement new features correctly it helps to keep regressions at bay. By default a code coverage report is generated after phpunit is run, the one of the latest release is [publicly](https://codeclimate.com/github/PrivateBin/PrivateBin/coverage) [available](https://www.codacy.com/app/PrivateBin/PrivateBin/files).
|
For unit tests of the php code the folder `tst/` can be used. Apart from helping to implement new features correctly it helps to keep regressions at bay. By default a code coverage report is generated after phpunit is run, the one of the latest release is [publicly](https://codeclimate.com/github/PrivateBin/PrivateBin/coverage) [available](https://www.codacy.com/app/PrivateBin/PrivateBin/files).
|
||||||
|
|
||||||
In order to run these tests, you will need to install the following packages
|
In order to run these tests, you will need to install the following packages
|
||||||
and its dependencies:
|
and their dependencies:
|
||||||
* phpunit
|
* phpunit
|
||||||
* php-gd
|
* php-gd
|
||||||
* php-sqlite3
|
* php-sqlite3
|
||||||
@ -40,17 +40,26 @@ Example for Debian and Ubuntu:
|
|||||||
$ sudo apt install phpunit php-gd php-sqlite php-xdebug
|
$ sudo apt install phpunit php-gd php-sqlite php-xdebug
|
||||||
```
|
```
|
||||||
|
|
||||||
To run the tests, just change into this directory and run phpunit:
|
To run the tests, change into the `tst` directory and run phpunit:
|
||||||
```console
|
```console
|
||||||
$ cd PrivateBin/tst
|
$ cd PrivateBin/tst
|
||||||
$ phpunit
|
$ phpunit
|
||||||
```
|
```
|
||||||
|
|
||||||
Additionally there is the `ConfigurationTestGenerator`. Based on the configurations defined in its constructor, it generates the unit test file `tst/ConfigurationCombinationsTest.php`, containing all possible combinations of these configurations and test for (most) valid combinations. Some of combinations can't be tested with this method, i.e. a valid option combined with an invalid one. Other very specific test cases (i.e. to trigger multiple errors) are covered in `tst/PrivateBinTest.php`. Here is how to generate the configuration test and run it:
|
Additionally there is the `ConfigurationTestGenerator`. Based on the
|
||||||
|
configurations defined in its constructor, it generates the unit test file
|
||||||
|
`tst/ConfigurationCombinationsTest.php`, containing all possible combinations
|
||||||
|
of these configurations and tests for (most of the) valid combinations. Some of
|
||||||
|
combinations can't be tested with this method, i.e. a valid option combined with
|
||||||
|
an invalid one. Other very specific test cases (i.e. to trigger multiple errors)
|
||||||
|
are covered in `tst/PrivateBinTest.php`. Here is how to generate the
|
||||||
|
configuration test and run it:
|
||||||
|
|
||||||
$ cd PrivateBin/tst
|
```console
|
||||||
$ php ConfigurationTestGenerator.php
|
$ cd PrivateBin/tst
|
||||||
$ phpunit ConfigurationCombinationsTest.php
|
$ php ConfigurationTestGenerator.php
|
||||||
|
$ phpunit ConfigurationCombinationsTest.php
|
||||||
|
```
|
||||||
|
|
||||||
Note that it can take an hour or longer to run the several thousand tests.
|
Note that it can take an hour or longer to run the several thousand tests.
|
||||||
|
|
||||||
@ -90,6 +99,48 @@ $ cd PrivateBin/js
|
|||||||
$ istanbul cover _mocha
|
$ istanbul cover _mocha
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Property based unit testing
|
||||||
|
|
||||||
|
In the JavaScript unit tests we use the JSVerify library to leverage property
|
||||||
|
based unit testing. Instead of artificially creating specific test cases to
|
||||||
|
cover all relevant paths of the tested code (with the generated coverage reports
|
||||||
|
providing means to check the tested paths), property based testing allows us to
|
||||||
|
describe the patterns of data that are valid input.
|
||||||
|
|
||||||
|
With each run of the tests, for each `jsc.property` 100 random inputs are
|
||||||
|
generated and tested. For example we tell the test to generate random strings,
|
||||||
|
which will include empty strings, numeric strings, long strings, unicode
|
||||||
|
sequences, etc. This is great for finding corner cases that one might not think
|
||||||
|
of when explicitly writing one test case at a time.
|
||||||
|
|
||||||
|
There is another benefit, too: When an error is found, JSVerify will try to find
|
||||||
|
the smallest, still failing test case for you and print this out including the
|
||||||
|
associated random number generator (RNG) state, so you can reproduce it easily:
|
||||||
|
|
||||||
|
```console
|
||||||
|
[...]
|
||||||
|
|
||||||
|
30 passing (3s)
|
||||||
|
1 failing
|
||||||
|
|
||||||
|
1) Helper getCookie returns the requested cookie:
|
||||||
|
Error: Failed after 30 tests and 11 shrinks. rngState: 88caf85079d32e416b; Counterexample: ["{", "9", "9", "YD8%fT"]; [" ", "_|K:"];
|
||||||
|
|
||||||
|
[...]
|
||||||
|
```
|
||||||
|
|
||||||
|
Of course it may just be that you need to adjust a test case if the random
|
||||||
|
pattern generated is ambiguous. In the above example the cookie string would
|
||||||
|
contain two identical keys "9", something that may not be valid, but that our
|
||||||
|
code could encounter and needs to be able to handle.
|
||||||
|
|
||||||
|
After you adjusted the code of the library or the test you can rerun the test
|
||||||
|
with the same RNG state as follows:
|
||||||
|
|
||||||
|
```console
|
||||||
|
$ istanbul cover _mocha -- test.js --jsverifyRngState 88caf85079d32e416b
|
||||||
|
```
|
||||||
|
|
||||||
## Data Model
|
## Data Model
|
||||||
|
|
||||||
If you want to create your own data models, you might want to know how the arrays, that you have to store, look like:
|
If you want to create your own data models, you might want to know how the arrays, that you have to store, look like:
|
||||||
|
Loading…
Reference in New Issue
Block a user