arno/trezor-firmware
1
0
Fork 0
mirror of https://github.com/trezor/trezor-firmware.git synced 2024-12-24 15:28:10 +00:00
Code Issues Releases Wiki Activity

testing: configure pytest.ini and update testing documentation

Browse Source 
as per https://github.com/trezor/python-trezor/pull/263
This commit is contained in:
matejcik 2018-05-21 18:22:06 +02:00 committed by Jan Pochyla
parent df967821e7
commit 003d77b209
3 changed files with 32 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
docs/testing.md
View File

@ -48,16 +48,37 @@ export TREZOR_PATH="udp:127.0.0.1:21324"
```
(You can find other devices with `trezorctl list`.)
Now you can run the test suite with:
Now you can run the test suite, either from `python-trezor` or `trezor-core` root directory:
```sh
pytest
```
Or from anywhere else:
```sh
pytest --pyargs trezorlib.tests.device_tests # this works from other locations
```
You can place your own tests in `trezorlib/tests/device_tests`. See test style guide (TODO).
If you only want to run a particular test, pick it with `-k <keyword>`:
If you only want to run a particular test, pick it with `-k <keyword>` or `-m <marker>`:
```sh
pytest -k nem # only runs tests that have "nem" in the name
pytest -m stellar # only runs tests marked with @pytest.mark.stellar
```
If you have tests marked `xfail` (expected to fail) but you want to run them as usual, run `pytest --runxfail`
If you want to see debugging information and protocol dumps, run with `-v`.
### 3. Submitting tests for new features
When you're happy with your tests, follow these steps:
1. Mark each of your tests with the name of your feature. E.g., `@pytest.mark.ultracoin2000`.
2. Also mark each of your tests with `@pytest.mark.xfail`. That means that the test is expected to fail.
If you want to run that test as usual, run `pytest --runxfail`
3. Submit a PR to `python-trezor`, containing these tests.
4. Edit the file `trezor-core/pytest.ini`, and add your marker to the `run_xfail` item:
```
run_xfail = lisk nem ultracoin2000
```
This will cause your PR to re-enable the `xfail`ed tests. That way we will see whether your feature actually implements what it claims.
5. Submit a PR to `trezor-core`.
6. Optionally, if you like to be extra nice: after both your PRs are accepted, submit a new one to `python-trezor` that removes the `xfail` markers, and one to `trezor-core` that removes the `run_xfail` entry.

6
pytest.ini Normal file
View File

@ -0,0 +1,6 @@
[pytest]
addopts = --pyargs trezorlib.tests.device_tests
xfail_strict = true
# List of markers that run as if not xfailed. See docs/testing.md for details
# run_xfail = stellar lisk nem

2
tests/run_tests_device_emu.sh
View File

@ -14,7 +14,7 @@ export TREZOR_PATH=udp:127.0.0.1:21324
# run tests
cd ../tests
error=0
if ! pytest -k 'not skip_t2' --pyargs trezorlib.tests.device_tests "$@"; then
if ! pytest -c ../pytest.ini "$@"; then
error=1
fi
kill $upy_pid