arno/trezor-firmware
1
0
Fork 0
mirror of https://github.com/trezor/trezor-firmware.git synced 2024-11-21 23:18:13 +00:00
Code Issues Releases Wiki Activity

feat(ci): run unit tests for TR in CI

Browse Source 
[no changelog]
This commit is contained in:
grdddj 2023-06-28 12:48:46 +02:00 committed by Vít Obrusník
parent a1a60ed320
commit dd38aff8e2
3 changed files with 89 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
ci/build.yml
View File

@ -205,6 +205,21 @@ core unix regular build:
- core/build/unix # most of it needed by test_rust - core/build/unix # most of it needed by test_rust
expire_in: 1 week expire_in: 1 week
# Non-frozen emulator build for model R.
core unix regular R build:
stage: build
<<: *gitlab_caching
needs: []
variables:
TREZOR_MODEL: "R"
script:
- $NIX_SHELL --run "poetry run make -C core build_unix"
artifacts:
name: "$CI_JOB_NAME-$CI_COMMIT_SHORT_SHA"
paths:
- core/build/unix # most of it needed by test_rust
expire_in: 1 week
core unix regular asan build: core unix regular asan build:
stage: build stage: build
<<: *gitlab_caching <<: *gitlab_caching

9
ci/test.yml
View File

@ -20,6 +20,15 @@ core unit python test:
script: script:
- $NIX_SHELL --run "poetry run make -C core test | ts -s" - $NIX_SHELL --run "poetry run make -C core test | ts -s"
# Python unit tests, checking core functionality. For model R.
core unit python R test:
stage: test
<<: *gitlab_caching
needs:
- core unix regular R build
script:
- $NIX_SHELL --run "poetry run make -C core test | ts -s"
# Rust unit tests. # Rust unit tests.
core unit rust test: core unit rust test:
stage: test stage: test

124
docs/ci/jobs.md
View File

@ -51,7 +51,7 @@ or contain `[no changelog]` in the commit message.
## BUILD stage - [build.yml](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml) ## BUILD stage - [build.yml](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml)
All builds are published as artifacts so they can be downloaded and used. All builds are published as artifacts so they can be downloaded and used.
Consists of **30 jobs** below: Consists of **31 jobs** below:
### [core fw regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L20) ### [core fw regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L20)
Build of Core into firmware. Regular version. Build of Core into firmware. Regular version.
@ -81,169 +81,175 @@ Build of Core into firmware. Bitcoin-only version.
Non-frozen emulator build. This means you still need Python files Non-frozen emulator build. This means you still need Python files
present which get interpreted. present which get interpreted.
### [core unix regular asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L208) ### [core unix regular R build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L209)
Non-frozen emulator build for model R.
### [core unix frozen regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L229) ### [core unix regular asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L223)
### [core unix frozen regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L244)
Build of Core into UNIX emulator. Something you can run on your laptop. Build of Core into UNIX emulator. Something you can run on your laptop.
Frozen version. That means you do not need any other files to run it, Frozen version. That means you do not need any other files to run it,
it is just a single binary file that you can execute directly. it is just a single binary file that you can execute directly.
### [core unix frozen btconly debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L248) ### [core unix frozen btconly debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L263)
Build of Core into UNIX emulator. Something you can run on your laptop. Build of Core into UNIX emulator. Something you can run on your laptop.
Frozen version. That means you do not need any other files to run it, Frozen version. That means you do not need any other files to run it,
it is just a single binary file that you can execute directly. it is just a single binary file that you can execute directly.
See [Emulator](../core/emulator/index.md) for more info. See [Emulator](../core/emulator/index.md) for more info.
Debug mode enabled, Bitcoin-only version. Debug mode enabled, Bitcoin-only version.
### [core unix frozen btconly debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L264) ### [core unix frozen btconly debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L279)
### [core unix frozen debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L287) ### [core unix frozen debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L302)
Build of Core into UNIX emulator. Something you can run on your laptop. Build of Core into UNIX emulator. Something you can run on your laptop.
Frozen version. That means you do not need any other files to run it, Frozen version. That means you do not need any other files to run it,
it is just a single binary file that you can execute directly. it is just a single binary file that you can execute directly.
**Are you looking for a Trezor T emulator? This is most likely it.** **Are you looking for a Trezor T emulator? This is most likely it.**
### [core unix frozen R debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L300) ### [core unix frozen R debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L315)
### [core unix frozen R debug build arm](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L315) ### [core unix frozen R debug build arm](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L330)
### [core unix frozen debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L333) ### [core unix frozen debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L348)
### [core unix frozen debug build arm](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L349) ### [core unix frozen debug build arm](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L364)
### [core macos frozen regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L371) ### [core macos frozen regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L386)
### [crypto build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L396) ### [crypto build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L411)
Build of our cryptographic library, which is then incorporated into the other builds. Build of our cryptographic library, which is then incorporated into the other builds.
### [legacy fw regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L426) ### [legacy fw regular build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L441)
### [legacy fw regular debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L442) ### [legacy fw regular debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L457)
### [legacy fw btconly build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L459) ### [legacy fw btconly build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L474)
### [legacy fw btconly debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L478) ### [legacy fw btconly debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L493)
### [legacy emu regular debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L499) ### [legacy emu regular debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L514)
Regular version (not only Bitcoin) of above. Regular version (not only Bitcoin) of above.
**Are you looking for a Trezor One emulator? This is most likely it.** **Are you looking for a Trezor One emulator? This is most likely it.**
### [legacy emu regular debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L514) ### [legacy emu regular debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L529)
### [legacy emu regular debug build arm](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L532) ### [legacy emu regular debug build arm](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L547)
### [legacy emu btconly debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L558) ### [legacy emu btconly debug build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L573)
Build of Legacy into UNIX emulator. Use keyboard arrows to emulate button presses. Build of Legacy into UNIX emulator. Use keyboard arrows to emulate button presses.
Bitcoin-only version. Bitcoin-only version.
### [legacy emu btconly debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L575) ### [legacy emu btconly debug asan build](https://github.com/trezor/trezor-firmware/blob/master/ci/build.yml#L590)
--- ---
## TEST stage - [test.yml](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml) ## TEST stage - [test.yml](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml)
All the tests run test cases on the freshly built emulators from the previous `BUILD` stage. All the tests run test cases on the freshly built emulators from the previous `BUILD` stage.
Consists of **38 jobs** below: Consists of **39 jobs** below:
### [core unit python test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L15) ### [core unit python test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L15)
Python unit tests, checking core functionality. Python unit tests, checking core functionality.
### [core unit rust test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L24) ### [core unit python R test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L24)
Python unit tests, checking core functionality. For model R.
### [core unit rust test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L33)
Rust unit tests. Rust unit tests.
### [core unit asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L33) ### [core unit asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L42)
### [core device test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L54) ### [core device test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L63)
Device tests for Core. Running device tests and also comparing screens Device tests for Core. Running device tests and also comparing screens
with the expected UI result. with the expected UI result.
See artifacts for a comprehensive report of UI. See artifacts for a comprehensive report of UI.
See [docs/tests/ui-tests](../tests/ui-tests.md) for more info. See [docs/tests/ui-tests](../tests/ui-tests.md) for more info.
### [core device R test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L85) ### [core device R test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L94)
### [core device asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L117) ### [core device asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L126)
### [core btconly device test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L136) ### [core btconly device test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L145)
Device tests excluding altcoins, only for BTC. Device tests excluding altcoins, only for BTC.
### [core btconly device asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L156) ### [core btconly device asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L165)
### [core monero test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L177) ### [core monero test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L186)
Monero tests. Monero tests.
### [core monero asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L197) ### [core monero asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L206)
### [core u2f test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L220) ### [core u2f test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L229)
Tests for U2F and HID. Tests for U2F and HID.
### [core u2f asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L239) ### [core u2f asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L248)
### [core fido2 test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L257) ### [core fido2 test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L266)
FIDO2 device tests. FIDO2 device tests.
### [core fido2 asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L280) ### [core fido2 asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L289)
### [core click test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L300) ### [core click test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L309)
Click tests - UI. Click tests - UI.
See [docs/tests/click-tests](../tests/click-tests.md) for more info. See [docs/tests/click-tests](../tests/click-tests.md) for more info.
### [core click R test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L332) ### [core click R test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L341)
Click tests. Click tests.
See [docs/tests/click-tests](../tests/click-tests.md) for more info. See [docs/tests/click-tests](../tests/click-tests.md) for more info.
### [core click asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L361) ### [core click asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L370)
### [core upgrade test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L382) ### [core upgrade test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L391)
Upgrade tests. Upgrade tests.
See [docs/tests/upgrade-tests](../tests/upgrade-tests.md) for more info. See [docs/tests/upgrade-tests](../tests/upgrade-tests.md) for more info.
### [core upgrade asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L401) ### [core upgrade asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L410)
### [core persistence test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L423) ### [core persistence test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L432)
Persistence tests - UI. Persistence tests - UI.
### [core persistence asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L453) ### [core persistence asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L462)
### [core hwi test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L471) ### [core hwi test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L480)
### [crypto test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L490) ### [crypto test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L499)
### [legacy device test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L522) ### [legacy device test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L531)
Legacy device test - UI. Legacy device test - UI.
### [legacy asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L550) ### [legacy asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L559)
### [legacy btconly test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L562) ### [legacy btconly test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L571)
### [legacy btconly asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L582) ### [legacy btconly asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L591)
### [legacy upgrade test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L597) ### [legacy upgrade test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L606)
### [legacy upgrade asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L616) ### [legacy upgrade asan test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L625)
### [legacy hwi test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L637) ### [legacy hwi test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L646)
### [python test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L657) ### [python test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L666)
### [python support test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L676) ### [python support test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L685)
### [rust test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L685) ### [rust test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L694)
### [storage test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L695) ### [storage test](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L704)
### [core unix memory profiler](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L719) ### [core unix memory profiler](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L728)
### [core firmware flash size checker](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L745) ### [core firmware flash size checker](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L754)
Finds out how much flash space we have left in the firmware build Finds out how much flash space we have left in the firmware build
Fails if the free space is less than certain threshold Fails if the free space is less than certain threshold
### [core firmware flash size compare master](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L758) ### [core firmware flash size compare master](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L767)
Compares the current flash space with the situation in the current master Compares the current flash space with the situation in the current master
Fails if the new binary is significantly larger than the master one Fails if the new binary is significantly larger than the master one
(the threshold is defined in the script, currently 5kb). (the threshold is defined in the script, currently 5kb).
Allowing fir failure, not to prevent the merge. Allowing fir failure, not to prevent the merge.
Also generates a report with the current situation Also generates a report with the current situation
### [connect test core](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L773) ### [connect test core](https://github.com/trezor/trezor-firmware/blob/master/ci/test.yml#L782)
--- ---
## TEST-NONENGLISH stage - [test-nonenglish.yml](https://github.com/trezor/trezor-firmware/blob/master/ci/test-nonenglish.yml) ## TEST-NONENGLISH stage - [test-nonenglish.yml](https://github.com/trezor/trezor-firmware/blob/master/ci/test-nonenglish.yml)