7f1a5ac4c1
WIP - refactor and extend font generation for non-ascii characters WIP - add czech characters mapping between UTF8 value and index WIP - regenerate font files with czech characters WIP - shorten czech button text, it was causing SHUTDOWN for some reason WIP - support UTF8 characters in fonts.c WIP - account for translation in tests WIP - small fixes WIP - fix last test WIP - support UTF8 also in Rust font operations WIP - add a script to find non-translated english strings in micropython code WIP - add a validator script for checking missing micropython translations WIP - translate remaining altcoins and other apps in core (fido, sdcard, TT layouts, ...) WIP - generate czech glyphs for TT fonts WIP - modify gen_font.py to account for negative bearing czech characters WIP - extend translation validation scripts, move them into core/tools WIP - translate TT layouts in Rust WIP - fix tests WIP - fix inverse coloring of nonprintable glyph WIP - add build and test pipelines for Czech language WIP - merge both JSON files together WIP - run new isort WIP - unify all the translation in Rust, expose to micropython TEMP - leave en_merged.json file, so it is accessible by translators with old link WIP - fixes WIP - add french characters and translation via Google Translator WIP - skip rustfmt in mako-created files WIP - revert all the font height changes causing false-positive UI diff WIP - fixes after rebase WIP - fix broken translations WIP - revert some wording changes causing UI diff WIP - improve validation and translate scripts, translate missing strings WIP - sort all keys alphabetically WIP - remove any usage of translation in bootloader WIP - add newline at the end of JSON file WIP - fix bitcoin-only strings check WIP - fix python support check WIP - add some missing translations WIP - fix SD card device test WIP - fix pystyle WIP - fix rust unittests WIP - fix click tests WIP - flag errors in french translations WIP - add script transferring translations data into a byte blob WIP - regenerate fr.rs WIP - store and read language translations from flash WIP - storing language name in storage WIP - sending language_data in apply_settings protobuf message WIP - separate protobuf message for translations, fixes WIP - set up translations area for TT as well WIP - get rid of TREZOR_LANG env variable during build WIP - make the firmware buildable for TT WIP - add basic device tests WIP - set language for tests WIP - counting with language when writing fixtures WIP - add todos WIP - fix CI WIP - unify translations, make titles CAPITAL WIP - translate missing english WIP - skip translations messages for T1 WIP - not changing tests names for english WIP - fix flake8 WIP - no test language setting for T1 WIP - clippy lint about complex data type WIP - fix some english UI diff for TR WIP - fix cstyle WIP - minimize the usage of #[cfg(feature = "micropython")] outside translations module WIP - minimize TT's UI diff WIP - fix ruststyle WIP - fix TR build WIP - advanced Shamir text change WIP - storing the language name as the first item in the translation data WIP - modify and extend tests after storing language name WIP - modify checklist sentence WIP - add TEST_LANG into Makefile for all the emu tests WIP - default arguments WIP - reimplement default arguments remove unneeded pub from get_info function WIP - Rust handling of object attributes lookups from upy - thanks Matejcik! WIP - generate mock interface for attribute-based translations lookups WIP - change function calls to object attributes WIP - symbolic link for unix/translations.c WIP - fix and improve the reading of translations - thanks Matejcik! WIP - add support for multiple languages in removing missing tests WIP - fix multiple-accounts warning in tests WIP - fix encoding of newlines in translations WIP - fix czech tutorial text WIP - fix czech click tests WIP - do not translate wire error messages WIP - add language options to click tests as well WIP - setup czech device tests in CI WIP - setup czech click tests in CI WIP - record czech device tests for TR WIP - record czech click tests for TR WIP - record czech device tests for TT WIP - record czech click tests for TT WIP - pystyle WIP - cstyle WIP - fix Rust micropython import dependency WIP - fix czech recordings WIP - support french translations in tests WIP - shorten some french words to fix the tests WIP - fix micropython cfg compilation WIP - record french click tests for TR WIP - record french device tests for TR WIP - record french device tests for TT WIP - record french click tests for TT WIP - fix french translations - shorten them WIP - translate missing french words WIP - fix click tests WIP - add french tests into CI WIP - pystyle WIP - allow for czech/french tests in update script WIP - update czech fixtures WIP - update french fixtures WIP - ruststyle WIP - disallow MPU to run it on hardware WIP - cstyle WIP - change translations delimiter from * to \x00 WIP - change translations protobufs WIP - remove language handling from storage WIP - add header into JSON files WIP - count with header in translations blob WIP - yml style fixes WIP - fix proto gen WIP - verify version and data hash WIP - fix loading test translations feat(core): allow access to translations area in firmware [no changelog] WIP - fixes after rebase WIP - increase the TT's translations area to 3 sectors WIP - dynamically read the maximum translations size WIP - record non-english tests from CI WIP - loading font data from translations blob WIP - bump translations version WIP - include czech and french glyph data WIP - whitelist another negative-bearing glyph WIP - remove czech/french glyphs from common font files WIP - fix language tests WIP - specific fonts for specific models WIP - revert the non-ascii font hardcoding WIP - include missing BIG font into nonprintable logic WIP - minor Rust code improvements WIP - include newlines at the end of json files WIP - move glyph Rust function to librust_fonts.h WIP - add all fonts into translations file WIP - move fonts into its own dir WIP - reflect separate dir for fonts WIP - not putting translations trezorhal into bootloader WIP - write and read multiple fonts into translations data WIP - silence pyright issue/notissue WIP - delete no more used translations/*.py imports WIP - fix bootloader builds by introducing translations feature and TRANSLATIONS flag WIP - fix TT's bootloader Rust build WIP - fix tests in non-english languages WIP - not search for UTF-8 when there are no translations data WIP - add colons to strings where missing WIP - fix language loading in tests WIP - fix signmessage input flow to work in all languages WIP - create offset table for translation strings WIP - code improvements WIP - record foreign language fixtures + sync with main in english WIP - do alignment check before reading u16 data WIP - allocate blob in RAM for translations data WIP - add TODO for blob generation WIP - record non-english device tests WIP - use bytes.align_to instead of messing with pointers WIP - fixtures WIP - remove unused import WIP - add order.py WIP - add order.json WIP - take order.json into account in creating general.rs WIP - take order.json into account in generating the blob WIP - style WIP - sort the language files WIP - remove unused file WIP - code improvements WIP - add TODO for homescreen notification WIP - translate plural forms WIP - translate time intervals WIP - sign translations with dev keys, validate signatures, improve robustness WIP - improve tests for translations WIP - add `trezorctl utils sign-translations` for production signing of the blob WIP - pyright fix WIP - changing TR progress loader offset - it was colliding with title WIP - show indeterminate loader when loading translations data WIP - record new and updated language tests WIP - show the change language title/prompt in the target language WIP - sort keys WIP - add crowdin-cli into shell.nix WIP - add crowdin sync script |
||
---|---|---|
.. | ||
tests | ||
.gitignore | ||
COPYING | ||
flash_common.c | ||
flash_common.h | ||
norcow.c | ||
norcow.h | ||
README.md | ||
storage.c | ||
storage.h |
Trezor Storage
The storage
folder contains the implementation of Trezor's internal storage, which is common for both Legacy (Trezor One) and Core (Trezor T). This README also contains a detailed description of the cryptographic design.
All tests are located in the tests
subdirectory, which also includes a Python implementation to run tests against this C production version and the Python one.
Summary
The PIN is no longer stored in the flash storage. A new entry is added to the flash storage consisting of a 256-bit encrypted data encryption key (EDEK) followed by a 128-bit encrypted storage authentication key (ESAK) and a 64-bit PIN verification code (PVC). The PIN is used to decrypt the EDEK and ESAK and the PVC is used to verify that the correct PIN was used. The resulting data encryption key (DEK) is then used to encrypt/decrypt protected entries in the flash storage. We use Chacha20Poly1305 as defined in RFC 7539 to encrypt the EDEK and the protected entries. The storage authentication key (SAK) is used to authenticate the list of (APP, KEY) values for all protected entries that have been set in the storage. This prevents an attacker from erasing or adding entries to the storage.
Storage format
Entries fall into three categories:
Category | Condition | Read | Write |
---|---|---|---|
Private | APP = 0 | Never | Never |
Protected | 1 ≤ APP ≤ 127 | Only when unlocked | Only when unlocked |
Public | 128 ≤ APP ≤ 191 | Always | Only when unlocked |
Writable | 192 ≤ APP ≤ 255 | Always | Always |
The format of public and writable entries has remained unchanged, that is:
Data | KEY | APP | LEN | DATA |
---|---|---|---|---|
Length (bytes) | 1 | 1 | 2 | LEN |
Private values are used to store storage-specific information and cannot be directly accessed through the storage interface. Protected entries have the following new format:
Data | KEY | APP | LEN | IV | TAG | ENCRDATA |
---|---|---|---|---|---|---|
Length (bytes) | 1 | 1 | 2 | 12 | 16 | LEN - 28 |
The LEN value thus indicates the total length of IV, TAG and ENCRDATA.
The random salt (32 bits), EDEK (256 bits), ESAK (128 bits) and PVC (64 bits) is stored in a single entry under APP=0, KEY=2:
Data | KEY | APP | LEN | SALT | EDEK | ESAK | PVC |
---|---|---|---|---|---|---|---|
Length (bytes) | 1 | 1 | 2 | 4 | 32 | 16 | 8 |
Value | 02 | 00 | 3C 00 |
The storage authentication tag (128 bits) is stored in a single entry under APP=0, KEY=5:
Data | KEY | APP | LEN | TAG |
---|---|---|---|---|
Length (bytes) | 1 | 1 | 2 | 16 |
Value | 05 | 00 | 20 00 |
Furthermore, if any entry is overwritten, the old entry is erased, i.e., overwritten with 0. We are also using APP=0, KEY=0 as marker that the entry is erased (this was formerly used for the PIN entry, which is not needed anymore).
PIN verification and decryption of protected entries in flash storage
-
From the flash storage read the entry containing the random salt, EDEK and PVC.
-
Gather constant data from various system resources such as the ProcessorID (aka Unique device ID) and any hardware serial numbers that are available. The concatenation of this data with the random salt will be referred to as salt.
-
Prompt the user to enter the PIN and compute:
PBKDF2(PRF = HMAC-SHA256, Password = pin, Salt = salt, iterations = 10000, dkLen = 352 bits)
The first 256 bits of the output will be used as the key encryption key (KEK) and the remaining 96 bits will be used as the key encryption initialization vector (KEIV).
Note: Since two blocks of output need to be produced in PBKDF2 the total number of iterations is 20000.
-
Compute:
(dek, tag) = ChaCha20Poly1305Decrypt(kek, keiv, edek)
-
Compare the PVC read from the flash storage with the first 64 bits of the computed tag value. If there is a mismatch, then fail. Otherwise store the DEK in a global variable.
-
When a protected entry needs to be decrypted, load the IV, ENCRDATA and TAG of the entry and compute:
(data, tag) = ChaCha20Poly1305Decrypt(dek, iv, (key || app), encrdata)
where the APP and KEY of the entry is used as two bytes of associated data. Compare the TAG read from the flash storage with the computed tag value. If there is a mismatch, then fail.
Initializing the EDEK
-
When the storage is initialized, generate the 32 bit random salt and 256 bit DEK using a cryptographically secure random number generator.
-
Set a boolean value in the storage denoting that the PIN has not been set. Use an empty PIN to derive the KEK and KEIV as described above.
-
Encrypt the DEK using the derived KEK and KEIV:
(edek, tag) = ChaCha20Poly1305Encrypt(kek, keiv, dek)
-
Store the random salt, EDEK value and the first 64 bits of the tag as the PVC.
Setting a new PIN
-
If the PIN has already been set, then prompt the user to enter the old PIN value, check the PVC and compute the DEK as described above in steps 1-4.
-
Generate a new 32 bit random salt and prompt the user to enter the new PIN value. Use these values to derive the new KEK and KEIV as described above.
-
Encrypt the DEK using the new KEK and KEIV:
(edek, tag) = ChaCha20Poly1305Encrypt(kek, keiv, dek)
-
Store the new EDEK value and the first 64 bits of the tag as the new PVC. This operation should be atomic, i.e. either both values should be stored or neither. Overwrite the old values of the EDEK and PVC with zeros.
Encryption of protected entries in flash storage
Whenever the value of an entry needs to be updated, a fresh IV is generated using a cryptographically secure random number generator and the data is encrypted as (encrdata, tag) = ChaCha20Poly1305Encrypt(dek, iv, (key || app), data)
.
Storage authentication
The storage authentication key (SAK) will be used to generate a storage authentication tag (SAT) for the list of all (APP, KEY) values of protected entries (1 ≤ APP ≤ 127) that have been set in the storage. The SAT will be checked during every get operation. When a new protected entry is added to the storage or when a protected entry is deleted from the storage, the value of the SAT will be updated. The value of the SAT is defined as the first 16 bytes of
HMAC-SHA-256(SAK, ⨁_i HMAC-SHA-256(SAK, KEY_i || APP_i))
where ⨁
denotes the n-ary bitwise XOR operation and KEYi || APPi is a two-byte encoding of the value of the i-th (APP, KEY) such that 1 ≤ APP ≤ 127.
Design rationale
-
The purpose of the PBKDF2 function is to thwart brute-force attacks in case the attacker is able to circumvent the PIN entry counter mechanism but does not have full access to the contents of the flash storage of the device, e.g. fault injection attacks. For an attacker that would be able to read the flash storage and obtain the salt, the PBKDF2 with 20000 iterations and a 4- to 9-digit PIN would not pose an obstacle.
-
The reason why we use a separate data encryption key rather than using the output of PBKDF2 directly to encrypt the sensitive entries is so that when the user decides to change their PIN, only the EDEK needs to be reencrypted, but the remaining entries do not need to be updated.
-
We use ChaCha20 for encryption, because as a stream cipher it has no padding overhead and its implementation is readily available in trezor-crypto. A possible alternative to using ChaCha20Poly1305 for DEK encryption is to use AES-CTR with HMAC in an encrypt-then-MAC scheme. A possible alternative to using ChaCha20 for encryption of other data entries is to use AES-XTS (XEX-based tweaked-codebook mode with ciphertext stealing), which was designed specifically for disk-encryption. The APP || KEY value would be used as the tweak.
- Advantages of AES-XTS:
- Does not require an initialization vector.
- Ensures better diffusion than a stream cipher, which eliminates the above concerns about malleability and fault injection attacks.
- Disadvantages of AES-XTS:
- Not implemented in trezor-crypto.
- Requires two keys of length at least 128 bits.
- Advantages of AES-XTS:
-
The 64 bit PVC means that there is less than a 1 in 1019 chance that a wrong PIN will happen to have the same PVC as the correct PIN. The existence of false PINs does not pose a security weakness since a false PIN cannot be used to decrypt the protected entries.
-
Instead of using separate IVs for each entry we considered using a single IV for the entire sector. Upon sector compaction a new IV would have to be generated and the encrypted data would have to be reencrypted under the new IV. A possible issue with this approach is that compaction cannot happen without the DEK, i.e. generally data could not be written to the flash storage without knowing the PIN. This property might not always be desirable.
New measures for PIN entry counter protection
The former implementation of the PIN entry counter was vulnerable to fault injection attacks.
Under the former implementation the PIN counter storage entry consisted of 32 words initialized to 0xFFFFFFFF. The first non-zero word in this area was the current PIN failure counter. Before verifying the PIN the lowest bit with value 1 was set to 0, i.e. a value of FFFFFFFC indicated two PIN entries. Upon successful PIN entry, the word was set to 0x00000000, indicating that the next word was the PIN failure counter. Allegedly, by manipulating the voltage on the USB input an attacker could convince the device to read the PIN entry counter as 0xFFFFFFFF even if some of the bits had been set to 0.
Design goals
- Make it easy to decrement the counter by changing a 1 bit to 0.
- Make it hard to reset the counter by a fault injection, i.e. counter values should not have an overly simple binary representation like 0xFFFFFFFF.
- If possible, use two or more different methods of checking the counter value so that an attacker has to mount different fault injection attacks to succeed.
- Optimize the format for successful PIN entry.
- Minimize the number of branching operations. Avoid loops, instead utilize bitwise and arithmetic operations when processing the PIN counter data.
Proposal summary
Under the former implementation, for every unsuccessful PIN entry we discarded one bit from the counter, while for every successful PIN entry we discard an entire word. In the new implementation we optimize the counter operations for successful PIN entry.
The basic idea is that there are two binary logs stored in the flash storage, e.g.:
...0001111111111111... pin_success_log
...0000001111111111... pin_entry_log
Before every PIN verification the highest 1-bit in the pin_entry_log
is set to 0. If the verification succeeds, then the corresponding bit in the pin_success_log
is also set to 0. The example above shows the status of the logs when the last three PIN entries were not successful.
In actual fact the logs are not written to the flash storage exactly as shown above, but they are stored in a form that should protect them against fault injection attacks. Only half of the stored bits carry information, the other half acts as "guard bits". So a stored value ...001110...
could look like ...0g0gg1g11g0g...
, where g
denotes a guard bit. The positions and the values of the guard bits are determined by a guard key. The guard_key
is a randomly generated uint32 value stored as an entry in the flash memory in cleartext. The assumption behind this is that an attacker attempting to reset or decrement the PIN counter by a fault injection is not able to read the flash storage. However, the value of guard_key
also needs to be protected against fault injection, so the set of valid guard_key
values should be limited by some condition which is easy to verify, such as guard_key mod M == C
, where M
and C
a suitably chosen constants. The constants should be chosen so that the binary representation of any valid guard_key
value has Hamming weight between 8 and 24. These conditions are discussed below.
Storage format
The PIN log has APP = 0 and KEY = 1. The DATA part of the entry consists of 33 words (132 bytes, assuming 32-bit words):
guard_key
(1 word)pin_success_log
(16 words)pin_entry_log
(16 words)
Each log is stored in big-endian word order. The byte order of each word is platform dependent.
Guard key validation
The guard_key is said to be valid if the following three conditions hold true:
- Each byte of the binary representation of the
guard_key
has a balanced number of zeros and ones at the positions corresponding to the guard values (that is those bits in the mask 0xAAAAAAAA). - The
guard_key
binary representation does not contain a run of 5 (or more) zeros or ones. - The
guard_key
integer representation is congruent to 15 modulo 6311.
Key validity can be checked with this function:
int key_validity(uint32_t guard_key)
{
uint32_t count = (guard_key & 0x22222222) + ((guard_key >> 2) & 0x22222222);
count = count + (count >> 4);
uint32_t zero_runs = ~guard_key;
zero_runs = zero_runs & (zero_runs >> 2);
zero_runs = zero_runs & (zero_runs >> 1);
zero_runs = zero_runs & (zero_runs >> 1);
uint32_t one_runs = guard_key;
one_runs = one_runs & (one_runs >> 2);
one_runs = one_runs & (one_runs >> 1);
one_runs = one_runs & (one_runs >> 1);
return ((count & 0x0e0e0e0e) == 0x04040404) & (one_runs == 0) & (zero_runs == 0) & (guard_key % 6311 == 15);
}
Key generation
The guard_key
may be generated in the following way:
- Generate a random integer r in such that 0 ≤ r ≤ 680552 with uniform probability.
- Set r = r * 6311 + 15.
- If key_validity(r) is not true go back to the step 1.
Note that on average steps 1 to 3 are repeated about one hundred times.
Key expansion
The guard_key
is read from storage, its value is checked for validity and used to compute the guard_mask
(indicating the positions of the guard bits) and guard value (indicating the values of the guard bits on their actual positions):
LOW_MASK = 0x55555555
guard_mask = ((guard_key & LOW_MASK) << 1) |
((~guard_key) & LOW_MASK)
guard = (((guard_key & LOW_MASK) << 1) & guard_key) |
(((~guard_key) & LOW_MASK) & (guard_key >> 1))
Explanation:
The guard_key
contains two pieces of information. The position of the guard bits but also their corresponding values. The bitwise format of the guard_key
is vpvpvp...vp
. The bits labelled p
indicate the position of each guard bit and the bits labelled v
indicate its value.
The guard_mask
is derived from the guard_key
and has the form xyxyxy...xy
where x+y = 1 (in other words, there is exactly one 1 bit in each pair xy). First, we set the x
bits:
(guard_key & LOW_MASK) << 1
and the y
bits to its corresponding complement:
(~guard_key) & LOW_MASK
That ensures that only one 1 bit is present in each pair xy
. The guard value is equal to the bits labelled v
in the guard_key
but only at the positions indicated by the guard_mask
. The guard value is therefore equal to:
-------- x bits mask --------- & -- guard_key --
guard = (((guard_key & LOW_MASK) << 1) & guard_key) |
----- y bits mask ---- & - guard_key shifted to v bits
(((~guard_key) & LOW_MASK) & (guard_key >> 1))
Log initialization
Each log is stored as 16 consecutive words each initialized to:
guard | ~guard_mask
Removing and adding guard bits
After reading a word from the flash storage we verify the format by checking the condition:
(word & guard_mask) == guard
and then remove the guard bits as follows:
word = word & ~guard_mask
word = ((word >> 1) | word ) & LOW_MASK
word = word | (word << 1)
This operation replaces each guard bit with the value of its neighbouring bit, e.g. …0g0gg1g11g0g…
is converted to …000011111100…
Thus each non-guard bit is duplicated.
The guard bits can be added back as follows:
word = (word & ~guard_mask) | guard
Determining the number of PIN failures
Remove the guard bits from the words of the pin_entry_log
using the operations described above and verify that the result has form 0*1* by checking the condition:
word & (word + 1) == 0
Then verify that the pin_entry_log
and pin_success_log
are in sync by checking the condition:
pin_entry_log & pin_success_log == pin_entry_log
Finally, determine the current number of PIN failures by counting the number of set bits in the evaluation of the following expression:
pin_success_log xor pin_entry_log
Note that the number of set bits in a word can be counted using bitwise and arithmetic operations. For a 32-bit word the following can be used:
count = word - ((word >> 1) & 0x55555555)
count = (count & 0x33333333) + ((count >> 2) & 0x33333333)
count = (count + (count >> 4)) & 0x0F0F0F0F
count = count + (count >> 8)
count = (count + (count >> 16)) & 0x3F