@ -462,9 +462,10 @@ Another method for making keys is _deterministic key generation_. Here you deriv
Wallets contain keys, not coins. The coins are stored on the blockchain in the form of transaction-outputs (often noted as _vout_ or _txout_). Each user has a wallet containing keys. Wallets are really keychains containing pairs of private/public keys (See <<public key>>). Users sign transactions with the keys, thereby proving they own the transaction outputs (their coins).
====
[[random wallet]]
==== Non-Deterministic (Random) Wallets
In the first implementations of bitcoin clients, wallets were simply collections of randomly generated private keys. The types of wallets are called _Type-0 Non-Deterministic Wallets_. For example, the Bitcoin Core Client pre-generates 100 random private keys when first started and generates more keys as needed, trying to use each key only once. This type of wallet is nicknamed "Just a Bunch Of Keys", or JBOK, and such wallets are being replaced with deterministic wallets because they are cumbersome to manage, backup and import. The disadvantage of random keys is that if you generate many of them you must keep copies of all of them, meaning that the wallet must be backed-up frequently. Each key must be backed-up, or the funds it controls are irrevocably lost if the wallet becomes inaccessible. This conflicts directly with the principle of avoiding address re-use, by using each bitcoin address for only one transaction. Address re-use reduces privacy by associating multiple transactions and addresses with each other. A Type-0 non-deterministic wallet is a poor choice of wallet, especially if you want to avoid address re-use as that means managing many keys, which creates the need for very frequent backups. The Bitcoin Core Client includes a wallet that is implemented as a Type-0 wallet, but the use of this wallet is actively discouraged by the Bitcoin Core developers.
In the first implementations of bitcoin clients, wallets were simply collections of randomly generated private keys. This type of wallet is called a _Type-0 Non-Deterministic Wallets_. For example, the Bitcoin Core Client pre-generates 100 random private keys when first started and generates more keys as needed, using each key only once. This type of wallet is nicknamed "Just a Bunch Of Keys", or JBOK, and such wallets are being replaced with deterministic wallets because they are cumbersome to manage, backup and import. The disadvantage of random keys is that if you generate many of them you must keep copies of all of them, meaning that the wallet must be backed-up frequently. Each key must be backed-up, or the funds it controls are irrevocably lost if the wallet becomes inaccessible. This conflicts directly with the principle of avoiding address re-use, by using each bitcoin address for only one transaction. Address re-use reduces privacy by associating multiple transactions and addresses with each other. A Type-0 non-deterministic wallet is a poor choice of wallet, especially if you want to avoid address re-use as that means managing many keys, which creates the need for frequent backups. While the Bitcoin Core Client includes a wallet that is implemented as a Type-0 wallet, the use of this wallet is actively discouraged by developers of the Bitcoin Core.
[[Type0_wallet]]
.Type-0 Non-Deterministic (Random) Wallet: A Collection of Randomly Generated Keys
Deterministic, or "seeded" wallets are wallets that contain private keys which are all derived from a common seed, through the use of a one-way hash function. The seed is a randomly generated number which is combined with other data, such as an index number or "chain code" (see <<hd_wallets>>) to derive the private keys. In a deterministic wallet, the seed is sufficient to recover all the derived keys and therefore a single backup at creation time is sufficient. The seed is also sufficient for a wallet export or import, allowing for easy migration of all the user's keys between different wallet implementations.
==== Mnemonic Code Words (DRAFT:BIP0039)
[[mnemonic_code_words]]
==== Mnemonic Code Words
Mnemonic codes are English word sequences that represent (encode) a random number used as a seed to derive a deterministic wallet. The sequence of words is sufficient to re-create the seed and from there re-create the wallet and all the derived keys. A wallet application that implements deterministic wallets with mnemonic code will show the user a sequence of 12-24 words when first creating a wallet. That sequence of words is the wallet backup and can be used to recover and re-create all the keys in the same or any compatible wallet application. Mnemonic code words make it easier for users to back up wallets, as they are easy to read and correctly transcribe, as compared to a random sequence of numbers.
@ -520,19 +522,157 @@ luggage oxygen faint major edit measure invite love trap field dilemma oblige
Deterministic wallets were developed to make it easy to derive many keys from a single "seed". The most advanced form of deterministic wallets is the _Hierarchical Deterministic Wallet_ or _HD Wallet_ defined by the BIP0032 standard. Hierarchical deterministic wallets contain keys derived in a tree structure, such that a parent key can derive a sequence of children keys, each of which can derive a sequence of grandchildren keys and so on to an infinite depth. This tree structure is illustrated below:
[[Type2_wallet]]
.Type-2 Hierarchical Deterministic Wallet: A Tree of Keys Generated from a Seed
image::images/HD_wallet.png["HD wallet"]
[TIP]
====
If you are implementing a bitcoin wallet, it should be built as an HD wallet following the BIP0032 and BIP0044 standards.
====
HD wallets offer two major advantages over random (non-deterministic) keys. First, the tree structure can be used to express additional organizational meaning, such as when a specific branch of sub-keys is used to receive incoming payments and a different branch is used to receive change from outgoing payments. Branches of keys can also be used in a corporate setting, allocating different branches to departments, subsidiaries, specific functions or accounting categories.
The second advantage of HD wallets is that users can create a sequence of public keys without having access to the corresponding private keys. This allows HD wallets to be used on an insecure server or in a receive-only capacity, issuing a different public key for each transaction. The public keys do not need to be pre-loaded or derived in advance, yet the server doesn't have the private keys that can spend the funds.
===== Creating an HD wallet from a seed
HD wallets are created from a single _root seed_, which is a 128, 256 or 512 bit random number. Everything else in the HD wallet is deterministically derived from this root seed, which makes it possible to re-create the entire HD wallet from that seed in any compatible HD wallet. This makes it easy to backup, restore, export and import HD wallets containing thousands or even millions of keys by simply transferring only the root seed. The root seed is most often represented by a _mnemonic word sequence_, as described in the previous section <<mnemonic_code_words>>, to make it easier for people to transcribe and store it.
The process of creating the master keys and master chain code for an HD wallet is shown below:
[[HDWalletFromSeed]]
.Hierarchical Deterministic Wallet: Creating master keys and chain code from a root seed
The root seed is input into the HMAC-SHA512 algorithm and the resulting hash is used to create a _master private key_ (m) and a _master chain code_. The master private key (m) then generates a corresponding master public key (M), using the normal elliptic curve multiplication process +m * G+ that we saw previously in this chapter. The chain code is used to introduce entropy in the function that creates child keys from parent keys, as we will see in the next section.
===== Deriving child private keys
Hierarchical Deterministic wallets use a _child key derivation_ (CKD) function to derive children keys from parent keys.
The child key derivation functions are based on one-way hash functions that combines:
* A parent private or public key (ECDSA uncompressed key)
* A seed called a chain code (256 bits)
* An index number (32 bits)
The chain code is used to introduce seemingly random data to the process, so that the index is not sufficient to derive other child keys. Thus, having a child key does not make it possible to find its siblings, unless you also have the chain code. The initial chain code seed (at the root of the tree) is made from random data, while subsequent chain codes are derived from each parent chain code.
These three items are combined and hashed to generate children keys, as follows:
The parent public key, chain code and the index number are combined and hashed with the HMAC-SHA512 algorithm to produce a 512 bit hash. The resulting hash is split into two halfs. The right-half 256 bits of the hash output become the chain code for the child. The left-half 256 bits of the hash and the index number are added to the parent private key to produce the child private key. In the diagram below, we see this illustrated with the index set to 0 to produce the 0'th (first by index) child of the parent.
[[CKDpriv]]
.Hierarchical Deterministic Wallet: Extending a parent private key to create a child private key
Changing the index allows us to extend the parent and create the other children in the sequence, e.g. Child 0, Child 1, Child 2 etc. Each parent key can have 2 billion children keys.
Repeating the process one level down the tree, each child can become a parent and in combination with the child chain code and an index can create it's own children, in an infinite number of generations.
===== Using derived child keys
Child private keys are indistinguishable from non-deterministic (random) keys. Because the derivation function is a one way function, the child key cannot be used to find the parent key. The child key can also not be used to find any siblings. If you have the n~th~ child, you cannot find its siblings, such as the n-1 child or the n+1 child, or any other children that are part of the sequence. Only the parent key and chain code can derive all the children. Without the child chain code, the child key cannot be used to derive any grandchildren either. You need both the child private key and the child chain code to start a new branch and derive grandchildren.
So what can the child private key be used for, on its own? It can be used to make a public key and a bitcoin address. Then, it can be used to sign transactions to spend anything paid to that address.
[TIP]
====
A child private key, the corresponding public key and the bitcoin address are all indistinguishable from keys and addresses created randomly. The fact that they are part of a sequence is not visible, outside of the HD wallet function that created them. Once created, they operate exactly as "normal" keys.
====
===== Extended keys
As we saw above, the key derivation function can be used to create children at any level of the tree, based on the three inputs: a key, a chain code and the index of the desired child. The two essential ingredients are the key and chain code and combined these are called an _extended key_. The term "extended key" could also be thought of as "extensible key" because such a key can be used to derive children.
Extended keys are stored and represented simply as the concatenation of the 256 bit key and 256 bit chain code into a 512 bit sequence. There are two types of extended keys: An extended private key is the combination of a private key and chain code and can be used to derive child private keys (and from them, child public keys). An extended public key is a public key and chain code, which can be used to create child public keys, as described in <<public_key_derivation>>.
Think of an extended key as the root of a branch in the tree structure of the HD wallet. With the root of the branch, you can derive the rest of the branch. The extended private key can create a complete branch, whereas the extended public key can only create a branch of public keys.
[TIP]
====
An extended key consists of a key and chain code and is the root of one of the branches in an HD wallet tree structure. With an extended key you can derive all the nodes in that branch.
====
[[public_key_derivation]]
===== Deriving child public keys
As mentioned above, a very useful characteristic of hierarchical deterministic wallets is the ability to derive public child keys from public parent keys, _without_ having the private keys. A child public key derived directly from the parent public key is exactly the same as if you first derived the child private key from the parent private key and then computed the child public key corresponding to it.
An extended public key can be used, therefore, to derive all of the _public_ keys (and only the public keys) in that branch of the HD wallet structure.
This shortcut can be used to create very secure public-key-only deployments where a server or application has a copy of an extended public key and no private keys whatsoever. That kind of deployment can produce an infinite number of public keys and bitcoin addresses but cannot spend any of the money sent to those addresses. Meanwhile, on another more secure server, the extended private key can derive all the corresponding private keys to sign transactions and spend the money.
One common application of this solution is to install an extended public key on a web server that serves an e-commerce application. The web server can use the public key derivation function to create a new bitcoin address for every transaction (e.g. for a customer shopping cart). The web server will not have any private keys that would be vulnerable to theft. Without HD wallets, the only way to do this is to generate thousands of bitcoin addresses on a separate secure server and then pre-load them on the e-commerce server. That approach is cumbersome and requires constant maintenance to ensure that the e-commerce server doesn't "run out" of keys.
Another common application of this solution is for cold-storage or hardware wallets. In that scenario, the extended private key can be stored on a paper wallet or hardware device (such as a Trezor hardware wallet), while the extended public key can be kept online. The user can create "receive" addresses at will, while the private keys are safely stored offline. To spend the funds, they can use the extended private key on an offline signing bitcoin client or sign transactions on the hardware wallet device (e.g. Trezor).
[[CKDpub]]
.Hierarchical Deterministic Wallet: Extending a parent public key to create a child public key
The ability to derive a branch of public keys from an extended public key is very useful, but it comes with a potential risk. Access to an extended public key does not give access to child private keys. However, because the extended public key contains the chain code, if a child private key is known, or somehow leaked, it can be used with the chain code to derive all the other child private keys. A single leaked child private key, together with a parent chain code, reveals all the private keys of all the children. Worse, the child private key together with a parent chain code can be used to deduce the parent private key.
To counter this risk, HD wallets use an alternative derivation function called _hardened derivation_, which "breaks" the relationship between parent public key and child chain code. The hardened derivation function uses the parent private key to derive the child chain code, instead of the parent public key. This creates a "firewall" in the parent/child sequence, with a chain code that cannot be used to compromise a parent or sibling private key. The hardened derivation function looks almost identical to the normal child private key derivation, except that the parent private key is used as input to the hash function, instead of the parent public key, as shown in the diagram below:
[[CKDprime]]
.Hierarchical Deterministic Wallet: Hardened derivation of a child key, omits the parent public key
When the hardened private derivation function is used, the resulting child private key and chain code are completely different from what would result from the normal derivation function. The resulting "branch" of keys can be used to produce extended public keys which are not vulnerable, since the chain code they contain cannot be exploited to reveal any private keys. Hardened derivation is therefore used to create a "gap" in the tree above the level where extended public keys are used. In simple terms, if you want to use the convenience of an extended public key to derive branches of public keys, without exposing yourself to the risk of a leaked chain code, you should derive it from a hardened parent, rather than a normal parent. As a best practice, the level-1 children of the master keys are always derived through the hardened derivation, to prevent compromise of the master keys.
===== Index numbers for normal and hardened derivation
The index number used int he derivation function is a 32-bit integer. To easily distinguish between keys derived through the normal derivation function versus keys derived through hardened derivation, this index number is split into two ranges. Index numbers between 0 and 2^31^ (0x0 to 0x80000000) are used _only_ for normal derivation. Index numbers between 2^31^ and 2^32^ (0x80000000 to 0x8FFFFFFF) are used _only_ for hardened derivation. Therefore, if the index number is less than 2^31^, that means the child is normal, whereas if the index number is above 2^31^, the child is hardened.
To make the index number easier to read and display, the index number for hardened children is displayed starting from zero, but with a prime symbol ('). The first normal child key is therefore displayed as 0, whereas the first hardened child (index 0x80000000) is displayed as 0'. In sequence then, the second hardened key would have index 0x80000001 and would be displayed as 1', and so on. When you see an HD wallet index i', that means 2^31^+i
===== HD key identifier (path)
Keys in an HD wallet are identified using a "path" naming convention, with each level of the tree separated by a slash (/) character. Private keys derived from the master private key start with "m". Public keys derived from the master public key start with "M". Each level of the tree shows the index number of the child key. The key identified by the path is the right most index with its "ancestry" read from right to left up to the master key from which it was derived. For example, identifier m/x/y/z describes the key which is the z-th child of key m/x/y, which is the y-th child of key m/x, which is the x-th child of m.
.HD wallet path indicating derived keys
|=======
| m/0 | The first (0) child private key from the master private key (m)
| m/0/0 | The first grandchild private key of the first child.
| m/0'/0 | The first normal grandchild of the first _hardened_ child (m/0')
| m/1/0 | The first grandchild private key of the second child (m/1).
| M/23/17/0/0 | The first great-great-grandchild public key of the first great-grandchild of the 18th grandchild of the 24th child.
|=======
===== Navigating the HD wallet tree structure
The HD wallet tree structure offers tremendous flexibility. Each parent extended key can have 4 billion children: 2 billion normal children and 2 billion hardened children. Each of those children can have another 4 billion children and so on. The tree can be as deep as you want, with an infinite number of generations. With all that flexibility, however, it becomes quite difficult to navigate this infinite tree. It is especially difficult to transfer HD wallets between implementations, because the possibilities for internal organization into branches and sub-branches are endless.
Two Bitcoin Improvement Proposals (BIPs) offer a solution to this complexity, by creating some proposed standards for the structure of HD wallet trees. BIP0043 proposes the use of the first hardened child index as a special identifier that signifies the "purpose" of the tree structure. Based on BIP0043, an HD wallet should use only one level-1 branch of the tree, with the index number identifying the structure and namespace of the rest of the tree by defining its purpose. For example, an HD wallet using only branch m/i'/, is intended to signify a specific purpose and that purpose is identified by index number "i".
Extending that specification, BIP0044 proposes a multi-account structure as "purpose" number +44'+ under BIP0043. All HD wallets following the BIP0044 structure are identified by the fact that they only used one branch of the tree: m/44'/.
BIP0044 specifies the structure as consisting of five pre-defined tree levels:
The first level "purpose" is always set to +44'+. The second level "coin_type" specifies the type of crypto-currency coin, allowing for multi-currency HD wallets where each currency has it's own subtree under the second level. There are three currencies defined for now: Bitcoin is m/44'/0', Bitcoin Testnet is m/44'/1' and Litecoin is m/44'/2'.
The third level of the tree is "account", which allows users to subdivide their wallets into separate logical sub-accounts, for accounting or organizational purposes. For example, an HD wallet might contain two bitcoin "accounts": m/44'/0'/0' and m/44'/0'/1'. Each account is the root of its own sub-tree.
On the fourth level "change", an HD wallet has two sub-trees, one for creating receiving addresses and one for creating change addresses. Note that whereas the previous levels used hardened derivation, this level uses normal derivation. This is to allow this level of the tree to export extended public keys for use in an non-secured environment. Usable addresses are derived by the HD wallet as children of the fourth level, making the fifth level of the tree the "address_index". For example, the third receiving address for bitcoin payments in the primary account would be M/44'/0'/0'/0/2. Here are a few more examples:
.BIP0044 HD wallet structure examples
|=======
| M/44'/0'/0'/0/2 | The third receiving public key for the primary bitcoin account
| M/44'/0'/3'/1/14 | The fifteenth change-address public key for the fourth bitcoin account
| m/44'/2'/0'/0/1 | The first private key in the Litecoin main account, for signing transactions
Andreas M. Antonopoulos
10 years ago
