Merging Third Edition from O'Reilly to Github

The third edition of Mastering Bitcoin, written by David Harding for O'Reilly Media was published in December 2023. It is now also available under a CC-BY-NC-ND license, here.

.gitattributes

@@ -1 +0,0 @@
-*.asciidoc linguist-detectable

.gitignore

@@ -5,3 +5,6 @@ code/python-env
 .debris
 _build/
 dist/
+_build/
+build/
+book.html

LICENSE

@@ -13,4 +13,4 @@ Different editions of this book are covered by different licenses. Contributions
 
 These "Free Culture" licenses were approved by the publisher O'Reilly Media (http://oreilly.com). O'Reilly Media is not just the world's best publisher of technical books but also a strong supporter of an open culture and the sharing of knowledge. 
 
-Thank you O'Reilly!
+Thank you O'Reilly!

README.md

@@ -14,20 +14,11 @@ If you know how to make a pull request to contribute a fix, please write the cor
 
 To read this book, see [book.asciidoc](https://github.com/bitcoinbook/bitcoinbook/blob/develop/book.asciidoc). Click on each of the chapters to read in your browser. Other parties may choose to release PDFs of the book online, for those editions where a PDF derivative is licensed. 
 
-## Chapters
-
-+ Chapter 1: '[Introduction](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch01.asciidoc)'
-+ Chapter 2: '[How Bitcoin Works](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch02.asciidoc)'
-+ Chapter 3: '[Bitcoin Core: The Reference Implementation](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch03.asciidoc)'
-+ Chapter 4: '[Keys, Addresses](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch04.asciidoc)'
-+ Chapter 5: '[Wallets](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch05.asciidoc)'
-+ Chapter 6: '[Transactions](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch06.asciidoc)'
-+ Chapter 7: '[Advanced Transactions and Scripting](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch07.asciidoc)'
-+ Chapter 8: '[The Bitcoin Network](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch08.asciidoc)'
-+ Chapter 9: '[The Blockchain](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch09.asciidoc)'
-+ Chapter 10: '[Mining and Consensus](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch10.asciidoc)'
-+ Chapter 11: '[Bitcoin Security](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch11.asciidoc)'
-+ Chapter 12: '[Blockchain Applications](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch12.asciidoc)'
+The 2nd edition of "Mastering Bitcoin" is available under a CC-BY-NC-ND license, not a CC-BY-SA license.
+
+It is deliberately not available as a PDF. Why? Because a PDF is a "derivative" product, which is what the ND prohibits. That's because the publisher (O'Reilly Media) is a for-profit publisher who puts considerable resources behind distributing the book. The book will eventually (within a year of publication) be released under a CC-BY-SA license, at which point the PDF format and translations will be allowed. Until then, making PDF copies violates the license and interferes with the publisher's (and the author's) ability to make a living. Furthermore, if you make it so the publisher can't recoup their investment, they will delay the release into CC-BY-SA.
+
+Please don't create or distribute PDFs until the license is changed to CC-BY-SA. It is rare for a publisher to even agree to a CC-BY-NC-ND license. Don't make it harder for free culture by violating even that, already generous, license. 
 
 # Published
 
@@ -47,9 +38,9 @@ The book's source code, found in this repository, is kept synchronized with the
 
 The tags [Edition1Print1](https://github.com/bitcoinbook/bitcoinbook/releases/tag/Edition1Print1), [Edition1Print2](https://github.com/bitcoinbook/bitcoinbook/releases/tag/Edition1Print2) correspond to the two existing prints of Mastering Bitcoin (First Edition) as published by O'Reilly Media.
 
-<a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">Mastering Bitcoin - First Edition</span> by <a xmlns:cc="http://creativecommons.org/ns#" href="https://antonopoulos.com/" property="cc:attributionName" rel="cc:attributionURL">Andreas M. Antonopoulos LLC</a> is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.
+<a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="https://purl.org/dc/terms/" href="https://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">Mastering Bitcoin - First Edition</span> by <a xmlns:cc="https://creativecommons.org/ns#" href="https://antonopoulos.com/" property="cc:attributionName" rel="cc:attributionURL">Andreas M. Antonopoulos LLC</a> is licensed under a <a rel="license" href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.
 
-This "Free Culture" compliant license was approved by my publisher O'Reilly Media (http://oreilly.com), who understands the value of open source. O'Reilly Media is not just the world's best publisher of technical books, but is also a strong supporter of this open culture and the sharing of knowledge.
+This "Free Culture" compliant license was approved by my publisher O'Reilly Media (https://oreilly.com), who understands the value of open source. O'Reilly Media is not just the world's best publisher of technical books, but is also a strong supporter of this open culture and the sharing of knowledge.
 
 Thank you O'Reilly Media!
 
@@ -57,8 +48,10 @@ Thank you O'Reilly Media!
 
 The tags, [second_edition_print_1](https://github.com/bitcoinbook/bitcoinbook/releases/tag/second_edition_print_1)  [second_edition_print2](https://github.com/bitcoinbook/bitcoinbook/releases/tag/second_edition_print2),  [second_edition_print3](https://github.com/bitcoinbook/bitcoinbook/releases/tag/second_edition_print3), correspond to the first (June 8th, 2017), second (July 20th, 2017) and third (March  23rd, 2018) print of Mastering Bitcoin (Second Edition), as published by O'Reilly Media.
 
-<a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">Mastering Bitcoin - Second Edition</span> by <a xmlns:cc="http://creativecommons.org/ns#" href="https://antonopoulos.com/" property="cc:attributionName" rel="cc:attributionURL">Andreas M. Antonopoulos LLC</a> is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.
+<a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-nd/4.0/88x31.png" /></a><br /><span xmlns:dct="https://purl.org/dc/terms/" property="dct:title">Mastering Bitcoin - Second Edition</span> by <a xmlns:cc="https://creativecommons.org/ns#" href="https://antonopoulos.com/" property="cc:attributionName" rel="cc:attributionURL">Andreas M. Antonopoulos LLC</a> is licensed under a <a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/">Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License</a>.
+
+It is expected that the second edition will be released under a CC-BY-SA license within a year of publication.
 
 # Translations
 
-If you are interested in translating this book, please join our team of volunteers at: https://www.transifex.com/aantonop/mastering-bitcoin
+If you are interested in translating this book, please join our team of volunteers at: https://www.transifex.com/bitcoinbook/mastering-bitcoin/

appa_whitepaper.adoc

@@ -1,10 +1,11 @@
 [[satoshi_whitepaper]]
 [appendix]
-== The Bitcoin Whitepaper by Satoshi Nakamoto
+== The Bitcoin Whitepaper [.keep-together]#by Satoshi Nakamoto#
+
 
 [NOTE]
 ====
-((("whitepaper", id="whitethirteen")))((("bitcoin whitepaper", id="BCwhitethirteen")))((("Nakamoto, Satoshi", id="nakamatothirteen")))This is the original whitepaper, reproduced in its entirety exactly as it was published by Satoshi Nakamoto in October 2008.
+This is the((("Bitcoin whitepaper", "original version", id="bitcoin-whitepaper-original")))((("whitepaper (Bitcoin)", "original version", id="whitepaper-original")))((("Nakamoto, Satoshi"))) original whitepaper, reproduced in its entirety exactly as it was published by Satoshi Nakamoto in October 2008.
 ====
 
 === Bitcoin - A Peer-to-Peer Electronic Cash System
@@ -23,23 +24,23 @@ Commerce on the Internet has come to rely almost exclusively on financial instit
 What is needed is an electronic payment system based on cryptographic proof instead of trust, allowing any two willing parties to transact directly with each other without the need for a trusted third party. Transactions that are computationally impractical to reverse would protect sellers from fraud, and routine escrow mechanisms could easily be implemented to protect buyers. In this paper, we propose a solution to the double-spending problem using a peer-to-peer distributed timestamp server to generate computational proof of the chronological order of transactions. The system is secure as long as honest nodes collectively control more CPU power than any cooperating group of attacker nodes.
 
 ==== Transactions
-We define an electronic coin as a chain of digital signatures. Each owner transfers the coin to the next by digitally signing a hash of the previous transaction and the public key of the next owner and adding these to the end of the coin. A payee can verify the signatures to verify the chain of ownership.
+We ((("transactions", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper", id="transaction-whitepaper")))define an electronic coin as a chain of digital signatures. Each owner transfers the coin to the next by digitally signing a hash of the previous transaction and the public key of the next owner and adding these to the end of the coin. A payee can verify the signatures to verify the chain of ownership.
 
-image::images/mbc2_abin01.png["Transactions"]
+image::images/mbc3_0401.png["Transactions"]
 
 The problem of course is the payee can't verify that one of the owners did not double-spend the coin. A common solution is to introduce a trusted central authority, or mint, that checks every transaction for double spending. After each transaction, the coin must be returned to the mint to issue a new coin, and only coins issued directly from the mint are trusted not to be double-spent. The problem with this solution is that the fate of the entire money system depends on the company running the mint, with every transaction having to go through them, just like a bank.
 
-We need a way for the payee to know that the previous owners did not sign any earlier transactions.  For our purposes, the earliest transaction is the one that counts, so we don't care about later attempts to double-spend. The only way to confirm the absence of a transaction is to be aware of all transactions. In the mint based model, the mint was aware of all transactions and decided which arrived first. To accomplish this without a trusted party, transactions must be publicly announced [1], and we need a system for participants to agree on a single history of the order in which they were received. The payee needs proof that at the time of each transaction, the majority of nodes agreed it was the first received.
+We need a way for the payee to know that the previous owners did not sign any earlier transactions.  For our purposes, the earliest transaction is the one that counts, so we don't care about later attempts to double-spend. The only way to confirm the absence of a transaction is to be aware of all transactions. In the mint based model, the mint was aware of all transactions and decided which arrived first. To accomplish this without a trusted party, transactions must be publicly announced [1], and we need a system for participants to agree on a single history of the order in which they were received. The payee needs proof that at the time of each transaction, the majority of nodes agreed it was the ((("transactions", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper", startref="transaction-whitepaper")))first received.
 
 ==== Timestamp Server
-The solution we propose begins with a timestamp server. A timestamp server works by taking a hash of a block of items to be timestamped and widely publishing the hash, such as in a newspaper or Usenet post [2-5]. The timestamp proves that the data must have existed at the time, obviously, in order to get into the hash. Each timestamp includes the previous timestamp in its hash, forming a chain, with each additional timestamp reinforcing the ones before it.
+The ((("timestamp servers")))solution we propose begins with a timestamp server. A timestamp server works by taking a hash of a block of items to be timestamped and widely publishing the hash, such as in a newspaper or Usenet post [2-5]. The timestamp proves that the data must have existed at the time, obviously, in order to get into the hash. Each timestamp includes the previous timestamp in its hash, forming a chain, with each additional timestamp reinforcing the ones before it.
 
-image::images/mbc2_abin02.png["timestamp server"]
+image::images/mbc3_aain02.png["timestamp server"]
 
 ==== Proof-of-Work
-To implement a distributed timestamp server on a peer-to-peer basis, we will need to use a proof-of-work system similar to Adam Back's Hashcash [6], rather than newspaper or Usenet posts. The proof-of-work involves scanning for a value that when hashed, such as with SHA-256, the hash begins with a number of zero bits. The average work required is exponential in the number of zero bits required and can be verified by executing a single hash.  For our timestamp network, we implement the proof-of-work by incrementing a nonce in the block until a value is found that gives the block's hash the required zero bits. Once the CPU effort has been expended to make it satisfy the proof-of-work, the block cannot be changed without redoing the work. As later blocks are chained after it, the work to change the block would include redoing all the blocks after it.
+To ((("proof-of-work algorithm", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper")))implement a distributed timestamp server on a peer-to-peer basis, we will need to use a proof-of-work system similar to Adam Back's Hashcash [6], rather than newspaper or Usenet posts. The proof-of-work involves scanning for a value that when hashed, such as with SHA-256, the hash begins with a number of zero bits. The average work required is exponential in the number of zero bits required and can be verified by executing a single hash.  For our timestamp network, we implement the proof-of-work by incrementing a nonce in the block until a value is found that gives the block's hash the required zero bits. Once the CPU effort has been expended to make it satisfy the proof-of-work, the block cannot be changed without redoing the work. As later blocks are chained after it, the work to change the block would include redoing all the blocks after it.
 
-image::images/mbc2_abin03.png["pow"]
+image::images/mbc3_aain03.png["pow"]
 
 The proof-of-work also solves the problem of determining representation in majority decision making. If the majority were based on one-IP-address-one-vote, it could be subverted by anyone able to allocate many IPs. Proof-of-work is essentially one-CPU-one-vote. The majority decision is represented by the longest chain, which has the greatest proof-of-work effort invested in it. If a majority of CPU power is controlled by honest nodes, the honest chain will grow the fastest and outpace any competing chains. To modify a past block, an attacker would have to redo the proof-of-work of the block and all blocks after it and then catch up with and surpass the work of the honest nodes. We will show later that the probability of a slower attacker catching up diminishes exponentially as subsequent blocks are added.
 
@@ -47,7 +48,7 @@ To compensate for increasing hardware speed and varying interest in running node
 
 ==== Network
 
-The steps to run the network are as follows:
+The steps((("nodes", "in Bitcoin", secondary-sortas="Bitcoin whitepaper")))((("Bitcoin network", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper"))) to run the network are as follows:
 
 1. New transactions are broadcast to all nodes.
 2. Each node collects new transactions into a block.
@@ -61,50 +62,50 @@ Nodes always consider the longest chain to be the correct one and will keep work
 New transaction broadcasts do not necessarily need to reach all nodes. As long as they reach many nodes, they will get into a block before long. Block broadcasts are also tolerant of dropped messages. If a node does not receive a block, it will request it when it receives the next block and realizes it missed one.
 
 ==== Incentive
-By convention, the first transaction in a block is a special transaction that starts a new coin owned by the creator of the block. This adds an incentive for nodes to support the network, and provides a way to initially distribute coins into circulation, since there is no central authority to issue them. The steady addition of a constant of amount of new coins is analogous to gold miners expending resources to add gold to circulation. In our case, it is CPU time and electricity that is expended.
+By((("incentives", id="incentive-whitepaper"))) convention, the first transaction in a block is a special transaction that starts a new coin owned by the creator of the block. This adds an incentive for nodes to support the network, and provides a way to initially distribute coins into circulation, since there is no central authority to issue them. The steady addition of a constant of amount of new coins is analogous to gold miners expending resources to add gold to circulation. In our case, it is CPU time and electricity that is expended.
 
 The incentive can also be funded with transaction fees. If the output value of a transaction is less than its input value, the difference is a transaction fee that is added to the incentive value of the block containing the transaction. Once a predetermined number of coins have entered circulation, the incentive can transition entirely to transaction fees and be completely inflation free.
 
-The incentive may help encourage nodes to stay honest. If a greedy attacker is able to assemble more CPU power than all the honest nodes, he would have to choose between using it to defraud people by stealing back his payments, or using it to generate new coins. He ought to find it more profitable to play by the rules, such rules that favour him with more new coins than everyone else combined, than to undermine the system and the validity of his own wealth.
+The incentive may help encourage nodes to stay honest. If a greedy attacker is able to assemble more CPU power than all the honest nodes, he would have to choose between using it to defraud people by stealing back his payments, or using it to generate new coins. He ought to find it more profitable to play by the rules, such rules that favour him with more new coins than everyone else combined, than to undermine the system and the validity of his own((("incentives", startref="incentive-whitepaper"))) wealth.
 
 ==== Reclaiming Disk Space
 
 ++++
-<p>Once the latest transaction in a coin is buried under enough blocks, the spent transactions before it can be discarded to save disk space. To facilitate this without breaking the block's hash, transactions are hashed in a Merkle Tree <a href="#ref_seven">[7]</a> <a href="#ref_two">[2]</a> <a href="#ref_five">[5]</a>, with only the root included in the block's hash. Old blocks can then be compacted by stubbing off branches of the tree. The interior hashes do not need to be stored.</p>
+<p>Once ((("disk space", "reclaiming")))((("reclaiming", "disk space")))((("blocks", "reclaiming disk space")))the latest transaction in a coin is buried under enough blocks, the spent transactions before it can be discarded to save disk space. To facilitate this without breaking the block's hash, transactions are hashed in a Merkle Tree <a href="#ref_seven">[7]</a> <a href="#ref_two">[2]</a> <a href="#ref_five">[5]</a>, with only the root included in the block's hash. Old blocks can then be compacted by stubbing off branches of the tree. The interior hashes do not need to be stored.</p>
 ++++
 
-image::images/mbc2_abin04.png["disk"]
+image::images/mbc3_aain04.png["disk"]
 
 A block header with no transactions would be about 80 bytes. If we suppose blocks are generated every 10 minutes, +80 bytes * 6 * 24 * 365 = 4.2MB+ per year. With computer systems typically selling with 2GB of RAM as of 2008, and Moore's Law predicting current growth of 1.2GB per year, storage should not be a problem even if the block headers must be kept in memory.
 
 ==== Simplified Payment Verification
-It is possible to verify payments without running a full network node. A user only needs to keep a copy of the block headers of the longest proof-of-work chain, which he can get by querying network nodes until he's convinced he has the longest chain, and obtain the Merkle branch linking the transaction to the block it's timestamped in. He can't check the transaction for himself, but by linking it to a place in the chain, he can see that a network node has accepted it, and blocks added after it further confirm the network has accepted it.
+It is ((("payment verification", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper", id="payment-verify")))((("verifying", "payment", "in Bitcoin whitepaper", tertiary-sortas="Bitcoin whitepaper", id="verify-payment")))possible to verify payments without running a full network node. A user only needs to keep a copy of the block headers of the longest proof-of-work chain, which he can get by querying network nodes until he's convinced he has the longest chain, and obtain the Merkle branch linking the transaction to the block it's timestamped in. He can't check the transaction for himself, but by linking it to a place in the chain, he can see that a network node has accepted it, and blocks added after it further confirm the network has accepted it.
 
-image::images/mbc2_abin05.png["spv"]
+image::images/mbc3_aain05.png["spv"]
 
-As such, the verification is reliable as long as honest nodes control the network, but is more vulnerable if the network is overpowered by an attacker. While network nodes can verify transactions for themselves, the simplified method can be fooled by an attacker's fabricated transactions for as long as the attacker can continue to overpower the network. One strategy to protect against this would be to accept alerts from network nodes when they detect an invalid block, prompting the user's software to download the full block and alerted transactions to confirm the inconsistency. Businesses that receive frequent payments will probably still want to run their own nodes for more independent security and quicker verification.
+As such, the verification is reliable as long as honest nodes control the network, but is more vulnerable if the network is overpowered by an attacker. While network nodes can verify transactions for themselves, the simplified method can be fooled by an attacker's fabricated transactions for as long as the attacker can continue to overpower the network. One strategy to protect against this would be to accept alerts from network nodes when they detect an invalid block, prompting the user's software to download the full block and alerted transactions to confirm the inconsistency. Businesses that receive frequent payments will probably still want to run their own nodes for more independent security and quicker ((("payment verification", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper", startref="payment-verify")))((("verifying", "payment", "in Bitcoin whitepaper", tertiary-sortas="Bitcoin whitepaper", startref="verify-payment")))verification.
 
 ==== Combining and Splitting Value
-Although it would be possible to handle coins individually, it would be unwieldy to make a separate transaction for every cent in a transfer. To allow value to be split and combined, transactions contain multiple inputs and outputs. Normally there will be either a single input from a larger previous transaction or multiple inputs combining smaller amounts, and at most two outputs: one for the payment, and one returning the change, if any, back to the sender.
+Although ((("transactions", "inputs", "in Bitcoin whitepaper", tertiary-sortas="Bitcoin whitepaper")))((("transactions", "outputs", "in Bitcoin whitepaper", tertiary-sortas="Bitcoin whitepaper")))((("inputs", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper")))((("outputs", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper")))it would be possible to handle coins individually, it would be unwieldy to make a separate transaction for every cent in a transfer. To allow value to be split and combined, transactions contain multiple inputs and outputs. Normally there will be either a single input from a larger previous transaction or multiple inputs combining smaller amounts, and at most two outputs: one for the payment, and one returning the change, if any, back to the sender.
 
-image::images/mbc2_abin06.png["combining-splitting"]
+image::images/mbc3_aain06.png["combining-splitting"]
 
 It should be noted that fan-out, where a transaction depends on several transactions, and those transactions depend on many more, is not a problem here. There is never the need to extract a complete standalone copy of a transaction's history.
 
 ==== Privacy
-The traditional banking model achieves a level of privacy by limiting access to information to the parties involved and the trusted third party. The necessity to announce all transactions publicly precludes this method, but privacy can still be maintained by breaking the flow of information in another place: by keeping public keys anonymous. The public can see that someone is sending an amount to someone else, but without information linking the transaction to anyone. This is similar to the level of information released by stock exchanges, where the time and size of individual trades, the "tape", is made public, but without telling who the parties were.
+The traditional((("privacy", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper"))) banking model achieves a level of privacy by limiting access to information to the parties involved and the trusted third party. The necessity to announce all transactions publicly precludes this method, but privacy can still be maintained by breaking the flow of information in another place: by keeping public keys anonymous. The public can see that someone is sending an amount to someone else, but without information linking the transaction to anyone. This is similar to the level of information released by stock exchanges, where the time and size of individual trades, the "tape", is made public, but without telling who the parties were.
 
-image::images/mbc2_abin07.png["privacy"]
+image::images/mbc3_aain07.png["privacy"]
 
 As an additional firewall, a new key pair should be used for each transaction to keep them from being linked to a common owner. Some linking is still unavoidable with multi-input transactions, which necessarily reveal that their inputs were owned by the same owner. The risk is that if the owner of a key is revealed, linking could reveal other transactions that belonged to the same owner.
 
 ==== Calculations
-We consider the scenario of an attacker trying to generate an alternate chain faster than the honest chain. Even if this is accomplished, it does not throw the system open to arbitrary changes, such as creating value out of thin air or taking money that never belonged to the attacker. Nodes are not going to accept an invalid transaction as payment, and honest nodes will never accept a block containing them. An attacker can only try to change one of his own transactions to take back money he recently spent.
+We consider ((("calculations", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper", id="calculate-whitepaper")))the scenario of an attacker trying to generate an alternate chain faster than the honest chain. Even if this is accomplished, it does not throw the system open to arbitrary changes, such as creating value out of thin air or taking money that never belonged to the attacker. Nodes are not going to accept an invalid transaction as payment, and honest nodes will never accept a block containing them. An attacker can only try to change one of his own transactions to take back money he recently spent.
 
-The race between the honest chain and an attacker chain can be characterized as a Binomial Random Walk. The success event is the honest chain being extended by one block, increasing its lead by +1, and the failure event is the attacker's chain being extended by one block, reducing the gap by -1.
+The race between the honest chain and an attacker chain can be characterized as a ((("Binomial Random Walk")))Binomial Random Walk. The success event is the honest chain being extended by one block, increasing its lead by +1, and the failure event is the attacker's chain being extended by one block, reducing the gap by -1.
 
 ++++
-<p>The probability of an attacker catching up from a given deficit is analogous to a Gambler's Ruin problem. Suppose a gambler with unlimited credit starts at a deficit and plays potentially an infinite number of trials to try to reach breakeven. We can calculate the probability he ever reaches breakeven, or that an attacker ever catches up with the honest chain, as follows <a href="#ref_eight">[8]</a>:</p>
+<p>The probability of an attacker catching up from a given deficit is analogous to a ((("Gambler&#x27;s Ruin problem")))Gambler's Ruin problem. Suppose a gambler with unlimited credit starts at a deficit and plays potentially an infinite number of trials to try to reach breakeven. We can calculate the probability he ever reaches breakeven, or that an attacker ever catches up with the honest chain, as follows <a href="#ref_eight">[8]</a>:</p>
 ++++
 
 p = probability an honest node finds the next block
@@ -113,7 +114,63 @@ q = probability the attacker finds the next block
 
 q~z~ = probability the attacker will ever catch up from z blocks behind
 
-image::images/mbc2_abin08.png["eq1"]
+
+++++
+<div data-type="equation">
+<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" >
+  <mstyle mathsize="1.2000em">
+    <msub>
+      <mi>q</mi>
+      <mi>z</mi>
+    </msub>
+    <mo>=</mo>
+    <mrow>
+      <mo fence="true" form="prefix">{</mo>
+      <mtable columnalign="center center">
+        <mtr>
+          <mtd>
+            <mn>1</mn>
+          </mtd>
+          <mtd>
+            <mrow>
+              <mtext>if</mtext>
+              <mspace width="0.2778em"></mspace>
+              <mi>p</mi>
+              <mo>≤</mo>
+              <mi>q</mi>
+            </mrow>
+          </mtd>
+        </mtr>
+        <mtr>
+          <mtd>
+            <mrow>
+              <mo form="prefix" stretchy="false">(</mo>
+              <mi>q</mi>
+              <mo lspace="0em" rspace="0em">⁄</mo>
+              <mi>p</mi>
+              <msup>
+                <mo form="postfix" stretchy="false">)</mo>
+                <mi>z</mi>
+              </msup>
+            </mrow>
+          </mtd>
+          <mtd>
+            <mrow>
+              <mtext>if</mtext>
+              <mspace width="0.2778em"></mspace>
+              <mi>p</mi>
+              <mo>&gt;</mo>
+              <mi>q</mi>
+            </mrow>
+          </mtd>
+        </mtr>
+      </mtable>
+      <mo fence="true" form="postfix">}</mo>
+    </mrow>
+  </mstyle>
+</math>
+</div>
+++++
 
 Given our assumption that p > q, the probability drops exponentially as the number of blocks the attacker has to catch up with increases. With the odds against him, if he doesn't make a lucky lunge forward early on, his chances become vanishingly small as he falls further behind.
 
@@ -123,15 +180,175 @@ The receiver generates a new key pair and gives the public key to the sender sho
 
 The recipient waits until the transaction has been added to a block and z blocks have been linked after it. He doesn't know the exact amount of progress the attacker has made, but assuming the honest blocks took the average expected time per block, the attacker's potential progress will be a Poisson distribution with expected value:
 
-image::images/mbc2_abin09.png["eq2"]
+++++
+<div data-type="equation">
+<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" >
+  <mstyle mathsize="1.2000em">
+    <mi>λ</mi>
+    <mo>=</mo>
+    <mi>z</mi>
+    <mfrac>
+      <mi>q</mi>
+      <mi>p</mi>
+    </mfrac>
+  </mstyle>
+</math>
+</div>
+++++
 
 To get the probability the attacker could still catch up now, we multiply the Poisson density for each amount of progress he could have made by the probability he could catch up from that point:
 
-image::images/mbc2_abin10.png["eq3"]
+++++
+<div data-type="equation">
+<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" >
+  <mstyle mathsize="1.2000em">
+    <mrow>
+      <munderover>
+        <mo movablelimits="false">∑</mo>
+        <mrow>
+          <mi>k</mi>
+          <mo>=</mo>
+          <mn>0</mn>
+        </mrow>
+        <mi>∞</mi>
+      </munderover>
+    </mrow>
+    <mfrac>
+      <mrow>
+        <msup>
+          <mi>λ</mi>
+          <mi>k</mi>
+        </msup>
+        <msup>
+          <mi>e</mi>
+          <mrow>
+            <mo>−</mo>
+            <mi>λ</mi>
+          </mrow>
+        </msup>
+      </mrow>
+      <mrow>
+        <mi>k</mi>
+        <mo form="postfix" stretchy="false">!</mo>
+      </mrow>
+    </mfrac>
+    <mo>⋅</mo>
+    <mrow>
+      <mo fence="true" form="prefix">{</mo>
+      <mtable columnalign="center center">
+        <mtr>
+          <mtd>
+            <mrow>
+              <mo form="prefix" stretchy="false">(</mo>
+              <mi>q</mi>
+              <mo lspace="0em" rspace="0em">⁄</mo>
+              <mi>p</mi>
+              <msup>
+                <mo form="postfix" stretchy="false">)</mo>
+                <mrow>
+                  <mo form="prefix" stretchy="false">(</mo>
+                  <mi>z</mi>
+                  <mo>−</mo>
+                  <mi>k</mi>
+                  <mo form="postfix" stretchy="false">)</mo>
+                </mrow>
+              </msup>
+            </mrow>
+          </mtd>
+          <mtd>
+            <mrow>
+              <mtext>if</mtext>
+              <mspace width="0.2778em"></mspace>
+              <mi>k</mi>
+              <mo>≤</mo>
+              <mi>z</mi>
+            </mrow>
+          </mtd>
+        </mtr>
+        <mtr>
+          <mtd>
+            <mn>1</mn>
+          </mtd>
+          <mtd>
+            <mrow>
+              <mtext>if</mtext>
+              <mspace width="0.2778em"></mspace>
+              <mi>k</mi>
+              <mo>&gt;</mo>
+              <mi>z</mi>
+            </mrow>
+          </mtd>
+        </mtr>
+      </mtable>
+      <mo fence="true" form="postfix">}</mo>
+    </mrow>
+  </mstyle>
+</math>
+</div>
+++++
 
 Rearranging to avoid summing the infinite tail of the distribution...
 
-image::images/mbc2_abin11.png["eq4"]
+++++
+<div data-type="equation">
+<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" >
+  <mstyle mathsize="1.2000em">
+    <mn>1</mn>
+    <mo>−</mo>
+    <mrow>
+      <munderover>
+        <mo movablelimits="false">∑</mo>
+        <mrow>
+          <mi>k</mi>
+          <mo>=</mo>
+          <mn>0</mn>
+        </mrow>
+        <mi>z</mi>
+      </munderover>
+    </mrow>
+    <mfrac>
+      <mrow>
+        <msup>
+          <mi>λ</mi>
+          <mi>k</mi>
+        </msup>
+        <msup>
+          <mi>e</mi>
+          <mrow>
+            <mo>−</mo>
+            <mi>λ</mi>
+          </mrow>
+        </msup>
+      </mrow>
+      <mrow>
+        <mi>k</mi>
+        <mo form="postfix" stretchy="false">!</mo>
+      </mrow>
+    </mfrac>
+    <mrow>
+      <mo fence="true" form="prefix">(</mo>
+      <mn>1</mn>
+      <mo>−</mo>
+      <mo form="prefix" stretchy="false">(</mo>
+      <mi>q</mi>
+      <mo lspace="0em" rspace="0em">⁄</mo>
+      <mi>p</mi>
+      <msup>
+        <mo form="postfix" stretchy="false">)</mo>
+        <mrow>
+          <mo form="prefix" stretchy="false">(</mo>
+          <mi>z</mi>
+          <mo>−</mo>
+          <mi>k</mi>
+          <mo form="postfix" stretchy="false">)</mo>
+        </mrow>
+      </msup>
+      <mo fence="true" form="postfix">)</mo>
+    </mrow>
+  </mstyle>
+</math>
+</div>
+++++
 
 Converting to C code...
 
@@ -198,7 +415,7 @@ q=0.45 z=340
 ----
 
 ==== Conclusion
-We have proposed a system for electronic transactions without relying on trust. We started with the usual framework of coins made from digital signatures, which provides strong control of ownership, but is incomplete without a way to prevent double-spending. To solve this, we proposed a peer-to-peer network using proof-of-work to record a public history of transactions that quickly becomes computationally impractical for an attacker to change if honest nodes control a majority of CPU power. The network is robust in its unstructured simplicity. Nodes work all at once with little coordination. They do not need to be identified, since messages are not routed to any particular place and only need to be delivered on a best effort basis. Nodes can leave and rejoin the network at will, accepting the proof-of-work chain as proof of what happened while they were gone. They vote with their CPU power, expressing their acceptance of valid blocks by working on extending them and rejecting invalid blocks by refusing to work on them. Any needed rules and incentives can be enforced with this consensus mechanism.
+We ((("calculations", "in Bitcoin whitepaper", secondary-sortas="Bitcoin whitepaper", startref="calculate-whitepaper")))have proposed a system for electronic transactions without relying on trust. We started with the usual framework of coins made from digital signatures, which provides strong control of ownership, but is incomplete without a way to prevent double-spending. To solve this, we proposed a peer-to-peer network using proof-of-work to record a public history of transactions that quickly becomes computationally impractical for an attacker to change if honest nodes control a majority of CPU power. The network is robust in its unstructured simplicity. Nodes work all at once with little coordination. They do not need to be identified, since messages are not routed to any particular place and only need to be delivered on a best effort basis. Nodes can leave and rejoin the network at will, accepting the proof-of-work chain as proof of what happened while they were gone. They vote with their CPU power, expressing their acceptance of valid blocks by working on extending them and rejecting invalid blocks by refusing to work on them. Any needed rules and incentives can be enforced with this consensus mechanism.
 
 ==== References
 ++++
@@ -235,7 +452,7 @@ We have proposed a system for electronic transactions without relying on trust.
 
 === License
 
-((("open source licenses")))This whitepaper was published in October 2008 by Satoshi Nakamoto. It was later (2009) added as supporting documentation to the bitcoin software, which carries the same MIT license. It has been reproduced in this book, without modification other than formatting, under the terms of the MIT license:
+This whitepaper was published in October 2008 by Satoshi Nakamoto. It was later (2009) added as supporting documentation to the bitcoin software and carries the same MIT license. It has been reproduced in this book, without modification other than formatting, under the terms of the MIT license:
 
 The MIT License (MIT)
 Copyright (c) 2008 Satoshi Nakamoto
@@ -244,4 +461,4 @@ Permission is hereby granted, free of charge, to any person obtaining a copy of
 
 The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
 
-THE SOFTWARE IS PROVIDED "AS IS," WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.((("", startref="whitethirteen")))((("", startref="nakamatothirteen")))((("", startref="BCwhitethirteen")))
+THE SOFTWARE IS PROVIDED "AS IS," WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN((("Bitcoin whitepaper", "original version", startref="bitcoin-whitepaper-original")))((("whitepaper (Bitcoin)", "original version", startref="whitepaper-original"))) THE SOFTWARE.

appb_errata.adoc

@@ -0,0 +1,245 @@
+[appendix]
+== Errata to the Bitcoin Whitepaper
+
+This ((("Bitcoin whitepaper", "errata", id="bitcoin-whitepaper-errata")))((("whitepaper (Bitcoin)", "errata", id="whitepaper-errata")))appendix contains a description of known problems in Satoshi Nakamoto’s paper, "Bitcoin:
+A Peer-to-Peer Electronic Cash System," as well as notes on terminology
+changes and how Bitcoin's implementation differs from that described in
+the paper.
+
+This document was originally published by a coauthor of this book in
+2016; it is reproduced here with updates.  The names of
+sections in this errata correspond to the names of the
+sections in Nakamoto's original paper.
+
+=== Abstract
+
+____
+"The longest chain not only serves as proof of the sequence of events
+witnessed, but proof that it came from the largest pool of CPU power."
+____
+
+* *Implementation detail:* If each link in the chain (called "blocks"
+in Bitcoin) was built using the same amount of _proof of work_ (PoW), the
+longest chain would be the one backed by the largest pool of
+computational power. However, Bitcoin was implemented in such a way that
+the amount of PoW can vary between blocks, so it became important not to
+check for the "the longest chain" but rather "the chain demonstrating
+the most PoW"; this is often shortened to "most-work chain."
++
+The
+https://oreil.ly/XYZzx[change]
+from checking for the longest chain to checking for the most-work chain
+occurred in July 2010, long after Bitcoin’s initial release:
++
+[source,diff]
+----
+-    if (pindexNew->nHeight > nBestHeight)
++    if (pindexNew->bnChainWork > bnBestChainWork)
+----
+
+[role="less_space pagebreak-before"]
+* *Terminology change:* General CPUs were used to generate the PoW for
+the earliest Bitcoin blocks, but PoW generation today is mostly performed
+by specialist Application Specific Integrated Circuits (ASICs), so
+instead of saying "CPU power" it is perhaps more correct to say
+"computational power" or, simply, "hash rate" for the hashing used
+in generating the PoW.
+
+____
+"As long as a majority of CPU power is controlled by nodes that are not
+cooperating to attack the network, they’ll generate the longest chain
+and outpace attackers."
+____
+
+* *Terminology change:* The term "nodes" today is used to refer to
+full validation nodes, which are programs that enforce all the rules of
+the system. Programs (and hardware) that extend the chain today are
+called "miners" based on Nakamoto’s analogy to gold miners in section
+6 of the paper. Nakamoto expected all miners to be nodes but the
+software he released did not require all nodes to be miners. In the
+original software, a simple menu item in the node GUI allowed toggling
+the mining function on or off.
++
+Today it is the case that the overwhelming number of nodes are not
+miners and that many individuals who own mining hardware do not use it
+with their own nodes (and even those that do mine with their own nodes
+often mine for short periods of time on top of newly discovered blocks
+without ensuring their node considers the new block valid). The early
+parts of the paper where "nodes" is mostly used without modification
+refer to mining using a full validation node; the later parts of the
+paper which refer to "network nodes" is mainly about what nodes can do
+even if they aren’t mining.
+* *Post-publication discovery:* When a new block is produced, the miner
+who produces that block can begin working on its sequel immediately but
+all other miners are unaware of the new block and cannot begin working
+on it until it has propagated across the
+network to them. This gives miners who produce many blocks an edge over
+miners who produce fewer blocks, and this can be exploited in what’s
+known as the _selfish mining attack_ to allow an attacker with around
+30% of total network hash rate to make other miners less profitable,
+perhaps driving them into following the attacking miner’s policy. So
+instead of saying "a majority of CPU power is controlled by nodes that
+are not cooperating to attack the network," it is perhaps more correct
+to say "as long as nodes cooperating to attack the network control less
+than about 30% of the network."
+
+=== Transactions
+
+____
+"We define((("transactions", "errata in Bitcoin whitepaper", id="transaction-errata"))) an electronic coin as a chain of digital signatures. Each
+owner transfers the coin to the next by digitally signing a hash of the
+previous transaction and the public key of the next owner and adding
+these to the end of the coin."
+____
+
+* *Implementation detail:* Bitcoin implements a more general version of
+this system where digital signatures are not used directly but rather a
+"deterministic expression" is used instead. Just as a signature that
+matches a known public key can be used to enable a payment, the data
+that satisfies a known expression can also enable a payment.
+Generically, the expression that must be satisfied in Bitcoin in order
+to spend a coin is known as an "encumbrance." Almost all encumbrances
+in Bitcoin to date require providing at least one signature. So instead
+of saying "a chain of digital signatures," it is more correct to say
+"a chain of encumbrances." Given that transactions often have more
+than one input and more than one output, the structure is not very
+chain-like; it’s more accurately described as a directed acyclic ((("transactions", "errata in Bitcoin whitepaper", startref="transaction-errata")))graph
+(DAG).
+
+=== Proof of Work
+
+____
+"...we((("proof-of-work algorithm", "errata in Bitcoin whitepaper", id="proof-errata"))) implement the proof-of-work by incrementing a nonce in the block
+until a value is found that gives the block’s hash the required zero
+bits."
+____
+
+* *Implementation detail:* Adam Back’s Hashcash implementation requires
+finding a hash with the required number of leading zero bits. Bitcoin
+treats the hash as an integer and requires that it be less than a
+specified integer, which effectively allows a fractional number of bits
+to be specified.
+
+____
+"Proof-of-work is essentially one-CPU-one-vote."
+____
+
+* *Important note:* The vote here is not on the rules of the system but
+merely on the ordering of the transactions in order to provide
+assurances that an "electronic coin" cannot be easily double spent.
+This is described in more detail in section 11 of the paper where it
+says, "We consider the scenario of an attacker trying to generate an
+alternate chain faster than the honest chain. Even if this is
+accomplished, it does not throw the system open to arbitrary changes,
+such as creating value out of thin air or taking money that never
+belonged to the attacker. Nodes are not going to accept an invalid
+transaction as payment, and honest nodes will never accept a block
+containing them."
+
+____
+"...proof-of-work difficulty is determined by a moving average targeting an
+average number of blocks per hour."
+____
+
+* *Implementation detail:* A moving average is not used. Instead, every
+2,016th block has its reported generation time compared to the
+generation time for an earlier block, and the difference between them is
+used to calculate the average used for adjustment.
++
+Further, the average implemented in Bitcoin targets an average number of
+blocks per two weeks (not per hour as might be implied by the text).
+Other implemented rules may further slow adjustments, such as a rule
+that the adjustment cannot increase block production speed by more than
+300% per period, nor slow it by more ((("proof-of-work algorithm", "errata in Bitcoin whitepaper", startref="proof-errata")))than 75%.
+
+=== Reclaiming Disk Space
+
+____
+"Once the ((("disk space, reclaiming")))((("reclaiming disk space")))((("blocks", "reclaiming disk space")))latest transaction in a coin is buried under enough blocks, the
+spent transactions before it can be discarded to save disk space."
+____
+
+* *Possible post-publication discovery:* Although the merkle tree
+structure described in this section can prove a transaction was included
+in a particular block, there is currently no way in Bitcoin to prove
+that a transaction has not been spent except to process all subsequent
+data in the blockchain. This means the method described here cannot be
+universally used for reclaiming disk space among all nodes, as all new
+nodes will need to process all transactions.
+
+=== Simplified Payment Verification
+
+____
+"One strategy((("payment verification", "errata in Bitcoin whitepaper")))((("verifying", "payment", "errata in Bitcoin whitepaper"))) to protect against this would be to accept alerts from
+network nodes when they detect an invalid block, prompting the user’s
+software to download the full block and alerted transactions to confirm
+the inconsistency."
+____
+
+* *Important Note:* Although software has been produced that implements
+some parts of this section and calls that Simplified Payment
+Verification (SPV), none of these programs currently accepts alerts from
+network nodes (full validation nodes) when invalid blocks have been
+detected. This has placed bitcoins in so-called SPV wallets at risk in
+the past.
+
+=== Privacy
+
+____
+"Some linking((("privacy", "errata in Bitcoin whitepaper"))) is still unavoidable with multi-input transactions, which
+necessarily reveal that their inputs were owned by the same owner."
+____
+
+* *Post-publication invention:* It isn't clear that different inputs
+in the same transaction have the same owner if owners often mix their
+inputs with
+inputs belonging to other owners. For example, there’s no public
+difference between Alice and Bob each contributing one of their inputs
+toward paying Charlie and Dan than there is between just Alice
+contributing two of her inputs toward paying Charlie and Dan.
++
+This technique is known today as
+https://oreil.ly/UBEJX[CoinJoin], and software implementing
+it has been in use since 2015.
+
+=== Calculations
+
+____
+"The receiver ((("calculations", "errata in Bitcoin whitepaper")))generates a new key pair and gives the public key to the
+sender shortly before signing. This prevents the sender from preparing a
+chain of blocks ahead of time by working on it continuously until he is
+lucky enough to get far enough ahead, then executing the transaction at
+that moment."
+____
+
+* *Post-publication discovery:* Nothing about the receiver generating a
+public key shortly before the spender signs a transaction prevents the
+spender from preparing a chain of blocks ahead of time. Early Bitcoin
+user Hal Finney discovered this attack and
+https://oreil.ly/kg_Xe[described
+it]: "Suppose the attacker is generating blocks occasionally. In each
+block he generates, he includes a transfer from address A to address B,
+both of which he controls.
++
+"To cheat you, when he generates a block, he doesn’t broadcast it.
+Instead, he runs down to your store and makes a payment to your address
+C with his address A. You wait a few seconds, don’t hear anything, and
+transfer the goods. He broadcasts his block now, and his transaction
+will take precedence over yours."
++
+The attack works for any number of confirmations, and is sometimes named
+the Finney Attack.
+
+'''''
+
+*Disclaimer:* The author of this document was not the first person to
+identify any of the problems described here—he has merely collected them
+into a single document.
+
+*License:* This errata document is released under the
+https://oreil.ly/xZeBR[CC0] 1.0 Universal
+Public Domain Dedication
+
+For updates made ((("Bitcoin whitepaper", "errata", startref="bitcoin-whitepaper-errata")))((("whitepaper (Bitcoin)", "errata", startref="whitepaper-errata")))after the publication of this book, please see the
+https://oreil.ly/ygExa[Original
+document].

appc_bips.adoc

@@ -0,0 +1,73 @@
+[[appdxbitcoinimpproposals]]
+[appendix]
+== Bitcoin Improvement Proposals
+
+Bitcoin Improvement Proposals are design documents providing information to the Bitcoin community or describing a new feature for Bitcoin or its processes or environment.
+
+As per BIP1 _BIP Purpose and Guidelines_, there are three((("BIPs (Bitcoin Improvement Proposals)", "types of"))) kinds of BIPs:
+
+_Standard_ BIP:: Describes any change that affects most or all Bitcoin implementations, such as a change to the network protocol, a change in block or transaction validity rules, or any change or addition that affects the interoperability of applications using Bitcoin.
+_Informational_ BIP:: Describes a Bitcoin design issue or provides general guidelines or information to the Bitcoin community, but does not propose a new feature. Informational BIPs do not necessarily represent a Bitcoin community consensus or recommendation, so users and implementors may ignore informational BIPs or follow their advice.
+_Process_ BIP:: Describes a Bitcoin process or proposes a change to (or an event in) a process. Process BIPs are like standard BIPs but apply to areas other than the Bitcoin protocol itself. They might propose an implementation but not to Bitcoin's codebase; they often require community consensus. Unlike informational BIPs, they are more than recommendations, and users are typically not free to ignore them. Examples include procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Bitcoin development. Any meta-BIP is also considered a process BIP.
+
+BIPs are recorded in a https://oreil.ly/jjO0R[versioned repository on GitHub].
+An MIT-licensed document from the open source Bitcoin Core project,
+reproduced here in edited form, describes which BIPs it implements, including listing
+the Pull Request (PR) and version of Bitcoin Core where support for each BIP was added or
+significantly changed.
+
+BIPs that are ((("BIPs (Bitcoin Improvement Proposals)", "implemented by Bitcoin Core", id="bips-implement")))((("Bitcoin Core", "BIPs implemented by", id="bitcoin-core-bips")))implemented by Bitcoin Core:
+
+-    BIP9: The changes allowing multiple soft forks to be deployed in parallel have been implemented since v0.12.1 (PR #7575).
+-    BIP11: Multisig outputs are standard since v0.6.0 (PR #669).
+-    BIP13: The address format for P2SH addresses has been implemented since v0.6.0 (PR #669).
+-    BIP14: The subversion string is being used as User Agent since v0.6.0 (PR #669).
+-    BIP16: The pay-to-script-hash evaluation rules have been implemented since v0.6.0, and took effect on April 1st 2012 (PR #748).
+-    BIP21: The URI format for Bitcoin payments has been implemented since v0.6.0 (PR #176).
+-    BIP22: The 'getblocktemplate' (GBT) RPC protocol for mining has been implemented since v0.7.0 (PR #936).
+-    BIP23: Some extensions to GBT have been implemented since v0.10.0rc1, including longpolling and block proposals (PR #1816).
+-    BIP30: The evaluation rules to forbid creating new transactions with the same txid as previous not-fully-spent transactions were implemented since v0.6.0, and the rule took effect on March 15th 2012 (PR #915).
+-    BIP31: The 'pong' protocol message (and the protocol version bump to 60001) has been implemented since v0.6.1 (PR #1081).
+-    BIP32: Hierarchical Deterministic Wallets has been implemented since v0.13.0 (PR #8035).
+-    BIP34: The rule that requires blocks to contain their height (number) in the coinbase input, and the introduction of version 2 blocks has been implemented since v0.7.0. The rule took effect for version 2 blocks as of block 224413 (March 5th 2013), and version 1 blocks are no longer allowed since block 227931 (March 25th 2013) (PR #1526).
+-    BIP35: The 'mempool' protocol message (and the protocol version bump to 60002) has been implemented since v0.7.0 (PR #1641). As of v0.13.0, this is only available for +NODE_BLOOM+ (BIP111) peers.
+
+[role="less_space pagebreak-before"]
+-    BIP37: The bloom filtering for transaction relaying, partial Merkle trees for blocks, and the protocol version bump to 70001 (enabling low-bandwidth lightweight clients) has been implemented since v0.8.0 (PR #1795). Disabled by default since v0.19.0, can be enabled by the +-peerbloomfilters+ option.
+-    BIP42: The bug that would have caused the subsidy schedule to resume after block 13440000 was fixed in v0.9.2 (PR #3842).
+-    BIP43: The experimental descriptor wallets introduced in v0.21.0 by default use the Hierarchical Deterministic Wallet derivation proposed by BIP43 (PR #16528).
+-    BIP44: The experimental descriptor wallets introduced in v0.21.0 by default use the Hierarchical Deterministic Wallet derivation proposed by BIP44 (PR #16528).
+-    BIP49: The experimental descriptor wallets introduced in v0.21.0 by default use the Hierarchical Deterministic Wallet derivation proposed by BIP49 (PR #16528).
+-    BIP61: The 'reject' protocol message (and the protocol version bump to 70002) was added in v0.9.0 (PR #3185). Starting v0.17.0, whether to send reject messages can be configured with the ++-enablebip61++ option, and support is deprecated (disabled by default) as of v0.18.0. Support was removed in v0.20.0 (PR #15437).
+-    BIP65: The ++CHECKLOCKTIMEVERIFY++ soft fork was merged in v0.12.0 (PR #6351), and backported to v0.11.2 and v0.10.4. Mempool-only +CLTV+ was added in PR #6124.
+-    BIP66: The strict DER rules and associated version 3 blocks have been implemented since v0.10.0 (PR #5713).
+-    BIP68: Sequence locks have been implemented as of v0.12.1 (PR #7184), and have been buried since v0.19.0 (PR #16060).
+-    BIP70 71 72: Payment Protocol support has been available in Bitcoin Core GUI since v0.9.0 (PR #5216). Support can be optionally disabled at build time since v0.18.0 (PR 14451), and it is disabled by default at build time since v0.19.0 (PR #15584). It has been removed as of v0.20.0 (PR 17165).
+-    BIP84: The experimental descriptor wallets introduced in v0.21.0 by default use the Hierarchical Deterministic Wallet derivation proposed by BIP84. (PR #16528)
+-    BIP86: Descriptor wallets by default use the Hierarchical Deterministic Wallet derivation proposed by BIP86 since v23.0 (PR #22364).
+-    BIP90: Trigger mechanism for activation of BIPs 34, 65, and 66 has been simplified to block height checks since v0.14.0 (PR #8391).
+-    BIP111: +NODE_BLOOM+ service bit added and enforced for all peer versions as of v0.13.0 (PR #6579 and PR #6641).
+-    BIP112: The +CHECKSEQUENCEVERIFY+ opcode has been implemented since v0.12.1 (PR #7524), and has been buried since v0.19.0 (PR #16060).
+-    BIP113: Median time past lock-time calculations have been implemented since v0.12.1 (PR #6566), and has been buried since v0.19.0 (PR #16060).
+-    BIP125: Opt-in full replace-by-fee signaling partially implemented. 
+-    BIP130: direct headers announcement is negotiated with peer versions ≥70012 as of v0.12.0 (PR 6494).
+-    BIP133: feefilter messages are respected and sent for peer versions ≥70013 as of v0.13.0 (PR 7542).
+-    BIP141: Segregated Witness (Consensus Layer) as of v0.13.0 (PR 8149), defined for mainnet as of v0.13.1 (PR 8937), and buried since v0.19.0 (PR #16060).
+-    BIP143: Transaction Signature Verification for Version 0 Witness Program as of v0.13.0 (PR 8149), defined for mainnet as of v0.13.1 (PR 8937), and buried since v0.19.0 (PR #16060).
+-    BIP144: Segregated Witness as of 0.13.0 (PR 8149).
+-    BIP145: getblocktemplate updates for Segregated Witness as of v0.13.0 (PR 8149).
+-    BIP147: +NULLDUMMY+ soft fork as of v0.13.1 (PR 8636 and PR 8937), buried since v0.19.0 (PR #16060).
+-    BIP152: Compact block transfer and related optimizations are used as of v0.13.0 (PR 8068).
+-    BIP155: The 'addrv2' and 'sendaddrv2' messages which enable relay of Tor V3 addresses (and other networks) are supported as of v0.21.0 (PR 19954).
+-    BIP157 158: Compact Block Filters for Light Clients can be indexed as of v0.19.0 (PR #14121) and served to peers on the P2P network as of v0.21.0 (PR #16442).
+-    BIP159: The +NODE_NETWORK_LIMITED+ service bit is signalled as of v0.16.0 (PR 11740), and such nodes are connected to as of v0.17.0 (PR 10387).
+-    BIP173: Bech32 addresses for native Segregated Witness outputs are supported as of v0.16.0 (PR 11167). Bech32 addresses are generated by default as of v0.20.0 (PR 16884).
+-    BIP174: RPCs to operate on Partially Signed Bitcoin Transactions (PSBT) are present as of v0.17.0 (PR 13557).
+-    BIP176: Bits Denomination [QT only] is supported as of v0.16.0 (PR 12035).
+-    BIP325: Signet test network is supported as of v0.21.0 (PR 18267).
+-    BIP339: Relay of transactions by wtxid is supported as of v0.21.0 (PR 18044).
+-    BIP340 341 342: Validation rules for Taproot (including Schnorr signatures and Tapscript leaves) are implemented as of v0.21.0 (PR 19953), with mainnet activation as of v0.21.1 (PR 21377, PR 21686).
+-    BIP350: Addresses for native v1+ segregated Witness outputs use bech32m instead of bech32 as of v22.0 (PR 20861).
+-    BIP371: Taproot fields for PSBT as of v24.0 (PR 22558).
+-    BIP380 381 382 383 384 385: Output Script Descriptors, and most of Script Expressions are implemented as of v0.17.0 (PR 13697).
+-    BIP386: +tr()+ Output Script Descriptors are implemented as((("BIPs (Bitcoin Improvement Proposals)", "implemented by Bitcoin Core", startref="bips-implement")))((("Bitcoin Core", "BIPs implemented by", startref="bitcoin-core-bips"))) of v22.0 (PR 22051).

appdx-bips.asciidoc

@@ -1,116 +0,0 @@
-[[appdxbitcoinimpproposals]]
-[appendix]
-== Bitcoin Improvement Proposals
-
-((("bitcoin improvement proposals", "types of")))Bitcoin Improvement Proposals are design documents providing information to the bitcoin community, or for describing a new feature for bitcoin or its processes or environment.
-
-As per BIP-01 _BIP Purpose and Guidelines_, there are three kinds of BIPs:
-
-_Standard_ BIP:: Describes any change that affects most or all bitcoin implementations, such as a change to the network protocol, a change in block or transaction validity rules, or any change or addition that affects the interoperability of applications using bitcoin.
-_Informational_ BIP:: Describes a bitcoin design issue, or provides general guidelines or information to the bitcoin community, but does not propose a new feature. Informational BIPs do not necessarily represent a bitcoin community consensus or recommendation, so users and implementors may ignore informational BIPs or follow their advice.
-_Process_ BIP:: Describes a bitcoin process, or proposes a change to (or an event in) a process. Process BIPs are like standard BIPs but apply to areas other than the Bitcoin protocol itself. They might propose an implementation, but not to bitcoin's codebase; they often require community consensus; and unlike informational BIPs, they are more than recommendations, and users are typically not free to ignore them. Examples include procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in bitcoin development. Any meta-BIP is also considered a process BIP.
-
-((("bitcoin improvement proposals", "repository of")))BIPs are recorded in a versioned repository on GitHub: https://github.com/bitcoin/bips[https://github.com/bitcoin/bips]. <<table_d-1>> shows a snapshot of the BIPs in April 2017. Consult the authoritative repository for up-to-date information on existing BIPs and their contents.((("bitcoin improvement proposals", "snapshot of", id="BIPsnap15")))
-
-[[table_d-1]]
-.Snapshot of BIPs
-[options="header"]
-|=======================================================================
-|BIP# | Title |Owner |Type |Status
-|[[bip-1]]https://github.com/bitcoin/bips/blob/master/bip-0001.mediawiki[BIP-1] |BIP Purpose and Guidelines |Amir Taaki |Process |Replaced
-|[[bip-2]]https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki[BIP-2] |BIP process, revised |Luke Dashjr |Process |Active
-|[[bip-8]]https://github.com/bitcoin/bips/blob/master/bip-0008.mediawiki[BIP-8] |Version bits with guaranteed lock-in |Shaolin Fry |Informational |Draft
-|[[bip-9]]https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki[BIP-9] |Version bits with timeout and delay |Pieter Wuille, Peter Todd, Greg Maxwell, Rusty Russell |Informational |Final
-|[[bip-10]]https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki[BIP-10] |Multi-Sig Transaction Distribution |Alan Reiner |Informational |Withdrawn
-|[[bip-11]]https://github.com/bitcoin/bips/blob/master/bip-0011.mediawiki[BIP-11] |M-of-N Standard Transactions |Gavin Andresen |Standard |Final
-|[[bip-12]]https://github.com/bitcoin/bips/blob/master/bip-0012.mediawiki[BIP-12] |OP_EVAL |Gavin Andresen |Standard |Withdrawn
-|[[bip-13]]https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki[BIP-13] |Address Format for pay-to-script-hash |Gavin Andresen |Standard |Final
-|[[bip-14]]https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki[BIP-14] |Protocol Version and User Agent |Amir Taaki, Patrick Strateman |Standard |Final
-|[[bip-15]]https://github.com/bitcoin/bips/blob/master/bip-0015.mediawiki[BIP-15] |Aliases |Amir Taaki |Standard |Deferred
-|[[bip-16]]https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki[BIP-16] |Pay to Script Hash |Gavin Andresen |Standard |Final
-|[[bip-17]]https://github.com/bitcoin/bips/blob/master/bip-0017.mediawiki[BIP-17] |OP_CHECKHASHVERIFY (CHV) |Luke Dashjr |Standard |Withdrawn
-|[[bip-18]]https://github.com/bitcoin/bips/blob/master/bip-0018.mediawiki[BIP-18] |hashScriptCheck |Luke Dashjr |Standard |Proposed
-|[[bip-19]]https://github.com/bitcoin/bips/blob/master/bip-0019.mediawiki[BIP-19] |M-of-N Standard Transactions (Low SigOp) |Luke Dashjr |Standard |Draft
-|[[bip-20]]https://github.com/bitcoin/bips/blob/master/bip-0020.mediawiki[BIP-20] |URI Scheme |Luke Dashjr |Standard |Replaced
-|[[bip-21]]https://github.com/bitcoin/bips/blob/master/bip-0021.mediawiki[BIP-21] |URI Scheme |Nils Schneider, Matt Corallo |Standard |Final
-|[[bip-22]]https://github.com/bitcoin/bips/blob/master/bip-0022.mediawiki[BIP-22] |getblocktemplate - Fundamentals |Luke Dashjr |Standard |Final
-|[[bip-23]]https://github.com/bitcoin/bips/blob/master/bip-0023.mediawiki[BIP-23] |getblocktemplate - Pooled Mining |Luke Dashjr |Standard |Final
-|[[bip-30]]https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki[BIP-30] |Duplicate transactions |Pieter Wuille |Standard |Final
-|[[bip-31]]https://github.com/bitcoin/bips/blob/master/bip-0031.mediawiki[BIP-31] |Pong message |Mike Hearn |Standard |Final
-|[[bip-32]]https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki[BIP-32] |Hierarchical Deterministic Wallets |Pieter Wuille |Informational |Final
-|[[bip-33]]https://github.com/bitcoin/bips/blob/master/bip-0033.mediawiki[BIP-33] |Stratized Nodes |Amir Taaki |Standard |Draft
-|[[bip-34]]https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki[BIP-34] |Block v2, Height in Coinbase |Gavin Andresen |Standard |Final
-|[[bip-35]]https://github.com/bitcoin/bips/blob/master/bip-0035.mediawiki[BIP-35] |mempool message |Jeff Garzik |Standard |Final
-|[[bip-36]]https://github.com/bitcoin/bips/blob/master/bip-0036.mediawiki[BIP-36] |Custom Services |Stefan Thomas |Standard |Draft
-|[[bip-37]]https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki[BIP-37] |Connection Bloom filtering |Mike Hearn, Matt Corallo |Standard |Final
-|[[bip-39]]https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki[BIP-39] |Mnemonic code for generating deterministic keys |Marek Palatinus, Pavol Rusnak, Aaron Voisine, Sean Bowe |Standard |Proposed
-|[[bip-40]]https://github.com/bitcoin/bips/blob/master/bip-0040.mediawiki[BIP-40] |Stratum wire protocol |Marek Palatinus |Standard |BIP number allocated
-|[[bip-41]]https://github.com/bitcoin/bips/blob/master/bip-0041.mediawiki[BIP-41] |Stratum mining protocol |Marek Palatinus |Standard |BIP number allocated
-|[[bip-42]]https://github.com/bitcoin/bips/blob/master/bip-0042.mediawiki[BIP-42] |A finite monetary supply for Bitcoin |Pieter Wuille |Standard |Draft
-|[[bip-43]]https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki[BIP-43] |Purpose Field for Deterministic Wallets |Marek Palatinus, Pavol Rusnak |Informational |Draft
-|[[bip-44]]https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki[BIP-44] |Multi-Account Hierarchy for Deterministic Wallets |Marek Palatinus, Pavol Rusnak |Standard |Proposed
-|[[bip-45]]https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki[BIP-45] |Structure for Deterministic P2SH Multisignature Wallets |Manuel Araoz, Ryan X. Charles, Matias Alejo Garcia |Standard |Proposed
-|[[bip-47]]https://github.com/bitcoin/bips/blob/master/bip-0047.mediawiki[BIP-47] |Reusable Payment Codes for Hierarchical Deterministic Wallets |Justus Ranvier |Informational |Draft
-|[[bip-49]]https://github.com/bitcoin/bips/blob/master/bip-0049.mediawiki[BIP-49] |Derivation scheme for P2WPKH-nested-in-P2SH based accounts |Daniel Weigl |Informational |Draft
-|[[bip-50]]https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki[BIP-50] |March 2013 Chain Fork Post-Mortem |Gavin Andresen |Informational |Final
-|[[bip-60]]https://github.com/bitcoin/bips/blob/master/bip-0060.mediawiki[BIP-60] |Fixed Length "version" Message (Relay-Transactions Field) |Amir Taaki |Standard |Draft
-|[[bip-61]]https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki[BIP-61] |Reject P2P message |Gavin Andresen |Standard |Final
-|[[bip-62]]https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki[BIP-62] |Dealing with malleability |Pieter Wuille |Standard |Withdrawn
-|[[bip-63]]https://github.com/bitcoin/bips/blob/master/bip-0063.mediawiki[BIP-63] |Stealth Addresses |Peter Todd |Standard |BIP number allocated
-|[[bip-64]]https://github.com/bitcoin/bips/blob/master/bip-0064.mediawiki[BIP-64] |getutxo message |Mike Hearn |Standard |Draft
-|[[bip-65]]https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki[BIP-65] |OP_CHECKLOCKTIMEVERIFY |Peter Todd |Standard |Final
-|[[bip-66]]https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki[BIP-66] |Strict DER signatures |Pieter Wuille |Standard |Final
-|[[bip-67]]https://github.com/bitcoin/bips/blob/master/bip-0067.mediawiki[BIP-67] |Deterministic Pay-to-script-hash multi-signature addresses through public key sorting |Thomas Kerin, Jean-Pierre Rupp, Ruben de Vries |Standard |Proposed
-|[[bip-68]]https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki[BIP-68] |Relative lock-time using consensus-enforced sequence numbers |Mark Friedenbach, BtcDrak, Nicolas Dorier, kinoshitajona |Standard |Final
-|[[bip-69]]https://github.com/bitcoin/bips/blob/master/bip-0069.mediawiki[BIP-69] |Lexicographical Indexing of Transaction Inputs and Outputs |Kristov Atlas |Informational |Proposed
-|[[bip-70]]https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki[BIP-70] |Payment Protocol |Gavin Andresen, Mike Hearn |Standard |Final
-|[[bip-71]]https://github.com/bitcoin/bips/blob/master/bip-0071.mediawiki[BIP-71] |Payment Protocol MIME types |Gavin Andresen |Standard |Final
-|[[bip-72]]https://github.com/bitcoin/bips/blob/master/bip-0072.mediawiki[BIP-72] |bitcoin: uri extensions for Payment Protocol |Gavin Andresen |Standard |Final
-|[[bip-73]]https://github.com/bitcoin/bips/blob/master/bip-0073.mediawiki[BIP-73] |Use "Accept" header for response type negotiation with Payment Request URLs |Stephen Pair |Standard |Final
-|[[bip-74]]https://github.com/bitcoin/bips/blob/master/bip-0074.mediawiki[BIP-74] |Allow zero value OP_RETURN in Payment Protocol |Toby Padilla |Standard |Draft
-|[[bip-75]]https://github.com/bitcoin/bips/blob/master/bip-0075.mediawiki[BIP-75] |Out of Band Address Exchange using Payment Protocol Encryption |Justin Newton, Matt David, Aaron Voisine, James MacWhyte |Standard |Draft
-|[[bip-80]]https://github.com/bitcoin/bips/blob/master/bip-0080.mediawiki[BIP-80] |Hierarchy for Non-Colored Voting Pool Deterministic Multisig Wallets |Justus Ranvier, Jimmy Song |Informational |Deferred
-|[[bip-81]]https://github.com/bitcoin/bips/blob/master/bip-0081.mediawiki[BIP-81] |Hierarchy for Colored Voting Pool Deterministic Multisig Wallets |Justus Ranvier, Jimmy Song |Informational |Deferred
-|[[bip-83]]https://github.com/bitcoin/bips/blob/master/bip-0083.mediawiki[BIP-83] |Dynamic Hierarchical Deterministic Key Trees |Eric Lombrozo |Standard |Draft
-|[[bip-90]]https://github.com/bitcoin/bips/blob/master/bip-0090.mediawiki[BIP-90] |Buried Deployments |Suhas Daftuar |Informational |Draft
-|[[bip-99]]https://github.com/bitcoin/bips/blob/master/bip-0099.mediawiki[BIP-99] |Motivation and deployment of consensus rule changes ([soft/hard]forks) |Jorge Timón |Informational |Draft
-|[[bip-101]]https://github.com/bitcoin/bips/blob/master/bip-0101.mediawiki[BIP-101] |Increase maximum block size |Gavin Andresen |Standard |Withdrawn
-|[[bip-102]]https://github.com/bitcoin/bips/blob/master/bip-0102.mediawiki[BIP-102] |Block size increase to 2MB |Jeff Garzik |Standard |Draft
-|[[bip-103]]https://github.com/bitcoin/bips/blob/master/bip-0103.mediawiki[BIP-103] |Block size following technological growth |Pieter Wuille |Standard |Draft
-|[[bip-104]]https://github.com/bitcoin/bips/blob/master/bip-0104.mediawiki[BIP-104] |'Block75' - Max block size like difficulty |t.khan |Standard |Draft
-|[[bip-105]]https://github.com/bitcoin/bips/blob/master/bip-0105.mediawiki[BIP-105] |Consensus based block size retargeting algorithm |BtcDrak |Standard |Draft
-|[[bip-106]]https://github.com/bitcoin/bips/blob/master/bip-0106.mediawiki[BIP-106] |Dynamically Controlled Bitcoin Block Size Max Cap |Upal Chakraborty |Standard |Draft
-|[[bip-107]]https://github.com/bitcoin/bips/blob/master/bip-0107.mediawiki[BIP-107] |Dynamic limit on the block size |Washington Y. Sanchez |Standard |Draft
-|[[bip-109]]https://github.com/bitcoin/bips/blob/master/bip-0109.mediawiki[BIP-109] |Two million byte size limit with sigop and sighash limits |Gavin Andresen |Standard |Rejected
-|[[bip-111]]https://github.com/bitcoin/bips/blob/master/bip-0111.mediawiki[BIP-111] |NODE_BLOOM service bit |Matt Corallo, Peter Todd |Standard |Proposed
-|[[bip-112]]https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki[BIP-112] |CHECKSEQUENCEVERIFY |BtcDrak, Mark Friedenbach, Eric Lombrozo |Standard |Final
-|[[bip-113]]https://github.com/bitcoin/bips/blob/master/bip-0113.mediawiki[BIP-113] |Median time-past as endpoint for lock-time calculations |Thomas Kerin, Mark Friedenbach |Standard |Final
-|[[bip-114]]https://github.com/bitcoin/bips/blob/master/bip-0114.mediawiki[BIP-114] |Merkelized Abstract Syntax Tree |Johnson Lau |Standard |Draft
-|[[bip-120]]https://github.com/bitcoin/bips/blob/master/bip-0120.mediawiki[BIP-120] |Proof of Payment |Kalle Rosenbaum |Standard |Draft
-|[[bip-121]]https://github.com/bitcoin/bips/blob/master/bip-0121.mediawiki[BIP-121] |Proof of Payment URI scheme |Kalle Rosenbaum |Standard |Draft
-|[[bip-122]]https://github.com/bitcoin/bips/blob/master/bip-0122.mediawiki[BIP-122] |URI scheme for Blockchain references / exploration |Marco Pontello |Standard |Draft
-|[[bip-123]]https://github.com/bitcoin/bips/blob/master/bip-0123.mediawiki[BIP-123] |BIP Classification |Eric Lombrozo |Process |Active
-|[[bip-124]]https://github.com/bitcoin/bips/blob/master/bip-0124.mediawiki[BIP-124] |Hierarchical Deterministic Script Templates |Eric Lombrozo, William Swanson |Informational |Draft
-|[[bip-125]]https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki[BIP-125] |Opt-in Full Replace-by-Fee Signaling |David A. Harding, Peter Todd |Standard |Proposed
-|[[bip-126]]https://github.com/bitcoin/bips/blob/master/bip-0126.mediawiki[BIP-126] |Best Practices for Heterogeneous Input Script Transactions |Kristov Atlas |Informational |Draft
-|[[bip-130]]https://github.com/bitcoin/bips/blob/master/bip-0130.mediawiki[BIP-130] |sendheaders message |Suhas Daftuar |Standard |Proposed
-|[[bip-131]]https://github.com/bitcoin/bips/blob/master/bip-0131.mediawiki[BIP-131] |"Coalescing Transaction" Specification (wildcard inputs) |Chris Priest |Standard |Draft
-|[[bip-132]]https://github.com/bitcoin/bips/blob/master/bip-0132.mediawiki[BIP-132] |Committee-based BIP Acceptance Process |Andy Chase |Process |Withdrawn
-|[[bip-133]]https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki[BIP-133] |feefilter message |Alex Morcos |Standard |Draft
-|[[bip-134]]https://github.com/bitcoin/bips/blob/master/bip-0134.mediawiki[BIP-134] |Flexible Transactions |Tom Zander |Standard |Draft
-|[[bip-140]]https://github.com/bitcoin/bips/blob/master/bip-0140.mediawiki[BIP-140] |Normalized TXID |Christian Decker |Standard |Draft
-|[[bip-141]]https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki[BIP-141] |Segregated Witness (Consensus layer) |Eric Lombrozo, Johnson Lau, Pieter Wuille |Standard |Draft
-|[[bip-142]]https://github.com/bitcoin/bips/blob/master/bip-0142.mediawiki[BIP-142] |Address Format for Segregated Witness |Johnson Lau |Standard |Deferred
-|[[bip-143]]https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki[BIP-143] |Transaction Signature Verification for Version 0 Witness Program |Johnson Lau, Pieter Wuille |Standard |Draft
-|[[bip-144]]https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki[BIP-144] |Segregated Witness (Peer Services) |Eric Lombrozo, Pieter Wuille |Standard |Draft
-|[[bip-145]]https://github.com/bitcoin/bips/blob/master/bip-0145.mediawiki[BIP-145] |getblocktemplate Updates for Segregated Witness |Luke Dashjr |Standard |Draft
-|[[bip-146]]https://github.com/bitcoin/bips/blob/master/bip-0146.mediawiki[BIP-146] |Dealing with signature encoding malleability |Johnson Lau, Pieter Wuille |Standard |Draft
-|[[bip-147]]https://github.com/bitcoin/bips/blob/master/bip-0147.mediawiki[BIP-147] |Dealing with dummy stack element malleability |Johnson Lau |Standard |Draft
-|[[bip-148]]https://github.com/bitcoin/bips/blob/master/bip-0148.mediawiki[BIP-148] |Mandatory activation of segwit deployment |Shaolin Fry |Standard |Draft
-|[[bip-150]]https://github.com/bitcoin/bips/blob/master/bip-0150.mediawiki[BIP-150] |Peer Authentication |Jonas Schnelli |Standard |Draft
-|[[bip-151]]https://github.com/bitcoin/bips/blob/master/bip-0151.mediawiki[BIP-151] |Peer-to-Peer Communication Encryption |Jonas Schnelli |Standard |Draft
-|[[bip-152]]https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki[BIP-152] |Compact Block Relay |Matt Corallo |Standard |Draft
-|[[bip-171]]https://github.com/bitcoin/bips/blob/master/bip-0171.mediawiki[BIP-171] |Currency/exchange rate information API |Luke Dashjr |Standard |Draft
-|[[bip-180]]https://github.com/bitcoin/bips/blob/master/bip-0180.mediawiki[BIP-180] |Block size/weight fraud proof |Luke Dashjr |Standard |Draft
-|[[bip-199]]https://github.com/bitcoin/bips/blob/master/bip-0199.mediawiki[BIP-199] |Hashed Time-Locked Contract transactions |Sean Bowe, Daira Hopwood |Standard |Draft((("", startref="BIPsnap15"))) 
-|=======================================================================

appdx-bitcore.asciidoc

@@ -1,116 +0,0 @@
-[[appdx_bitcore]]
-[appendix]
-
-== Bitcore
-
-
-((("Bitcore", id="bitcore16")))Bitcore is a suite of tools provided by BitPay. Its goal is to provide easy-to-use tools for Bitcoin developers. Almost all of Bitcore's code is written in JavaScript. There are some modules written specifically for NodeJS. Finally, the "node" module of Bitcore includes Bitcoin Core's C++ code. Please see https://bitcore.io for more information.
-
-=== Bitcore's Feature List
-
-* Bitcoin full node (bitcore-node)
-* Block explorer (insight)
-* Block, transaction, and wallet utilities (bitcore-lib)
-* Communicating directly with Bitcoin's P2P network (bitcore-p2p)
-* Seed entropy mnemonic generation (bitcore-mnemonic)
-* Payment protocol (bitcore-payment-protocol)
-* Message verification and signing (bitcore-message)
-* Elliptic curve Integrated Encryption Scheme (bitcore-ecies)
-* Wallet service (bitcore-wallet-service)
-* Wallet client (bitcore-wallet-client)
-* Integrating services directly with Bitcoin Core (bitcore-node)
-
-=== Bitcore Library Examples
-
-==== Prerequisites
-
-* NodeJS >= 4.x
-
-If using NodeJS and the node REPL:
-
-[source,bash]
-----
-$ npm install -g bitcore-lib bitcore-p2p
-----
-
-==== Wallet Examples using bitcore-lib
-
-Creating a new Bitcoin address with associated private key:
-
-----
-> bitcore = require('bitcore-lib')
-> privateKey = new bitcore.PrivateKey()
-> address = privateKey.toAddress().toString()
-----
-
-Creating a hierarchical deterministic private key and address:
-
-----
-> hdPrivateKey = bitcore.HDPrivateKey()
-> hdPublicKey = bitcore.HDPublicKey(hdPrivateKey)
-> hdAddress = new bitcore.Address(hdPublicKey.publicKey).toString()
-----
-
-Creating and signing a transaction from an UTXO:
-
-----
-> utxo = {
-  txId: txId, // transaction id containing an unspent output
-  outputIndex: outputIndex, // output index (e.g. 0)
-  address: addressOfUtxo,
-  script: bitcore.Script.buildPublicKeyHashOut(addressOfUtxo).toString(),
-  satoshis: satoshis // amount sent to the address
-}
-> fee = 3000 //set appropriately for conditions on the network
-> tx = new bitcore.Transaction()
-        .from(utxo)
-        .to(address, 35000)
-        .fee(fee)
-        .enableRBF()
-        .sign(privateKeyOfUtxo)
-----
-
-Replace the last transaction in the mempool (replace-by-fee):
-
-----
-> rbfTx = new Transaction()
-           .from(utxo)
-           .to(address, 35000)
-           .fee(fee*2)
-           .enableRBF()
-           .sign(privateKeyOfUtxo);
-> tx.serialize();
-> rbfTx.serialize();
-----
-
-Broadcasting a transaction to the Bitcoin network
-(note: broadcast valid transactions only; refer to https://bitnodes.21.co/nodes[] for peer hosts):
-
-1. Copy the code below into a file called _broadcast.js_.
-2. The +tx+ and +rbfTx+ variables are the output of +tx.serialize()+ and +rbfTx.serialize()+, respectively.
-3. In order to replace-by-fee, the peer must support bitcoind option +mempoolreplace+ and have it set to +1+.
-4. Run the file node _broadcast.js_((("", startref="bitcore16"))):
-
-----
-var p2p = require('bitcore-p2p');
-var bitcore = require('bitcore-lib');
-var tx = new bitcore.Transaction('output from serialize function');
-var rbfTx = new bitcore.Transaction('output from serialize function');
-var host = 'ip address'; //use valid peer listening on tcp 8333
-var peer = new p2p.Peer({host: host});
-var messages = new p2p.Messages();
-peer.on('ready', function() {
-  var txs = [messages.Transaction(tx), messages.Transaction(rbfTx)];
-  var index = 0;
-  var interval = setInterval(function() {
-    peer.sendMessage(txs[index++]);
-    console.log('tx: ' + index + ' sent');
-    if (index === txs.length) {
-      clearInterval(interval);
-      console.log('disconnecting from peer: ' + host);
-      peer.disconnect();
-    }
-  }, 2000);
-});
-peer.connect();
-----

appdx-bx.asciidoc

@@ -1,181 +0,0 @@
-[[appdx_bx]]
-[appendix]
-
-== Bitcoin Explorer (bx) Commands
-
-((("Bitcoin Explorer (bx) commands", id="BX18_1", range="startofrange")))Bitcoin Explorer (bx) is a command-line tool that offers a variety of commands for key management and transaction construction. It is part of the libbitcoin bitcoin library.
-
-----
-Usage: bx COMMAND [--help]
-
-Info: The bx commands are:
-
-address-decode
-address-embed
-address-encode
-address-validate
-base16-decode
-base16-encode
-base58-decode
-base58-encode
-base58check-decode
-base58check-encode
-base64-decode
-base64-encode
-bitcoin160
-bitcoin256
-btc-to-satoshi
-ec-add
-ec-add-secrets
-ec-multiply
-ec-multiply-secrets
-ec-new
-ec-to-address
-ec-to-public
-ec-to-wif
-fetch-balance
-fetch-header
-fetch-height
-fetch-history
-fetch-stealth
-fetch-tx
-fetch-tx-index
-hd-new
-hd-private
-hd-public
-hd-to-address
-hd-to-ec
-hd-to-public
-hd-to-wif
-help
-input-set
-input-sign
-input-validate
-message-sign
-message-validate
-mnemonic-decode
-mnemonic-encode
-ripemd160
-satoshi-to-btc
-script-decode
-script-encode
-script-to-address
-seed
-send-tx
-send-tx-node
-send-tx-p2p
-settings
-sha160
-sha256
-sha512
-stealth-decode
-stealth-encode
-stealth-public
-stealth-secret
-stealth-shared
-tx-decode
-tx-encode
-uri-decode
-uri-encode
-validate-tx
-watch-address
-wif-to-ec
-wif-to-public
-wrap-decode
-wrap-encode
-----
-
-For more information, see the https://github.com/libbitcoin/libbitcoin-explorer[Bitcoin Explorer homepage] and https://github.com/libbitcoin/libbitcoin-explorer/wiki[Bitcoin Explorer user documentation].
-
-=== Examples of bx Command Use
-
-Let's look at some examples of using Bitcoin Explorer commands to experiment with keys and addresses.
-
-Generate a random "seed" value using the +seed+ command, which uses the operating system's random number generator. Pass the seed to the +ec-new+ command to generate a new private key. We save the standard output into the file _private_key_:
-
-----
-$ bx seed | bx ec-new > private_key
-$ cat private_key
-73096ed11ab9f1db6135857958ece7d73ea7c30862145bcc4bbc7649075de474
-----
-
-Now, generate the public key from that private key using the +ec-to-public+ command. We pass the _private_key_ file into the standard input and save the standard output of the command into a new file _public_key_:
-
-----
-$ bx ec-to-public < private_key > public_key
-$ cat public_key
-02fca46a6006a62dfdd2dbb2149359d0d97a04f430f12a7626dd409256c12be500
-----
-
-We can reformat the +public_key+ as an address using the +ec-to-address+ command. We pass the _public_key_ into standard input:
-
-----
-$ bx ec-to-address < public_key
-17re1S4Q8ZHyCP8Kw7xQad1Lr6XUzWUnkG
-----
-
-Keys generated in this manner produce a type-0 nondeterministic wallet. That means that each key is generated from an independent seed. Bitcoin Explorer commands can also generate keys deterministically, in accordance with BIP-32. In this case, a "master" key is created from a seed and then extended deterministically to produce a tree of subkeys, resulting in a type-2 deterministic wallet.
-
-First, we use the +seed+ and +hd-new+ commands to generate a master key that will be used as the basis to derive a hierarchy of keys:
-
-----
-$ bx seed > seed
-$ cat seed
-eb68ee9f3df6bd4441a9feadec179ff1
-
-$ bx hd-new < seed > master
-$ cat master
-xprv9s21ZrQH143K2BEhMYpNQoUvAgiEjArAVaZaCTgsaGe6LsAnwubeiTcDzd23mAoyizm9cApe51gNfLMkBqkYoWWMCRwzfuJk8RwF1SVEpAQ
-----
-
-We now use the +hd-private+ command to generate a hardened "account" key and a sequence of two private keys within the account:
-
-----
-$ bx hd-private --hard < master > account
-$ cat account
-xprv9vkDLt81dTKjwHB8fsVB5QK8cGnzveChzSrtCfvu3aMWvQaThp59ueufuyQ8Qi3qpjk4aKsbmbfxwcgS8PYbgoR2NWHeLyvg4DhoEE68A1n
-
-$ bx hd-private --index 0 < account
-xprv9xHfb6w1vX9xgZyPNXVgAhPxSsEkeRcPHEUV5iJcVEsuUEACvR3NRY3fpGhcnBiDbvG4LgndirDsia1e9F3DWPkX7Tp1V1u97HKG1FJwUpU
-
-$ bx hd-private --index 1 < account
-xprv9xHfb6w1vX9xjc8XbN4GN86jzNAZ6xHEqYxzbLB4fzHFd6VqCLPGRZFsdjsuMVERadbgDbziCRJru9n6tzEWrASVpEdrZrFidt1RDfn4yA3
-----
-
-Next, we use the +hd-public+ command to generate the corresponding sequence of two public keys:
-
-----
-$ bx hd-public --index 0 < account
-xpub6BH1zcTuktiFu43rUZ2gXqLgzu5F3tLEeTQ5t6iE3aQtM2VMTxMcyLN9fYHiGhGpQe9QQYmqL2eYPFJ3vezHz5wzaSW4FiGrseNDR4LKqTy
-
-$ bx hd-public --index 1 < account
-xpub6BH1zcTuktiFx6CzhPbGjG3UYQ13WR16CmtbPiagEKpEVtpyjshWyMaMV1cn7nUPUkgQHPVXJVqsrA8xWbGQDhohEcDFTEYMvYzwRD7Juf8
-----
-
-The public keys can also be derived from their corresponding private keys using the +hd-to-public+ command:
-
-----
-$ bx hd-private --index 0 < account | bx hd-to-public
-xpub6BH1zcTuktiFu43rUZ2gXqLgzu5F3tLEeTQ5t6iE3aQtM2VMTxMcyLN9fYHiGhGpQe9QQYmqL2eYPFJ3vezHz5wzaSW4FiGrseNDR4LKqTy
-
-$ bx hd-private --index 1 < account | bx hd-to-public
-xpub6BH1zcTuktiFx6CzhPbGjG3UYQ13WR16CmtbPiagEKpEVtpyjshWyMaMV1cn7nUPUkgQHPVXJVqsrA8xWbGQDhohEcDFTEYMvYzwRD7Juf8
-----
-
-We can generate a practically limitless number of keys in a deterministic chain, all derived from a single seed. This technique is used in many wallet applications to generate keys that can be backed up and restored with a single seed value. This is easier than having to back up the wallet with all its randomly generated keys every time a new key is created.
-
-The seed can be encoded using the +mnemonic-encode+ command:
-
-----
-$ bx hd-mnemonic < seed > words
-adore repeat vision worst especially veil inch woman cast recall dwell appreciate
-----
-
-The seed can then be decoded using the +mnemonic-decode+ command:
-
-----
-$ bx mnemonic-decode < words
-eb68ee9f3df6bd4441a9feadec179ff1
-----
-
-Mnemonic encoding can make the seed easier to record and even remember.(((range="endofrange", startref="BX18_1")))

appdx-pycoin.asciidoc

@@ -1,424 +0,0 @@
-[[appdx-pycoin]]
-[appendix]
-== pycoin, ku, and tx
-
-
-((("pycoin library")))The Python library https://github.com/richardkiss/pycoin[+pycoin+], originally written and maintained by Richard Kiss, is a Python-based library that supports manipulation of bitcoin keys and transactions, even supporting the scripting language enough to properly deal with nonstandard transactions.
-
-The pycoin library supports both Python 2 (2.7.x) and Python 3 (3.3 and later) and comes with some handy command-line utilities, +ku+ and +tx+.
-
-=== Key Utility (KU)
-
-((("key utility (ku)", id="keyutil17")))The command-line utility +ku+ ("key utility") is a Swiss Army knife for manipulating keys. It supports BIP-32 keys, WIF, and addresses (bitcoin and alt coins). Following are some examples.
-
-Create a BIP-32 key using the default entropy sources of GPG and _/dev/random_:
-
-
-----
-$ ku create
-
-input           : create
-network         : Bitcoin
-wallet key      : xprv9s21ZrQH143K3LU5ctPZTBnb9kTjA5Su9DcWHvXJemiJBsY7VqXUG7hipgdWaU
-                    m2nhnzdvxJf5KJo9vjP2nABX65c5sFsWsV8oXcbpehtJi
-public version  : xpub661MyMwAqRbcFpYYiuvZpKjKhnJDZYAkWSY76JvvD7FH4fsG3Nqiov2CfxzxY8
-                    DGcpfT56AMFeo8M8KPkFMfLUtvwjwb6WPv8rY65L2q8Hz
-tree depth      : 0
-fingerprint     : 9d9c6092
-parent f'print  : 00000000
-child index     : 0
-chain code      : 80574fb260edaa4905bc86c9a47d30c697c50047ed466c0d4a5167f6821e8f3c
-private key     : yes
-secret exponent : 112471538590155650688604752840386134637231974546906847202389294096567806844862
- hex            : f8a8a28b28a916e1043cc0aca52033a18a13cab1638d544006469bc171fddfbe
-wif             : L5Z54xi6qJusQT42JHA44mfPVZGjyb4XBRWfxAzUWwRiGx1kV4sP
- uncompressed   : 5KhoEavGNNH4GHKoy2Ptu4KfdNp4r56L5B5un8FP6RZnbsz5Nmb
-public pair x   : 76460638240546478364843397478278468101877117767873462127021560368290114016034
-public pair y   : 59807879657469774102040120298272207730921291736633247737077406753676825777701
- x as hex       : a90b3008792432060fa04365941e09a8e4adf928bdbdb9dad41131274e379322
- y as hex       : 843a0f6ed9c0eb1962c74533795406914fe3f1957c5238951f4fe245a4fcd625
-y parity        : odd
-key pair as sec : 03a90b3008792432060fa04365941e09a8e4adf928bdbdb9dad41131274e379322
- uncompressed   : 04a90b3008792432060fa04365941e09a8e4adf928bdbdb9dad41131274e379322
-                    843a0f6ed9c0eb1962c74533795406914fe3f1957c5238951f4fe245a4fcd625
-hash160         : 9d9c609247174ae323acfc96c852753fe3c8819d
- uncompressed   : 8870d869800c9b91ce1eb460f4c60540f87c15d7
-Bitcoin address : 1FNNRQ5fSv1wBi5gyfVBs2rkNheMGt86sp
- uncompressed   : 1DSS5isnH4FsVaLVjeVXewVSpfqktdiQAM
-----
-
-Create a BIP-32 key from a passphrase:
-
-[WARNING]
-====
-The passphrase in this example is way too easy to guess.
-====
-
-----
-$ ku P:foo
-
-input           : P:foo
-network         : Bitcoin
-wallet key      : xprv9s21ZrQH143K31AgNK5pyVvW23gHnkBq2wh5aEk6g1s496M8ZMjxncCKZKgb5j
-                    ZoY5eSJMJ2Vbyvi2hbmQnCuHBujZ2WXGTux1X2k9Krdtq
-public version  : xpub661MyMwAqRbcFVF9ULcqLdsEa5WnCCugQAcgNd9iEMQ31tgH6u4DLQWoQayvtS
-                    VYFvXz2vPPpbXE1qpjoUFidhjFj82pVShWu9curWmb2zy
-tree depth      : 0
-fingerprint     : 5d353a2e
-parent f'print  : 00000000
-child index     : 0
-chain code      : 5eeb1023fd6dd1ae52a005ce0e73420821e1d90e08be980a85e9111fd7646bbc
-private key     : yes
-secret exponent : 65825730547097305716057160437970790220123864299761908948746835886007793998275
- hex            : 91880b0e3017ba586b735fe7d04f1790f3c46b818a2151fb2def5f14dd2fd9c3
-wif             : L26c3H6jEPVSqAr1usXUp9qtQJw6NHgApq6Ls4ncyqtsvcq2MwKH
- uncompressed   : 5JvNzA5vXDoKYJdw8SwwLHxUxaWvn9mDea6k1vRPCX7KLUVWa7W
-public pair x   : 81821982719381104061777349269130419024493616650993589394553404347774393168191
-public pair y   : 58994218069605424278320703250689780154785099509277691723126325051200459038290
- x as hex       : b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f
- y as hex       : 826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52
-y parity        : even
-key pair as sec : 02b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f
- uncompressed   : 04b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f
-                    826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52
-hash160         : 5d353a2ecdb262477172852d57a3f11de0c19286
- uncompressed   : e5bd3a7e6cb62b4c820e51200fb1c148d79e67da
-Bitcoin address : 19Vqc8uLTfUonmxUEZac7fz1M5c5ZZbAii
- uncompressed   : 1MwkRkogzBRMehBntgcq2aJhXCXStJTXHT
-----
-
-
-Get info as JSON:
-
-
-----
-$ ku P:foo -P -j
-----
-[source,json]
-----
-{
-   "y_parity": "even",
-   "public_pair_y_hex": "826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52",
-   "private_key": "no",
-   "parent_fingerprint": "00000000",
-   "tree_depth": "0",
-   "network": "Bitcoin",
-   "btc_address_uncompressed": "1MwkRkogzBRMehBntgcq2aJhXCXStJTXHT",
-   "key_pair_as_sec_uncompressed": "04b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52",
-   "public_pair_x_hex": "b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f",
-   "wallet_key": "xpub661MyMwAqRbcFVF9ULcqLdsEa5WnCCugQAcgNd9iEMQ31tgH6u4DLQWoQayvtSVYFvXz2vPPpbXE1qpjoUFidhjFj82pVShWu9curWmb2zy",
-   "chain_code": "5eeb1023fd6dd1ae52a005ce0e73420821e1d90e08be980a85e9111fd7646bbc",
-   "child_index": "0",
-   "hash160_uncompressed": "e5bd3a7e6cb62b4c820e51200fb1c148d79e67da",
-   "btc_address": "19Vqc8uLTfUonmxUEZac7fz1M5c5ZZbAii",
-   "fingerprint": "5d353a2e",
-   "hash160": "5d353a2ecdb262477172852d57a3f11de0c19286",
-   "input": "P:foo",
-   "public_pair_x": "81821982719381104061777349269130419024493616650993589394553404347774393168191",
-   "public_pair_y": "58994218069605424278320703250689780154785099509277691723126325051200459038290",
-   "key_pair_as_sec": "02b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f"
-}
-----
-
-Public BIP32 key:
-
-----
-$ ku -w -P P:foo
-xpub661MyMwAqRbcFVF9ULcqLdsEa5WnCCugQAcgNd9iEMQ31tgH6u4DLQWoQayvtSVYFvXz2vPPpbXE1qpjoUFidhjFj82pVShWu9curWmb2zy
-----
-
-Generate a subkey:
-
-----
-$ ku -w -s3/2 P:foo
-xprv9wTErTSkjVyJa1v4cUTFMFkWMe5eu8ErbQcs9xajnsUzCBT7ykHAwdrxvG3g3f6BFk7ms5hHBvmbdutNmyg6iogWKxx6mefEw4M8EroLgKj
-----
-
-Hardened subkey:
-
-----
-$ ku -w -s3/2H P:foo
-xprv9wTErTSu5AWGkDeUPmqBcbZWX1xq85ZNX9iQRQW9DXwygFp7iRGJo79dsVctcsCHsnZ3XU3DhsuaGZbDh8iDkBN45k67UKsJUXM1JfRCdn1
-----
-
-WIF:
-
-----
-$ ku -W P:foo
-L26c3H6jEPVSqAr1usXUp9qtQJw6NHgApq6Ls4ncyqtsvcq2MwKH
-----
-
-Address:
-
-
-----
-$ ku -a P:foo
-19Vqc8uLTfUonmxUEZac7fz1M5c5ZZbAii
-----
-
-
-
-Generate a bunch of subkeys:
-
-
-----
-$ ku P:foo -s 0/0-5 -w
-xprv9xWkBDfyBXmZjBG9EiXBpy67KK72fphUp9utJokEBFtjsjiuKUUDF5V3TU8U8cDzytqYnSekc8bYuJS8G3bhXxKWB89Ggn2dzLcoJsuEdRK
-xprv9xWkBDfyBXmZnzKf3bAGifK593gT7WJZPnYAmvc77gUQVej5QHckc5Adtwxa28ACmANi9XhCrRvtFqQcUxt8rUgFz3souMiDdWxJDZnQxzx
-xprv9xWkBDfyBXmZqdXA8y4SWqfBdy71gSW9sjx9JpCiJEiBwSMQyRxan6srXUPBtj3PTxQFkZJAiwoUpmvtrxKZu4zfsnr3pqyy2vthpkwuoVq
-xprv9xWkBDfyBXmZsA85GyWj9uYPyoQv826YAadKWMaaEosNrFBKgj2TqWuiWY3zuqxYGpHfv9cnGj5P7e8EskpzKL1Y8Gk9aX6QbryA5raK73p
-xprv9xWkBDfyBXmZv2q3N66hhZ8DAcEnQDnXML1J62krJAcf7Xb1HJwuW2VMJQrCofY2jtFXdiEY8UsRNJfqK6DAdyZXoMvtaLHyWQx3FS4A9zw
-xprv9xWkBDfyBXmZw4jEYXUHYc9fT25k9irP87n2RqfJ5bqbjKdT84Mm7Wtc2xmzFuKg7iYf7XFHKkSsaYKWKJbR54bnyAD9GzjUYbAYTtN4ruo
-----
-
-[role="pagebreak-before"]
-Generate the corresponding addresses:
-
-
-----
-$ ku P:foo -s 0/0-5 -a
-1MrjE78H1R1rqdFrmkjdHnPUdLCJALbv3x
-1AnYyVEcuqeoVzH96zj1eYKwoWfwte2pxu
-1GXr1kZfxE1FcK6ZRD5sqqqs5YfvuzA1Lb
-116AXZc4bDVQrqmcinzu4aaPdrYqvuiBEK
-1Cz2rTLjRM6pMnxPNrRKp9ZSvRtj5dDUML
-1WstdwPnU6HEUPme1DQayN9nm6j7nDVEM
-----
-
-
-Generate the corresponding WIFs:
-
-
-----
-$ ku P:foo -s 0/0-5 -W
-L5a4iE5k9gcJKGqX3FWmxzBYQc29PvZ6pgBaePLVqT5YByEnBomx
-Kyjgne6GZwPGB6G6kJEhoPbmyjMP7D5d3zRbHVjwcq4iQXD9QqKQ
-L4B3ygQxK6zH2NQGxLDee2H9v4Lvwg14cLJW7QwWPzCtKHdWMaQz
-L2L2PZdorybUqkPjrmhem4Ax5EJvP7ijmxbNoQKnmTDMrqemY8UF
-L2oD6vA4TUyqPF8QG4vhUFSgwCyuuvFZ3v8SKHYFDwkbM765Nrfd
-KzChTbc3kZFxUSJ3Kt54cxsogeFAD9CCM4zGB22si8nfKcThQn8C
-----
-
-
-
-Check that it works by choosing a BIP32 string (the one corresponding to subkey 0/3):
-
-
-
-----
-$ ku -W xprv9xWkBDfyBXmZsA85GyWj9uYPyoQv826YAadKWMaaEosNrFBKgj2TqWuiWY3zuqxYGpHfv9cnGj5P7e8EskpzKL1Y8Gk9aX6QbryA5raK73p
-L2L2PZdorybUqkPjrmhem4Ax5EJvP7ijmxbNoQKnmTDMrqemY8UF
-$ ku -a xprv9xWkBDfyBXmZsA85GyWj9uYPyoQv826YAadKWMaaEosNrFBKgj2TqWuiWY3zuqxYGpHfv9cnGj5P7e8EskpzKL1Y8Gk9aX6QbryA5raK73p
-116AXZc4bDVQrqmcinzu4aaPdrYqvuiBEK
-----
-
-
-Yep, looks familiar.
-
-From secret exponent:
-
-
-----
-$ ku 1
-
-input           : 1
-network         : Bitcoin
-secret exponent : 1
- hex            : 1
-wif             : KwDiBf89QgGbjEhKnhXJuH7LrciVrZi3qYjgd9M7rFU73sVHnoWn
- uncompressed   : 5HpHagT65TZzG1PH3CSu63k8DbpvD8s5ip4nEB3kEsreAnchuDf
-public pair x   : 55066263022277343669578718895168534326250603453777594175500187360389116729240
-public pair y   : 32670510020758816978083085130507043184471273380659243275938904335757337482424
- x as hex       : 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
- y as hex       : 483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
-y parity        : even
-key pair as sec : 0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
- uncompressed   : 0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
-                    483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
-hash160         : 751e76e8199196d454941c45d1b3a323f1433bd6
- uncompressed   : 91b24bf9f5288532960ac687abb035127b1d28a5
-Bitcoin address : 1BgGZ9tcN4rm9KBzDn7KprQz87SZ26SAMH
- uncompressed   : 1EHNa6Q4Jz2uvNExL497mE43ikXhwF6kZm
-----
-
-
-Litecoin version:
-
-
-----
-$ ku -nL 1
-
-input            : 1
-network          : Litecoin
-secret exponent  : 1
- hex             : 1
-wif              : T33ydQRKp4FCW5LCLLUB7deioUMoveiwekdwUwyfRDeGZm76aUjV
- uncompressed    : 6u823ozcyt2rjPH8Z2ErsSXJB5PPQwK7VVTwwN4mxLBFrao69XQ
-public pair x    : 55066263022277343669578718895168534326250603453777594175500187360389116729240
-public pair y    : 32670510020758816978083085130507043184471273380659243275938904335757337482424
- x as hex        : 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
- y as hex        : 483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
-y parity         : even
-key pair as sec  : 0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
- uncompressed    : 0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
-                     483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
-hash160          : 751e76e8199196d454941c45d1b3a323f1433bd6
- uncompressed    : 91b24bf9f5288532960ac687abb035127b1d28a5
-Litecoin address : LVuDpNCSSj6pQ7t9Pv6d6sUkLKoqDEVUnJ
- uncompressed    : LYWKqJhtPeGyBAw7WC8R3F7ovxtzAiubdM
-----
-
-
-Dogecoin WIF:
-
-
-----
-$ ku -nD -W 1
-QNcdLVw8fHkixm6NNyN6nVwxKek4u7qrioRbQmjxac5TVoTtZuot
-----
-
-
-From public pair (on Testnet):
-
-
-----
-$ ku -nT 55066263022277343669578718895168534326250603453777594175500187360389116729240,even
-
-input                   : 550662630222773436695787188951685343262506034537775941755001873603
-                            89116729240,even
-network                 : Bitcoin testnet
-public pair x           : 55066263022277343669578718895168534326250603453777594175500187360389116729240
-public pair y           : 32670510020758816978083085130507043184471273380659243275938904335757337482424
- x as hex               : 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
- y as hex               : 483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
-y parity                : even
-key pair as sec         : 0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
- uncompressed           : 0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
-                            483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
-hash160                 : 751e76e8199196d454941c45d1b3a323f1433bd6
- uncompressed           : 91b24bf9f5288532960ac687abb035127b1d28a5
-Bitcoin testnet address : mrCDrCybB6J1vRfbwM5hemdJz73FwDBC8r
- uncompressed           : mtoKs9V381UAhUia3d7Vb9GNak8Qvmcsme
-----
-
-
-From hash160:
-
-
-----
-$ ku 751e76e8199196d454941c45d1b3a323f1433bd6
-
-input           : 751e76e8199196d454941c45d1b3a323f1433bd6
-network         : Bitcoin
-hash160         : 751e76e8199196d454941c45d1b3a323f1433bd6
-Bitcoin address : 1BgGZ9tcN4rm9KBzDn7KprQz87SZ26SAMH
-----
-
-
-((("", startref="pycoin library")))As a Dogecoin address:
-
-
-----
-$ ku -nD 751e76e8199196d454941c45d1b3a323f1433bd6
-
-input            : 751e76e8199196d454941c45d1b3a323f1433bd6
-network          : Dogecoin
-hash160          : 751e76e8199196d454941c45d1b3a323f1433bd6
-Dogecoin address : DFpN6QqFfUm3gKNaxN6tNcab1FArL9cZLE
-----
-
-==== Transaction Utility (TX)
-
-((("transaction utility (TX)", id="TX17")))
-The command-line utility +tx+ will display transactions in human-readable form, fetch base transactions from pycoin's transaction cache or from web services (blockchain.info, blockcypher.com, blockr.io and chain.so are currently supported), merge transactions, add or delete inputs or outputs, and sign transactions.
-
-Following are some examples.
-
-
-View the famous "pizza" transaction:
-
-
-----
-$ tx 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
-warning: consider setting environment variable PYCOIN_CACHE_DIR=~/.pycoin_cache to cache transactions fetched via web services
-warning: no service providers found for get_tx; consider setting environment variable PYCOIN_BTC_PROVIDERS
-usage: tx [-h] [-t TRANSACTION_VERSION] [-l LOCK_TIME] [-n NETWORK] [-a]
-          [-i address] [-f path-to-private-keys] [-g GPG_ARGUMENT]
-          [--remove-tx-in tx_in_index_to_delete]
-          [--remove-tx-out tx_out_index_to_delete] [-F transaction-fee] [-u]
-          [-b BITCOIND_URL] [-o path-to-output-file]
-          argument [argument ...]
-tx: error: can't find Tx with id 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
-----
-
-
-Oops! We don't have web services set up. Let's do that now:
-
-
-[source,bash]
-----
-$ PYCOIN_CACHE_DIR=~/.pycoin_cache
-$ PYCOIN_BTC_PROVIDERS="block.io blockchain.info blockexplorer.com"
-$ export PYCOIN_CACHE_DIR PYCOIN_BTC_PROVIDERS
-----
-
-
-It's not done automatically so a command-line tool won't leak potentially private information about what transactions you're interested in to a third-party website. If you don't care, you could put these lines into your _.profile_.
-
-Let's try again:
-
-----
-$ tx 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
-Version:  1  tx hash 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a  159 bytes
-TxIn count: 1; TxOut count: 1
-Lock time: 0 (valid anytime)
-Input:
-  0:                          (unknown) from 1e133f7de73ac7d074e2746a3d6717dfc99ecaa8e9f9fade2cb8b0b20a5e0441:0
-Output:
-  0: 1CZDM6oTttND6WPdt3D6bydo7DYKzd9Qik receives 10000000.00000 mBTC
-Total output 10000000.00000 mBTC
-including unspents in hex dump since transaction not fully signed
-010000000141045e0ab2b0b82cdefaf9e9a8ca9ec9df17673d6a74e274d0c73ae77d3f131e000000004a493046022100a7f26eda874931999c90f87f01ff1ffc76bcd058fe16137e0e63fdb6a35c2d78022100a61e9199238eb73f07c8f209504c84b80f03e30ed8169edd44f80ed17ddf451901ffffffff010010a5d4e80000001976a9147ec1003336542cae8bded8909cdd6b5e48ba0ab688ac00000000
-
-** can't validate transaction as source transactions missing
-----
-
-The final line appears because to validate the transactions' signatures, you technically need the source transactions. So let's add +-a+ to augment the transactions with source information:
-
-----
-$ tx -a 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
-warning: transaction fees recommendations casually calculated and estimates may be incorrect
-warning: transaction fee lower than (casually calculated) expected value of 0.1 mBTC, transaction might not propagate
-Version:  1  tx hash 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a  159 bytes
-TxIn count: 1; TxOut count: 1
-Lock time: 0 (valid anytime)
-Input:
-  0: 17WFx2GQZUmh6Up2NDNCEDk3deYomdNCfk from 1e133f7de73ac7d074e2746a3d6717dfc99ecaa8e9f9fade2cb8b0b20a5e0441:0 10000000.00000 mBTC  sig ok
-Output:
-  0: 1CZDM6oTttND6WPdt3D6bydo7DYKzd9Qik receives 10000000.00000 mBTC
-Total input  10000000.00000 mBTC
-Total output 10000000.00000 mBTC
-Total fees        0.00000 mBTC
-
-010000000141045e0ab2b0b82cdefaf9e9a8ca9ec9df17673d6a74e274d0c73ae77d3f131e000000004a493046022100a7f26eda874931999c90f87f01ff1ffc76bcd058fe16137e0e63fdb6a35c2d78022100a61e9199238eb73f07c8f209504c84b80f03e30ed8169edd44f80ed17ddf451901ffffffff010010a5d4e80000001976a9147ec1003336542cae8bded8909cdd6b5e48ba0ab688ac00000000
-
-all incoming transaction values validated
-----
-
-((("", startref="TX17")))Now, let's look at unspent outputs for a specific address (UTXO). In block #1, we see a coinbase transaction to +12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX+. Let's use +fetch_unspent+ to find all coins in this address:
-
-----
-$ fetch_unspent 12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX
-a3a6f902a51a2cbebede144e48a88c05e608c2cce28024041a5b9874013a1e2a/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/333000
-cea36d008badf5c7866894b191d3239de9582d89b6b452b596f1f1b76347f8cb/31/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/10000
-065ef6b1463f552f675622a5d1fd2c08d6324b4402049f68e767a719e2049e8d/86/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/10000
-a66dddd42f9f2491d3c336ce5527d45cc5c2163aaed3158f81dc054447f447a2/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/10000
-ffd901679de65d4398de90cefe68d2c3ef073c41f7e8dbec2fb5cd75fe71dfe7/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/100
-d658ab87cc053b8dbcfd4aa2717fd23cc3edfe90ec75351fadd6a0f7993b461d/5/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/911
-36ebe0ca3237002acb12e1474a3859bde0ac84b419ec4ae373e63363ebef731c/1/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/100000
-fd87f9adebb17f4ebb1673da76ff48ad29e64b7afa02fda0f2c14e43d220fe24/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/1
-dfdf0b375a987f17056e5e919ee6eadd87dad36c09c4016d4a03cea15e5c05e3/1/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/1337
-cb2679bfd0a557b2dc0d8a6116822f3fcbe281ca3f3e18d3855aa7ea378fa373/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/1337
-d6be34ccf6edddc3cf69842dce99fe503bf632ba2c2adb0f95c63f6706ae0c52/1/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/2000000
-    0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098/0/410496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858eeac/5000000000
-----

appdx-scriptops.asciidoc

@@ -1,200 +0,0 @@
-[[tx_script_ops]]
-[appendix]
-== Transaction Script Language Operators, Constants, and Symbols
-
-[NOTE]
-====
-Tables and descriptions sourced from https://en.bitcoin.it/wiki/Script[].
-====
-
-<<tx_script_ops_table_pushdata>> shows operators for pushing values onto the stack.((("transactions", "scripts and Script language", id="TRlang14")))((("scripting", "Script language operators, constants, and symbols", id="SCRlang14")))
-
-[[tx_script_ops_table_pushdata]]
-.Push value onto stack
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_0 or OP_FALSE | 0x00 | An empty array is pushed onto the stack
-| 1 to 75 | 0x01 to 0x4b | Push the next N bytes onto the stack, where N is 1 to 75 bytes
-| OP_PUSHDATA1 | 0x4c | The next script byte contains N, push the following N bytes onto the stack
-| OP_PUSHDATA2 | 0x4d | The next two script bytes contain N, push the following N bytes onto the stack
-| OP_PUSHDATA4 | 0x4e | The next four script bytes contain N, push the following N bytes onto the stack
-| OP_1NEGATE | 0x4f | Push the value "–1" onto the stack
-| OP_RESERVED | 0x50 | Halt&#x2014;Invalid transaction unless found in an unexecuted OP_IF clause
-| OP_1 or OP_TRUE| 0x51 | Push the value "1" onto the stack
-| OP_2 to OP_16 | 0x52 to 0x60 |  For OP_N, push the value "N" onto the stack, e.g., OP_2 pushes "2"
-|=======
-
-[role="pagebreak-before"]
-<<tx_script_ops_table_control>> shows conditional flow control operators.
-
-[[tx_script_ops_table_control]]
-.Conditional flow control
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_NOP | 0x61 | Do nothing
-| OP_VER | 0x62 | Halt&#x2014;Invalid transaction unless found in an unexecuted OP_IF clause
-| OP_IF | 0x63 | Execute the statements following if top of stack is not 0
-| OP_NOTIF | 0x64 | Execute the statements following if top of stack is 0
-| OP_VERIF | 0x65 | Halt&#x2014;Invalid transaction
-| OP_VERNOTIF | 0x66 | Halt&#x2014;Invalid transaction
-| OP_ELSE | 0x67 | Execute only if the previous statements were not executed
-| OP_ENDIF | 0x68 | End the OP_IF, OP_NOTIF, OP_ELSE block
-| OP_VERIFY | 0x69 | Check the top of the stack, halt and invalidate transaction if not TRUE
-| OP_RETURN | 0x6a | Halt and invalidate transaction
-|=======
-
-<<tx_script_ops_table_timelock>> shows operators used for timelocks.
-
-[[tx_script_ops_table_timelock]]
-.Timelock operations
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_CHECKLOCKTIMEVERIFY (previously OP_NOP2) | 0xb1 | Marks transaction as invalid if the top stack item is greater than the transaction's nLockTime field, otherwise script evaluation continues as though an OP_NOP was executed. Transaction is also invalid if 1. the stack is empty; or 2. the top stack item is negative; or 3. the top stack item is greater than or equal to 500000000 while the transaction's nLockTime field is less than 500000000, or vice versa; or 4. the input's nSequence field is equal to 0xffffffff. The precise semantics are described in BIP-65 
-| OP_CHECKSEQUENCEVERIFY (previously OP_NOP3) | 0xb2 | Marks transaction as invalid if the relative lock time of the input (enforced by BIP 0068 with nSequence) is not equal to or longer than the value of the top stack item. The precise semantics are described in BIP-112|
-|=======
-
-<<tx_script_ops_table_stack>> shows operators used to manipulate the stack.
-
-[[tx_script_ops_table_stack]]
-.Stack operations
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_TOALTSTACK | 0x6b | Pop top item from stack and push to alternative stack
-| OP_FROMALTSTACK | 0x6c | Pop top item from alternative stack and push to stack
-| OP_2DROP | 0x6d | Pop top two stack items
-| OP_2DUP | 0x6e | Duplicate top two stack items
-| OP_3DUP | 0x6f | Duplicate top three stack items
-| OP_2OVER | 0x70 | Copy the third and fourth items in the stack to the top
-| OP_2ROT | 0x71 | Move the fifth and sixth items in the stack to the top
-| OP_2SWAP | 0x72 | Swap the two top pairs of items in the stack
-| OP_IFDUP | 0x73 | Duplicate the top item in the stack if it is not 0
-| OP_DEPTH | 0x74 | Count the items on the stack and push the resulting count
-| OP_DROP | 0x75 | Pop the top item in the stack
-| OP_DUP | 0x76 | Duplicate the top item in the stack
-| OP_NIP | 0x77 | Pop the second item in the stack
-| OP_OVER | 0x78 | Copy the second item in the stack and push it onto the top
-| OP_PICK | 0x79 | Pop value N from top, then copy the Nth item to the top of the stack
-| OP_ROLL | 0x7a | Pop value N from top, then move the Nth item to the top of the stack
-| OP_ROT | 0x7b | Rotate the top three items in the stack
-| OP_SWAP | 0x7c | Swap the top two items on the stack
-| OP_TUCK | 0x7d | Copy the top item and insert it between the top and second item
-|=======
-
-<<tx_script_ops_table_splice>> shows string operators.
-
-[[tx_script_ops_table_splice]]
-.String splice operations
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| _OP_CAT_ | 0x7e | Disabled (concatenates top two items)
-| _OP_SUBSTR_ | 0x7f | Disabled (returns substring)
-| _OP_LEFT_ | 0x80 | Disabled (returns left substring)
-| _OP_RIGHT_ | 0x81 | Disabled (returns right substring)
-| OP_SIZE | 0x82 | Calculate string length of top item and push the result 
-|=======
-
-<<tx_script_ops_table_binmath>> shows binary arithmetic and boolean logic operators.
-
-[[tx_script_ops_table_binmath]]
-.Binary arithmetic and conditionals
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| _OP_INVERT_ | 0x83 | Disabled (Flip the bits of the top item)
-| _OP_AND_ | 0x84 | Disabled (Boolean AND of two top items)
-| _OP_OR_ | 0x85 | Disabled (Boolean OR of two top items)
-| _OP_XOR_ | 0x86 | Disabled (Boolean XOR of two top items)
-| OP_EQUAL | 0x87 | Push TRUE (1) if top two items are exactly equal, push FALSE (0) otherwise
-| OP_EQUALVERIFY | 0x88 | Same as OP_EQUAL, but run OP_VERIFY after to halt if not TRUE
-| OP_RESERVED1 | 0x89 | Halt&#x2014;Invalid transaction unless found in an unexecuted OP_IF clause
-| OP_RESERVED2 | 0x8a | Halt&#x2014;Invalid transaction unless found in an unexecuted OP_IF clause
-|=======
-
-[role="pagebreak-before"]
-<<tx_script_ops_table_numbers>> shows numeric (arithmetic) operators.
-
-[[tx_script_ops_table_numbers]]
-.Numeric operators
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_1ADD | 0x8b | Add 1 to the top item   
-| OP_1SUB | 0x8c | Subtract 1 from the top item
-| _OP_2MUL_ | 0x8d | Disabled (multiply top item by 2)
-| _OP_2DIV_ | 0x8e | Disabled (divide top item by 2)
-| OP_NEGATE | 0x8f | Flip the sign of top item
-| OP_ABS | 0x90 | Change the sign of the top item to positive
-| OP_NOT | 0x91 | If top item is 0 or 1 Boolean flip it, otherwise return 0
-| OP_0NOTEQUAL | 0x92 | If top item is 0 return 0, otherwise return 1
-| OP_ADD | 0x93 | Pop top two items, add them and push result
-| OP_SUB | 0x94 | Pop top two items, subtract first from second, push result
-| _OP_MUL_ | 0x95 | Disabled (multiply top two items)
-| _OP_DIV_ | 0x96 | Disabled (divide second item by first item)
-| _OP_MOD_ | 0x97 | Disabled (remainder divide second item by first item)
-| _OP_LSHIFT_ | 0x98 | Disabled (shift second item left by first item number of bits)
-| _OP_RSHIFT_ | 0x99 | Disabled (shift second item right by first item number of bits)
-| OP_BOOLAND | 0x9a | Boolean AND of top two items
-| OP_BOOLOR | 0x9b | Boolean OR of top two items
-| OP_NUMEQUAL | 0x9c | Return TRUE if top two items are equal numbers
-| OP_NUMEQUALVERIFY | 0x9d | Same as NUMEQUAL, then OP_VERIFY to halt if not TRUE
-| OP_NUMNOTEQUAL | 0x9e | Return TRUE if top two items are not equal numbers
-| OP_LESSTHAN | 0x9f | Return TRUE if second item is less than top item
-| OP_GREATERTHAN | 0xa0 | Return TRUE if second item is greater than top item
-| OP_LESSTHANOREQUAL | 0xa1 | Return TRUE if second item is less than or equal to top item
-| OP_GREATERTHANOREQUAL | 0xa2 | Return TRUE if second item is great than or equal to top item
-| OP_MIN | 0xa3 | Return the smaller of the two top items 
-| OP_MAX | 0xa4 | Return the larger of the two top items
-| OP_WITHIN | 0xa5 | Return TRUE if the third item is between the second item (or equal) and first item
-|=======
-
-[role="pagebreak-before"]
-<<tx_script_ops_table_crypto>> shows cryptographic function operators.
-
-[[tx_script_ops_table_crypto]]
-.Cryptographic and hashing operations
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_RIPEMD160 | 0xa6 | Return RIPEMD160 hash of top item
-| OP_SHA1 | 0xa7 | Return SHA1 hash of top item
-| OP_SHA256 | 0xa8 | Return SHA256 hash of top item
-| OP_HASH160 | 0xa9 | Return RIPEMD160(SHA256(x)) hash of top item
-| OP_HASH256 | 0xaa | Return SHA256(SHA256(x)) hash of top item
-| OP_CODESEPARATOR | 0xab | Mark the beginning of signature-checked data
-| OP_CHECKSIG | 0xac | Pop a public key and signature and validate the signature for the transaction's hashed data, return TRUE if matching
-| OP_CHECKSIGVERIFY | 0xad | Same as CHECKSIG, then OP_VERIFY to halt if not TRUE
-| OP_CHECKMULTISIG | 0xae | Run CHECKSIG for each pair of signature and public key provided. All must match. Bug in implementation pops an extra value, prefix with OP_0 as workaround
-| OP_CHECKMULTISIGVERIFY | 0xaf | Same as CHECKMULTISIG, then OP_VERIFY to halt if not TRUE
-|=======
-
-<<tx_script_ops_table_nop>> shows nonoperator symbols.
-
-[[tx_script_ops_table_nop]]
-.Nonoperators
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_NOP1 to OP_NOP10 | 0xb0 to 0xb9 | Does nothing, ignored
-|=======
-
-
-<<tx_script_ops_table_internal>> shows operator codes reserved for use by the internal script parser.((("", startref="TRlang14")))((("", startref="SCRlang14")))
-
-[[tx_script_ops_table_internal]]
-.Reserved OP codes for internal use by the parser
-[options="header"]
-|=======
-| Symbol | Value (hex) | Description
-| OP_SMALLDATA | 0xf9 | Represents small data field 
-| OP_SMALLINTEGER | 0xfa | Represents small integer data field
-| OP_PUBKEYS | 0xfb | Represents public key fields
-| OP_PUBKEYHASH | 0xfd | Represents a public key hash field
-| OP_PUBKEY | 0xfe | Represents a public key field
-| OP_INVALIDOPCODE | 0xff | Represents any OP code not currently assigned
-|=======
-

atlas.json

@@ -2,30 +2,29 @@
   "branch": "master",
   "files": [
     "cover.html",
+    "praise.html",
     "titlepage.html",
     "copyright.html",
     "dedication.html",
     "toc.html",
     "preface.asciidoc",
-    "glossary.asciidoc",
-    "ch01.asciidoc",
-    "ch02.asciidoc",
-    "ch03.asciidoc",
-    "ch04.asciidoc",
-    "ch05.asciidoc",
-    "ch06.asciidoc",
-    "ch07.asciidoc",
-    "ch08.asciidoc",
-    "ch09.asciidoc",
-    "ch10.asciidoc",
-    "ch11.asciidoc",
-    "ch12.asciidoc",
-    "appdx-bitcoinwhitepaper.asciidoc",
-    "appdx-scriptops.asciidoc",
-    "appdx-bips.asciidoc",
-    "appdx-bitcore.asciidoc",
-    "appdx-pycoin.asciidoc",
-    "appdx-bx.asciidoc",
+    "ch01_intro.adoc",
+    "ch02_overview.adoc",
+    "ch03_bitcoin-core.adoc",
+    "ch04_keys.adoc",
+    "ch05_wallets.adoc",
+    "ch06_transactions.adoc",
+    "ch07_authorization-authentication.adoc",
+    "ch08_signatures.adoc",
+    "ch09_fees.adoc",
+    "ch10_network.adoc",
+    "ch11_blockchain.adoc",
+    "ch12_mining.adoc",
+    "ch13_security.adoc",
+    "ch14_applications.adoc",
+    "appa_whitepaper.adoc",
+    "appb_errata.adoc",
+    "appc_bips.adoc",
     "ix.html",
     "author_bio.html",
     "colo.html"
@@ -35,7 +34,7 @@
       "version": "print",
       "toc": true,
       "index": true,
-      "antennahouse_version": "AHFormatterV62_64-MR4",
+      "antennahouse_version": "AHFormatterV71_64-MR2",
       "syntaxhighlighting": true,
       "show_comments": false,
       "color_count": "1",
@@ -47,7 +46,8 @@
       "epubcheck": true,
       "syntaxhighlighting": true,
       "show_comments": false,
-      "downsample_images": false
+      "downsample_images": true,
+      "mathmlreplacement": false
     },
     "mobi": {
       "index": true,
@@ -66,8 +66,9 @@
   },
   "theme": "oreillymedia/animal_theme_sass",
   "title": "Mastering Bitcoin",
-  "print_isbn13": "9781491954386",
+  "print_isbn13": "9781098150099",
   "templating": false,
   "lang": "en",
-  "accent_color": "cmyk(26%, 89%, 0%, 0%)"
+  "accent_color": "",
+  "preprocessing": "none"
 }

author_bio.html

@@ -1,10 +1,13 @@
-<section data-type="colophon" class="abouttheauthor">
-  <h1>About the Author</h1>
-  <p><strong>Andreas M. Antonopoulos</strong> is a noted technologist and serial entrepreneur who has become one of the most well-known and well-respected figures in bitcoin. As an engaging public speaker, teacher, and writer, Andreas makes complex subjects accessible and easy to understand. As an advisor, he helps startups recognize, evaluate, and navigate security and business risks.</p>
+<section class="abouttheauthor" data-type="colophon" xmlns="http://www.w3.org/1999/xhtml">
+<h1>About the Authors</h1>
 
-<p>Andreas grew up with the internet, starting his first company, an early BBS and proto-ISP, as a teenager in his home in Greece. He earned degrees in computer science, data communications, and distributed systems from University College London (UCL)&#x2014;recently ranked among the world&#x2019;s top 10 universities. After moving to the United States, Andreas cofounded and managed a successful technology research company, and in that role advised dozens of Fortune 500 company executives on networking, security, data centers, and cloud computing. More than 200 of his articles on security, cloud computing, and data centers have been published in print and syndicated worldwide. He holds two patents in networking and security.</p>
+<p><strong>Andreas M. Antonopoulos</strong> is a noted technologist and serial entrepreneur who has become one of the most well-known and well-respected figures in Bitcoin. As an engaging public speaker, teacher, and writer, Andreas makes complex subjects accessible and easy to understand. As an advisor, he helps startups recognize, evaluate, and navigate security and business risks.</p>
 
-<p>In 1990, Andreas started teaching various IT topics in private, professional, and academic environments. He honed his speaking skills in front of audiences ranging in size from five executives in a boardroom to thousands of people in large conferences. With more than 400 speaking engagements under his belt he is considered a world-class and charismatic public speaker and teacher. In 2014, he was appointed as a teaching fellow with the University of Nicosia, the first university in the world to offer a masters degree in digital currency. In this role, he helped develop the curriculum and cotaught the Introduction to Digital Currencies course, offered as a massive open online course (MOOC) through the university.</p>
+<p>Andreas grew up with the internet, starting his first company, an early BBS and proto-ISP, as a teenager in his home in Greece. He earned degrees in computer science, data communications, and distributed systems from University College London (UCL)—recently ranked among the world’s top 10 universities. After moving to the United States, Andreas cofounded and managed a successful technology research company, and in that role advised dozens of Fortune 500 company executives on networking, security, data centers, and cloud computing. More than two hundred of his articles on security, cloud computing, and data centers have been published in print and syndicated worldwide. He holds two patents in networking and security.</p>
 
-<p>As a bitcoin entrepreneur, Andreas has founded a number of bitcoin businesses and launched several community open source projects. He serves as an advisor to several bitcoin and cryptocurrency companies. He is a widely published author of articles and blog posts on bitcoin, a permanent host on the popular Let&#x2019;s Talk Bitcoin podcast, and a frequent speaker at technology and security conferences worldwide.</p>
+<p>In 1990, Andreas started teaching various IT topics in private, professional, and academic environments. He honed his speaking skills in front of audiences ranging in size from five executives in a boardroom to thousands of people in large conferences. With more than four hundred speaking engagements under his belt, he is considered a world-class and charismatic public speaker and teacher. In 2014, he was appointed as a teaching fellow with the University of Nicosia, the first university in the world to offer a master's degree in digital currency. In this role, he helped develop the curriculum and cotaught the "Introduction to Digital Currencies" course, offered as a massive open online course (MOOC) through the university.</p>
+
+<p>As a Bitcoin entrepreneur, Andreas has founded a number of Bitcoin businesses and launched several community open source projects. He serves as an advisor to several Bitcoin and cryptocurrency companies. He is a widely published author of articles and blog posts on Bitcoin, a permanent host on the popular <em>Let’s Talk Bitcoin</em> podcast, and a frequent speaker at technology and security conferences worldwide.</p>
+
+<p><strong>David A. Harding</strong> is a technical writer focused on creating documentation for open source software. He is the coauthor of the <em>Bitcoin Optech</em> weekly newsletter <span class='keep-together'>(2018–2023),</span> <em>21.co Bitcoin Computer</em> tutorials (2015–2017), and Bitcoin.org developer documentation (2014–2015). He is also a Brink.dev grant committee member (2022–2023) and former board member (2020–2022).</p>
 </section>

book.adoc

@@ -0,0 +1,51 @@
+= Mastering Bitcoin
+
+include::preface.asciidoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::intro.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::overview.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::bitcoin-core.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::keys.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::wallets.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::transactions.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::authorization-authentication.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::signatures.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::fees.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::network.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::blockchain.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::mining.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::security.adoc[]
+
+//IMAGE_REPORT:NEW_CHAPTER
+include::applications.adoc[]
+
+include::whitepaper.adoc[]
+
+include::errata.adoc[]
+
+include::bips.adoc[]

book.asciidoc

@@ -1,45 +0,0 @@
-= Mastering Bitcoin
-
-include::preface.asciidoc[]
-
-include::glossary.asciidoc[]
-
-include::ch01.asciidoc[]
-
-include::ch02.asciidoc[]
-
-include::ch03.asciidoc[]
-
-include::ch04.asciidoc[]
-
-include::ch05.asciidoc[]
-
-include::ch06.asciidoc[]
-
-include::ch07.asciidoc[]
-
-include::ch08.asciidoc[]
-
-include::ch09.asciidoc[]
-
-include::ch10.asciidoc[]
-
-include::ch11.asciidoc[]
-
-include::ch12.asciidoc[]
-
-include::appdx-bitcoinwhitepaper.asciidoc[]
-
-include::appdx-scriptops.asciidoc[]
-
-include::appdx-bips.asciidoc[]
-
-include::appdx-bitcore.asciidoc[]
-
-include::appdx-pycoin.asciidoc[]
-
-include::appdx-bx.asciidoc[]
-
-include::ix.html[]
-
-include::colo.html[]

ch01.asciidoc

@@ -1,246 +0,0 @@
-[role="pagenumrestart"]
-[[ch01_intro_what_is_bitcoin]]
-== Introduction
-
-=== What Is Bitcoin?
-
-((("bitcoin", "defined", id="GSdefine01")))Bitcoin is a collection of concepts and technologies that form the basis of a digital money ecosystem. Units of currency called bitcoin are used to store and transmit value among participants in the Bitcoin network. Bitcoin users communicate with each other using the Bitcoin protocol primarily via the internet, although other transport networks can also be used. The Bitcoin protocol stack, available as open source software, can be run on a wide range of computing devices, including laptops and smartphones, making the technology easily accessible.
-
-[TIP]
-====
-Notice how the unit of currency is called "bitcoin" with a small _b_, and the system is called "Bitcoin", with a capital _B_.
-====
-
-Users can transfer bitcoin over the network to do just about anything that can be done with conventional currencies, including buying and selling goods, sending money to people or organizations, or extending credit. Bitcoin can be purchased, sold, and exchanged for other currencies at specialized currency exchanges. Bitcoin in a sense is the perfect form of money for the internet because it is fast, secure, and borderless.
-
-Unlike traditional currencies, bitcoin is entirely virtual. There are no physical coins or even digital coins per se. The coins are implied in transactions that transfer value from sender to recipient. Users of Bitcoin own keys that allow them to prove ownership of bitcoin in the Bitcoin network. With these keys, they can sign transactions to unlock the value and spend it by transferring it to a new owner. Keys are often stored in a digital wallet on each user’s computer or smartphone. Possession of the key that can sign a transaction is the only prerequisite to spending bitcoin, putting the control entirely in the hands of each user.
-
-Bitcoin is a distributed, peer-to-peer system. As such, there is no "central" server or point of control. Bitcoins, i.e. units of bitcoin, are created through a process called "mining," which involves competing to find solutions to a mathematical problem while processing Bitcoin transactions. Any participant in the Bitcoin network (i.e., anyone using a device running the full Bitcoin protocol stack) may operate as a miner, using their computer's processing power to verify and record transactions. Every 10 minutes, on average, a Bitcoin miner can validate the transactions of the past 10 minutes and is rewarded with brand new bitcoin. Essentially, Bitcoin mining decentralizes the currency-issuance and clearing functions of a central bank and replaces the need for any central bank.
-
-The Bitcoin protocol includes built-in algorithms that regulate the mining function across the network. The difficulty of the processing task that miners must perform is adjusted dynamically so that, on average, someone succeeds every 10 minutes regardless of how many miners (and how much processing) are competing at any moment. The protocol also halves the rate at which new bitcoin is created every 4 years, and limits the total number of bitcoin that will be created to a fixed total just below 21 million coins. The result is that the number of bitcoin in circulation closely follows an easily predictable curve that approaches 21 million by the year 2140. Due to bitcoin's diminishing rate of issuance, over the long term, the Bitcoin currency is deflationary. Furthermore, bitcoin cannot be inflated by "printing" new money above and beyond the expected issuance rate.
-
-Behind the scenes, Bitcoin is also the name of the protocol, a peer-to-peer network, and a distributed computing innovation. The bitcoin currency is really only the first application of this invention. Bitcoin represents the culmination of decades of research in cryptography and distributed systems and includes four key innovations brought together in a unique and powerful combination. Bitcoin consists of:
-
-* A decentralized peer-to-peer network (the Bitcoin protocol)
-* A public transaction ledger (the blockchain)
-* ((("mining and consensus", "consensus rules", "satisfying")))A set of rules for independent transaction validation and currency issuance (consensus rules)
-* A mechanism for reaching global decentralized consensus on the valid blockchain (Proof-of-Work algorithm)
-
-As a developer, I see Bitcoin as akin to the internet of money, a network for propagating value and securing the ownership of digital assets via distributed computation. There's a lot more to Bitcoin than first meets the eye.
-
-In this chapter we'll get started by explaining some of the main concepts and terms, getting the necessary software, and using Bitcoin for simple transactions. In the following chapters, we'll start unwrapping the layers of technology that make Bitcoin possible and examine the inner workings of the Bitcoin network and protocol.((("", startref="GSdefine01")))
-
-[role="pagebreak-before less_space"]
-.Digital Currencies Before Bitcoin
-****
-
-((("digital currencies", "prior to bitcoin")))The emergence of viable digital money is closely linked to developments in cryptography. This is not surprising when one considers the fundamental challenges involved with using bits to represent value that can be exchanged for goods and services. Three basic questions for anyone accepting digital money are:
-
-1.     Can I trust that the money is authentic and not counterfeit?
-2.     Can I trust that the digital money can only be spent once (known as the “double-spend” problem)?
-3.     Can I be sure that no one else can claim this money belongs to them and not me?
-
-Issuers of paper money are constantly battling the counterfeiting problem by using increasingly sophisticated papers and printing technology. Physical money addresses the double-spend issue easily because the same paper note cannot be in two places at once. Of course, conventional money is also often stored and transmitted digitally. In these cases, the counterfeiting and double-spend issues are handled by clearing all electronic transactions through central authorities that have a global view of the currency in circulation. For digital money, which cannot take advantage of esoteric inks or holographic strips, cryptography provides the basis for trusting the legitimacy of a user’s claim to value. Specifically, cryptographic digital signatures enable a user to sign a digital asset or transaction proving the ownership of that asset. With the appropriate architecture, digital signatures also can be used to address the double-spend issue.
-
-When cryptography started becoming more broadly available and understood in the late 1980s, many researchers began trying to use cryptography to build digital currencies. These early digital currency projects issued digital money, usually backed by a national currency or precious metal such as gold.
-
-((("decentralized systems", "vs. centralized", secondary-sortas="centralized")))Although these earlier digital currencies worked, they were centralized and, as a result, were easy to attack by governments and hackers. Early digital currencies used a central clearinghouse to settle all transactions at regular intervals, just like a traditional banking system. Unfortunately, in most cases, these nascent digital currencies were targeted by worried governments and eventually litigated out of existence. Some failed in spectacular crashes when the parent company liquidated abruptly. To be robust against intervention by antagonists, whether legitimate governments or criminal elements, a _decentralized_ digital currency was needed to avoid a single point of attack. Bitcoin is such a system, decentralized by design, and free of any central authority or point of control that can be attacked or corrupted.
-
-****
-
-=== History of Bitcoin
-
-((("Nakamoto, Satoshi")))((("distributed computing")))((("bitcoin", "history of")))Bitcoin was invented in 2008 with the publication of a paper titled "Bitcoin: A Peer-to-Peer Electronic Cash System,"footnote:["Bitcoin: A Peer-to-Peer Electronic Cash System," Satoshi Nakamoto (https://bitcoin.org/bitcoin.pdf).] written under the alias of Satoshi Nakamoto (see <<satoshi_whitepaper>>). Nakamoto combined several prior inventions such as b-money and HashCash to create a completely decentralized electronic cash system that does not rely on a central authority for currency issuance or settlement and validation of transactions. ((("Proof-of-Work algorithm")))((("decentralized systems", "consensus in")))((("mining and consensus", "Proof-of-Work algorithm")))The key innovation was to use a distributed computation system (called a "Proof-of-Work" algorithm) to conduct a global "election" every 10 minutes, allowing the decentralized network to arrive at _consensus_ about the state of transactions. ((("double-spend problem")))((("spending bitcoin", "double-spend problem")))This elegantly solves the issue of double-spend where a single currency unit can be spent twice. Previously, the double-spend problem was a weakness of digital currency and was addressed by clearing all transactions through a central clearinghouse.
-
-The Bitcoin network started in 2009, based on a reference implementation published by Nakamoto and since revised by many other programmers. The implementation of the Proof-of-Work algorithm (mining) that provides security and resilience for Bitcoin has increased in power exponentially, and now exceeds the combined processing power of the world's top supercomputers. Bitcoin's total market value has at times exceeded $1 trillion US dollars, depending on the bitcoin-to-dollar exchange rate. The largest transaction processed so far by the network was $1.1 billion US dollars, transmitted instantly and processed for a fee of only $0.68.
-
-Satoshi Nakamoto withdrew from the public in April 2011, leaving the responsibility of developing the code and network to a thriving group of volunteers. The identity of the person or people behind Bitcoin is still unknown. ((("open source licenses")))However, neither Satoshi Nakamoto nor anyone else exerts individual control over the Bitcoin system, which operates based on fully transparent mathematical principles, open source code, and consensus among participants. The invention itself is groundbreaking and has already spawned new science in the fields of distributed computing, economics, and econometrics.
-
-
-.A Solution to a Distributed Computing Problem
-****
-((("Byzantine Generals&#x27; Problem")))Satoshi Nakamoto's invention is also a practical and novel solution to a problem in distributed computing, known as the "Byzantine Generals' Problem." Briefly, the problem consists of trying to agree on a course of action or the state of a system by exchanging information over an unreliable and potentially compromised network. ((("central trusted authority")))Satoshi Nakamoto's solution, which uses the concept of Proof-of-Work to achieve consensus _without a central trusted authority_, represents a breakthrough in distributed computing and has wide applicability beyond currency. It can be used to achieve consensus on decentralized networks to prove the fairness of elections, lotteries, asset registries, digital notarization, and more.
-****
-
-
-[[user-stories]]
-=== Bitcoin Uses, Users, and Their Stories
-
-((("bitcoin", "use cases", id="GSuses01")))Bitcoin is an innovation in the ancient technology of money. At its core, money simply facilitates the exchange of value between people. Therefore, in order to fully understand Bitcoin and its uses, we'll examine it from the perspective of people using it. Each of the people and their stories, as listed here, illustrates one or more specific use cases. We'll be seeing them throughout the book:
-
-North American low-value retail::
-((("use cases", "retail sales")))Alice lives in Northern California's Bay Area. She has heard about Bitcoin from her techie friends and wants to start using it. We will follow her story as she learns about Bitcoin, acquires some, and then spends some of her bitcoin to buy a cup of coffee at Bob's Cafe in Palo Alto. This story will introduce us to the software, the exchanges, and basic transactions from the perspective of a retail consumer.
-
-North American high-value retail::
-Carol is an art gallery owner in San Francisco. She sells expensive paintings for Bitcoin. This story will introduce the risks of a "51%" consensus attack for retailers of high-value items.
-
-Offshore contract services::
-((("offshore contract services")))((("use cases", "offshore contract services")))Bob, the cafe owner in Palo Alto, is building a new website. He has contracted with an Indian web developer, Gopesh, who lives in Bangalore, India. Gopesh has agreed to be paid in bitcoin. This story will examine the use of Bitcoin for outsourcing, contract services, and international wire transfers.
-
-Web store::
-((("use cases", "web store")))Gabriel is an enterprising young teenager in Rio de Janeiro, running a small web store that sells Bitcoin-branded t-shirts, coffee mugs, and stickers. Gabriel is too young to have a bank account, but his parents are encouraging his entrepreneurial spirit.
-
-Charitable donations::
-((("charitable donations")))((("use cases", "charitable donations")))Eugenia is the director of a children's charity in the Philippines. Recently she has discovered Bitcoin and wants to use it to reach a whole new group of foreign and domestic donors to fundraise for her charity. She's also investigating ways to use Bitcoin to distribute funds quickly to areas of need. This story will show the use of Bitcoin for global fundraising across currencies and borders and the use of an open ledger for transparency in charitable organizations.
-
-Import/export::
-((("use cases", "import/export")))Mohammed is an electronics importer in Dubai. He's trying to use Bitcoin to buy electronics from the United States and China for import into the UAE to accelerate the process of payments for imports. This story will show how Bitcoin can be used for large business-to-business international payments tied to physical goods.
-
-Mining for bitcoin::
-((("use cases", "mining for bitcoin")))Jing is a computer engineering student in Shanghai. He has built a "mining" rig to mine for bitcoin using his engineering skills to supplement his income. This story will examine the "industrial" base of Bitcoin: the specialized equipment used to secure the Bitcoin network and issue new currency.
-
-Each of these stories is based on the real people and real industries currently using Bitcoin to create new markets, new industries, and innovative solutions to global economic issues.((("", startref="GSuses01")))
-
-=== Getting Started
-
-((("getting started", "wallet selection", id="GSwallet01")))((("wallets", "selecting", id="Wselect01")))((("bitcoin", "getting started", id="BCbasic01")))Bitcoin is a protocol that can be accessed using a client application that speaks the protocol. A "Bitcoin wallet" is the most common user interface to the Bitcoin system, just like a web browser is the most common user interface for the HTTP protocol. There are many implementations and brands of Bitcoin wallets, just like there are many brands of web browsers (e.g., Chrome, Safari, Firefox, and Internet Explorer). And just like we all have our favorite browsers (Mozilla Firefox, Yay!) and our villains (Internet Explorer, Yuck!), Bitcoin wallets vary in quality, performance, security, privacy, and reliability. There is also a reference implementation of the Bitcoin protocol that includes a wallet, known as the "Satoshi Client" or "Bitcoin Core," which is derived from the original implementation written by Satoshi Nakamoto.
-
-==== Choosing a Bitcoin Wallet
-
-((("security", "wallet selection")))Bitcoin wallets are one of the most actively developed applications in the Bitcoin ecosystem. There is intense competition, and while a new wallet is probably being developed right now, several wallets from last year are no longer actively maintained. Many wallets focus on specific platforms or specific uses and some are more suitable for beginners while others are filled with features for advanced users. Choosing a wallet is highly subjective and depends on the use and user expertise. Therefore it would be pointless to recommend a specific brand or wallet. However, we can categorize Bitcoin wallets according to their platform and function and provide some clarity about all the different types of wallets that exist. Better yet, moving keys or seeds between Bitcoin wallets is relatively easy, so it is worth trying out several different wallets until you find one that fits your needs.
-
-[role="pagebreak-before"]
-Bitcoin wallets can be categorized as follows, according to the platform:
-
-Desktop wallet:: A desktop wallet was the first type of Bitcoin wallet created as a reference implementation and many users run desktop wallets for the features, autonomy, and control they offer. Running on general-use operating systems such as Windows and Mac OS has certain security disadvantages, however, as these platforms are often insecure and poorly configured.
-
-Mobile wallet:: A mobile wallet is the most common type of Bitcoin wallet. Running on smart-phone operating systems such as Apple iOS and Android, these wallets are often a great choice for new users. Many are designed for simplicity and ease-of-use, but there are also fully featured mobile wallets for power users.
-
-Web wallet:: Web wallets are accessed through a web browser and store the user's wallet on a server owned by a third party. This is similar to webmail in that it relies entirely on a third-party server. Some of these services operate using client-side code running in the user's browser, which keeps control of the Bitcoin keys in the hands of the user. Most, however, present a compromise by taking control of the Bitcoin keys from users in exchange for ease-of-use. It is inadvisable to store large amounts of bitcoin on third-party systems.
-
-Hardware wallet:: Hardware wallets are devices that operate a secure self-contained Bitcoin wallet on special-purpose hardware. They usually connect to a desktop or mobile device via USB cable or near-field-communication (NFC), and are operated with a web browser or accompanying software. By handling all Bitcoin-related operations on the specialized hardware, these wallets are considered very secure and suitable for storing large amounts of bitcoin.
-
-Another way to categorize bitcoin wallets is by their degree of autonomy and how they interact with the Bitcoin network:
-
-Full-node client:: ((("full-node clients")))A full client, or "full node," is a client that stores the entire history of Bitcoin transactions (every transaction by every user, ever), manages users' wallets, and can initiate transactions directly on the Bitcoin network. A full node handles all aspects of the protocol and can independently validate the entire blockchain and any transaction. A full-node client consumes substantial computer resources (e.g., more than 125 GB of disk, 2 GB of RAM) but offers complete autonomy and independent transaction verification.
-
-Lightweight client:: ((("lightweight clients")))((("simplified-payment-verification (SPV)")))A lightweight client, also known as a simplified-payment-verification (SPV) client, connects to Bitcoin full nodes (mentioned previously) for access to the Bitcoin transaction information, but stores the user wallet locally and independently creates, validates, and transmits transactions. Lightweight clients interact directly with the Bitcoin network, without an intermediary.
-
-Third-party API client:: ((("third-party API clients")))A third-party API client is one that interacts with Bitcoin through a third-party system of application programming interfaces (APIs), rather than by connecting to the Bitcoin network directly. The wallet may be stored by the user or by third-party servers, but all transactions go through a third party.
-
-Combining these categorizations, many Bitcoin wallets fall into a few groups, with the three most common being desktop full client, mobile lightweight wallet, and web third-party wallet. The lines between different categories are often blurry, as many wallets run on multiple platforms and can interact with the network in different ways.
-
-For the purposes of this book, we will be demonstrating the use of a variety of downloadable Bitcoin clients, from the reference implementation (Bitcoin Core) to mobile and web wallets. Some of the examples will require the use of Bitcoin Core, which, in addition to being a full client, also exposes APIs to the wallet, network, and transaction services. If you are planning to explore the programmatic interfaces into the Bitcoin system, you will need to run Bitcoin Core, or one of the alternative clients.((("", startref="GSwallet01")))((("", startref="Wselect01")))
-
-==== Quick Start
-
-((("getting started", "quick start example", id="GSquick01")))((("wallets", "quick start example", id="Wquick01")))((("use cases", "buying coffee", id="aliceone")))Alice, who we introduced in <<user-stories>>, is not a technical user and only recently heard about Bitcoin from her friend Joe. While at a party, Joe is once again enthusiastically explaining Bitcoin to all around him and is offering a demonstration. Intrigued, Alice asks how she can get started with Bitcoin. Joe says that a mobile wallet is best for new users and he recommends a few of his favorite wallets. Alice downloads "Bluewallet" (available for iOS and Android) and installs it on her phone.
-
-When Alice runs her wallet application for the first time, she chooses the option to create a new Bitcoin wallet, and takes a moment **away from Joe and all other parties** to write down a secret mnemonic phrase _in order_ on a piece of paper. As explained by the mobile wallet and by Joe earlier, the mnemonic phrase allows Alice to restore her wallet in case she loses her mobile device and grants her access to her funds on another device. After creating her wallet and securing her mnemonic phrase, Alice can tap on her wallet to see her bitcoin amount, transaction history, as well as two buttons that allow her to either _receive_ or _send_ bitcoin, shown in <<bluewallet-welcome>>.
-
-==== Mnemonic Words
-
-A modern Bitcoin wallet will provide a _mnemonic phrase_ (also sometimes called a "seed" or "seed phrase") for Alice to back up. The mnemonic phrase consists of 12-24 English words, selected randomly by the software, and used as the basis for the keys that are generated by the wallet. The mnemonic phrase can be used by Alice to restore all the transactions and funds in her wallet in the case of an event such as a lost mobile device, a software bug, or memory corruption.
-
-[TIP]
-====
-The correct term for these backup words is "mnemonic phrase". We avoid the use of the term "seed" to refer to a mnemonic phrase, because even though its use is common it is incorrect.
-====
-
-[[mnemonic-storage]]
-==== Storing the Mnemonic Safely
-
-Alice needs to be careful to store the mnemonic phrase in a way that balances the need to prevent theft and accidental loss. If she doesn't protect it enough, her mnemonic will be at risk of being stolen. If she protects it too much, her mnemonic will be at risk of being permanently lost. The recommended way to properly balance these risks is to write two copies of the mnemonic phrase on paper, with each of the words numbered as the order matters.
-
-Once Alice has recorded the mnemonic phrase, she should plan to store each copy in a separate secure location such as a locked desk drawer or a fireproof safe.
-
-[WARNING]
-====
-Never attempt a "DIY" security scheme that deviates in any way from the best practice recommendation in <<mnemonic-storage>>. Do not cut your mnemonic in half, make screenshots, store on USB drives, email or cloud drives, encrypt it, or try any other non-standard method. You will tip the balance in such a way as to risk permanent loss or theft. Many people have lost funds, not from theft but because they tried a non-standard solution without having the expertise to balance the risks involved. The best practice recommendation is carefully balanced by experts and suitable for the vast majority of users.
-====
-****
-[[bluewallet-welcome]]
-.Alice uses the Receive screen on her Bluewallet mobile Bitcoin wallet, and displays her address to Bob in a QR code format
-image::images/mbc2_0101.png["BluewalletWelcome"]
-
-((("addresses", "bitcoin wallet quick start example")))((("QR codes", "bitcoin wallet quick start example")))((("addresses", see="also keys and addresses"))) The main wallet view displays the bitcoin amount, transaction history, and _Receive_ and _Send_ buttons. In addition, many wallets feature the ability to purchase Bitcoin directly through an exchange or similar service where you can offer fiat money in return for cryptocurrency, which is done by <<bitcoin_price>> and selling to the wallet user at or above this price. The _Buy Bitcoin_ button would allow Alice to purchase Bitcoin in this fashion.
-
-Alice is now ready to start using her new bitcoin wallet. ((("", startref="GSquick01")))((("", startref="Wquick01"))) Her wallet application randomly generated a private key (described in more detail in <<private_keys>>) which will be used to derive Bitcoin addresses that direct to her wallet. At this point, her Bitcoin addresses are not known to the Bitcoin network or "registered" with any part of the Bitcoin system. Her Bitcoin addresses are simply random numbers that correspond to her private key that she can use to control access to the funds. The addresses are generated independently by her wallet without reference or registration with any service. In fact, in most wallets, there is no association between a Bitcoin address and any externally identifiable information including the user's identity. Until the moment an address is referenced as the recipient of value in a transaction posted on the bitcoin ledger, the Bitcoin address is simply part of the vast number of possible addresses that are valid in bitcoin. Only once an address has been associated with a transaction does it become part of the known addresses in the network.
-
-Alice uses the _Receive_ button, which displays a QR code along with a Bitcoin address. The QR code is the square with a pattern of black and white dots, serving as a form of barcode that contains the same information in a format that can be scanned by Joe's smartphone camera. Next to the wallet's QR code is the Bitcoin address it encodes, and Alice may choose to manually send her address to Joe by copying it onto her clipboard with a tap. Of note, when receiving funds to a new mobile wallet for the first time, many wallets will often re-verify that you have indeed secured your mnemonic phrase. This can range from a simple prompt to requiring the user to manually re-enter the phrase.
-
-[TIP]
-====
-((("addresses", "security of")))((("security", "Bitcoin addresses")))Bitcoin addresses start with 1, 3, or bc1. Like email addresses, they can be shared with other bitcoin users who can use them to send bitcoin directly to your wallet. There is nothing sensitive, from a security perspective, about the Bitcoin address. It can be posted anywhere without risking the security of the account. Unlike email addresses, you can create new addresses as often as you like, all of which will direct funds to your wallet. In fact, many modern wallets automatically create a new address for every transaction to maximize privacy. A wallet is simply a collection of addresses and the keys that unlock the funds within.
-====
-
-
-[[getting_first_bitcoin]]
-==== Getting Her First Bitcoin
-
-There are several ways Alice can acquire bitcoin:
-
-* She can exchange some of her national currency (e.g. USD) at a cryptocurrency exchange
-* She can buy some from a friend, or an acquaintance from a Bitcoin Meetup, in exchange for cash
-* She can find a _Bitcoin ATM_ in her area, which acts as a vending machine, selling bitcoin for cash
-* She can offer her skills or a product she sells and accepts payment in bitcoin
-* She can ask her employer or clients to pay her in bitcoin
-
-All of these methods have varying degrees of difficulty, and many will involve paying a fee. Some financial institutions will also require Alice to provide identification documents to comply with local banking regulations/anti-money laundering (AML) practices, a process which is known as Know Your Customer (KYC). However, with all these methods, Alice will be able to receive bitcoin.
-
-[TIP]
-====
-((("privacy, maintaining")))((("security", "maintaining privacy")))((("digital currencies", "currency exchanges")))((("currency exchanges")))((("digital currencies", "benefits of bitcoin")))((("bitcoin", "benefits of")))One of the advantages of bitcoin over other payment systems is that, when used correctly, it affords users much more privacy. Acquiring, holding, and spending bitcoin does not require you to divulge sensitive and personally identifiable information to third parties. However, where bitcoin touches traditional systems, such as currency exchanges, national and international regulations often apply. In order to exchange bitcoin for your national currency, you will often be required to provide proof of identity and banking information. Users should be aware that once a Bitcoin address is attached to an identity, all associated bitcoin transactions are also easy to identify and track. This is one reason many users choose to maintain dedicated exchange accounts unlinked to their wallets.
-====
-
-Alice was introduced to bitcoin by a friend so she has an easy way to acquire her first bitcoin. Next, we will look at how she buys bitcoin from her friend Joe and how Joe sends the bitcoin to her wallet.
-
-[[bitcoin_price]]
-==== Finding the Current Price of Bitcoin
-
-((("getting started", "exchange rates")))((("exchange rates", "determining")))Before Alice can buy bitcoin from Joe, they have to agree on the _exchange rate_ between bitcoin and US dollars. This brings up a common question for those new to bitcoin: "Who sets the bitcoin price?" The short answer is that the price is set by markets.
-
-((("exchange rates", "floating")))((("floating exchange rate")))Bitcoin, like most other currencies, has a _floating exchange rate_. That means that the value of bitcoin vis-a-vis any other currency fluctuates according to supply and demand in the various markets where it is traded. For example, the "price" of bitcoin in US dollars is calculated in each market based on the most recent trade of bitcoin and US dollars. As such, the price tends to fluctuate minutely several times per second. A pricing service will aggregate the prices from several markets and calculate a volume-weighted average representing the broad market exchange rate of a currency pair (e.g., BTC/USD).
-
-There are hundreds of applications and websites that can provide the current market rate. Here are some of the most popular:
-
-https://bitcoinaverage.com/[Bitcoin Average]:: ((("BitcoinAverage")))A site that provides a simple view of the volume-weighted-average for each currency.
-https://coincap.io/[CoinCap]:: A service listing the market capitalization and exchange rates of hundreds of crypto-currencies, including bitcoin.
-https://bit.ly/cmebrr[Chicago Mercantile Exchange Bitcoin Reference Rate]:: A reference rate that can be used for institutional and contractual reference, provided as part of investment data feeds by the CME.
-
-In addition to these various sites and applications, most bitcoin wallets will automatically convert amounts between bitcoin and other currencies. Joe will use his wallet to convert the price automatically before sending bitcoin to Alice.
-
-[[sending_receiving]]
-==== Sending and Receiving Bitcoin
-
-((("getting started", "sending and receiving bitcoin", id="GSsend01")))((("spending bitcoin", "bitcoin wallet quick start example")))((("spending bitcoin", see="also transactions")))Alice has decided to exchange $10 US dollars for bitcoin, so as not to risk too much money on this new technology. She gives Joe $10 in cash, opens her Bluewallet mobile wallet application, and selects Receive. This displays a QR code with Alice's first Bitcoin address.
-
-Joe then selects Send on his Bluewallet smartphone wallet and is presented with a screen containing the following inputs:
-
-* The amount to send, in bitcoin (BTC) or his local currency (USD)
-* A destination Bitcoin address
-* A transaction note (description)
-* A transaction fee
-
-In the input field for the Bitcoin address, there is a small _Scan_ button. This allows Joe to scan the QR code with his smartphone camera so that he doesn't have to type in Alice's Bitcoin address, which is quite long and difficult to type. Joe taps the _Scan_ button and it activates the smartphone camera, scanning the QR code displayed on Alice's smartphone.
-
-Joe now has Alice's Bitcoin address set as the recipient. Joe enters the amount as $10 US dollars and his wallet converts it by accessing the most recent exchange rate from an online service. The exchange rate at the time is $100 US dollars per bitcoin, so $10 US dollars is worth 0.10 bitcoin (BTC), or 100 millibitcoin (mBTC) as shown in the screenshot from Joe's wallet (see <<bluewallet-mobile-send>>).
-
-In the transaction note/description input, Joe enters "Alice". He can use this field to add some information regarding his transaction for future reference. This function is for his record keeping only. The transaction note will be stored in his wallet and only Joe will be able to see it. It will not be sent to Alice, nor stored on the blockchain.
-
-He also selects a transaction fee for his transaction. The higher the transaction fee, the faster his transaction will be confirmed (included in a block by a miner). He selects the minimum transaction fee possible at that time (0 satoshis/byte).
-
-[TIP]
-====
-The price of bitcoin has changed a lot over time, and an incredible amount since the first edition of this book was written. As of March 2021, a person would need approximately $54,000 USD to purchase one whole bitcoin. Many examples in this book reference real-life past transactions, when the price of bitcoin was much lower and transactions with zero fees were still possible. Think about how generous of a friend Joe would have been if he had made the same agreement with Alice today!
-====
-
-[[bluewallet-mobile-send]]
-[role="smallereighty"]
-.Bluewallet mobile bitcoin wallet send screen
-image::images/mbc2_0102.png["bluewallet mobile send screen"]
-
-Using Bluewallet, Joe carefully checks to make sure he has entered the correct amount, because he is about to transmit money and mistakes are irreversible. For simplicity sake, we will assume that Joe does not pay any transaction fees. The purpose and setting of transaction fees are covered in subsequent chapters.  After double-checking the address and amount, he presses _Send_ to transmit the transaction. Joe's mobile bitcoin wallet constructs a transaction that assigns 0.10 BTC to the address provided by Alice, sourcing the funds from Joe's wallet and signing the transaction with Joe's private keys. This tells the Bitcoin network that Joe has authorized a transfer of value to Alice's new address. As the transaction is transmitted via the peer-to-peer protocol, it quickly propagates across the Bitcoin network. In less than a second, most of the well-connected nodes in the network receive the transaction and see Alice's address for the first time.
-
-Meanwhile, Alice's wallet is constantly "listening" to published transactions on the Bitcoin network, looking for any that match the addresses it contains. A few seconds after Joe's wallet transmits the transaction, Alice's wallet will indicate that it is receiving 0.10 BTC.
-
-[TIP]
-====
-Each bitcoin can be subdivided into 100 million units, each called a "satoshi" (singular) or "satoshis" (plural). Named for bitcoin's creator, the Satoshi is the smallest unit of bitcoin, equivalent to 0.00000001 BTC.
-====
-
-[[confirmations]]
-.Confirmations
-((("getting started", "confirmations")))((("confirmations", "bitcoin wallet quick start example")))((("confirmations", see="also mining and consensus; transactions")))((("clearing", seealso="confirmations")))At first, Alice's wallet will show the transaction from Joe as "Unconfirmed." This means that the transaction has been propagated to the network but has not yet been recorded in the bitcoin transaction ledger, known as the blockchain. To be confirmed, a transaction must be included in a block and added to the blockchain, which happens every 10 minutes, on average. In traditional financial terms this is known as _clearing_. This book will cover in detail the propagation, validation, and clearing (or confirmation) of bitcoin transactions.
-
-Alice is now the proud owner of 0.10 BTC that she can spend. In the next chapter we will look at her first purchase with bitcoin, and examine the underlying transaction and propagation technologies in more detail.((("", startref="BCbasic01")))((("use cases", "buying coffee", startref="aliceone")))

ch01_intro.adoc

@@ -0,0 +1,505 @@
+[role="pagenumrestart"]
+[[ch01_intro_what_is_bitcoin]]
+== Introduction
+
+Bitcoin((("Bitcoin", "operational overview", id="bitcoin-operational-overview"))) is a collection of concepts and technologies that form the basis of a digital money ecosystem. Units of currency called bitcoin are used to store and transmit value among participants in the Bitcoin network. Bitcoin users communicate with each other using the Bitcoin protocol primarily via the internet, although other transport networks can also be used. The Bitcoin protocol stack, available as open source software, can be run on a wide range of computing devices, including laptops and smartphones, making the technology easily accessible.
+
+[TIP]
+====
+In this book, the ((("bitcoins", "defined")))unit of currency is called "bitcoin" with a small _b_,
+and the system is called "Bitcoin," with a capital _B_.
+====
+
+Users can transfer bitcoin over the network to do just about anything
+that can be done with conventional currencies, including buying and selling
+goods, sending money to people or organizations, or extending credit. Bitcoin
+can be purchased, sold, and exchanged for other currencies at
+specialized currency exchanges. Bitcoin is arguably the perfect form
+of money for the internet because it is fast, secure, and borderless.
+
+Unlike traditional currencies, the bitcoin currency is entirely virtual. There are no
+physical coins or even individual digital coins. The coins are implied in
+transactions that transfer value from spender to receiver. Users of
+Bitcoin control keys that allow them to prove ownership of bitcoin in the
+Bitcoin network. With these keys, they can sign transactions to unlock
+the value and spend it by transferring it to a new owner. Keys are often
+stored in a digital wallet on each user’s computer or smartphone.
+Possession of the key that can sign a transaction is the only
+prerequisite to spending bitcoin, putting the control entirely in the
+hands of each user.
+
+Bitcoin is a distributed, peer-to-peer system. As such, there is no
+central server or point of control. Units of bitcoin
+are created through a process((("bitcoins", "mining")))((("mining"))) called "mining," which involves repeatedly
+performing a computational task that references a list of recent Bitcoin
+transactions. Any participant in the Bitcoin network may operate as a
+miner, using their computing devices to help secure
+transactions. Every 10 minutes, on average, one Bitcoin miner can add security to
+past transactions and is rewarded with both brand new
+bitcoins and the fees paid by recent transactions. Essentially, Bitcoin
+mining decentralizes the currency-issuance
+and clearing functions of a central bank and replaces the need for any
+central bank.
+
+//-- Math for following paragraph --
+//total_btc = 0
+//for i in range(0, 10_000_000):
+//  total_btc += (50 /  (2**int(i/210000)) )
+//  if total_btc / 21e6 > 0.99:
+//     print(i)
+//     break
+
+The Bitcoin protocol includes built-in algorithms that regulate the
+mining function across the network. The difficulty of the computational
+task that miners must perform is adjusted dynamically so that, on
+average, someone succeeds every 10 minutes regardless of how many miners
+(and how much processing) are competing at any moment. The protocol also
+periodically decreases the number of new bitcoins that are created,
+limiting the total number of bitcoins that will ever be created to a fixed total
+just below 21 million coins. The result is that the number of bitcoins in
+circulation closely follows an easily predictable curve where half of
+the remaining coins are added to circulation every four years.  At
+approximately block 1,411,200, which is expected to be produced around
+the year 2035, 99% of all bitcoins
+that will ever exist will have been issued.  Due to Bitcoin's
+diminishing rate of issuance, over the long term, the Bitcoin currency
+is deflationary.  Furthermore, nobody can force you to accept
+any bitcoins that were created beyond the
+expected issuance rate.
+
+Behind the scenes, Bitcoin is also the name of the protocol, a peer-to-peer network, and a distributed computing innovation. Bitcoin builds on decades of research in cryptography and distributed systems and includes at least four key innovations brought together in a unique and powerful combination. Bitcoin consists of:
+
+* A decentralized peer-to-peer network (the Bitcoin protocol)
+* A public transaction journal (the blockchain)
+* A set of rules for independent transaction validation and currency issuance (consensus rules)
+* A mechanism for reaching global decentralized consensus on the valid blockchain (proof-of-work algorithm)
+
+As a developer, I see Bitcoin as akin to the internet of money, a network for propagating value and securing the ownership of digital assets via distributed computation. There's a lot more to Bitcoin than first meets ((("Bitcoin", "operational overview", startref="bitcoin-operational-overview")))the eye.
+
+In this chapter we'll get started by explaining some of the main concepts and terms, getting the necessary software, and using Bitcoin for simple transactions. In the following chapters, we'll start unwrapping the layers of technology that make Bitcoin possible and examine the inner workings of the Bitcoin network and protocol.
+
+.Digital Currencies Before Bitcoin
+****
+
+The ((("digital currencies, history of")))((("history", "of digital currencies", secondary-sortas="digital currencies")))((("cryptography")))emergence of viable digital money is closely linked to developments in cryptography. This is not surprising when one considers the fundamental challenges involved with using bits to represent value that can be exchanged for goods and services. Three basic questions for anyone accepting digital money are:
+
+*     Can I trust that the money is authentic and not counterfeit?
+*     Can I trust that the digital money can only be spent once (known as the “double-spend” problem)?
+*     Can I be sure that no one else can claim this money belongs to them and not me?
+
+Issuers of paper money are constantly battling the counterfeiting problem by using increasingly sophisticated papers and printing technology.  Physical money addresses the double-spend issue easily because the same paper note cannot be in two places at once. Of course, conventional money is also often stored and transmitted digitally. In these cases, the counterfeiting and double-spend issues are handled by clearing all electronic transactions through central authorities that have a global view of the currency in circulation. For digital money, which cannot take advantage of esoteric inks or holographic strips, cryptography provides the basis for trusting the legitimacy of a user’s claim to value.  Specifically, cryptographic digital signatures enable a user to sign a digital asset or transaction proving the ownership of that asset. With the appropriate architecture, digital signatures also can be used to address the double-spend issue.
+
+When cryptography started becoming more broadly available and understood in the late 1980s, many researchers began trying to use cryptography to build digital currencies. These early digital currency projects issued digital money, usually backed by a national currency or precious metal such as gold.
+
+Although these earlier digital currencies worked, they were centralized and, as a result, were easy to attack by governments and hackers. Early digital currencies used a central clearinghouse to settle all transactions at regular intervals, just like a traditional banking system. Unfortunately, in most cases these nascent digital currencies were targeted by worried governments and eventually litigated out of existence. Some failed in spectacular crashes when the parent company liquidated abruptly. To be robust against intervention by antagonists, whether legitimate governments or criminal elements, a _decentralized_ digital currency was needed to avoid a single point of attack. Bitcoin is such a system, decentralized by design, and free of any central authority or point of control that can be attacked or corrupted.
+
+****
+
+=== History of Bitcoin
+
+Bitcoin was((("Bitcoin", "history of")))((("history", "of Bitcoin", secondary-sortas="Bitcoin")))((("Nakamoto, Satoshi"))) first described in 2008 with the publication of a
+paper titled "Bitcoin: A Peer-to-Peer Electronic Cash
+System,"footnote:[https://oreil.ly/KUaBM["Bitcoin: A Peer-to-Peer Electronic Cash System"],
+Satoshi Nakamoto.] written under the
+alias of Satoshi Nakamoto (see <<satoshi_whitepaper>>). Nakamoto
+combined several prior inventions such as digital signatures and Hashcash to create
+a completely decentralized electronic cash system that does not rely on
+a central authority for currency issuance or settlement and validation
+of transactions. A key innovation was to use a distributed computation
+system (called a "proof-of-work" algorithm) to conduct a global
+lottery every 10 minutes on average, allowing the decentralized network to
+arrive at _consensus_ about the state of transactions. This
+elegantly solves the issue of double-spend where a single currency unit
+can be spent twice. Previously, the double-spend problem was a weakness
+of digital currency and was addressed by clearing all transactions
+through a central clearinghouse.
+
+The Bitcoin network started in 2009, based on a reference implementation
+published by Nakamoto and since revised by many other programmers. The
+number and power of machines running the((("proof-of-work algorithm", seealso="mining"))) proof-of-work algorithm
+(mining) that provides security and resilience for Bitcoin have
+increased exponentially, and their combined computational power now
+exceeds the combined number of computing operations of the
+world's top supercomputers.
+
+
+Satoshi Nakamoto withdrew from the public in April 2011, leaving the responsibility of developing the code and network to a thriving group of volunteers. The identity of the person or people behind Bitcoin is still unknown. However, neither Satoshi Nakamoto nor anyone else exerts individual control over the Bitcoin system, which operates based on fully transparent mathematical principles, open source code, and consensus among participants. The invention itself is groundbreaking and has already spawned new science in the fields of distributed computing, economics, and econometrics.
+
+.A Solution to a Distributed Computing Problem
+****
+Satoshi Nakamoto's invention ((("Byzantine Generals&#x27; Problem")))((("distributed computing problem")))is
+also a practical and novel solution to a problem in distributed
+computing, known as the "Byzantine Generals' Problem." Briefly, the
+problem consists of trying to get multiple participants without a leader
+to agree on a course of action by exchanging information over an
+unreliable and potentially compromised network. Satoshi Nakamoto's solution, which uses the concept of
+proof of work to achieve consensus _without a central trusted
+authority_, represents a breakthrough in distributed computing.
+****
+
+=== Getting Started
+
+Bitcoin is((("Bitcoin", "wallets", see="wallets")))((("wallets", "explained"))) a protocol that can be accessed using an
+application that speaks the protocol. A "Bitcoin wallet" is the
+most common user interface to the Bitcoin system, just like a web
+browser is the most common user interface for the HTTP protocol. There
+are many implementations and brands of Bitcoin wallets, just like there
+are many brands of web browsers (e.g., Chrome, Safari, and Firefox).
+And just like we all have our favorite browsers,
+Bitcoin wallets vary in quality, performance, security, privacy, and
+reliability. There is also a reference implementation of the Bitcoin
+protocol that includes a wallet, known as "Bitcoin Core," which is
+derived from the original implementation written by Satoshi Nakamoto.
+
+==== Choosing a Bitcoin Wallet
+
+Bitcoin wallets ((("wallets", "choosing")))((("selecting", "wallets")))((("choosing", see="selecting")))are one of the most actively developed applications in the Bitcoin ecosystem. There is intense competition, and while a new wallet is probably being developed right now, several wallets from last year are no longer actively maintained. Many wallets focus on specific platforms or specific uses and some are more suitable for beginners while others are filled with features for advanced users. Choosing a wallet is highly subjective and depends on the use and user expertise. Therefore, it would be pointless to recommend a specific brand or wallet. However, we can categorize Bitcoin wallets according to their platform and function and provide some clarity about all the different types of wallets that exist. It is worth trying out several different wallets until you find one that fits your needs.
+
+===== Types of Bitcoin wallets
+Bitcoin wallets ((("wallets", "types of", id="wallet-type")))can be categorized as follows, according to the platform:
+
+Desktop wallet:: A ((("desktop wallets")))desktop wallet was the first type of Bitcoin wallet created as a reference implementation. Many users run desktop wallets for the features, autonomy, and control they offer. Running on general-use operating systems such as Windows and macOS has certain security disadvantages, however, as these platforms are often insecure and poorly configured.
+
+Mobile wallet:: A ((("mobile wallets")))mobile wallet is the most common type of Bitcoin
+wallet. Running on smart-phone operating systems such as Apple iOS and
+Android, these wallets are often a great choice for new users. Many are
+designed for simplicity and ease-of-use, but there are also fully
+featured mobile wallets for power users.  To avoid downloading and
+storing large amounts of data, most mobile wallets retrieve information
+from remote servers, reducing your privacy by disclosing to third
+parties information about your Bitcoin addresses and balances.
+
+Web wallet:: Web wallets ((("web wallets")))are accessed through a web browser and store
+the user's wallet on a server owned by a third party. This is similar to
+webmail in that it relies entirely on a third-party server. Some of
+these services operate using client-side code running in the user's
+browser, which keeps control of the Bitcoin keys in the hands of the
+user, although the user's dependence on the server still compromises
+their privacy. Most, however, take control of the Bitcoin keys from
+users in exchange for ease-of-use. It is inadvisable
+to store large amounts of bitcoin on third-party systems.
+
+Hardware signing devices:: Hardware signing devices((("hardware signing devices"))) are devices that can
+store keys and sign transactions using special-purpose hardware and
+firmware. They usually
+connect to a desktop, mobile, or web wallet via USB cable,
+near-field-communication (NFC), or a camera with QR codes.  By handling
+all Bitcoin-related operations on the specialized hardware, these
+wallets are less vulnerable to many types of attacks.  Hardware signing
+devices are sometimes called "hardware wallets", but they need to be
+paired with a full-featured wallet to send and receive transactions, and
+the security and privacy offered by that paired wallet plays a critical
+role in how much security and privacy the user obtains when using the
+hardware signing device.
+
+===== Full node versus Lightweight
+Another way to categorize Bitcoin wallets is by their degree of autonomy and how they interact with the Bitcoin network:
+
+Full node:: A full node ((("full nodes")))is a program that validates the
+entire history of Bitcoin transactions (every transaction by every user, ever).  Optionally, full nodes can also store previously validated
+transactions and serve data to other Bitcoin programs, either on the
+same computer or over the internet.  A full node uses substantial
+computer resources--about the same as watching an hour-long streaming
+video for each day of Bitcoin transactions--but the full node offers
+complete autonomy to its users.
+
+Lightweight client:: 
+A lightweight client,((("lightweight clients")))((("simplified-payment-verification (SPV) clients")))((("SPV (simplified-payment-verification) clients"))) also known as a simplified-payment-verification (SPV) client,
+connects to a full node or other remote server for receiving and sending
+Bitcoin transaction information, but stores the user wallet locally,
+partially validates the transactions it receives, and independently
+creates outgoing transactions.
+
+Third-party API client:: A third-party
+API client ((("third-party API clients")))is one that interacts with Bitcoin through a third-party
+system of APIs rather than by
+connecting to the Bitcoin network directly. The wallet may be stored by
+the user or by third-party servers, but the client trusts the remote
+server to provide it with accurate information and protect its ((("wallets", "types of", startref="wallet-type")))privacy.
+
+[TIP]
+====
+Bitcoin ((("Bitcoin", "as peer-to-peer network", secondary-sortas="peer-to-peer network")))((("peer-to-peer networks, Bitcoin as")))((("peers")))((("clients")))is a peer-to-peer (P2P) network.  Full nodes are the _peers:_
+each peer individually validates every confirmed transaction and can
+provide data to its user with complete authority.  Lightweight wallets
+and other software are _clients:_ each client depends on one or more peers
+to provide it with valid data.  Bitcoin clients can perform secondary
+validation on some of the data they receive and make connections to
+multiple peers to reduce their dependence on the integrity of a single
+peer, but the security of a client ultimately relies on the integrity of
+its peers.
+====
+
+===== Who controls the keys
+
+A very((("wallets", "key control")))((("keys, control of")))((("bitcoins", "key control"))) important additional consideration is _who controls the keys_. As
+we will see in subsequent chapters, access to bitcoins is
+controlled by "private keys," which are like very long PINs. If
+you are the only one to have control over these private
+keys, you are in control of your bitcoins. Conversely, if you do not have
+control, then your bitcoins are managed by a third-party who
+ultimately controls your funds on your behalf. Key management software falls into two
+important categories based on control: _wallets_, where you
+control the keys, and the funds and accounts with custodians where some
+third-party controls the keys. To emphasize this point, I (Andreas)
+coined the phrase: _Your keys, your coins. Not your keys, not your coins_.
+
+Combining these categorizations, many Bitcoin wallets fall into a few
+groups, with the three most common being desktop full node
+(you control the keys), mobile lightweight wallet (you control the keys), and web-based
+accounts with third parties (you don't control the keys). The lines between different categories
+are sometimes blurry, as software runs on multiple platforms and can
+interact with the network in different ways.
+
+==== Quick Start
+
+Alice is not a
+technical user and only recently heard about Bitcoin from her friend
+Joe. While at a party, Joe is enthusiastically explaining
+Bitcoin to everyone around him and is offering a demonstration. Intrigued,
+Alice asks how she can get started with Bitcoin. Joe says that a mobile
+wallet is best for new users and he recommends a few of his favorite
+wallets. Alice downloads one of Joe's recommendations
+and installs it on her phone.
+
+When Alice runs her wallet application for the first time, she chooses
+the option to create a new Bitcoin wallet. Because the wallet she has
+chosen is a ((("wallets", "noncustodial")))((("noncustodial wallets")))noncustodial wallet, Alice (and only Alice) will be in
+control of her keys. Therefore, she bears responsibility for backing
+them up, since losing the keys means she loses access to her bitcoins. To
+facilitate this, her wallet produces a _recovery code_ that can be used
+to restore her wallet.
+
+[[recovery_code_intro]]
+==== Recovery Codes
+
+Most ((("wallets", "recovery codes", id="wallet-recovery")))((("recovery codes", id="recovery-code")))((("backing up", "recovery codes", see="recovery codes")))modern noncustodial Bitcoin wallets will provide a recovery
+code for their user
+to back up.  The recovery code usually consists of numbers, letters, or words
+selected randomly by the software, and is used as the basis for the keys
+that are generated by the wallet. See <<recovery_code_sample>> for
+examples.
+
+++++
+<table id="recovery_code_sample">
+<caption>Sample recovery codes</caption>
+<thead>
+<tr>
+<th>Wallet</th>
+<th>Recovery code</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><p>BlueWallet</p></td>
+<td><p>(1) media (2) suspect (3) effort (4) dish (5) album (6) shaft (7) price (8) junk (9) pizza (10) situate (11) oyster (12) rib</p></td>
+</tr>
+<tr>
+<td><p>Electrum</p></td>
+<td><p>nephew dog crane clever quantum crazy purse traffic repeat fruit old clutch</p></td>
+</tr>
+<tr>
+<td><p>Muun</p></td>
+<td><p>LAFV TZUN V27E NU4D WPF4 BRJ4 ELLP BNFL</p></td>
+</tr>
+</tbody>
+</table>
+++++
+
+[TIP]
+====
+A recovery code is ((("mnemonic phrases", see="recovery codes")))((("seed phrases", see="recovery codes")))sometimes called a "mnemonic" or "mnemonic phrase,"
+which implies you should memorize the phrase, but writing the phrase
+down on paper takes less work and tends to be more reliable than most
+people's memories.  Another alternative name is "seed phrase" because
+it provides the input ("seed") to the function that generates all of
+a wallet's keys.
+====
+
+If something happens to Alice's wallet, she can download a new copy of
+her wallet software and enter this recovery code to rebuild the wallet
+database of all the onchain transactions she's ever sent or received.
+However, recovering from the recovery code will not by itself restore any additional
+data Alice entered into her wallet, such as the labels she associated
+with particular addresses or transactions.  Although losing access to
+that metadata isn't as important as losing access to money, it can
+still be important in its own way.  Imagine you need to review an old
+bank or credit card statement and the name of every entity you paid (or
+who paid you) has been blanked out.  To prevent losing metadata, many
+wallets provide an additional backup feature beyond recovery codes.
+
+For some wallets, that additional backup feature is even more important
+today than it used to be.  Many Bitcoin payments ((("offchain technology")))are now made using
+_offchain_ technology, where not every payment is stored in the public blockchain.  This reduces user's costs and improves privacy, among other
+benefits, but it means that a mechanism like recovery codes that depends on
+onchain data can't guarantee recovery of all of a user's bitcoins.  For
+applications with offchain support, it's important to make frequent
+backups of the wallet database.
+
+Of note, when receiving funds to a new mobile wallet for the first time,
+many wallets will often re-verify that you have securely backed-up your
+recovery code. This can range from a simple prompt to requiring the
+user to manually re-enter the code.
+
+[WARNING]
+====
+Although many legitimate wallets will prompt you to re-enter
+your recovery code, there are also many malware applications that mimic the
+design of a wallet, insist you enter your recovery code, and then
+relay any entered code to the malware developer so they can steal
+your funds.  This is the equivalent of phishing websites that try to
+trick you into giving them your bank passphrase.  For most wallet
+applications, the only times they will ask for your recovery code are during
+the initial set up (before you have received any bitcoins) and during
+recovery (after you lost access to your original wallet).  If the application
+asks for your recovery code any other time, consult with an expert to
+ensure you aren't ((("wallets", "recovery codes", startref="wallet-recovery")))((("recovery codes", startref="recovery-code")))being phished.
+====
+
+==== Bitcoin Addresses
+
+Alice is ((("addresses", "explained")))now ready to start using her new Bitcoin wallet.  Her wallet application randomly generated a private key (described in more detail in <<private_keys>>) that will be used to derive Bitcoin addresses that direct to her wallet. At this point, her Bitcoin addresses are not known to the Bitcoin network or "registered" with any part of the Bitcoin system. Her Bitcoin addresses are simply numbers that correspond to her private key that she can use to control access to the funds. The addresses are generated independently by her wallet without reference or registration with any service.
+
+[TIP]
+====
+There
+are a variety ((("invoices")))of Bitcoin addresses and invoice formats.  Addresses and
+invoices can be shared with other Bitcoin users
+who can use them to send bitcoins directly to your wallet.  You can share
+an address or invoice with other people without worrying about the
+security of your bitcoins.  Unlike a bank account number, nobody who
+learns one of your Bitcoin addresses can withdraw money from your wallet--you
+must initiate all spends.  However, if you give two people the same
+address, they will be able to see how many bitcoins the other person sent
+you.  If you post your address publicly, everyone will be able to see
+how much bitcoin other people sent to that address.  To protect your privacy, you
+should generate a new invoice with a new address each time you request a
+payment.
+====
+
+==== Receiving Bitcoin
+
+Alice((("bitcoins", "receiving")))((("receiving bitcoins"))) uses the _Receive_ button, which displays a QR code, shown in <<wallet_receive>>.
+
+[role="width-50"]
+[[wallet_receive]]
+.Alice uses the Receive screen on her mobile Bitcoin wallet and displays her address in a QR code format.
+image::images/mbc3_0101.png["Wallet receive screen with QR code displayed.  Image derived from Bitcoin Design Guide CC-BY"]
+
+The QR code is the square with a pattern of black and white dots, serving as a form of barcode that contains the same information in a format that can be scanned by Joe's smartphone camera.
+
+[WARNING]
+====
+Any funds sent to the addresses in this book will be lost.  If you want
+to test sending bitcoins, please consider donating it to a
+bitcoin-accepting charity.
+====
+
+[[getting_first_bitcoin]]
+==== Getting Your First Bitcoin
+
+The((("bitcoins", "acquiring", id="bitcoin-acquire")))((("acquiring bitcoins", id="acquire-bitcoin"))) first task for new users is to acquire some bitcoin.
+
+Bitcoin transactions are irreversible. Most electronic payment networks such as credit cards, debit cards, PayPal, and bank account transfers are reversible. For someone selling bitcoin, this difference introduces a very high risk that the buyer will reverse the electronic payment after they have received bitcoin, in effect defrauding the seller. To mitigate this risk, companies accepting traditional electronic payments in return for bitcoin usually require buyers to undergo identity verification and credit-worthiness checks, which may take several days or weeks. As a new user, this means you cannot buy bitcoin instantly with a credit card. With a bit of patience and creative thinking, however, you won't need to.
+
+
+Here are some methods for acquiring bitcoin as a new user:
+
+* Find a friend who has bitcoins and buy some from him or her directly. Many Bitcoin users start this way. This method is the least complicated. One way to meet people with bitcoins is to attend a local Bitcoin meetup listed at pass:[<a href="https://meetup.com" class="orm:hideurl">Meetup.com</a>].
+* Earn bitcoin by selling a product or service for bitcoin. If you are a programmer, sell your programming skills. If you're a hairdresser, cut hair for bitcoins.
+* Use a Bitcoin ATM in your city.  A Bitcoin ATM is a machine that accepts cash and sends bitcoins to your smartphone Bitcoin wallet.
+* Use((("bitcoins", "currency exchanges")))((("currency exchanges"))) a Bitcoin currency exchange linked to your bank account. Many countries now have currency exchanges that offer a market for buyers and sellers to swap bitcoins with local currency. Exchange-rate listing services, such as https://bitcoinaverage.com[BitcoinAverage], often show a list of Bitcoin exchanges for each currency.
+
+[TIP]
+====
+One of the advantages of
+Bitcoin over other payment systems is that, when used correctly, it
+affords users much more privacy. Acquiring, holding, and spending
+bitcoin does not require you to divulge sensitive and personally
+identifiable information to third parties. However, where bitcoin
+touches traditional systems, such as currency exchanges, national and
+international regulations often apply. In order to exchange bitcoin for
+your national currency, you will often be required to provide proof of
+identity and banking information. Users should be aware that once a
+Bitcoin address is attached to an identity, other associated Bitcoin
+transactions may also become easy to identify and track--including
+transactions made earlier. This is one reason
+many users choose to maintain dedicated exchange accounts independent from
+their wallets.
+====
+
+Alice was introduced to Bitcoin by a friend, so she has an easy way to acquire her first bitcoins. Next, we will look at how she buys bitcoins from her friend Joe and how Joe sends the bitcoins to her ((("bitcoins", "acquiring", startref="bitcoin-acquire")))((("acquiring bitcoins", startref="acquire-bitcoin")))wallet.
+
+[[bitcoin_price]]
+==== Finding the Current Price of Bitcoin
+
+Before ((("bitcoins", "exchange rate", id="bitcoin-exchange-rate")))((("exchange rate", id="exchange-rate")))((("current price of bitcoins", id="current-price")))Alice can buy bitcoin from Joe, they have to agree on the _exchange rate_ between bitcoin and US dollars. This brings up a common question for those new to Bitcoin: "Who sets the price
+of bitcoins?" The short answer is that the price is set by markets.
+
+Bitcoin, like most other currencies, has a _floating exchange rate_. That means that the value of bitcoin fluctuates according to supply and demand in the various markets where it is traded. For example, the "price" of bitcoin in US dollars is calculated in each market based on the most recent trade of bitcoins and US dollars. As such, the price tends to fluctuate minutely several times per second. A pricing service will aggregate the prices from several markets and calculate a volume-weighted average representing the broad market exchange rate of a currency pair (e.g., BTC/USD).
+
+There are hundreds of applications and websites that can provide the current market rate. Here are some of the most popular:
+
+https://bitcoinaverage.com[Bitcoin Average]:: A site that provides a simple view of the volume-weighted average for each currency.
+https://coincap.io[CoinCap]:: A service listing the market capitalization and exchange rates of hundreds of cryptocurrencies, including bitcoins.
+https://oreil.ly/ACieC[Chicago Mercantile Exchange Bitcoin Reference Rate]:: A reference rate that can be used for institutional and contractual reference, provided as part of investment data feeds by the CME.
+
+In addition to these various sites and applications, some Bitcoin
+wallets will automatically convert amounts between bitcoin and other currencies.
+
+[[sending_receiving]]
+==== Sending and Receiving Bitcoin
+
+Alice ((("bitcoins", "spending", id="bitcoin-send")))((("spending bitcoins", id="send-bitcoin")))((("bitcoins", "receiving", id="bitcoin-receive")))((("receiving bitcoins", id="receive-bitcoin")))has
+decided to buy 0.001 bitcoins. After she and Joe check the exchange rate,
+she gives Joe an appropriate amount of cash, opens her mobile wallet
+application, and selects Receive. This
+displays a QR code with Alice's first Bitcoin address.
+
+Joe then selects Send on his smartphone wallet and opens the QR code
+scanner.  This allows Joe to scan the barcode with his smartphone camera
+so that he doesn't have to type in Alice's Bitcoin address, which is
+quite long.
+
+Joe now has Alice's Bitcoin address set as the recipient. Joe enters the amount as 0.001 bitcoins (BTC); see <<wallet-send>>.  Some wallets may
+show the amount in a different denomination: 0.001 BTC is 1 millibitcoin
+(mBTC) or 100,000 satoshis (sats).
+
+Some wallets may also suggest Joe enter a label for this transaction; if
+so, Joe enters "Alice".  Weeks or months from now, this will help Joe
+remember why he sent these 0.001 bitcoins.  Some wallets may also prompt
+Joe about fees.  Depending on the wallet and how the transaction is
+being sent, the wallet may ask Joe to either enter a transaction fee rate or
+prompt him with a suggested fee (or fee rate).  The higher the transaction fee, the
+faster the transaction will be confirmed (see <<confirmations>>).
+
+[[wallet-send]]
+.Bitcoin wallet send screen.
+image::images/mbc3_0102.png["Wallet send screen.  Image derived from Bitcoin Design Guide CC-BY"]
+
+Joe then carefully checks to make sure he has entered the correct
+amount, because he is about to transmit money and mistakes will soon become
+irreversible. After double-checking the address and amount, he presses
+Send to transmit the transaction. Joe's mobile Bitcoin wallet constructs
+a transaction that assigns 0.001 BTC to the address provided by Alice,
+sourcing the funds from Joe's wallet, and signing the transaction with
+Joe's private keys. This tells the Bitcoin network that Joe has
+authorized a transfer of value to Alice's new address. As the
+transaction is transmitted via the peer-to-peer protocol, it quickly
+propagates across the Bitcoin network. After just a few seconds, most of
+the well-connected nodes in the network receive the transaction and see
+Alice's address for the first time.
+
+Meanwhile, Alice's wallet is constantly "listening" for new
+transactions on the Bitcoin network, looking for any that match the
+addresses it contains. A few seconds after Joe's wallet transmits the
+transaction, Alice's wallet will indicate that it is receiving
+0.001 BTC.
+
+[[confirmations]]
+.Confirmations
+****
+At first, ((("bitcoins", "clearing transactions")))((("clearing transactions")))((("transactions", "clearing")))((("confirmations")))Alice's address will show the transaction from Joe as "Unconfirmed." This means that the transaction has been propagated to the network but has not yet been recorded in the Bitcoin transaction journal, known as the blockchain. To be confirmed, a transaction must be included in a block and added to the blockchain, which happens every 10 minutes, on average. In traditional financial terms this is known as _clearing_. For more details on propagation, validation, and clearing (confirmation) of bitcoin transactions, see <<mining>>.
+****
+
+Alice is now the proud owner of 0.001 BTC that she can spend. Over the next few days, Alice buys more bitcoin using an ATM and((("bitcoins", "spending", startref="bitcoin-send")))((("spending bitcoins", startref="send-bitcoin")))((("bitcoins", "receiving", startref="bitcoin-receive")))((("receiving bitcoins", startref="receive-bitcoin"))) an exchange. In the next chapter we will look at her first purchase with Bitcoin, and examine the underlying transaction and propagation technologies in more detail.

ch02.asciidoc

@@ -1,288 +0,0 @@
-[[ch02_bitcoin_overview]]
-== How Bitcoin Works
-
-=== Transactions, Blocks, Mining, and the Blockchain
-
-((("bitcoin", "overview of", id="BCover02")))((("central trusted authority")))((("decentralized systems", "bitcoin overview", id="DCSover02")))The Bitcoin system, unlike traditional banking and payment systems, is based on decentralized trust. Instead of a central trusted authority, in bitcoin, trust is achieved as an emergent property from the interactions of different participants in the Bitcoin system. In this chapter, we will examine bitcoin from a high level by tracking a single transaction through the Bitcoin system and watch as it becomes "trusted" and accepted by the bitcoin mechanism of distributed consensus and is finally recorded on the blockchain, the distributed ledger of all transactions. Subsequent chapters will delve into the technology behind transactions, the network, and mining.
-
-==== Bitcoin Overview
-
-In the overview diagram shown in <<bitcoin-overview>>, we see that the Bitcoin system consists of users with wallets containing keys, transactions that are propagated across the network, and miners who produce (through competitive computation) the consensus blockchain, which is the authoritative ledger of all transactions.
-
-
-((("blockchain explorer sites")))Each example in this chapter is based on an actual transaction made on the Bitcoin network, simulating the interactions between the users (Joe, Alice, Bob, and Gopesh) by sending funds from one wallet to another. While tracking a transaction through the Bitcoin network to the blockchain, we will use a _blockchain explorer_ site to visualize each step. A blockchain explorer is a web application that operates as a bitcoin search engine, in that it allows you to search for addresses, transactions, and blocks and see the relationships and flows between them.
-
-[[bitcoin-overview]]
-.Bitcoin overview
-image::images/mbc2_0201.png["Bitcoin Overview"]
-
-((("Bitcoin Block Explorer")))((("BlockCypher Explorer")))((("Blockchain.com")))((("BitPay Insight")))Popular blockchain explorers include:
-
-* https://live.blockcypher.com[BlockCypher Explorer]
-* https://blockchain.com[Blockchain.com]
-* https://insight.bitpay.com[BitPay Insight]
-* https://blockstream.info[Blockstream Explorer]
-
-Each of these has a search function that can take a Bitcoin address, transaction hash, block number, or block hash and retrieve corresponding information from the Bitcoin network. With each transaction or block example, we will provide a URL so you can look it up yourself and study it in detail.
-
-
-[[cup_of_coffee]]
-==== Buying a Cup of Coffee
-
-((("use cases", "buying coffee", id="UCcoffee02")))Alice, introduced in the previous chapter, is a new user who has just acquired her first bitcoin. In <<getting_first_bitcoin>>, Alice met with her friend Joe to exchange some cash for bitcoin. The transaction created by Joe funded Alice's wallet with 0.10 BTC. Now Alice will make her first retail transaction, buying a cup of coffee at Bob's coffee shop in Palo Alto, California.
-
-((("exchange rates", "determining")))Bob's Cafe recently started accepting bitcoin payments by adding a bitcoin option to its point-of-sale system. The prices at Bob's Cafe are listed in the local currency (US dollars), but at the register, customers have the option of paying in either dollars or bitcoin. Alice places her order for a cup of coffee and Bob enters it into the register, as he does for all transactions.  The point-of-sale system automatically converts the total price from US dollars to bitcoin at the prevailing market rate and displays the price in both currencies:
-
-----
-Total:
-$1.50 USD
-0.015 BTC
-----
-
-
-((("millibits")))Bob says, "That's one-dollar-fifty, or fifteen millibits."
-
-((("payment requests")))((("QR codes", "payment requests")))Bob's point-of-sale system will also automatically create a special QR code containing a _payment request_ (see <<payment-request-QR>>).
-
-Unlike a QR code that simply contains a destination Bitcoin address, a payment request is a QR-encoded URL that contains a destination address, a payment amount, and a generic description such as "Bob's Cafe." This allows a bitcoin wallet application to prefill the information used to send the payment while showing a human-readable description to the user. You can scan the QR code with a bitcoin wallet application to see what Alice would see.
-
-
-[[payment-request-QR]]
-.Payment request QR code
-image::images/mbc2_0202.png["payment-request"]
-
-[TIP]
-====
-((("QR codes", "warnings and cautions")))((("transactions", "warnings and cautions")))((("warnings and cautions", "avoid sending money to addresses appearing in book")))Try to scan this with your wallet to see the address and amount but DO NOT SEND MONEY.
-====
-[[payment-request-URL]]
-.The payment request QR code encodes the following URL, defined in BIP-21:
-----
-bitcoin:1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA?
-amount=0.015&
-label=Bob%27s%20Cafe&
-message=Purchase%20at%20Bob%27s%20Cafe
-
-Components of the URL
-
-A Bitcoin address: "1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA"
-The payment amount: "0.015"
-A label for the recipient address: "Bob's Cafe"
-A description for the payment: "Purchase at Bob's Cafe"
-----
-
-Alice uses her smartphone to scan the barcode on display. Her smartphone shows a payment of +0.0150 BTC+ to +Bob's Cafe+ and she selects Send to authorize the payment. Within a few seconds (about the same amount of time as a credit card authorization), Bob sees the transaction on the register, completing the transaction.
-
-In the following sections, we will examine this transaction in more detail. We'll see how Alice's wallet constructed it, how it was propagated across the network, how it was verified, and finally, how Bob can spend that amount in subsequent transactions.
-
-[NOTE]
-====
-((("fractional values")))((("milli-bitcoin")))((("satoshis")))The Bitcoin network can transact in fractional values, e.g., from millibitcoin (1/1000th of a bitcoin) down to 1/100,000,000th of a bitcoin, which is known as a satoshi.  Throughout this book, we’ll use the term “bitcoin” to refer to any quantity of bitcoin currency, from the smallest unit (1 satoshi) to the total number (21,000,000) of all bitcoin that will ever be mined.
-====
-
-You can examine Alice's transaction to Bob's Cafe on the blockchain using a block explorer site (<<view_alice_transaction>>):
-
-[[view_alice_transaction]]
-.View Alice's transaction on https://www.blockchain.com/btc/tx/0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2[blockchain.com]
-====
-----
-https://www.blockchain.com/btc/tx/0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2
-----
-====
-
-=== Bitcoin Transactions
-
-((("transactions", "defined")))In simple terms, a transaction tells the network that the owner of some bitcoin value has authorized the transfer of that value to another owner. The new owner can now spend the bitcoin by creating another transaction that authorizes the transfer to another owner, and so on, in a chain of ownership.
-
-==== Transaction Inputs and Outputs
-
-((("transactions", "overview of", id="Tover02")))((("outputs and inputs", "basics of")))Transactions are like lines in a double-entry bookkeeping ledger.  Each transaction contains one or more "inputs," which are like debits against a bitcoin account. On the other side of the transaction, there are one or more "outputs," which are like credits added to a bitcoin account. ((("fees", "transaction fees")))The inputs and outputs (debits and credits) do not necessarily add up to the same amount. Instead, outputs add up to slightly less than inputs and the difference represents an implied _transaction fee_, which is a small payment collected by the miner who includes the transaction in the ledger. A bitcoin transaction is shown as a bookkeeping ledger entry in <<transaction-double-entry>>.
-
-The transaction also contains proof of ownership for each amount of bitcoin (inputs) whose value is being spent, in the form of a digital signature from the owner, which can be independently validated by anyone. ((("spending bitcoin", "defined")))In bitcoin terms, "spending" is signing a transaction that transfers value from a previous transaction over to a new owner identified by a Bitcoin address.
-
-[[transaction-double-entry]]
-.Transaction as double-entry bookkeeping
-image::images/mbc2_0203.png["Transaction Double-Entry"]
-
-==== Transaction Chains
-
-((("chain of transactions")))Alice's payment to Bob's Cafe uses a previous transaction's output as its input. In the previous chapter, Alice received bitcoin from her friend Joe in return for cash. That transaction created a bitcoin value locked by Alice's key. Her new transaction to Bob's Cafe references the previous transaction as an input and creates new outputs to pay for the cup of coffee and receive change. The transactions form a chain, where the inputs from the latest transaction correspond to outputs from previous transactions. Alice's key provides the signature that unlocks those previous transaction outputs, thereby proving to the Bitcoin network that she owns the funds. She attaches the payment for coffee to Bob's address, thereby "encumbering" that output with the requirement that Bob produces a signature in order to spend that amount. This represents a transfer of value between Alice and Bob. This chain of transactions, from Joe to Alice to Bob, is illustrated in <<blockchain-mnemonic>>.
-
-[[blockchain-mnemonic]]
-.A chain of transactions, where the output of one transaction is the input of the next transaction
-image::images/mbc2_0204.png["Transaction chain"]
-
-==== Making Change
-
-((("change, making")))((("change addresses")))((("addresses", "change addresses")))Many bitcoin transactions will include outputs that reference both an address of the new owner and an address of the current owner, called the _change_ address. This is because transaction inputs, like currency notes, cannot be divided. If you purchase a $5 US dollar item in a store but use a $20 US dollar bill to pay for the item, you expect to receive $15 US dollars in change. The same concept applies to bitcoin transaction inputs. If you purchased an item that costs 5 bitcoin but only had a 20 bitcoin input to use, your wallet would create a single transaction that sends two outputs, one output of 5 bitcoin to the store owner and one output of 15 bitcoin back to yourself as change (less any applicable transaction fee). Importantly, the change address does not have to be the same address as that of the input and for privacy reasons is often a new address from the owner's wallet.
-
-Different wallets may use different strategies when aggregating inputs to make a payment requested by the user. They might aggregate many small inputs, or use one that is equal to or larger than the desired payment. Unless the wallet can aggregate inputs in such a way to exactly match the desired payment plus transaction fees, the wallet will need to generate some change. This is very similar to how people handle cash. If you always use the largest bill in your pocket, you will end up with a pocket full of loose change. If you only use the loose change, you'll always have only big bills. People subconsciously find a balance between these two extremes, and bitcoin wallet developers strive to program this balance.
-
-((("transactions", "defined")))((("outputs and inputs", "defined")))((("inputs", see="outputs and inputs")))In summary, _transactions_ move value from _transaction inputs_ to _transaction outputs_. An input is a reference to a previous transaction's output, showing where the value is coming from. A transaction usually includes an output that directs a specific value to a new owner's Bitcoin address and a change output back to the original owner. Outputs from one transaction can be used as inputs in a new transaction, thus creating a chain of ownership as the value is moved from owner to owner (see <<blockchain-mnemonic>>).
-
-==== Common Transaction Forms
-
-The most common form of transaction is a simple payment from one address to another, which often includes some "change" returned to the original owner. This type of transaction has one input and two outputs and is shown in <<transaction-common>>.
-
-[[transaction-common]]
-.Most common transaction
-image::images/mbc2_0205.png["Common Transaction"]
-
-Another common form of transaction is one that aggregates several inputs into a single output (see <<transaction-aggregating>>). This represents the real-world equivalent of exchanging a pile of coins and currency notes for a single larger note. Transactions like these are sometimes generated by wallet applications to clean up lots of smaller amounts that were received as change for payments.
-
-[[transaction-aggregating]]
-.Transaction aggregating funds
-image::images/mbc2_0206.png["Aggregating Transaction"]
-
-Finally, another transaction form that is seen often on the bitcoin ledger is a batched transaction, which distributes one input to multiple outputs representing multiple recipients, a technique called "transaction batching" (see <<transaction-distributing>>). Since this type of transaction is useful for saving in transaction fees, it is commonly used by commercial entities to distribute funds, such as when a company is processing payroll payments to multiple employees or when a bitcoin exchange is processing multiple customers' withdrawals in a single transaction.((("", startref="Tover02")))
-
-[[transaction-distributing]]
-.Transaction distributing funds
-image::images/mbc2_0207.png["Distributing Transaction"]
-
-=== Constructing a Transaction
-
-((("transactions", "constructing", id="Tconstruct02")))((("wallets", "constructing transactions")))Alice's wallet application contains all the logic for selecting appropriate inputs and outputs to build a transaction to Alice's specification. Alice only needs to specify a destination and an amount, and the rest happens in the wallet application without her seeing the details. Importantly, a wallet application can construct transactions even if it is completely offline. Like writing a check at home and later sending it to the bank in an envelope, the transaction does not need to be constructed and signed while connected to the Bitcoin network.
-
-==== Getting the Right Inputs
-
-((("outputs and inputs", "locating and tracking inputs")))Alice's wallet application will first have to find inputs that can pay the amount she wants to send to Bob. Most wallets keep track of all the available outputs belonging to addresses in the wallet. Therefore, Alice's wallet would contain a copy of the transaction output from Joe's transaction, which was created in exchange for cash (see <<getting_first_bitcoin>>). A bitcoin wallet application that runs as a full-node client actually contains a copy of every unspent output from every transaction in the blockchain. This allows a wallet to construct transaction inputs as well as quickly verify incoming transactions as having correct inputs. However, because a full-node client takes up a lot of disk space, most user wallets run "lightweight" clients that track only the user's own unspent outputs.
-
-If the wallet application does not maintain a copy of unspent transaction outputs, it can query the Bitcoin network to retrieve this information using a variety of APIs available by different providers or by asking a full-node using an application programming interface (API) call. <<example_2-2>> shows an API request, constructed as an HTTP GET command to a specific URL. This URL will return all the unspent transaction outputs for an address, giving any application the information it needs to construct transaction inputs for spending. We use the simple command-line HTTP client _cURL_ to retrieve the response.
-
-[[example_2-2]]
-.Look up all the unspent outputs for Alice's Bitcoin address
-====
-[source,bash]
-----
-$ curl https://blockchain.info/unspent?active=1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK
-----
-====
-
-[source,json]
-----
-{
-
-	"unspent_outputs":[
-
-		{
-			"tx_hash":"186f9f998a5...2836dd734d2804fe65fa35779",
-			"tx_index":104810202,
-			"tx_output_n": 0,
-			"script":"76a9147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a888ac",
-			"value": 10000000,
-			"value_hex": "00989680",
-			"confirmations":0
-		}
-
-	]
-}
-----
-
-
-The response in <<example_2-2>> shows one unspent output (one that has not been redeemed yet) under the ownership of Alice's address +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. The response includes the reference to the transaction in which this unspent output is contained (the payment from Joe) and its value in satoshis, at 10 million, equivalent to 0.10 bitcoin. With this information, Alice's wallet application can construct a transaction to transfer that value to new owner addresses.
-
-[TIP]
-====
-View the https://www.blockchain.com/btc/tx/7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18[transaction from Joe to Alice].
-====
-
-As you can see, Alice's wallet contains enough bitcoin in a single unspent output to pay for the cup of coffee. Had this not been the case, Alice's wallet application might have to "rummage" through a pile of smaller unspent outputs, like picking coins from a purse until it could find enough to pay for the coffee. In both cases, there might be a need to get some change back, which we will see in the next section, as the wallet application creates the transaction outputs (payments).
-
-
-==== Creating the Outputs
-
-((("outputs and inputs", "creating outputs")))A transaction output is created in the form of a script that creates an encumbrance on the value and can only be redeemed by the introduction of a solution to the script. In simpler terms, Alice's transaction output will contain a script that says something like, "This output is payable to whoever can present a signature from the key corresponding to Bob's address." Because only Bob has the wallet with the keys corresponding to that address, only Bob's wallet can present such a signature to redeem this output. Alice will therefore "encumber" the output value with a demand for a signature from Bob.
-
-This transaction will also include a second output, because Alice's funds are in the form of a 0.10 BTC output, too much money for the 0.015 BTC cup of coffee. Alice will need 0.085 BTC in change. Alice's change payment is created by Alice's wallet as an output in the very same transaction as the payment to Bob. Essentially, Alice's wallet breaks her funds into two payments: one to Bob and one back to herself. She can then use (spend) the change output in a subsequent transaction.
-
-Finally, for the transaction to be processed by the network in a timely fashion, Alice's wallet application will add a small fee. This is not explicit in the transaction; it is implied by the difference between inputs and outputs. If instead of taking 0.085 in change, Alice creates only 0.0845 as the second output, there will be 0.0005 BTC (half a millibitcoin) left over. The input's 0.10 BTC is not fully spent with the two outputs, because they will add up to less than 0.10. The resulting difference is the _transaction fee_ that is collected by the miner as a fee for validating and including the transaction in a block to be recorded on the blockchain.
-
-The resulting transaction can be seen using a blockchain explorer web application, as shown in <<transaction-alice>>.
-
-[[transaction-alice]]
-[role="smallerseventyfive"]
-.Alice's transaction to Bob's Cafe
-image::images/mbc2_0208.png["Alice Coffee Transaction"]
-
-[[transaction-alice-url]]
-[TIP]
-====
-View the https://www.blockchain.com/btc/tx/0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2[transaction from Alice to Bob's Cafe].
-====
-
-==== Adding the Transaction to the Ledger
-
-The transaction created by Alice's wallet application is 258 bytes long and contains everything necessary to confirm ownership of the funds and assign new owners. Now, the transaction must be transmitted to the Bitcoin network where it will become part of the blockchain. In the next section we will see how a transaction becomes part of a new block and how the block is "mined." Finally, we will see how the new block, once added to the blockchain, is increasingly trusted by the network as more blocks are added.
-
-===== Transmitting the transaction
-
-((("propagation", "process of")))Because the transaction contains all the information necessary to process, it does not matter how or where it is transmitted to the Bitcoin network. The Bitcoin network is a peer-to-peer network, with each Bitcoin client participating by connecting to several other Bitcoin clients. The purpose of the Bitcoin network is to propagate transactions and blocks to all participants.
-
-===== How it propagates
-
-((("Bitcoin nodes", "defined")))((("nodes", see="Bitcoin nodes")))Any system, such as a server, desktop application, or wallet, that participates in the Bitcoin network by "speaking" the Bitcoin protocol is called a _Bitcoin node_. Alice's wallet application can send the new transaction to any Bitcoin node it is connected to over any type of connection: wired, WiFi, mobile, etc.  Her bitcoin wallet does not have to be connected to Bob's bitcoin wallet directly and she does not have to use the internet connection offered by the cafe, though both those options are possible, too. ((("propagation", "flooding technique")))((("flooding technique")))Any Bitcoin node that receives a valid transaction it has not seen before will immediately forward it to all other nodes to which it is connected, a propagation technique known as _flooding_. Thus, the transaction rapidly propagates out across the peer-to-peer network, reaching a large percentage of the nodes within a few seconds.
-
-===== Bob's view
-
-If Bob's bitcoin wallet application is directly connected to Alice's wallet application, Bob's wallet application might be the first node to receive the transaction. However, even if Alice's wallet sends the transaction through other nodes, it will reach Bob's wallet within a few seconds. Bob's wallet will immediately identify Alice's transaction as an incoming payment because it contains outputs redeemable by Bob's keys. Bob's wallet application can also independently verify that the transaction is well formed, uses previously unspent outputs, and contains sufficient transaction fees to be included in the next block. At this point Bob can assume, with little risk, that the transaction will shortly be included in a block and confirmed.
-
-[TIP]
-====
-((("confirmations", "of small-value transactions", secondary-sortas="small-value transactions")))A common misconception about bitcoin transactions is that they must be "confirmed" by waiting 10 minutes for a new block, or up to 60 minutes for a full six confirmations. Although confirmations ensure the transaction has been accepted by the whole network, such a delay is unnecessary for small-value items such as a cup of coffee. A merchant may accept a valid small-value transaction with no confirmations, with no more risk than a credit card payment made without an ID or a signature, as merchants routinely accept today.((("", startref="Tconstruct02")))
-====
-
-=== Bitcoin Mining
-
-((("mining and consensus", "overview of", id="MACover02")))((("blockchain (the)", "overview of mining", id="BToverview02")))Alice's transaction is now propagated on the Bitcoin network. It does not become part of the _blockchain_ until it is verified and included in a block by a process called _mining_. See <<mining>> for a detailed explanation.
-
-The Bitcoin system of trust is based on computation. Transactions are bundled into _blocks_, which require an enormous amount of computation to prove, but only a small amount of computation to verify as proven. The mining process serves two purposes in bitcoin:
-
-* ((("mining and consensus", "consensus rules", "security provided by")))((("consensus", see="mining and consensus")))Mining nodes validate all transactions by reference to bitcoin's _consensus rules_. Therefore, mining provides security for bitcoin transactions by rejecting invalid or malformed transactions.
-* Mining creates new bitcoin in each block, almost like a central bank printing new money. The amount of bitcoin created per block is limited and diminishes with time, following a fixed issuance schedule.
-
-
-Mining achieves a fine balance between cost and reward. Mining uses electricity to solve a mathematical problem. A successful miner will collect a _reward_ in the form of new bitcoin and transaction fees. However, the reward will only be collected if the miner has correctly validated all the transactions, to the satisfaction of the rules of _consensus_. This delicate balance provides security for bitcoin without a central authority.
-
-A good way to describe mining is like a giant competitive game of sudoku that resets every time someone finds a solution and whose difficulty automatically adjusts so that it takes approximately 10 minutes to find a solution. Imagine a giant sudoku puzzle, several thousand rows and columns in size. If I show you a completed puzzle you can verify it quite quickly. However, if the puzzle has a few squares filled and the rest are empty, it takes a lot of work to solve! The difficulty of the sudoku can be adjusted by changing its size (more or fewer rows and columns), but it can still be verified quite easily even if it is very large. The "puzzle" used in bitcoin is based on a cryptographic hash and exhibits similar characteristics: it is asymmetrically hard to solve but easy to verify, and its difficulty can be adjusted.
-
-((("mining and consensus", "mining farms and pools")))In <<user-stories>>, we introduced ((("use cases", "mining for bitcoin")))Jing, an entrepreneur in Shanghai. Jing runs a _mining farm_, which is a business that runs thousands of specialized mining computers, competing for the reward. Every 10 minutes or so, Jing's mining computers compete against thousands of similar systems in a global race to find a solution to a block of transactions. ((("Proof-of-Work algorithm")))((("mining and consensus", "Proof-of-Work algorithm")))Finding such a solution, the so-called _Proof-of-Work_ (PoW), requires quadrillions of hashing operations per second across the entire Bitcoin network. The algorithm for Proof-of-Work involves repeatedly hashing the header of the block and a random number with the SHA256 cryptographic algorithm until a solution matching a predetermined pattern emerges. The first miner to find such a solution wins the round of competition and publishes that block into the blockchain.
-
-Jing started mining in 2010 using a very fast desktop computer to find a suitable Proof-of-Work for new blocks. As more miners started joining the Bitcoin network, the difficulty of the problem increased rapidly. Soon, Jing and other miners upgraded to more specialized hardware, with high-end dedicated graphical processing units (GPUs), often used in gaming desktops or consoles. At the time of this writing, the difficulty is so high that it is profitable only to mine with ((("application-specific integrated circuits (ASIC)")))application-specific integrated circuits (ASIC), essentially hundreds of mining algorithms printed in hardware, running in parallel on a single silicon chip. ((("mining pools", "defined")))Jing's company also participates in a _mining pool_, which much like a lottery pool allows several participants to share their efforts and rewards. Jing's company now runs a warehouse containing thousands of  ASIC miners to mine for bitcoin 24 hours a day. The company pays its electricity costs by selling the bitcoin it is able to generate from mining, creating some income from the profits.
-
-=== Mining Transactions in Blocks
-
-((("blocks", "mining transactions in")))New transactions are constantly flowing into the network from user wallets and other applications. As these are seen by the Bitcoin network nodes, they get added to a temporary pool of unverified transactions maintained by each node. As miners construct a new block, they add unverified transactions from this pool to the new block and then attempt to prove the validity of that new block, with the mining algorithm (Proof-of-Work). The process of mining is explained in detail in <<mining>>.
-
-Transactions are added to the new block, prioritized by the highest-fee transactions first and a few other criteria. Each miner starts the process of mining a new block of transactions as soon as they receive the previous block from the network, knowing they have lost that previous round of competition. They immediately create a new block, fill it with transactions and the fingerprint of the previous block, and start calculating the Proof-of-Work for the new block. Each miner includes a special transaction in their block, one that pays their own Bitcoin address the block reward (currently 6.25 newly created bitcoin) plus the sum of transaction fees from all the transactions included in the block. If they find a solution that makes that block valid, they "win" this reward because their successful block is added to the global blockchain and the reward transaction they included becomes spendable. ((("mining pools", "operation of")))Jing, who participates in a mining pool, has set up his software to create new blocks that assign the reward to a pool address. From there, a share of the reward is distributed to Jing and other miners in proportion to the amount of work they contributed in the last round.
-
-((("candidate blocks")))((("blocks", "candidate blocks")))Alice's transaction was picked up by the network and included in the pool of unverified transactions. Once validated by the mining software it was included in a new block, called a _candidate block_, generated by Jing's mining pool. All the miners participating in that mining pool immediately start computing Proof-of-Work for the candidate block. Approximately five minutes after the transaction was first transmitted by Alice's wallet, one of Jing's ASIC miners found a solution for the candidate block and announced it to the network. Once other miners validated the winning block they started the race to generate the next block.
-
-Jing's winning block became part of the blockchain as block #277316, containing 419 transactions, including Alice's transaction. The block containing Alice's transaction is counted as one "confirmation" of that transaction.
-
-[TIP]
-====
-You can see the block that includes https://www.blockchain.com/btc/block/277316[Alice's transaction].
-====
-
-((("confirmations", "role in transactions")))Approximately 20 minutes later, a new block, #277317, is mined by another miner. Because this new block is built on top of block #277316 that contained Alice's transaction, it added even more computation to the blockchain, thereby strengthening the trust in those transactions. Each block mined on top of the one containing the transaction counts as an additional confirmation for Alice's transaction. As the blocks pile on top of each other, it becomes exponentially harder to reverse the transaction, thereby making it more and more trusted by the network.
-
-((("genesis block")))((("blocks", "genesis block")))((("blockchain (the)", "genesis block")))In the diagram in <<block-alice1>>,  we can see block #277316, which contains Alice's transaction. Below it are 277,316 blocks (including block #0), linked to each other in a chain of blocks (blockchain) all the way back to block #0, known as the _genesis block_. Over time, as the "height" in blocks increases, so does the computation difficulty for each block and the chain as a whole. The blocks mined after the one that contains Alice's transaction act as further assurance, as they pile on more computation in a longer and longer chain. By convention, any block with more than six confirmations is considered irrevocable, because it would require an immense amount of computation to invalidate and recalculate six blocks. We will examine the process of mining and the way it builds trust in more detail in <<mining>>.((("", startref="BToverview02")))((("", startref="MACover02")))
-
-[[block-alice1]]
-.Alice's transaction included in block #277316
-image::images/mbc2_0209.png["Alice's transaction included in a block"]
-
-=== Spending the Transaction
-
-((("spending bitcoin", "simplified-payment-verification (SPV)")))((("simplified-payment-verification (SPV)")))Now that Alice's transaction has been embedded in the blockchain as part of a block, it is part of the distributed ledger of bitcoin and visible to all bitcoin applications. Each Bitcoin client can independently verify the transaction as valid and spendable. Full-node clients can track the source of the funds from the moment the bitcoin were first generated in a block, incrementally from transaction to transaction, until they reach Bob's address. Lightweight clients can do what is called a simplified payment verification (see <<spv_nodes>>) by confirming that the transaction is in the blockchain and has several blocks mined after it, thus providing assurance that the miners accepted it as valid.
-
-Bob can now spend the output from this and other transactions. For example, Bob can pay a contractor or supplier by transferring value from Alice's coffee cup payment to these new owners. Most likely, Bob's bitcoin software will aggregate many small payments into a larger payment, perhaps concentrating all the day's bitcoin revenue into a single transaction. This would aggregate the various payments into a single output (and a single address). For a diagram of an aggregating transaction, see <<transaction-aggregating>>.
-
-As Bob spends the payments received from Alice and other customers, he extends the chain of transactions. Let's assume that Bob pays his web designer Gopesh((("use cases", "offshore contract services"))) in Bangalore for a new web page. Now the chain of transactions will look like <<block-alice2>>.
-
-[[block-alice2]]
-.Alice's transaction as part of a transaction chain from Joe to Gopesh, where the output of one transaction is spent as the input of the next transaction
-image::images/mbc2_0210.png["Alice's transaction as part of a transaction chain"]
-
-In this chapter, we saw how transactions build a chain that moves value from owner to owner. We also tracked Alice's transaction, from the moment it was created in her wallet, through the Bitcoin network and to the miners who recorded it on the blockchain. In the rest of this book, we will examine the specific technologies behind wallets, addresses, signatures, transactions, the network, and finally mining.((("", startref="BCover02")))((("", startref="DCSover02"))) ((("", startref="UCcoffee02")))

ch02_overview.adoc

@@ -0,0 +1,629 @@
+[[ch02_bitcoin_overview]]
+== How Bitcoin Works
+
+The Bitcoin system, unlike traditional banking and
+payment systems, does not require trust in third parties. Instead of a central
+trusted authority, in Bitcoin, each user can use software running on
+their own computer to verify the correct operation of every
+aspect of the Bitcoin system.
+In this chapter, we will examine Bitcoin from a high level by tracking a
+single transaction through the Bitcoin system and watch as it
+is recorded on the blockchain, the distributed journal of all
+transactions. Subsequent chapters will delve into the technology behind
+transactions, the network, and ((("Bitcoin", "operational overview", id="bitcoin-operational-overview-ch2")))((("blockchain explorers", id="blockchain-explorers")))mining.
+
+=== Bitcoin Overview
+
+++++
+<p class="fix_tracking">
+The Bitcoin system consists of users with wallets containing keys,
+transactions that are propagated across the network, and miners who
+produce (through competitive computation) the consensus blockchain,
+which is the authoritative journal of all transactions.
+</p>
+
+<p class="fix_tracking2">
+Each example in this chapter is based
+on an actual transaction made on the Bitcoin network, simulating the
+interactions between several users by sending
+funds from one wallet to another. While tracking a transaction through
+the Bitcoin network to the blockchain, we will use a <em>blockchain
+explorer</em> site to visualize each step. A blockchain explorer is a web
+application that operates as a Bitcoin search engine, in that it allows
+you to search for addresses, transactions, and blocks and see the
+relationships and flows between them.
+</p>
+++++
+
+Popular blockchain explorers include the following:
+
+* https://blockstream.info[Blockstream Explorer]
+* https://mempool.space[Mempool.Space]
+* https://live.blockcypher.com[BlockCypher Explorer]
+
+Each of these has a search function that can take a Bitcoin address,
+transaction hash, block number, or block hash and retrieve corresponding
+information from the Bitcoin network. With each transaction or block
+example, we will provide a URL so you can look it up yourself and study
+it in detail.
+
+[[block-explorer-privacy]]
+.Block Explorer Privacy Warning
+[WARNING]
+====
+Searching information((("privacy", "blockchain explorers"))) on a block explorer may disclose to its operator
+that you're interested in that information, allowing them to associate
+it with your IP address, browser details, past searches, or other
+identifiable information.  If you look up the transactions in this book,
+the operator of the block explorer might guess that you're learning
+about Bitcoin, which shouldn't be a problem.  But if you look up your
+own transactions, the operator may be able to guess how many bitcoins
+you've received, spent, and ((("Bitcoin", "operational overview", startref="bitcoin-operational-overview-ch2")))((("blockchain explorers", startref="blockchain-explorers")))currently own.
+====
+
+[[spending_bitcoin]]
+=== Buying from an Online Store
+
+Alice, ((("transactions", "spending bitcoins", id="transaction-spend")))((("bitcoins", "spending", id="bitcoin-spend")))((("spending bitcoins", id="spend-bitcoin")))introduced in the previous chapter, is a new user who has just
+acquired her first bitcoins. In <<getting_first_bitcoin>>, Alice met with
+her friend Joe to exchange some cash for bitcoins. Since then, Alice has
+bought additional bitcoins.  Now Alice will make
+her first spending transaction, buying access to a premium podcast episode from Bob's online store.
+
+Bob's web store recently started accepting bitcoin payments by adding a
+Bitcoin option to its website. The prices at Bob's store are listed in
+the local currency (US dollars), but at checkout, customers have the
+option of paying in either dollars or bitcoin.
+
+Alice finds the podcast episode she wants to buy and proceeds to the checkout page. At checkout,
+Alice is offered the option to pay with bitcoin in addition to the
+usual options. The checkout cart displays the price in US dollars and
+also in bitcoin (BTC), at Bitcoin's prevailing exchange rate.
+
+Bob's
+ecommerce system will automatically create a QR code((("invoices")))((("QR codes"))) containing an
+_invoice_ (<<invoice-QR>>).
+
+////
+TODO: Replace QR code with test-BTC address
+////
+
+[[invoice-QR]]
+.Invoice QR code.
+image::images/mbc3_0201.png["payment-request"]
+
+Unlike a QR code that simply contains a destination Bitcoin address, this
+invoice is a QR-encoded URI that contains a destination address,
+a payment amount, and a description.
+This allows a Bitcoin wallet application to prefill the
+information used to send the payment while showing a human-readable
+description to the user. You can scan the QR code with a bitcoin wallet
+application to see what Alice would see:
+
+
+[[invoice-URI]]
+.The invoice QR code encodes the following URI, defined in BIP21:
+----
+bitcoin:bc1qk2g6u8p4qm2s2lh3gts5cpt2mrv5skcuu7u3e4?amount=0.01577764&
+label=Bob%27s%20Store&
+message=Purchase%20at%20Bob%27s%20Store
+
+Components of the URI
+
+A Bitcoin address: "bc1qk2g6u8p4qm2s2lh3gts5cpt2mrv5skcuu7u3e4"
+The payment amount: "0.01577764"
+A label for the recipient address: "Bob's Store"
+A description for the payment: "Purchase at Bob's Store"
+----
+
+[TIP]
+====
+Try to scan this with your wallet to see
+the address and amount but DO NOT SEND MONEY.
+====
+
+Alice uses her smartphone to scan the barcode on display. Her smartphone
+shows a payment for the correct amount to +Bob's Store+ and she selects Send to
+authorize the payment. Within a few seconds (about the same amount of
+time as a credit card authorization), Bob sees the transaction on the
+register.
+
+[NOTE]
+====
+The
+Bitcoin network((("bitcoins", "fractional values")))((("fractional values of bitcoins")))((("satoshis")))((("millibitcoins"))) can transact in fractional values, e.g., from
+millibitcoin (1/1000th of a bitcoin) down to 1/100,000,000th of a
+bitcoin, which is known as a satoshi.  This book uses the same
+pluralization rules used for dollars and other traditional currencies
+when talking about amounts greater than one bitcoin and when using
+decimal notation, such as "10 bitcoins" or "0.001 bitcoins."  The same
+rules also apply to other bitcoin bookkeeping units, such as
+millibitcoins and satoshis.
+====
+
+You can use a block explorer to examine blockchain data, such as the payment made to Bob in Alice's((("transactions", "spending bitcoins", startref="transaction-spend")))((("bitcoins", "spending", startref="bitcoin-spend")))((("spending bitcoins", startref="spend-bitcoin"))) https://oreil.ly/hAeyh[transaction].
+
+In the following sections, we will examine this transaction in more
+detail. We'll see how Alice's wallet constructed it, how it was
+propagated across the network, how it was verified, and finally, how Bob
+can spend that amount in subsequent transactions.
+
+=== Bitcoin Transactions
+
+In ((("transactions", "defined")))((("bitcoins", "transactions", see="transactions")))simple terms, a transaction tells the
+network that the owner of certain bitcoins has authorized the transfer
+of that value to another owner. The new owner can now spend the bitcoin
+by creating another transaction that authorizes the transfer to another
+owner, and so on, in a chain of ownership.
+
+==== Transaction Inputs and Outputs
+
+Transactions((("transactions", "inputs", id="transaction-input-ch2")))((("transactions", "outputs", id="transaction-output-ch2")))((("inputs", id="input")))((("outputs", id="output"))) are like lines in a double-entry
+bookkeeping ledger.  Each transaction contains one or more _inputs_,
+which spend funds. On the other side of
+the transaction, there are one or more _outputs_, which receive funds.
+The inputs
+and outputs do not necessarily add up to the same
+amount. Instead, outputs add up to slightly less than inputs and the
+difference represents an ((("transaction fees")))implied _transaction fee_, which is a small
+payment collected by the miner who includes the transaction in the
+blockchain. A Bitcoin transaction is shown as a bookkeeping ledger entry in
+<<transaction-double-entry>>.
+
+The transaction also contains proof of ownership for each amount of
+bitcoins (inputs) whose value is being spent, in the form of a digital
+signature from the owner, which can be independently validated by
+anyone. In Bitcoin terms, spending
+is signing a transaction that transfers value from a previous
+transaction over to a new owner identified by a Bitcoin address.
+
+[[transaction-double-entry]]
+.Transaction as double-entry bookkeeping.
+image::images/mbc3_0202.png["Transaction Double-Entry"]
+
+==== Transaction Chains
+
+Alice's ((("transaction chains", id="transaction-chains")))payment to Bob's Store uses a
+previous transaction's output as its input. In the previous chapter,
+Alice received bitcoins from her friend Joe in return for cash.
+We've labeled that as _Transaction 1_ (Tx1) in <<transaction-chain>>.
+
+Tx1 sent 0.001 bitcoins (100,000 satoshis) to an output locked by
+Alice's key. Her new transaction to Bob's Store (Tx2) references the
+previous output as an input.  In the illustration, we show that
+reference using an arrow and by labeling the input as "Tx1:0".  In an
+actual transaction, the reference is the 32-byte transaction identifier
+(txid) for the transaction where Alice received the money from Joe.  The
+":0" indicates the position of the output where Alice received the
+money; in this case, the first position (position 0).
+
+As shown, actual Bitcoin transactions don't
+explicitly include the value of their input.  To determine the value of
+an input, software needs to use the input's reference to find the
+previous transaction output being spent.
+
+Alice's Tx2 contains two new outputs, one paying 75,000 satoshis for the
+podcast and another paying 20,000 satoshis back to Alice to receive
+change.
+
+////
+@startditaa
+      Transaction 1             Tx2                   Tx3
+     Inputs  Outputs         In     Out           In      Out
+   +-------+---------+   +-------+--------+    +-------+--------+
+   |       |         |   |       | cDDD   |    |       |        |
+<--+ Tx0꞉0 | 100,000 |<--+ Tx1꞉0 | 20,000 |  +-+ Tx2꞉1 | 67,000 |
+   |       |         |   |       |        |  | |       |        |
+   +-------+---------+   +-------+--------+  | +-------+--------+
+   |       | cDDD    |   |       |        |  | |       |        |
+   |       | 500,000 |   |       | 75,000 |<-+ |       |        |
+   |       |         |   |       |        |    |       |        |
+   +-------+---------+   +-------+--------+    +-------+--------+
+        Fee꞉ (unknown)           Fee꞉ 5,000            Fee꞉ 8,000
+@enddittaa
+////
+
+[[transaction-chain]]
+.A chain of transactions, where the output of one transaction is the input of the next transaction.
+image::images/mbc3_0203.png["Transaction chain"]
+
+[TIP]
+====
+Serialized Bitcoin transactions--the data format that software uses for
+sending transactions--encodes the value to transfer using an integer
+of the smallest defined onchain unit of value.  When Bitcoin was first
+created, this unit didn't have a name and some developers simply called
+it the _base unit._  Later many users began calling this unit a
+_satoshi_ (sat) in honor of Bitcoin's creator.  In <<transaction-chain>>
+and some other illustrations in this book, we use satoshi values because
+that's what the protocol itself ((("satoshis")))uses.
+====
+
+==== Making Change
+
+In addition((("transactions", "change output", id="transaction-change-output")))((("change output", id="change-output")))((("outputs", "change output", id="output-change"))) to one or more outputs that pay the receiver of
+bitcoins, many transactions will also include an output that pays the
+spender of the bitcoins, called a _change_ output.
+This is because transaction inputs,
+like currency notes, cannot be partly spent. If you purchase a $5 US item in a store but use a $20 bill to pay for the item, you
+expect to receive $15 in change. The same concept applies to
+Bitcoin transaction inputs. If you purchased an item that costs 5
+bitcoins but only had an input worth 20 bitcoins to use, you would send one
+output of 5 bitcoins to the store owner and one output of 15 bitcoins back
+to yourself as change (not counting your transaction fee).
+
+At the level of the Bitcoin protocol, there is no difference between a
+change output (and the address it pays, called a _change address_) and a
+payment output.
+
+Importantly, the change address does not have to be the
+same address as that of the input and, for privacy reasons, is often a new
+address from the owner's wallet.  In ideal circumstances, the two
+different uses of outputs both use never-before-seen addresses and
+otherwise look identical, preventing any third party from determining
+which outputs are change and which are payments.  However, for
+illustration purposes, we've added shading to the change outputs in
+<<transaction-chain>>.
+
+Not every transaction has a change output.  Those that don't are ((("changeless transactions")))((("transactions", "changeless")))called
+_changeless transactions_, and they can have only a single output.
+Changeless transactions are only a practical option if the amount being
+spent is roughly the same as the amount available in the transaction
+inputs minus the anticipated transaction fee.  In <<transaction-chain>>,
+we see Bob creating Tx3 as a changeless transaction that spends the
+output he received in Tx2.
+
+==== Coin Selection
+
+Different wallets((("transactions", "coin selection")))((("coin selection in transactions")))((("selecting", "coins in transactions"))) use different strategies when choosing which
+inputs to use in a payment, called _coin selection_.
+
+They might aggregate many small
+inputs, or use one that is equal to or larger than the desired payment.
+Unless the wallet can aggregate inputs in such a way to exactly match
+the desired payment plus transaction fees, the wallet will need to
+generate some change. This is very similar to how people handle cash. If
+you always use the largest bill in your pocket, you will end up with a
+pocket full of loose change. If you only use the loose change, you'll
+often have only big bills. People subconsciously find a balance between
+these two extremes, and Bitcoin wallet developers strive to program this
+balance.
+
+==== Common Transaction Forms
+
+A very ((("transactions", "common types", id="transaction-common-ch2")))common form of transaction is a simple payment. This type of
+transaction has one input and two outputs and is shown in
+<<transaction-common>>.
+
+[[transaction-common]]
+.Most common transaction.
+image::images/mbc3_0204.png["Common Transaction"]
+
+Another common form of transaction ((("consolidation transactions")))is a _consolidation transaction_, which spends several inputs
+into a single output (<<transaction-consolidating>>). This represents
+the real-world equivalent of exchanging a pile of coins and currency
+notes for a single larger note. Transactions like these are sometimes
+generated by wallets and businesses to clean up lots of smaller amounts.
+
+[[transaction-consolidating]]
+.Consolidation transaction aggregating funds.
+image::images/mbc3_0205.png["Aggregating Transaction"]
+
+Finally, another transaction form that is seen often on the
+blockchain ((("payment batching")))is _payment batching_, which pays to multiple outputs
+representing multiple recipients (<<transaction-distributing>>).
+This type of transaction is sometimes used by commercial entities to
+distribute funds, such as when processing payroll payments to multiple employees.
+
+[[transaction-distributing]]
+.Batch transaction distributing funds.
+image::images/mbc3_0206.png["Distributing Transaction"]
+
+=== Constructing a Transaction
+
+Alice's wallet((("transactions", "constructing", id="transaction-construct"))) application contains all
+the logic for selecting inputs and generating outputs to build a
+transaction to Alice's specification. Alice only needs to choose a
+destination, amount, and transaction fee, and the rest happens in the wallet
+application without her seeing the details. Importantly, if a wallet
+already knows what inputs it controls, it can construct transactions
+even if it is completely offline.
+Like writing a check at home and later sending it to the bank in an
+envelope, the transaction does not need to be constructed and signed
+while connected to the Bitcoin network.
+
+==== Getting the Right Inputs
+
+Alice's wallet
+application ((("inputs", "constructing transactions")))will first have to find inputs that can pay the amount she
+wants to send to Bob. Most wallets keep track of all the available
+outputs belonging to addresses in the wallet. Therefore, Alice's wallet
+would contain a copy of the transaction output from Joe's transaction,
+which was created in exchange for cash (see <<getting_first_bitcoin>>).
+A Bitcoin wallet application that runs on a full node actually
+contains a copy of every confirmed transaction's ((("UTXOs (unspent transaction outputs)")))unspent outputs, called
+_unspent transaction outputs_ (UTXOs).
+However, because full nodes use more resources, many
+user wallets run lightweight clients that track only the user's own
+UTXOs.
+
+In this case, this single
+UTXO is sufficient to pay for the podcast. Had this not been the case,
+Alice's wallet application might have to combine several
+smaller UTXOs, like picking coins from a purse, until it could
+find enough to pay for the podcast. In both cases, there might be a need
+to get some change back, which we will see in the next section, as the
+wallet application creates the transaction outputs (payments).
+
+
+==== Creating the Outputs
+
+A transaction output((("outputs", "constructing transactions"))) is
+created with a
+script that says something like, "This output is paid to whoever can
+present a signature from the key corresponding to Bob's public address."
+Because only Bob has the wallet with the keys corresponding to that
+address, only Bob's wallet can present such a signature to later spend this
+output. Alice will therefore _encumber_ the output value with a demand
+for a signature from Bob.
+
+This transaction will also include a second output ((("change output")))because Alice's
+funds contain more money than the cost of the
+podcast. Alice's change
+output is created in the very same
+transaction as the payment to Bob. Essentially, Alice's wallet breaks
+her funds into two outputs: one to Bob and one back to herself. She can
+then spend the change output in a subsequent transaction.
+
+Finally, for the transaction to be processed by the network in a((("transaction fees"))) timely
+fashion, Alice's wallet application will add a small fee. The fee is not
+explicitly stated in the transaction; it is implied by the difference in value between
+inputs and outputs.  This transaction fee is collected by the
+miner as a fee for including the transaction in a block
+that gets recorded on the blockchain.
+
+[[transaction-alice-url]]
+[TIP]
+====
+View the https://oreil.ly/GwBq1[transaction from Alice to Bob's Store].
+====
+
+==== Adding the Transaction to the Blockchain
+
+The ((("transactions", "adding to blockchain", id="transaction-add-blockchain")))((("blockchain", "adding transactions to", id="blockchain-add-transaction")))transaction created by Alice's wallet application
+contains everything necessary to confirm ownership of the funds and
+assign new owners. Now, the transaction must be transmitted to the
+Bitcoin network where it will become part of the blockchain. In the next
+section we will see how a transaction becomes part of a new block and
+how the block is mined. Finally, we will see how the new block, once
+added to the blockchain, is increasingly trusted by the network as more
+blocks are added.
+
+===== Transmitting the transaction
+
+Because the transaction contains all
+the information necessary for it to be processed, it does not matter how or where it
+is transmitted to the Bitcoin network. The Bitcoin network is a
+peer-to-peer network, with each Bitcoin peer participating by
+connecting to several other Bitcoin peers. The purpose of the Bitcoin
+network is to propagate transactions and blocks to all participants.
+
+===== How it propagates
+
+
+Peers in ((("Bitcoin", "as peer-to-peer network", secondary-sortas="peer-to-peer network")))((("peer-to-peer networks, Bitcoin as")))the Bitcoin peer-to-peer network are programs that have both
+the software logic and the data necessary for them to fully verify the
+correctness of a new transaction.  The connections between peers are
+often visualized as edges (lines) in a graph, with the peers themselves
+being the nodes (dots).  For that reason, Bitcoin peers are commonly
+called "full verification nodes," ((("full nodes")))or _full nodes_ for short.
+
+Alice's wallet application can send the new
+transaction to any Bitcoin node over any type of
+connection: wired, WiFi, mobile, etc.  It can also send the transaction
+to another program (such as a block explorer) that will relay it to a
+node.  Her Bitcoin wallet does not have
+to be connected to Bob's Bitcoin wallet directly and she does not have
+to use the internet connection offered by Bob, though both those
+options are possible too. Any Bitcoin node that receives a
+valid transaction it has not seen before will forward it to
+all other nodes to which it is connected, a propagation technique known
+((("gossiping")))as _gossiping_. Thus, the transaction rapidly propagates out across the
+peer-to-peer network, reaching a large percentage of the nodes within a
+few seconds.
+
+===== Bob's view
+
+If Bob's Bitcoin wallet application is directly connected to Alice's
+wallet application, Bob's wallet application might be the first to
+receive the transaction. However, even if Alice's wallet sends the
+transaction through other nodes, it will reach Bob's wallet within a few
+seconds. Bob's wallet will immediately identify Alice's transaction as
+an incoming payment because it contains an output redeemable by Bob's
+keys. Bob's wallet application can also independently verify that the
+transaction is well formed.  If Bob is using his own full node, his
+wallet can further verify Alice's transaction only spends((("transactions", "constructing", startref="transaction-construct")))((("transactions", "adding to blockchain", startref="transaction-add-blockchain")))((("blockchain", "adding transactions to", startref="blockchain-add-transaction"))) valid UTXOs.
+
+=== Bitcoin Mining
+
+Alice's transaction((("mining", "operational overview", id="mining-operational-overview"))) is now propagated on the Bitcoin
+network. It does not become part of the _blockchain_ until it is
+included in a block by a process called _mining_ and that block has been
+validated by full nodes. See
+<<mining>> for a detailed explanation.
+
+Bitcoin's system of counterfeit protection is based on computation.
+Transactions are bundled((("transactions", "in blocks", secondary-sortas="blocks")))((("blocks"))) into _blocks_.  Blocks have a very small header
+that must be formed in a very specific way, requiring an enormous
+amount of computation to get right--but only a small amount of
+computation to verify as correct.
+The mining process serves two purposes in Bitcoin:
+
+[role="less_space pagebreak-before"]
+* Miners can only
+receive honest income from creating blocks that follow all of((("consensus rules"))) Bitcoin's
+_consensus rules_.  Therefore, miners are normally incentivized to
+only include valid transactions in their blocks and the blocks they
+build upon.  This allows users to optionally make a trust-based
+assumption that any transaction in a block is a valid transaction.
+
+* Mining currently creates new bitcoins in each block, almost like a central bank
+printing new money. The amount of bitcoin created per block is limited
+and diminishes with time, following a fixed issuance schedule.
+
+
+Mining achieves a fine balance between cost and reward. Mining uses
+electricity to solve a computational problem. A successful miner will
+collect ((("rewards")))a _reward_ in the form of new bitcoins and transaction fees.
+However, the reward will only be collected if the miner has only
+included valid transactions, with the Bitcoin protocol's rules for
+_consensus_ determining what is valid.  This delicate balance provides
+security for Bitcoin without a central authority.
+
+Mining is designed to be a decentralized lottery.  Each miner can create
+their own lottery ticket by creating ((("candidate blocks")))a _candidate block_ that includes
+the new transactions they want to mine plus some additional data fields.
+The miner inputs their candidate into a specially designed algorithm that
+scrambles((("hash functions"))) (or "hashes") the data, producing output that looks nothing
+like the input data.  This _hash_ function will always produce the same
+output for the same input--but nobody can predict what the output will
+look like for a new input, even if it is only slightly different from a
+previous input.  If the output of the hash function matches a template
+determined by the Bitcoin protocol, the miner wins the lottery and
+Bitcoin users will accept the block with its transactions as a
+valid block.  If the output doesn't match the template, the miner makes
+a small change to their candidate block and tries again.  As of this
+writing, the number of candidate blocks miners need to try before finding
+a winning combination is about 168 billion trillion.  That's also how
+many times the hash function needs to be run.
+
+However, once a winning combination has been found, anyone can verify
+the block is valid by running the hash function just once.  That makes a
+valid block something that requires an incredible amount of work to
+create but only a trivial amount of work to verify.  The simple
+verification process is able to probabalistically prove the work was
+done, so the data necessary to generate that proof--in this case, the
+block--is called((("proof-of-work algorithm"))) _proof of work (PoW)_.
+
+Transactions are added to the new block, prioritized by the highest fee rate
+transactions first and a few other criteria. Each miner starts the
+process of mining a new candidate block of transactions as soon as they receive the
+previous block from the network, knowing that some other miner won that
+iteration of the lottery.  They immediately create a new candidate block
+with a commitment to the previous block, fill it with transactions, and start
+calculating the PoW for the candidate block. Each miner includes a
+special transaction in their candidate blocks, one that pays their own Bitcoin address
+the block reward plus the sum of
+transaction fees from all the transactions included in the candidate block. If they
+find a solution that makes the candidate into a valid block, they receive this reward
+after their successful block is added to the global blockchain and the
+reward transaction they included becomes spendable. Miners who participate in a mining pool have set up their
+software to create candidate blocks that assign the reward to a pool address.
+From there, a share of the reward is distributed to members of the pool
+miners in proportion to the amount of work they contributed.
+
+Alice's
+transaction was picked up by the network and included in the pool of
+unverified transactions. Once validated by a full node, it was
+included in a candidate block.
+Approximately five minutes after the transaction was first transmitted
+by Alice's wallet, a miner finds a solution for the
+block and announces it to the network. After each other miner
+validates the winning block, they start a new lottery to generate the next
+block.
+
+The winning block containing Alice's transaction became part of the
+blockchain. The block containing Alice's transaction is counted as ((("confirmations")))((("clearing transactions")))((("transactions", "clearing")))one
+_confirmation_ of that transaction.  After the block containing Alice's
+transaction has propagated through the network, creating an alternative
+block with a different version of Alice's transaction (such as a
+transaction that doesn't pay Bob) would require performing the same
+amount of work as it will take all Bitcoin miners to create an entirely
+new block.  When there are multiple alternative blocks to choose from,
+Bitcoin full nodes choose the chain of valid blocks with the most total
+PoW, called the _best blockchain_.  For the entire network to
+accept an alternative block, an additional new block would need to be
+mined on top of the alternative.
+
+That means miners have a choice.  They can work with Alice on an
+alternative to the transaction where she pays Bob, perhaps with
+Alice paying miners a share of the money she previously paid Bob.  This
+dishonest behavior will require they expend the effort required to
+create two new blocks.  Instead, miners who behave honestly can create a
+single new block and receive all of the fees from the transactions
+they include in it, plus the block subsidy.  Normally, the high cost of
+dishonestly creating two blocks for a small additional payment is much
+less profitable than honestly creating a new block, making it unlikely
+that a confirmed transaction will be deliberately changed.  For Bob, this
+means that he can begin to believe that the payment from Alice can be
+relied upon.
+
+[TIP]
+====
+You can see the block that includes
+https://oreil.ly/7v_lH[Alice's transaction].
+====
+
+Approximately 19 minutes
+after the block containing Alice's transaction is broadcast, a new block
+is mined by another miner. Because this
+new block is built on top of the block that contained Alice's
+transaction (giving Alice's transaction two confirmations), Alice's
+transaction can now only be changed if two alternative blocks are
+mined--plus a new block built on top of them--for a total of three
+blocks that would need to be mined for Alice to take back the money she
+sent Bob.  Each block mined on top of the one containing Alice's
+transaction counts as an additional confirmation.  As the blocks pile on
+top of each other, it becomes harder to reverse the transaction, thereby
+giving Bob more and more confidence that Alice's payment is secure.
+
+In <<block-alice1>>,  we can see the block that contains Alice's transaction. Below it are
+hundreds of thousands of blocks, linked to each other in a chain of
+blocks (blockchain) all the way back to block #0, known as((("genesis block"))) the _genesis
+block_. Over time, as the "height" of new blocks increases, so does the
+computation difficulty for the chain as a whole.
+By convention, any block with more than six confirmations
+is considered very hard to change, because it would require an immense amount of
+computation to recalculate six blocks (plus one new block). We will examine
+the process of mining and the way it builds confidence in more ((("mining", "operational overview", startref="mining-operational-overview")))detail in
+<<mining>>.
+
+[[block-alice1]]
+.Alice's transaction included in a block.
+image::images/mbc3_0207.png["Alice's transaction included in a block"]
+
+[role="less_space pagebreak-before"]
+=== Spending the Transaction
+
+Now((("transactions", "spending bitcoins", id="transaction-spend2")))((("bitcoins", "spending", id="bitcoin-spend2")))((("spending bitcoins", id="spend-bitcoin2"))) that Alice's
+transaction has been embedded in the blockchain as part of a block, it
+is visible to all Bitcoin
+applications. Each Bitcoin full node can independently verify the
+transaction as valid and spendable. Full nodes validate every transfer
+of the funds from the moment the bitcoins were first generated in
+a block through each subsequent transaction until they reach
+Bob's address. Lightweight clients can partially verify payments
+by confirming that the
+transaction is in the blockchain and has several blocks mined after it,
+thus providing assurance that the miners expended significant effort
+committing to it (see <<spv_nodes>>).
+
+Bob can now spend the output from this and other transactions. For
+example, Bob can pay a contractor or supplier by transferring value from
+Alice's podcast payment to these new owners.
+As Bob spends the payments received from Alice and other customers, he
+extends the chain of transactions. Let's assume that Bob pays his web
+designer Gopesh
+for a new website page. Now the chain of transactions will
+look like <<block-alice2>>.
+
+[[block-alice2]]
+.Alice's transaction as part of a transaction chain from Joe to Gopesh.
+image::images/mbc3_0208.png["Alice's transaction as part of a transaction chain"]
+
+In this chapter, we saw how transactions build a chain that moves value
+from owner to owner. We also tracked Alice's transaction from the
+moment it was created in her wallet, through the Bitcoin network, and to
+the miners who recorded it on the blockchain. In the rest of this book,
+we will examine the specific technologies behind wallets, addresses,
+signatures, transactions, the network, and finally, mining.

ch03.asciidoc

@@ -1,800 +0,0 @@
-[[ch03_bitcoin_client]]
-== Bitcoin Core: The Reference Implementation
-
-((("open source licenses")))((("Nakamoto, Satoshi")))Bitcoin is an _open source_ project and the source code is available under an open (MIT) license, free to download and use for any purpose. Open source means more than simply free to use. It also means that bitcoin is developed by an open community of volunteers. At first, that community consisted of only Satoshi Nakamoto. By 2016, bitcoin's source code had more than 400 contributors with about a dozen developers working on the code almost full-time and several dozen more on a part-time basis. Anyone can contribute to the code—including you!
-
-
-((("bitcoin whitepaper")))((("Satoshi client")))((("reference implementation", see="Bitcoin Core")))((("Bitcoin Core", "reference implementation")))When bitcoin was created by Satoshi Nakamoto, the software was actually completed before the whitepaper reproduced in <<satoshi_whitepaper>> was written. Satoshi wanted to make sure it worked before writing about it. That first implementation, then simply known as "Bitcoin" or "Satoshi client," has been heavily modified and improved. It has evolved into what is known as _Bitcoin Core_, to differentiate it from other compatible implementations. Bitcoin Core is the _reference implementation_ of the Bitcoin system, meaning that it is the authoritative reference on how each part of the technology should be implemented. Bitcoin Core implements all aspects of bitcoin, including wallets, a transaction and block validation engine, and a full network node in the peer-to-peer Bitcoin network.
-
-[WARNING]
-====
-((("wallets", "best practices for")))((("bitcoin improvement proposals", "Mnemonic Code Words (BIP-39)")))((("bitcoin improvement proposals", "Hierarchical Deterministic Wallets (BIP-32/BIP-44)")))Even though Bitcoin Core includes a reference implementation of a wallet, this is not intended to be used as a production wallet for users or for applications. Application developers are advised to build wallets using modern standards such as BIP-39 and BIP-32 (see <<mnemonic_code_words>> and <<hd_wallets>>). BIP stands for _Bitcoin Improvement Proposal_.
-====
-
-<<bitcoin_core_architecture>> shows the architecture of Bitcoin Core.((("Bitcoin Core", "architecture")))
-
-[[bitcoin_core_architecture]]
-.Bitcoin Core architecture (Source: Eric Lombrozo)
-image::images/mbc2_0301.png["Bitcoin Core Architecture"]
-
-
-=== Bitcoin Development Environment
-
-((("development environment", "setup", see="Bitcoin Core")))If you're a developer, you will want to set up a development environment with all the tools, libraries, and support software for writing bitcoin applications. In this highly technical chapter, we'll walk through that process step-by-step. If the material becomes too dense (and you're not actually setting up a development environment) feel free to skip to the next chapter, which is less technical.
-
-[[compiling_core]]
-=== Compiling Bitcoin Core from the Source Code
-
-((("Bitcoin Core", "compiling from source code", id="BCsource03")))((("Bitcoin Core", "compiling from source code", "downloading")))((("code examples, obtaining and using")))Bitcoin Core's source code can be downloaded as an archive or by cloning the authoritative source repository from GitHub.  ((("Bitcoin Core downloads")))On the https://bitcoincore.org/bin/[Bitcoin Core download page], select the most recent version and download the compressed archive of the source code, e.g., +bitcoin-0.15.0.2.tar.gz+. ((("GitHub bitcoin page")))Alternatively, use the git command line to create a local copy of the source code from the https://github.com/bitcoin/bitcoin[GitHub bitcoin page].
-
-[TIP]
-====
-((("$ symbol")))((("shell commands")))((("terminal applications")))In many of the examples in this chapter we will be using the operating system's command-line interface (also known as a "shell"), accessed via a "terminal" application. The shell will display a prompt; you type a command; and the shell responds with some text and a new prompt for your next command. The prompt may look different on your system, but in the following examples it is denoted by a +$+ symbol. In the examples, when you see text after a +$+ symbol, don't type the +$+ symbol but type the command immediately following it, then press Enter to execute the command. In the examples, the lines below each command are the operating system's responses to that command. When you see the next +$+ prefix, you'll know it's a new command and you should repeat the process.
-====
-
-((("cloning source code")))((("source code, cloning", seealso="Bitcoin Core")))In this example, we are using the +git+ command to create a local copy ("clone") of the source code:
-
-----
-$ git clone https://github.com/bitcoin/bitcoin.git
-Cloning into 'bitcoin'...
-remote: Counting objects: 102071, done.
-remote: Compressing objects: 100% (10/10), done.
-Receiving objects: 100% (102071/102071), 86.38 MiB | 730.00 KiB/s, done.
-remote: Total 102071 (delta 4), reused 5 (delta 1), pack-reused 102060
-Resolving deltas: 100% (76168/76168), done.
-Checking connectivity... done.
-$
-----
-
-[TIP]
-====
-((("distributed version control systems")))Git is the most widely used distributed version control system, an essential part of any software developer's toolkit. You may need to install the +git+ command, or a graphical user interface for git, on your operating system if you do not have it already.
-====
-
-When the git cloning operation has completed, you will have a complete local copy of the source code repository in the directory _bitcoin_. Change to this directory by typing ++**cd bitcoin**++ at the prompt:
-
-----
-$ cd bitcoin
-----
-
-==== Selecting a Bitcoin Core Release
-((("Bitcoin Core", "compiling from source code", "version selection")))By default, the local copy will be synchronized with the most recent code, which might be an unstable or beta version of bitcoin. Before compiling the code, select a specific version by checking out a release _tag_. This will synchronize the local copy with a specific snapshot of the code repository identified by a keyword tag. Tags are used by the developers to mark specific releases of the code by version number. First, to find the available tags, we use the +git tag+ command:
-
-----
-$ git tag
-v0.1.5
-v0.1.6test1
-v0.10.0
-...
-v0.11.2
-v0.11.2rc1
-v0.12.0rc1
-v0.12.0rc2
-...
-----
-
-The list of tags shows all the released versions of bitcoin. By convention, _release candidates_, which are intended for testing, have the suffix "rc." Stable releases that can be run on production systems have no suffix. From the preceding list, select the highest version release, which at the time of writing was v0.15.0. To synchronize the local code with this version, use the +git checkout+ command:
-
-----
-$ git checkout v0.15.0
-HEAD is now at 3751912... Merge #11295: doc: Old fee_estimates.dat are discarded by 0.15.0
-----
-
-You can confirm you have the desired version "checked out" by issuing the command +git status+:
-
-----
-$ git status
-HEAD detached at v0.15.0
-nothing to commit, working directory clean
-----
-
-==== Configuring the Bitcoin Core Build
-
-((("Bitcoin Core", "compiling from source code", "build configuration")))((("documentation")))((("build documentation", seealso="Bitcoin Core")))The source code includes documentation, which can be found in a number of files. Review the main documentation located in _README.md_ in the _bitcoin_ directory by typing ++**more README.md**++ at the prompt and using the spacebar to progress to the next page. In this chapter, we will build the command-line Bitcoin client, also known as +bitcoind+ on Linux. Review the instructions for compiling the +bitcoind+ command-line client on your platform by typing ++**more doc/build-unix.md**++. Alternative instructions for macOS and Windows can be found in the _doc_ directory, as _build-osx.md_ or _build-windows.md_, respectively.
-
-Carefully review the build prerequisites, which are in the first part of the build documentation. These are libraries that must be present on your system before you can begin to compile bitcoin. If these prerequisites are missing, the build process will fail with an error. If this happens because you missed a prerequisite, you can install it and then resume the build process from where you left off. Assuming the prerequisites are installed, you start the build process by generating a set of build scripts using the _autogen.sh_ script.
-
-----
-$ ./autogen.sh
-...
-glibtoolize: copying file 'build-aux/m4/libtool.m4'
-glibtoolize: copying file 'build-aux/m4/ltoptions.m4'
-glibtoolize: copying file 'build-aux/m4/ltsugar.m4'
-glibtoolize: copying file 'build-aux/m4/ltversion.m4'
-...
-configure.ac:10: installing 'build-aux/compile'
-configure.ac:5: installing 'build-aux/config.guess'
-configure.ac:5: installing 'build-aux/config.sub'
-configure.ac:9: installing 'build-aux/install-sh'
-configure.ac:9: installing 'build-aux/missing'
-Makefile.am: installing 'build-aux/depcomp'
-...
-----
-
-The _autogen.sh_ script creates a set of automatic configuration scripts that will interrogate your system to discover the correct settings and ensure you have all the necessary libraries to compile the code. The most important of these is the +configure+ script that offers a number of different options to customize the build process. Type ++**./configure --help**++ to see the various options:
-
-----
-$ ./configure --help
-`configure' configures Bitcoin Core 0.15.0 to adapt to many kinds of systems.
-
-Usage: ./configure [OPTION]... [VAR=VALUE]...
-
-...
-Optional Features:
-  --disable-option-checking  ignore unrecognized --enable/--with options
-  --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
-  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
-
-  --enable-wallet         enable wallet (default is yes)
-
-  --with-gui[=no|qt4|qt5|auto]
-...
-----
-
-The +configure+ script allows you to enable or disable certain features of +bitcoind+ through the use of the +--enable-FEATURE+ and +--disable-FEATURE+ flags, where pass:[<span class="keep-together"><code>FEATURE</code></span>] is replaced by the feature name, as listed in the help output. In this chapter, we will build the +bitcoind+ client with all the default features. We won't be using the configuration flags, but you should review them to understand what optional features are part of the client. If you are in an academic setting, computer lab restrictions may require you to install applications in your home directory (e.g., using +--prefix=$HOME+).
-
-Here are some useful options that override the default behavior of the configure script:
-
-++++
-<dl>
-<dt><code>--prefix=$HOME</code></dt>
-<dd><p>This overrides the default installation location (which is <em>/usr/local/</em>) for the resulting executable. Use <code>$HOME</code> to put everything in your home directory, or a different path.</p></dd>
-
-<dt><code>--disable-wallet</code></dt>
-<dd><p>This is used to disable the reference wallet implementation.</p></dd>
-
-<dt><code>--with-incompatible-bdb</code></dt>
-<dd><p>If you are building a wallet, allow the use of an incompatible version of the Berkeley DB library.</p></dd>
-
-<dt><code>--with-gui=no</code></dt>
-<dd><p>Don't build the graphical user interface, which requires the Qt library. This builds server and command-line bitcoin only.</p></dd>
-</dl>
-++++
-
-Next, run the +configure+ script to automatically discover all the necessary libraries and create a customized build script for your system:
-
-----
-$ ./configure
-checking build system type... x86_64-unknown-linux-gnu
-checking host system type... x86_64-unknown-linux-gnu
-checking for a BSD-compatible install... /usr/bin/install -c
-checking whether build environment is sane... yes
-checking for a thread-safe mkdir -p... /bin/mkdir -p
-checking for gawk... gawk
-checking whether make sets $(MAKE)... yes
-...
-[many pages of configuration tests follow]
-...
-$
-----
-
-
-If all went well, the +configure+ command will end by creating the customized build scripts that will allow us to compile +bitcoind+. If there are any missing libraries or errors, the +configure+ command will terminate with an error instead of creating the build scripts. If an error occurs, it is most likely because of a missing or incompatible library. Review the build documentation again and make sure you install the missing prerequisites. Then run +configure+ again and see if that fixes the error.
-
-==== Building the Bitcoin Core Executables
-
-((("Bitcoin Core", "compiling from source code", "core executables")))((("core executables", seealso="Bitcoin Core")))Next, you will compile the source code, a process that can take up to an hour to complete, depending on the speed of your CPU and available memory. During the compilation process you should see output every few seconds or every few minutes, or an error if something goes wrong. If an error occurs, or the compilation process is interrupted, it can be resumed any time by typing +make+ again. Type ++**make**++ to start compiling the executable application:
-
-----
-$ make
-Making all in src
-  CXX      crypto/libbitcoinconsensus_la-hmac_sha512.lo
-  CXX      crypto/libbitcoinconsensus_la-ripemd160.lo
-  CXX      crypto/libbitcoinconsensus_la-sha1.lo
-  CXX      crypto/libbitcoinconsensus_la-sha256.lo
-  CXX      crypto/libbitcoinconsensus_la-sha512.lo
-  CXX      libbitcoinconsensus_la-hash.lo
-  CXX      primitives/libbitcoinconsensus_la-transaction.lo
-  CXX      libbitcoinconsensus_la-pubkey.lo
-  CXX      script/libbitcoinconsensus_la-bitcoinconsensus.lo
-  CXX      script/libbitcoinconsensus_la-interpreter.lo
-
-[... many more compilation messages follow ...]
-
-$
-----
-
-On a fast system with more than one CPU, you might want to set the number of parallel compile jobs. For instance, +make -j 2+ will use two cores if they are available. If all goes well, Bitcoin Core is now compiled. You should run the unit test suite with +make check+ to ensure the linked libraries are not broken in obvious ways. The final step is to install the various executables on your system using the +make install+ command. You may be prompted for your user password, because this step requires administrative privileges:
-
-----
-$ make check && sudo make install
-Password:
-Making install in src
- ../build-aux/install-sh -c -d '/usr/local/lib'
-libtool: install: /usr/bin/install -c bitcoind /usr/local/bin/bitcoind
-libtool: install: /usr/bin/install -c bitcoin-cli /usr/local/bin/bitcoin-cli
-libtool: install: /usr/bin/install -c bitcoin-tx /usr/local/bin/bitcoin-tx
-...
-$
-----
-
-((("", startref="BCsource03")))The default installation of +bitcoind+ puts it in _/usr/local/bin_. You can confirm that Bitcoin Core is correctly installed by asking the system for the path of the executables, as follows:
-
-----
-$ which bitcoind
-/usr/local/bin/bitcoind
-
-$ which bitcoin-cli
-/usr/local/bin/bitcoin-cli
-----
-
-=== Running a Bitcoin Core Node
-
-((("Bitcoin Core", "running core nodes", id="BCnode03")))((("Bitcoin nodes", "running core nodes", id="BNcore03")))Bitcoin's peer-to-peer network is composed of network "nodes," run mostly by volunteers and some of the businesses that build bitcoin applications. Those running Bitcoin nodes have a direct and authoritative view of the Bitcoin blockchain, with a local copy of all the transactions, independently validated by their own system. By running a node, you don't have to rely on any third party to validate a transaction. Moreover, by running a Bitcoin node you contribute to the Bitcoin network by making it more robust.
-
-Running a node, however, requires a permanently connected system with enough resources to process all bitcoin transactions. Depending on whether you choose to index all transactions and keep a full copy of the blockchain, you may also need a lot of disk space and RAM. As of early 2021, a full-index node needs 2 GB of RAM and a minimum of 360 GB of disk space (see https://www.blockchain.com/charts/blocks-size[]). Bitcoin nodes also transmit and receive bitcoin transactions and blocks, consuming internet bandwidth. If your internet connection is limited, has a low data cap, or is metered (charged by the gigabit), you should probably not run a Bitcoin node on it, or run it in a way that constrains its bandwidth (see <<constrained_resources>>).
-
-[TIP]
-====
-((("warnings and cautions", "core node resource requirements")))((("resource requirements")))Bitcoin Core keeps a full copy of the blockchain by default, with every transaction that has ever occurred on the Bitcoin network since its inception in 2009. This dataset is dozens of gigabytes in size and is downloaded incrementally over several days or weeks, depending on the speed of your CPU and internet connection. Bitcoin Core will not be able to process transactions or update account balances until the full blockchain dataset is downloaded. Make sure you have enough disk space, bandwidth, and time to complete the initial synchronization. You can configure Bitcoin Core to reduce the size of the blockchain by discarding old blocks (see <<constrained_resources>>), but it will still download the entire dataset before discarding data.
-====
-
-Despite these resource requirements, thousands of volunteers run Bitcoin nodes. Some are running on systems as simple as a Raspberry Pi (a $35 USD computer the size of a pack of cards). Many volunteers also run Bitcoin nodes on rented servers, usually some variant of Linux. A _Virtual Private Server_ (VPS) or _Cloud Computing Server_ instance can be used to run a Bitcoin node. Such servers can be rented for $25 to $50 USD per month from a variety of providers.
-
-Why would you want to run a node? Here are some of the most common reasons:
-
-* If you are developing bitcoin software and need to rely on a Bitcoin node for programmable (API) access to the network and blockchain.
-
-* If you are building applications that must validate transactions according to bitcoin's consensus rules. Typically, bitcoin software companies run several nodes.
-
-* If you want to support bitcoin. Running a node makes the network more robust and able to serve more wallets, more users, and more transactions.
-
-* If you do not want to rely on any third party to process or validate your transactions.
-
-If you're reading this book and interested in developing bitcoin software, you should be running your own node.
-
-==== Configuring the Bitcoin Core Node
-
-((("Bitcoin Core", "running core nodes", "configuring")))((("warnings and cautions", "password creation")))((("passwords", "creating")))((("security", "passwords")))Bitcoin Core will look for a configuration file in its data directory on every start. In this section we will examine the various configuration options and set up a configuration file. To locate the configuration file, run +bitcoind -printtoconsole+ in your terminal and look for the first couple of lines.
-
-----
-$ bitcoind -printtoconsole
-Bitcoin version v0.15.0
-Using the 'standard' SHA256 implementation
-Using data directory /home/ubuntu/.bitcoin/
-Using config file /home/ubuntu/.bitcoin/bitcoin.conf
-...
-[a lot more debug output]
-...
-----
-
-You can hit Ctrl-C to shut down the node once you determine the location of the config file. Usually the configuration file is inside the _.bitcoin_ data directory under your user's home directory. It is not created automatically, but you can create a starter config file by copying and pasting from the <<#full_index_node>> example, below. You can create or modify the configuration file in your preferred editor.
-
-Bitcoin Core offers more than 100 configuration options that modify the behavior of the network node, the storage of the blockchain, and many other aspects of its operation. To see a listing of these options, run +bitcoind  --help+:
-
-----
-$ bitcoind --help
-Bitcoin Core Daemon version v0.15.0
-
-Usage:
-  bitcoind [options]                     Start Bitcoin Core Daemon
-
-Options:
-
-  -?
-       Print this help message and exit
-
-  -version
-       Print version and exit
-
-  -alertnotify=<cmd>
-       Execute command when a relevant alert is received or we see a really
-       long fork (%s in cmd is replaced by message)
-...
-[many more options]
-...
-
-  -rpcthreads=<n>
-       Set the number of threads to service RPC calls (default: 4)
-----
-
-((("configuration options", seealso="Bitcoin Core")))Here are some of the most important options that you can set in the configuration file, or as command-line parameters to +bitcoind+:
-
-alertnotify:: Run a specified command or script to send emergency alerts to the owner of this node, usually by email.
-
-conf:: An alternative location for the configuration file. This only makes sense as a command-line parameter to +bitcoind+, as it can't be inside the configuration file it refers to.
-
-datadir:: Select the directory and filesystem in which to put all the blockchain data. By default this is the _.bitcoin_ subdirectory of your home directory. Make sure this filesystem has several gigabytes of free space.
-
-prune:: Reduce the disk space requirements to this many megabytes, by deleting old blocks. Use this on a resource-constrained node that can't fit the full blockchain.
-
-txindex:: Maintain an index of all transactions. This means a complete copy of the blockchain that allows you to programmatically retrieve any transaction by ID.
-
-dbcache:: The size of the UTXO cache. The default is 450 MiB. Increase this on high-end hardware and reduce the size on low-end hardware to save memory at the expense of slow disk IO.
-
-maxconnections:: Set the maximum number of nodes from which to accept connections. Reducing this from the default will reduce your bandwidth consumption. Use if you have a data cap or pay by the gigabyte.
-
-maxmempool:: Limit the transaction memory pool to this many megabytes. Use it to reduce memory use on memory-constrained nodes.
-
-maxreceivebuffer/maxsendbuffer:: Limit per-connection memory buffer to this many multiples of 1000 bytes. Use on memory-constrained nodes.
-
-minrelaytxfee:: Set the minimum fee rate for transaction you will relay. Below this value, the transaction is treated nonstandard, rejected from the transaction pool and not relayed.
-
-
-[[txindex]]
-.Transaction Database Index and txindex Option
-****
-((("Bitcoin Core", "running core nodes", "database options")))((("transactions", "database configuration options")))((("txindex option")))((("full indexing option")))By default, Bitcoin Core builds a database containing _only_ the transactions related to the user's wallet. If you want to be able to access _any_ transaction with commands like +getrawtransaction+ (see <<exploring_and_decoding_transanctions>>), you need to configure Bitcoin Core to build a complete transaction index, which can be achieved with the +txindex+ option. Set +txindex=1+ in the Bitcoin Core configuration file. If you don't set this option at first and later set it to full indexing, you need to restart +bitcoind+ with the +-reindex+ option and wait for it to rebuild the index.
-****
-
-<<full_index_node>> shows how you might combine the preceding options, with a fully indexed node, running as an API backend for a bitcoin application.
-
-[[full_index_node]]
-.Sample configuration of a full-index node
-====
-----
-alertnotify=myemailscript.sh "Alert: %s"
-datadir=/lotsofspace/bitcoin
-txindex=1
-----
-====
-
-<<constrained_resources>> shows a resource-constrained node running on a smaller server.
-
-[[constrained_resources]]
-.Sample configuration of a resource-constrained system
-====
-----
-alertnotify=myemailscript.sh "Alert: %s"
-maxconnections=15
-prune=5000
-dbcache=150
-maxmempool=150
-maxreceivebuffer=2500
-maxsendbuffer=500
-----
-====
-
-Once you've edited the configuration file and set the options that best represent your needs, you can test +bitcoind+ with this configuration. Run Bitcoin Core with the option +printtoconsole+ to run in the foreground with output to the console:
-
-----
-$ bitcoind -printtoconsole
-
-Bitcoin version v0.15.0
-InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
-Assuming ancestors of block 0000000000000000003b9ce759c2a087d52abc4266f8f4ebd6d768b89defa50a have valid signatures.
-Using the 'standard' SHA256 implementation
-Default data directory /home/ubuntu/.bitcoin
-Using data directory /lotsofspace/.bitcoin
-Using config file /home/ubuntu/.bitcoin/bitcoin.conf
-Using at most 125 automatic connections (1048576 file descriptors available)
-Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
-Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
-Using 2 threads for script verification
-HTTP: creating work queue of depth 16
-No rpcpassword set - using random cookie authentication
-Generated RPC authentication cookie /lotsofspace/.bitcoin/.cookie
-HTTP: starting 4 worker threads
-init message: Verifying wallet(s)...
-Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
-Using wallet wallet.dat
-CDBEnv::Open: LogDir=/lotsofspace/.bitcoin/database ErrorFile=/lotsofspace/.bitcoin/db.log
-scheduler thread start
-Cache configuration:
-* Using 250.0MiB for block index database
-* Using 8.0MiB for chain state database
-* Using 1742.0MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)
-init message: Loading block index...
-Opening LevelDB in /lotsofspace/.bitcoin/blocks/index
-Opened LevelDB successfully
-
-[... more startup messages ...]
-----
-
-You can hit Ctrl-C to interrupt the process once you are satisfied that it is loading the correct settings and running as you expect.
-
-To run Bitcoin Core in the background as a process, start it with the +daemon+ option, as +bitcoind -daemon+.
-
-To monitor the progress and runtime status of your Bitcoin node, use the command +bitcoin-cli getblockchaininfo+:
-
-----
-$ bitcoin-cli getblockchaininfo
-----
-
-[source,json]
-----
-{
-  "chain": "main",
-  "blocks": 0,
-  "headers": 83999,
-  "bestblockhash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
-  "difficulty": 1,
-  "mediantime": 1231006505,
-  "verificationprogress": 3.783041623201835e-09,
-  "chainwork": "0000000000000000000000000000000000000000000000000000000100010001",
-  "pruned": false,
-  [...]
-}
-----
-
-This shows a node with a blockchain height of 0 blocks and 83999 headers. The node currently fetches the block headers of the best chain and afterward continues to download the full blocks.
-
-Once you are happy with the configuration options you have selected, you should add bitcoin to the startup scripts in your operating system, so that it runs continuously and restarts when the operating system restarts. You will find a number of example startup scripts for various operating systems in bitcoin's source directory under _contrib/init_ and a _README.md_ file showing which system uses which script.((("", startref="BCnode03")))((("", startref="BNcore03")))
-
-=== Bitcoin Core Application Programming Interface (API)
-
-((("Bitcoin Core", "Bitcoin Core API", id="BCapi03")))The Bitcoin Core client implements a JSON-RPC interface that can also be accessed using the command-line helper +bitcoin-cli+. The command line allows us to experiment interactively with the capabilities that are also available programmatically via the API. ((("Bitcoin Core", "Bitcoin Core API", "RPC commands")))To start, invoke the +help+ command to see a list of the available bitcoin RPC commands:
-
-[[bitcoind_commands]]
-
-----
-$ bitcoin-cli help
-addmultisigaddress nrequired ["key",...] ( "account" )
-addnode "node" "add|remove|onetry"
-backupwallet "destination"
-createmultisig nrequired ["key",...]
-createrawtransaction [{"txid":"id","vout":n},...] {"address":amount,...}
-decoderawtransaction "hexstring"
-...
-...
-verifymessage "bitcoinaddress" "signature" "message"
-walletlock
-walletpassphrase "passphrase" timeout
-walletpassphrasechange "oldpassphrase" "newpassphrase"
-----
-
-Each of these commands may take a number of parameters. To get additional help, a detailed description, and information on the parameters, add the command name after +help+. For example, to see help on the +getblockhash+ RPC command:
-
-----
-$ bitcoin-cli help getblockhash
-getblockhash height
-
-Returns hash of block in best-block-chain at height provided.
-
-Arguments:
-1. height         (numeric, required) The height index
-
-Result:
-"hash"         (string) The block hash
-
-Examples:
-> bitcoin-cli getblockhash 1000
-> curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getblockhash", "params": [1000] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/
-----
-
-At the end of the help information you will see two examples of the RPC command, using the +bitcoin-cli+ helper or the HTTP client +curl+. These examples demonstrate how you might call the command. Copy the first example and see the result:
-
-----
-$ bitcoin-cli getblockhash 1000
-00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09
-----
-
-The result is a block hash, which is described in more detail in the following chapters. But for now, this command should return the same result on your system, demonstrating that your Bitcoin Core node is running, is accepting commands, and has information about block 1000 to return to you.
-
-In the next sections we will demonstrate some very useful RPC commands and their expected output.
-
-==== Getting Information on the Bitcoin Core Client Status
-
-((("Bitcoin Core", "Bitcoin Core API", "status information")))Bitcoin Core provides status reports on different modules through the JSON-RPC interface. The most important commands include +getblockchaininfo+, +getmempoolinfo+, +getnetworkinfo+ and +getwalletinfo+.
-
-Bitcoin's +getblockchaininfo+ RPC command was introduced earlier. The +getnetworkinfo+ command displays basic information about the status of the Bitcoin network node. Use +bitcoin-cli+ to run it:
-
-----
-$ bitcoin-cli getnetworkinfo
-----
-[source,json]
-----
-{
-  "version": 150000,
-  "subversion": "/Satoshi:0.15.0/",
-  "protocolversion": 70015,
-  "localservices": "000000000000000d",
-  "localrelay": true,
-  "timeoffset": 0,
-  "networkactive": true,
-  "connections": 8,
-  "networks": [
-    ...
-    detailed information about all networks (ipv4, ipv6 or onion)
-    ...
-  ],
-  "relayfee": 0.00001000,
-  "incrementalfee": 0.00001000,
-  "localaddresses": [
-  ],
-  "warnings": ""
-}
-
-----
-
-The data is returned in JavaScript Object Notation (JSON), a format that can easily be "consumed" by all programming languages but is also quite human-readable. Among this data we see the version numbers for the bitcoin software client (150000) and Bitcoin protocol (70015).  We see the current number of connections (8) and various information about the Bitcoin network and the settings related to this client.
-
-[TIP]
-====
-It will take some time, perhaps more than a day, for the +bitcoind+ client to "catch up" to the current blockchain height as it downloads blocks from other Bitcoin clients. You can check its progress using +getblockchaininfo+ to see the number of known blocks.
-====
-
-[[exploring_and_decoding_transanctions]]
-==== Exploring and Decoding Transactions
-
-((("Bitcoin Core", "Bitcoin Core API", "exploring and decoding transactions")))((("transactions", "exploring with Bitcoin Core API")))Commands: +getrawtransaction+, +decoderawtransaction+
-
-
-
-In <<cup_of_coffee>>, ((("use cases", "buying coffee", id="alicethree")))Alice bought a cup of coffee from Bob's Cafe. Her transaction was recorded on the blockchain with transaction ID (+txid+) +0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2+. Let's use the API to retrieve and examine that transaction by passing the transaction ID as a parameter:
-
-++++
-<pre data-type="programlisting">
-$ bitcoin-cli getrawtransaction 0627052b6f28912f2703066a912ea577f2ce4da4caa5a&#x21b5;
-5fbd8a57286c345c2f2
-
-0100000001186f9f998a5aa6f048e51dd8419a14d8a0f1a8a2836dd734d2804fe65fa35779000&#x21b5;
-000008b483045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4&#x21b5;
-ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813014&#x21b5;
-10484ecc0d46f1918b30928fa0e4ed99f16a0fb4fde0735e7ade8416ab9fe423cc54123363767&#x21b5;
-89d172787ec3457eee41c04f4938de5cc17b4a10fa336a8d752adfffffffff0260e3160000000&#x21b5;
-0001976a914ab68025513c3dbd2f7b92a94e0581f5d50f654e788acd0ef8000000000001976a9&#x21b5;
-147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a888ac00000000
-</pre>
-++++
-
-
-[TIP]
-====
-((("transaction IDs (txd)")))((("malleability")))A transaction ID is not authoritative until a transaction has been confirmed. Absence of a transaction hash in the blockchain does not mean the transaction was not processed. This is known as "transaction malleability," because transaction hashes can be modified prior to confirmation in a block. After confirmation, the +txid+ is immutable and authoritative.
-====
-
-The command +getrawtransaction+ returns a serialized transaction in hexadecimal notation. To decode that, we use the +decoderawtransaction+ command, passing the hex data as a parameter. You can copy the hex returned by +getrawtransaction+ and paste it as a parameter to +decoderawtransaction+:
-
-++++
-<pre data-type="programlisting">
-$ bitcoin-cli decoderawtransaction 0100000001186f9f998a5aa6f048e51dd8419a14d8&#x21b5;
-a0f1a8a2836dd734d2804fe65fa35779000000008b483045022100884d142d86652a3f47ba474&#x21b5;
-6ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298&#x21b5;
-cad530a863ea8f53982c09db8f6e381301410484ecc0d46f1918b30928fa0e4ed99f16a0fb4fd&#x21b5;
-e0735e7ade8416ab9fe423cc5412336376789d172787ec3457eee41c04f4938de5cc17b4a10fa&#x21b5;
-336a8d752adfffffffff0260e31600000000001976a914ab68025513c3dbd2f7b92a94e0581f5&#x21b5;
-d50f654e788acd0ef8000000000001976a9147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a8&#x21b5;
-88ac00000000
-
-</pre>
-++++
-
-++++
-<pre data-type="programlisting" data-code-language="json">
-{
-  "txid": "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2",
-  "size": 258,
-  "version": 1,
-  "locktime": 0,
-  "vin": [
-    {
-      "txid": "7957a35fe64f80d234d76d83a2...8149a41d81de548f0a65a8a999f6f18",
-      "vout": 0,
-      "scriptSig": {
-        "asm":"3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1decc...",
-        "hex":"483045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1de..."
-      },
-      "sequence": 4294967295
-    }
-  ],
-  "vout": [
-    {
-      "value": 0.01500000,
-      "n": 0,
-      "scriptPubKey": {
-        "asm": "OP_DUP OP_HASH160 ab68...5f654e7 OP_EQUALVERIFY OP_CHECKSIG",
-        "hex": "76a914ab68025513c3dbd2f7b92a94e0581f5d50f654e788ac",
-        "reqSigs": 1,
-        "type": "pubkeyhash",
-        "addresses": [
-          "1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA"
-        ]
-      }
-    },
-    {
-      "value": 0.08450000,
-      "n": 1,
-      "scriptPubKey": {
-        "asm": "OP_DUP OP_HASH160 7f9b1a...025a8 OP_EQUALVERIFY OP_CHECKSIG",
-        "hex": "76a9147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a888ac",
-        "reqSigs": 1,
-        "type": "pubkeyhash",
-        "addresses": [
-          "1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK"
-        ]
-      }
-    }
-  ]
-}
-
-</pre>
-++++
-
-The transaction decode shows all the components of this transaction, including the transaction inputs and outputs. In this case we see that the transaction that credited our new address with 15 millibits used one input and generated two outputs. The input to this transaction was the output from a previously confirmed transaction (shown as the vin +txid+ starting with +7957a35fe+). The two outputs correspond to the 15 millibit credit and an output with change back to the sender.
-
-We can further explore the blockchain by examining the previous transaction referenced by its +txid+ in this transaction using the same commands (e.g., +getrawtransaction+). Jumping from transaction to transaction we can follow a chain of transactions back as the coins are transmitted from owner address to owner address.
-
-
-
-==== Exploring Blocks
-
-((("Bitcoin Core", "Bitcoin Core API", "exploring blocks")))((("blocks", "exploring with Bitcoin Core API")))Commands: +getblock+, +getblockhash+
-
-((("blocks", "block height")))((("blocks", "block hash")))Exploring blocks is similar to exploring transactions. However, blocks can be referenced either by the block _height_ or by the block _hash_. First, let's find a block by its height. In <<cup_of_coffee>>, we saw that Alice's transaction was included in block 277316.
-
-We use the +getblockhash+ command, which takes the block height as the parameter and returns the block hash for that block:
-
-++++
-<pre data-type="programlisting">
-$ bitcoin-cli getblockhash 277316
-0000000000000001b6b9a13b095e96db41c4a928b97ef2d944a9b31b2cc7bdc4
-</pre>
-++++
-
-Now that we know which block Alice's transaction was included in, we can query that block. We use the +getblock+ command with the block hash as the parameter:
-
-++++
-<pre data-type="programlisting">
-$ bitcoin-cli getblock 0000000000000001b6b9a13b095e96db41c4a928b97ef2d944a9b3&#x21b5;
-1b2cc7bdc4
-</pre>
-++++
-
-++++
-<pre data-type="programlisting" data-code-language="json">
-{
-  "hash": "0000000000000001b6b9a13b095e96db41c4a928b97ef2d944a9b31b2cc7bdc4",
-  "confirmations": 37371,
-  "size": 218629,
-  "height": 277316,
-  "version": 2,
-  "merkleroot": "c91c008c26e50763e9f548bb8b2fc323735f73577effbc55502c51eb4cc7cf2e",
-  "tx": [
-    "d5ada064c6417ca25c4308bd158c34b77e1c0eca2a73cda16c737e7424afba2f",
-    "b268b45c59b39d759614757718b9918caf0ba9d97c56f3b91956ff877c503fbe",
-    "04905ff987ddd4cfe603b03cfb7ca50ee81d89d1f8f5f265c38f763eea4a21fd",
-    "32467aab5d04f51940075055c2f20bbd1195727c961431bf0aff8443f9710f81",
-    "561c5216944e21fa29dd12aaa1a45e3397f9c0d888359cb05e1f79fe73da37bd",
-[... hundreds of transactions ...]
-    "78b300b2a1d2d9449b58db7bc71c3884d6e0579617e0da4991b9734cef7ab23a",
-    "6c87130ec283ab4c2c493b190c20de4b28ff3caf72d16ffa1ce3e96f2069aca9",
-    "6f423dbc3636ef193fd8898dfdf7621dcade1bbe509e963ffbff91f696d81a62",
-    "802ba8b2adabc5796a9471f25b02ae6aeee2439c679a5c33c4bbcee97e081196",
-    "eaaf6a048588d9ad4d1c092539bd571dd8af30635c152a3b0e8b611e67d1a1af",
-    "e67abc6bd5e2cac169821afc51b207127f42b92a841e976f9b752157879ba8bd",
-    "d38985a6a1bfd35037cb7776b2dc86797abbb7a06630f5d03df2785d50d5a2ac",
-    "45ea0a3f6016d2bb90ab92c34a7aac9767671a8a84b9bcce6c019e60197c134b",
-    "c098445d748ced5f178ef2ff96f2758cbec9eb32cb0fc65db313bcac1d3bc98f"
-  ],
-  "time": 1388185914,
-  "mediantime": 1388183675,
-  "nonce": 924591752,
-  "bits": "1903a30c",
-  "difficulty": 1180923195.258026,
-  "chainwork": "000000000000000000000000000000000000000000000934695e92aaf53afa1a",
-  "previousblockhash": "0000000000000002a7bbd25a417c0374cc55261021e8a9ca74442b01284f0569",
-  "nextblockhash": "000000000000000010236c269dd6ed714dd5db39d36b33959079d78dfd431ba7"
-}
-</pre>
-++++
-
-The block contains 419 transactions and the 64th transaction listed (+0627052b...+) is Alice's coffee payment. The +height+ entry tells us this is the 277316th block in the blockchain.
-
-==== Using Bitcoin Core's Programmatic Interface
-
-((("Bitcoin Core", "Bitcoin Core API", "using programmatic interface")))((("programmatic interface", id="progint03")))The +bitcoin-cli+ helper is very useful for exploring the Bitcoin Core API and testing functions. But the whole point of an application programming interface is to access functions programmatically. In this section we will demonstrate accessing Bitcoin Core from another program.
-
-Bitcoin Core's API is a JSON-RPC interface. JSON stands for JavaScript Object Notation and it is a very convenient way to present data that both humans and programs can easily read. RPC stands for Remote Procedure Call, which means that we are calling procedures (functions) that are remote (on the Bitcoin Core node) via a network protocol. In this case, the network protocol is HTTP, or HTTPS (for encrypted connections).
-
-When we used the +bitcoin-cli+ command to get help on a command, it showed us an example of using +curl+, the versatile command-line HTTP client to construct one of these JSON-RPC calls:
-
-----
-$ curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/
-----
-
-This command shows that +curl+ submits an HTTP request to the local host (127.0.0.1), connecting to the default bitcoin port (8332), and submitting a  +jsonrpc+ request for the +getblockchaininfo+ method using +text/plain+ encoding.
-
-You might notice that curl will ask for credentials to be sent along with the request. Bitcoin Core will create a random password on each start and place it in the data directory under the name +.cookie+. The +bitcoin-cli+ helper can read this password file given the data directory. Similarly, you can copy the password and pass it to curl (or any higher level Bitcoin Core RPC wrappers). Alternatively, you can create a static password with the helper script provided in _./share/rpcauth/rpcauth.py_ in Bitcoin Core's source directory.
-
-If you're implementing a JSON-RPC call in your own program, you can use a generic HTTP library to construct the call, similar to what is shown in the preceding +curl+ example.
-
-However, there are libraries in most every programming language that "wrap" the Bitcoin Core API in a way that makes this a lot simpler. We will use the +python-bitcoinlib+ library to simplify API access. Remember, this requires you to have a running Bitcoin Core instance, which will be used to make JSON-RPC calls.
-
-The Python script in <<rpc_example>> makes a simple +getblockchaininfo+ call and prints the +blocks+ parameter from the data returned by Bitcoin Core (full node required).
-
-[[rpc_example]]
-.Running getblockchaininfo via Bitcoin Core's JSON-RPC API
-====
-[source,python]
-----
-include::code/rpc_example.py[]
-----
-====
-
-Running it gives us the following result:
-
-----
-$ python rpc_example.py
-394075
-----
-
-It tells us that our local Bitcoin Core node has 394075 blocks in its blockchain. Not a spectacular result, but it demonstrates the basic use of the library as a simplified interface to Bitcoin Core's JSON-RPC API.
-
-Next, let's use the +getrawtransaction+ and +decodetransaction+ calls to retrieve the details of Alice's coffee payment. In <<rpc_transaction>>, we retrieve Alice's transaction and list the transaction's outputs. For each output, we show the recipient address and value. As a reminder, Alice's transaction had one output paying Bob's Cafe and one output for change back to Alice.
-
-[[rpc_transaction]]
-.Retrieving a transaction and iterating its outputs
-====
-[source,python]
-----
-include::code/rpc_transaction.py[]
-----
-====
-
-Running this code, we get:
-
-----
-$ python rpc_transaction.py
-([u'1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA'], Decimal('0.01500000'))
-([u'1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK'], Decimal('0.08450000'))
-----
-
-Both of the preceding examples are rather simple. You don't really need a program to run them; you could just as easily use the +bitcoin-cli+ helper. The next example, however, requires several hundred RPC calls and more clearly demonstrates the use of a programmatic interface.
-
-In <<rpc_block>>, we first retrieve block 277316, then retrieve each of the 419 transactions within by reference to each transaction ID. Next, we iterate through each of the transaction's outputs and add up the value.((("", startref="alicethree")))
-
-[[rpc_block]]
-.Retrieving a block and adding all the transaction outputs
-====
-[source,python]
-----
-include::code/rpc_block.py[]
-----
-====
-
-Running this code, we get:
-
-----
-$ python rpc_block.py
-
-('Total value in block: ', Decimal('10322.07722534'))
-----
-
-Our example code calculates that the total value transacted in this block is 10,322.07722534 BTC (including 25 BTC reward and 0.0909 BTC in fees). Compare that to the amount reported by a block explorer site by searching for the block hash or height. Some block explorers report the total value excluding the reward and excluding the fees. See if you can spot the difference.((("", startref="BCapi03")))((("", startref="progint03")))
-
-[[alt_libraries]]
-=== Alternative Clients, Libraries, and Toolkits
-
-((("Bitcoin Core", "alternatives to", id="BCalt03")))((("clients, libraries, and toolkits", id="clients03")))((("libraries, clients, and toolkits", id="librar03")))((("toolkits, libraries, and clients", id="toolkit03")))((("third-party API clients", id="thirdpart03")))There are many alternative clients, libraries, toolkits, and even full-node implementations in the bitcoin ecosystem. These are implemented in a variety of programming languages, offering programmers native interfaces in their preferred language.
-
-The following sections list some of the best libraries, clients, and toolkits, organized by programming languages.
-
-==== C/C++
-https://github.com/bitcoin/bitcoin[Bitcoin Core] :: The reference implementation of bitcoin
-https://github.com/libbitcoin/libbitcoin-system[libbitcoin]:: Cross-platform C++ development toolkit, node, and consensus library
-https://github.com/libbitcoin/libbitcoin-explorer[bitcoin explorer]:: Libbitcoin's command-line tool
-https://github.com/jgarzik/picocoin[picocoin]:: A C language lightweight client library for bitcoin by Jeff Garzik
-
-==== JavaScript
-https://bcoin.io/[bcoin]:: A modular and scalable full-node implementation with API
-https://bitcore.io/[Bitcore] :: Full node, API, and library by Bitpay
-https://github.com/bitcoinjs/bitcoinjs-lib[BitcoinJS] :: A pure JavaScript Bitcoin library for node.js and browsers
-
-==== Java
-https://bitcoinj.github.io[bitcoinj]:: A Java full-node client library
-
-==== PHP
-https://github.com/bit-wasp/bitcoin-php[bitwasp/bitcoin]:: A PHP bitcoin library, and related projects
-
-==== Python
-https://github.com/petertodd/python-bitcoinlib[python-bitcoinlib]::  A Python bitcoin library, consensus library, and node by Peter Todd
-https://github.com/richardkiss/pycoin[pycoin]:: A Python bitcoin library by Richard Kiss
-https://github.com/primal100/pybitcointools[pybitcointools]:: An archived fork of Python bitcoin library by Vitalik Buterin
-
-==== Ruby
-https://github.com/sinisterchipmunk/bitcoin-client[bitcoin-client]:: A Ruby library wrapper for the JSON-RPC API
-
-==== Go
-https://github.com/btcsuite/btcd[btcd]:: A Go language full-node Bitcoin client
-
-==== Rust
-https://github.com/rust-bitcoin/rust-bitcoin[rust-bitcoin]:: Rust bitcoin library for serialization, parsing, and API calls
-
-==== C#
-https://github.com/MetacoSA/NBitcoin[NBitcoin]:: Comprehensive bitcoin library for the .NET framework
-
-==== Objective-C
-https://github.com/oleganza/CoreBitcoin[CoreBitcoin]:: Bitcoin toolkit for ObjC and Swift
-
-Many more libraries exist in a variety of other programming languages and more are created all the time.((("", startref="BCalt03")))((("", startref="clients03")))((("", startref="thirdpart03")))((("", startref="toolkit03")))((("", startref="librar03")))

ch04.asciidoc

@@ -1,801 +0,0 @@
-[[ch04_keys_addresses]]
-== Keys, Addresses
-
-((("cryptography", "defined")))((("cryptography", see="also keys and addresses")))You may have heard that bitcoin is based on _cryptography_, which is a branch of mathematics used extensively in computer security. Cryptography means "secret writing" in Greek, but the science of cryptography encompasses more than just secret writing, which is referred to as encryption. Cryptography can also be used to prove knowledge of a secret without revealing that secret (digital signature), or prove the authenticity of data (digital fingerprint). These types of cryptographic proofs are the mathematical tools critical to bitcoin and used extensively in bitcoin applications. ((("encryption")))((("encryption", see="also keys and addresses")))Ironically, encryption is not an important part of bitcoin, as its communications and transaction data are not encrypted and do not need to be encrypted to protect the funds. In this chapter we will introduce some of the cryptography used in bitcoin to control ownership of funds, in the form of keys, addresses, and wallets.
-
-=== Introduction
-
-((("digital keys", see="keys and addresses")))((("keys and addresses", "overview of", id="KAover04")))((("digital signatures", "purpose of")))Ownership of bitcoin is established through _digital keys_, _Bitcoin addresses_, and _digital signatures_. The digital keys are not actually stored in the network, but are instead created and stored by users in a file, or simple database, called a _wallet_. The digital keys in a user's wallet are completely independent of the Bitcoin protocol and can be generated and managed by the user's wallet software without reference to the blockchain or access to the internet. Keys enable many of the interesting properties of bitcoin, including decentralized trust and control, ownership attestation, and the cryptographic-proof security model.
-
-Most bitcoin transactions require a valid digital signature to be included in the blockchain, which can only be generated with a secret key; therefore, anyone with a copy of that key has control of the bitcoin. ((("witnesses")))The digital signature used to spend funds is also referred to as a _witness_, a term used in cryptography. The witness data in a bitcoin transaction testifies to the true ownership of the funds being spent.
-
-((("public and private keys", "key pairs")))((("public and private keys", see="also keys and addresses")))Keys come in pairs consisting of a private (secret) key and a public key. Think of the public key as similar to a bank account number and the private key as similar to the secret PIN, or signature on a check, that provides control over the account. These digital keys are very rarely seen by the users of bitcoin. For the most part, they are stored inside the wallet file and managed by the bitcoin wallet software.
-
-In the payment portion of a bitcoin transaction, the recipient's public key is represented by its digital fingerprint, called a _Bitcoin address_, which is used in the same way as the beneficiary name on a check (i.e., "Pay to the order of"). In most cases, a Bitcoin address is generated from and corresponds to a public key. However, not all Bitcoin addresses represent public keys; they can also represent other beneficiaries such as scripts, as we will see later in this chapter. This way, Bitcoin addresses abstract the recipient of funds, making transaction destinations flexible, similar to paper checks: a single payment instrument that can be used to pay into people's accounts, pay into company accounts, pay for bills, or pay to cash. The Bitcoin address is the only representation of the keys that users will routinely see, because this is the part they need to share with the world.
-
-First, we will introduce cryptography and explain the mathematics used in bitcoin. Next, we will look at how keys are generated, stored, and managed.  We will review the various encoding formats used to represent private and public keys, addresses, and script addresses. Finally, we will look at advanced use of keys and addresses: vanity, multisignature, and script addresses and paper wallets.
-
-==== Public Key Cryptography and Cryptocurrency
-
-((("keys and addresses", "overview of", "public key cryptography")))((("digital currencies", "cryptocurrency")))Public key cryptography was invented in the 1970s and is a mathematical foundation for computer and information security.
-
-Since the invention of public key cryptography, several suitable mathematical functions, such as prime number exponentiation and elliptic curve multiplication, have been discovered. These mathematical functions are practically irreversible, meaning that they are easy to calculate in one direction and infeasible to calculate in the opposite direction. Based on these mathematical functions, cryptography enables the creation of digital secrets and unforgeable digital signatures. Bitcoin uses elliptic curve multiplication as the basis for its cryptography.
-
-In bitcoin, we use public key cryptography to create a key pair that controls access to bitcoin. The key pair consists of a private key and--derived from it--a unique public key. The public key is used to receive funds, and the private key is used to sign transactions to spend the funds.
-
-There is a mathematical relationship between the public and the private key that allows the private key to be used to generate signatures on messages. These signatures can be validated against the public key without revealing the private key.
-
-When spending bitcoin, the current bitcoin owner presents her public key and a signature (different each time, but created from the same private key) in a transaction to spend those bitcoin. Through the presentation of the public key and signature, everyone in the Bitcoin network can verify and accept the transaction as valid, confirming that the person transferring the bitcoin owned them at the time of the transfer.
-
-[TIP]
-====
-((("keys and addresses", "overview of", "key pairs")))In most wallet implementations, the private and public keys are stored together as a _key pair_ for convenience. However, the public key can be calculated from the private key, so storing only the private key is also possible.
-====
-
-[[private_public_keys]]
-==== Private and Public Keys
-
-((("keys and addresses", "overview of", "private and public key pairs")))((("elliptic curve cryptography")))((("cryptography", "elliptic curve cryptography")))A bitcoin wallet contains a collection of key pairs, each consisting of a private key and a public key. The private key (k) is a number, usually picked at random. From the private key, we use elliptic curve multiplication, a one-way cryptographic function, to generate a public key (K). From the public key (K), we use a one-way cryptographic hash function to generate a Bitcoin address (A). In this section, we will start with generating the private key, look at the elliptic curve math that is used to turn that into a public key, and finally, generate a Bitcoin address from the public key. The relationship between private key, public key, and Bitcoin address is shown in <<k_to_K_to_A>>.
-
-[[k_to_K_to_A]]
-.Private key, public key, and Bitcoin address
-image::images/mbc2_0401.png["privk_to_pubK_to_addressA"]
-
-.Why Use Asymmetric Cryptography (Public/Private Keys)?
-****
-((("cryptography", "asymmetric")))((("digital signatures", "asymmetric cryptography and")))((("asymmetric cryptography")))Why is asymmetric cryptography used in bitcoin? It's not used to "encrypt" (make secret) the transactions. Rather, the useful property of asymmetric cryptography is the ability to generate _digital signatures_. A private key can be applied to the digital fingerprint of a transaction to produce a numerical signature. This signature can only be produced by someone with knowledge of the private key. However, anyone with access to the public key and the transaction fingerprint can use them to _verify_ the signature. This useful property of asymmetric cryptography makes it possible for anyone to verify every signature on every transaction, while ensuring that only the owners of private keys can produce valid signatures.
-****
-
-[[private_keys]]
-==== Private Keys
-
-((("keys and addresses", "overview of", "private key generation")))((("warnings and cautions", "private key protection")))A private key is simply a number, picked at random. Ownership and control over the private key is the root of user control over all funds associated with the corresponding Bitcoin address. The private key is used to create signatures that are required to spend bitcoin by proving ownership of funds used in a transaction. The private key must remain secret at all times, because revealing it to third parties is equivalent to giving them control over the bitcoin secured by that key. The private key must also be backed up and protected from accidental loss, because if it's lost it cannot be recovered and the funds secured by it are forever lost, too.
-
-[TIP]
-====
-The bitcoin private key is just a number. You can pick your private keys randomly using just a coin, pencil, and paper: toss a coin 256 times and you have the binary digits of a random private key you can use in a bitcoin wallet. The public key can then be generated from the private key.
-====
-
-===== Generating a private key from a random number
-
-The first and most important step in generating keys is to find a secure source of entropy, or randomness. Creating a bitcoin key is essentially the same as "Pick a number between 1 and 2^256^." The exact method you use to pick that number does not matter as long as it is not predictable or repeatable. Bitcoin software uses the underlying operating system's random number generators to produce 256 bits of entropy (randomness). Usually, the OS random number generator is initialized by a human source of randomness, which is why you may be asked to wiggle your mouse around for a few seconds.
-
-More precisely, the private key can be any number between +0+ and +n - 1+ inclusive, where n is a constant (n = 1.1578 * 10^77^, slightly less than 2^256^) defined as the order of the elliptic curve used in bitcoin (see <<elliptic_curve>>). To create such a key, we randomly pick a 256-bit number and check that it is less than +n+. In programming terms, this is usually achieved by feeding a larger string of random bits, collected from a cryptographically secure source of randomness, into the SHA256 hash algorithm, which will conveniently produce a 256-bit number. If the result is less than +n+, we have a suitable private key. Otherwise, we simply try again with another random number.
-
-[WARNING]
-====
-((("random numbers", "random number generation")))((("entropy", "random number generation")))Do not write your own code to create a random number or use a "simple" random number generator offered by your programming language. Use a cryptographically secure pseudorandom number generator (CSPRNG) with a seed from a source of sufficient entropy. Study the documentation of the random number generator library you choose to make sure it is cryptographically secure. Correct implementation of the CSPRNG is critical to the security of the keys.
-====
-
-The following is a randomly generated private key (k) shown in hexadecimal format (256 bits shown as 64 hexadecimal digits, each 4 bits):
-
-----
-1E99423A4ED27608A15A2616A2B0E9E52CED330AC530EDCC32C8FFC6A526AEDD
-----
-
-
-[TIP]
-====
-The size of bitcoin's private key space, (2^256^) is an unfathomably large number. It is approximately 10^77^ in decimal. For comparison, the visible universe is estimated to contain 10^80^ atoms.
-====
-
-((("dumpprivkey command")))To generate a new key with the Bitcoin Core client (see <<ch03_bitcoin_client>>), use the +getnewaddress+ command. For security reasons it displays the address only, not the private key. To ask +bitcoind+ to expose the private key, use the +dumpprivkey+ command. The +dumpprivkey+ command shows the private key in a Base58 checksum-encoded format called the _Wallet Import Format_ (WIF), which we will examine in more detail in <<priv_formats>>. Here's an example of generating and displaying a private key using these two commands:
-
-----
-$ bitcoin-cli getnewaddress
-1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy
-$ bitcoin-cli dumpprivkey 1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy
-KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ
-----
-
-The +dumpprivkey+ command opens the wallet and extracts the private key that was generated by the +getnewaddress+ command. It is not possible for +bitcoind+ to know the private key from the address unless they are both stored in the wallet.
-
-[TIP]
-=====================================================================
-The +dumpprivkey+ command does not generate a private key from an address, as this is impossible. The command simply reveals the private key that is already known to the wallet and which was generated by the +getnewaddress+ command.
-=====================================================================
-
-[role="pagebreak-before"]
-You can also use the Bitcoin Explorer command-line tool (see <<appdx_bx>>) to generate and display private keys with the commands +seed+, +ec-new+, and +ec-to-wif+:
-
-----
-$ bx seed | bx ec-new | bx ec-to-wif
-5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
-----
-
-[[pubkey]]
-==== Public Keys
-
-((("keys and addresses", "overview of", "public key calculation")))((("generator point")))The public key is calculated from the private key using elliptic curve multiplication, which is irreversible: _K_ = _k_ * _G_, where _k_ is the private key, _G_ is a constant point called the _generator point_, and _K_ is the resulting public key. The reverse operation, known as "finding the discrete logarithm"—calculating _k_ if you know __K__—is as difficult as trying all possible values of _k_, i.e., a brute-force search. Before we demonstrate how to generate a public key from a private key, let's look at elliptic curve cryptography in a bit more detail.
-
-[TIP]
-====
-Elliptic curve multiplication is a type of function that cryptographers call a "one-way" function: it is easy to do in one direction (multiplication) and impossible to do in the reverse direction ("division", or finding the discrete logarithm). The owner of the private key can easily create the public key and then share it with the world knowing that no one can reverse the function and calculate the private key from the public key. This mathematical trick becomes the basis for unforgeable and secure digital signatures that prove ownership of bitcoin funds.
-====
-
-[[elliptic_curve]]
-==== Elliptic Curve Cryptography Explained
-
-((("keys and addresses", "overview of", "elliptic curve cryptography")))((("elliptic curve cryptography", id="eliptic04")))((("cryptography", "elliptic curve cryptography", id="Celliptic04")))Elliptic curve cryptography is a type of asymmetric or public key cryptography based on the discrete logarithm problem as expressed by addition and multiplication on the points of an elliptic curve.
-
-<<ecc-curve>> is an example of an elliptic curve, similar to that used by bitcoin.
-
-[[ecc-curve]]
-[role="smallerthirty"]
-.An elliptic curve
-image::images/mbc2_0402.png["ecc-curve"]
-
-Bitcoin uses a specific elliptic curve and set of mathematical constants, as defined in a standard called +secp256k1+, established by the National Institute of Standards and Technology (NIST). The +secp256k1+ curve is defined by the following function, which produces an elliptic curve:
-
-[latexmath]
-++++
-\begin{equation}
-{y^2 = (x^3 + 7)}~\text{over}~(\mathbb{F}_p)
-\end{equation}
-++++
-
-or
-
-[latexmath]
-++++
-\begin{equation}
-{y^2 \mod p = (x^3 + 7) \mod p}
-\end{equation}
-++++
-
-The _mod p_ (modulo prime number p) indicates that this curve is over a finite field of prime order _p_, also written as latexmath:[\( \mathbb{F}_p \)], where p = 2^256^ – 2^32^ – 2^9^ – 2^8^ – 2^7^ – 2^6^ – 2^4^ – 1, a very large prime number.
-
-Because this curve is defined over a finite field of prime order instead of over the real numbers, it looks like a pattern of dots scattered in two dimensions, which makes it difficult to visualize. However, the math is identical to that of an elliptic curve over real numbers. As an example, <<ecc-over-F17-math>> shows the same elliptic curve over a much smaller finite field of prime order 17, showing a pattern of dots on a grid. The +secp256k1+ bitcoin elliptic curve can be thought of as a much more complex pattern of dots on a unfathomably large grid.
-
-[[ecc-over-F17-math]]
-[role="smallersixty"]
-.Elliptic curve cryptography: visualizing an elliptic curve over F(p), with p=17
-image::images/mbc2_0403.png["ecc-over-F17-math"]
-
-So, for example, the following is a point P with coordinates (x,y) that is a point on the +secp256k1+ curve:
-
-----
-P = (55066263022277343669578718895168534326250603453777594175500187360389116729240, 32670510020758816978083085130507043184471273380659243275938904335757337482424)
-----
-
-<<example_4_1>> shows how you can check this yourself using Python:
-
-[[example_4_1]]
-.Using Python to confirm that this point is on the elliptic curve
-====
-[source, pycon]
-----
-Python 3.4.0 (default, Mar 30 2014, 19:23:13)
-[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
-Type "help", "copyright", "credits" or "license" for more information.
->>> p = 115792089237316195423570985008687907853269984665640564039457584007908834671663
->>> x = 55066263022277343669578718895168534326250603453777594175500187360389116729240
->>> y = 32670510020758816978083085130507043184471273380659243275938904335757337482424
->>> (x ** 3 + 7 - y**2) % p
-0
-----
-====
-
-In elliptic curve math, there is a point called the "point at infinity," which roughly corresponds to the role of zero in addition. On computers, it's sometimes represented by x = y = 0 (which doesn't satisfy the elliptic curve equation, but it's an easy separate case that can be checked).
-
-There is also a pass:[+] operator, called "addition," which has some properties similar to the traditional addition of real numbers that gradeschool children learn. Given two points P~1~ and P~2~ on the elliptic curve, there is a third point P~3~ = P~1~ + P~2~, also on the elliptic curve.
-
-Geometrically, this third point P~3~ is calculated by drawing a line between P~1~ and P~2~. This line will intersect the elliptic curve in exactly one additional place. Call this point P~3~' = (x, y). Then reflect in the x-axis to get P~3~ = (x, –y).
-
-There are a couple of special cases that explain the need for the "point at infinity."
-
-If P~1~ and P~2~ are the same point, the line "between" P~1~ and P~2~ should extend to be the tangent on the curve at this point P~1~. This tangent will intersect the curve in exactly one new point. You can use techniques from calculus to determine the slope of the tangent line. These techniques curiously work, even though we are restricting our interest to points on the curve with two integer coordinates!
-
-In some cases (i.e., if P~1~ and P~2~ have the same x values but different y values), the line between P~1~ and P~2~ will be exactly vertical, in which case P~3~ = "point at infinity."
-
-If P~1~ is the "point at infinity," then P~1~ + P~2~ = P~2~. Similarly, if P~2~ is the point at infinity, then P~1~ + P~2~ = P~1~. This shows how the point at infinity plays the role of zero.
-
-It turns out that pass:[+] is associative, which means that (A pass:[+] B) pass:[+] C = A pass:[+] (B pass:[+] C). That means we can write A pass:[+] B pass:[+] C without parentheses and without ambiguity.
-
-Now that we have defined addition, we can define multiplication in the standard way that extends addition. For a point P on the elliptic curve, if k is a whole number, then kP = P + P + P + ... + P (k times). Note that k is sometimes confusingly called an "exponent" in this case.((("", startref="eliptic04")))((("", startref="Celliptic04")))
-
-[[public_key_derivation]]
-==== Generating a Public Key
-
-((("keys and addresses", "overview of", "public key generation")))((("generator point")))Starting with a private key in the form of a randomly generated number _k_, we multiply it by a predetermined point on the curve called the _generator point_ _G_ to produce another point somewhere else on the curve, which is the corresponding public key _K_. The generator point is specified as part of the +secp256k1+ standard and is always the same for all keys in bitcoin:
-
-[latexmath]
-++++
-\begin{equation}
-{K = k * G}
-\end{equation}
-++++
-
-where _k_ is the private key, _G_ is the generator point, and _K_ is the resulting public key, a point on the curve. Because the generator point is always the same for all bitcoin users, a private key _k_ multiplied with _G_ will always result in the same public key _K_. The relationship between _k_ and _K_ is fixed, but can only be calculated in one direction, from _k_ to _K_. That's why a Bitcoin address (derived from _K_) can be shared with anyone and does not reveal the user's private key (_k_).
-
-[TIP]
-====
-A private key can be converted into a public key, but a public key cannot be converted back into a private key because the math only works one way.
-====
-
-Implementing the elliptic curve multiplication, we take the private key _k_ generated previously and multiply it with the generator point G to find the public key _K_:
-
-----
-K = 1E99423A4ED27608A15A2616A2B0E9E52CED330AC530EDCC32C8FFC6A526AEDD * G
-----
-
-Public key _K_ is defined as a point +K = (x,y)+:
-
-----
-K = (x, y)
-
-where,
-
-x = F028892BAD7ED57D2FB57BF33081D5CFCF6F9ED3D3D7F159C2E2FFF579DC341A
-y = 07CF33DA18BD734C600B96A72BBC4749D5141C90EC8AC328AE52DDFE2E505BDB
-----
-
-To visualize multiplication of a point with an integer, we will use the simpler elliptic curve over real numbers&#x2014;remember, the math is the same. Our goal is to find the multiple _kG_ of the generator point _G_, which is the same as adding _G_ to itself, _k_ times in a row. In elliptic curves, adding a point to itself is the equivalent of drawing a tangent line on the point and finding where it intersects the curve again, then reflecting that point on the x-axis.
-
-<<ecc_illustrated>> shows the process for deriving _G_, _2G_, _4G_, and _8G_ as a geometric operation on the curve.
-
-[TIP]
-====
-((("secp256k1 optimized C library")))Bitcoin uses the https://github.com/bitcoin-core/secp256k1[secp256k1 optimized C library] to do the elliptic curve math.((("", startref="KAover04")))
-====
-
-[[ecc_illustrated]]
-.Elliptic curve cryptography: visualizing the multiplication of a point G by an integer k on an elliptic curve
-image::images/mbc2_0404.png["ecc_illustrated"]
-
-=== Bitcoin Addresses
-
-((("keys and addresses", "Bitcoin addresses", id="KAaddress04")))A Bitcoin address is a string of digits and characters that can be shared with anyone who wants to send you money. Addresses produced from public keys consist of a string of numbers and letters, beginning with the digit "1". Here's an example of a Bitcoin address:
-
-----
-1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy
-----
-
-
-The Bitcoin address is what appears most commonly in a transaction as the "recipient" of the funds. If we compare a bitcoin transaction to a paper check, the Bitcoin address is the beneficiary, which is what we write on the line after "Pay to the order of." On a paper check, that beneficiary can sometimes be the name of a bank account holder, but can also include corporations, institutions, or even cash. Because paper checks do not need to specify an account, but rather use an abstract name as the recipient of funds, they are very flexible payment instruments. Bitcoin transactions use a similar abstraction, the Bitcoin address, to make them very flexible. A Bitcoin address can represent the owner of a private/public key pair, or it can represent something else, such as a payment script, as we will see in <<p2sh>>. For now, let's examine the simple case, a Bitcoin address that represents, and is derived from, a public key.
-
-((("addresses", "algorithms used to create")))The Bitcoin address is derived from the public key through the use of one-way cryptographic hashing. A "hashing algorithm" or simply "hash algorithm" is a one-way function that produces a fingerprint or "hash" of an arbitrary-sized input. Cryptographic hash functions are used extensively in bitcoin: in Bitcoin addresses, in script addresses, and in the mining Proof-of-Work algorithm. The algorithms used to make a Bitcoin address from a public key are the Secure Hash Algorithm (SHA) and the RACE Integrity Primitives Evaluation Message Digest (RIPEMD), specifically SHA256 and RIPEMD160.
-
-Starting with the public key _K_, we compute the SHA256 hash and then compute the RIPEMD160 hash of the result, producing a 160-bit (20-byte) number:
-
-[latexmath]
-++++
-\begin{equation}
-{A = RIPEMD160(SHA256(K))}
-\end{equation}
-++++
-
-where _K_ is the public key and _A_ is the resulting Bitcoin address.
-
-
-[TIP]
-====
-A Bitcoin address is _not_ the same as a public key. Bitcoin addresses are derived from a public key using a one-way function.
-====
-
-Bitcoin addresses are almost always encoded as "Base58Check" (see <<base58>>), which uses 58 characters (a Base58 number system) and a checksum to help human readability, avoid ambiguity, and protect against errors in address transcription and entry. Base58Check is also used in many other ways in bitcoin, whenever there is a need for a user to read and correctly transcribe a number, such as a Bitcoin address, a private key, an encrypted key, or a script hash. In the next section we will examine the mechanics of Base58Check encoding and decoding and the resulting representations. <<pubkey_to_address>> illustrates the conversion of a public key into a Bitcoin address.
-
-[[pubkey_to_address]]
-.Public key to Bitcoin address: conversion of a public key into a Bitcoin address
-image::images/mbc2_0405.png["pubkey_to_address"]
-
-[[base58]]
-==== Base58 and Base58Check Encoding
-
-((("keys and addresses", "Bitcoin addresses", "Base58 and Base58check encoding")))((("Base58 and Base58check encoding", id="base5804")))((("addresses", "Base58 and Base58check encoding", id="Abase5804")))In order to represent long numbers in a compact way, using fewer symbols, many computer systems use mixed-alphanumeric representations with a base (or radix) higher than 10. For example, whereas the traditional decimal system uses the 10 numerals 0 through 9, the hexadecimal system uses 16, with the letters A through F as the six additional symbols. A number represented in hexadecimal format is shorter than the equivalent decimal representation. Even more compact, Base64 representation uses 26 lowercase letters, 26 capital letters, 10 numerals, and 2 more characters such as &#x201c;`+`&#x201d; and "/" to transmit binary data over text-based media such as email. Base64 is most commonly used to add binary attachments to email. Base58 is a text-based binary-encoding format developed for use in bitcoin and used in many other cryptocurrencies. It offers a balance between compact representation, readability, and error detection and prevention. Base58 is a subset of Base64, using upper- and lowercase letters and numbers, but omitting some characters that are frequently mistaken for one another and can appear identical when displayed in certain fonts. Specifically, Base58 is Base64 without the 0 (number zero), O (capital o), l (lower L), I (capital i), and the symbols &#x201c;`+`&#x201d; and "/". Or, more simply, it is a set of lowercase and capital letters and numbers without the four (0, O, l, I) just mentioned. <<base58alphabet>> shows the full Base58 alphabet.
-
-[[base58alphabet]]
-.Bitcoin's Base58 alphabet
-====
-----
-123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz
-----
-====
-
-
-To add extra security against typos or transcription errors, Base58Check is a Base58 encoding format, frequently used in bitcoin, which has a built-in error-checking code. The checksum is an additional four bytes added to the end of the data that is being encoded. The checksum is derived from the hash of the encoded data and can therefore be used to detect and prevent transcription and typing errors. When presented with Base58Check code, the decoding software will calculate the checksum of the data and compare it to the checksum included in the code. If the two do not match, an error has been introduced and the Base58Check data is invalid. This prevents a mistyped Bitcoin address from being accepted by the wallet software as a valid destination, an error that would otherwise result in loss of funds.
-
-To convert data (a number) into a Base58Check format, we first add a prefix to the data, called the "version byte," which serves to easily identify the type of data that is encoded. For example, in the case of a Bitcoin address the prefix is zero (0x00 in hex), whereas the prefix used when encoding a private key is 128 (0x80 in hex). A list of common version prefixes is shown in <<base58check_versions>>.
-
-Next, we compute the "double-SHA" checksum, meaning we apply the SHA256 hash-algorithm twice on the previous result (prefix and data):
-
-----
-checksum = SHA256(SHA256(prefix+data))
-----
-
-From the resulting 32-byte hash (hash-of-a-hash), we take only the first four bytes. These four bytes serve as the error-checking code, or checksum. The checksum is concatenated (appended) to the end.
-
-The result is composed of three items: a prefix, the data, and a checksum. This result is encoded using the Base58 alphabet described previously. <<base58check_encoding>> illustrates the Base58Check encoding process.
-
-[[base58check_encoding]]
-.Base58Check encoding: a Base58, versioned, and checksummed format for unambiguously encoding bitcoin data
-image::images/mbc2_0406.png["Base58CheckEncoding"]
-
-In bitcoin, most of the data presented to the user is Base58Check-encoded to make it compact, easy to read, and easy to detect errors. The version prefix in Base58Check encoding is used to create easily distinguishable formats, which when encoded in Base58 contain specific characters at the beginning of the Base58Check-encoded payload. These characters make it easy for humans to identify the type of data that is encoded and how to use it. This is what differentiates, for example, a Base58Check-encoded Bitcoin address that starts with a 1 from a Base58Check-encoded private key WIF that starts with a 5. Some example version prefixes and the resulting Base58 characters are shown in <<base58check_versions>>.
-
-[[base58check_versions]]
-.Base58Check version prefix and encoded result examples
-[options="header"]
-|=======
-|Type| Version prefix (hex)| Base58 result prefix
-| Bitcoin Address | 0x00 | 1
-| Pay-to-Script-Hash Address | 0x05 | 3
-| Bitcoin Testnet Address | 0x6F | m or n
-| Private Key WIF |  0x80 | 5, K, or L
-| BIP-38 Encrypted Private Key | 0x0142 | 6P
-| BIP-32 Extended Public Key | 0x0488B21E | xpub
-|=======
-
-==== Key Formats
-
-((("keys and addresses", "Bitcoin addresses", "key formats")))Both private and public keys can be represented in a number of different formats. These representations all encode the same number, even though they look different. These formats are primarily used to make it easy for people to read and transcribe keys without introducing errors.
-
-[[priv_formats]]
-===== Private key formats
-
-((("public and private keys", "private key formats")))The private key can be represented in a number of different formats, all of which correspond to the same 256-bit number. <<table_4-2>> shows three common formats used to represent private keys. Different formats are used in different circumstances. Hexadecimal and raw binary formats are used internally in software and rarely shown to users. The WIF is used for import/export of keys between wallets and often used in QR code (barcode) representations of private keys.
-
-[[table_4-2]]
-.Private key representations (encoding formats)
-[options="header"]
-|=======
-|Type|Prefix|Description
-| Raw | None | 32 bytes
-| Hex | None | 64 hexadecimal digits
-| WIF |  5 | Base58Check encoding: Base58 with version prefix of 0x80 and 4-byte checksum
-| WIF-compressed | K or L | As above, with added suffix 0x01 before encoding
-|=======
-
-<<table_4-3>> shows the private key generated in these three formats.
-
-[[table_4-3]]
-.Example: Same key, different formats
-[options="header"]
-|=======
-|Format | Private key
-| Hex | 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd
-| WIF | 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
-| WIF-compressed | KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ
-|=======
-
-All of these representations are different ways of showing the same number, the same private key. They look different, but any one format can easily be converted to any other format. Note that the "raw binary" is not shown in <<table_4-3>> as any encoding for display here would, by definition, not be raw binary data.
-
-We use the +wif-to-ec+ command from Bitcoin Explorer (see <<appdx_bx>>) to show that both WIF keys represent the same private key:
-
-----
-$ bx wif-to-ec 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
-1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd
-
-$ bx wif-to-ec KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ
-1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd
-----
-
-===== Decode from Base58Check
-
-The Bitcoin Explorer commands (see <<appdx_bx>>) make it easy to write shell scripts and command-line "pipes" that manipulate bitcoin keys, addresses, and transactions. You can use Bitcoin Explorer to decode the Base58Check format on the command line.
-
-We use the +base58check-decode+ command to decode the uncompressed key:
-
-----
-$ bx base58check-decode 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
-wrapper
-{
-    checksum 4286807748
-    payload 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd
-    version 128
-}
-----
-
-The result contains the key as payload, the WIF version prefix 128, and a checksum.
-
-Notice that the "payload" of the compressed key is appended with the suffix +01+, signalling that the derived public key is to be compressed:
-
-----
-$ bx base58check-decode KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ
-wrapper
-{
-    checksum 2339607926
-    payload 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd01
-    version 128
-}
-----
-
-===== Encode from hex to Base58Check
-
-To encode into Base58Check (the opposite of the previous command), we use the +base58check-encode+ command from Bitcoin Explorer (see <<appdx_bx>>) and provide the hex private key, followed by the WIF version prefix 128:
-
-----
-bx base58check-encode 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd --version 128
-5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
-----
-
-===== Encode from hex (compressed key) to Base58Check
-
-To encode into Base58Check as a "compressed" private key (see <<comp_priv>>), we append the suffix +01+ to the hex key and then encode as in the preceding section:
-
-----
-$ bx base58check-encode 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd01 --version 128
-KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ
-----
-
-The resulting WIF-compressed format starts with a "K." This denotes that the private key within has a suffix of "01" and will be used to produce compressed public keys only (see <<comp_pub>>).
-
-===== Public key formats
-
-((("public and private keys", "public key formats")))Public keys are also presented in different ways, usually as either _compressed_ or _uncompressed_ public keys.
-
-As we saw previously, the public key is a point on the elliptic curve consisting of a pair of coordinates +(x,y)+. It is usually presented with the prefix +04+ followed by two 256-bit numbers: one for the _x_ coordinate of the point, the other for the _y_ coordinate. The prefix +04+ is used to distinguish uncompressed public keys from compressed public keys that begin with a +02+ or a +03+.
-
-Here's the public key generated by the private key we created earlier, shown as the coordinates +x+ and +y+:
-
-----
-x = F028892BAD7ED57D2FB57BF33081D5CFCF6F9ED3D3D7F159C2E2FFF579DC341A
-y = 07CF33DA18BD734C600B96A72BBC4749D5141C90EC8AC328AE52DDFE2E505BDB
-----
-
-Here's the same public key shown as a 520-bit number (130 hex digits) with the prefix +04+ followed by +x+ and then +y+ coordinates, as +04 x y+:
-
-++++
-<pre data-type="programlisting">
-K = 04F028892BAD7ED57D2FB57BF33081D5CFCF6F9ED3D3D7F159C2E2FFF579DC341A&#x21b5;
-07CF33DA18BD734C600B96A72BBC4749D5141C90EC8AC328AE52DDFE2E505BDB
-</pre>
-++++
-
-[[comp_pub]]
-===== Compressed public keys
-
-((("public and private keys", "compressed public keys")))Compressed public keys were introduced to bitcoin to reduce the size of transactions and conserve disk space on nodes that store the Bitcoin blockchain database. Most transactions include the public key, which is required to validate the owner's credentials and spend the bitcoin. Each public key requires 520 bits (prefix + x + y), which when multiplied by several hundred transactions per block, or tens of thousands of transactions per day, adds a significant amount of data to the blockchain.
-
-As we saw in the section <<pubkey>>, a public key is a point (x,y) on an elliptic curve. Because the curve expresses a mathematical function, a point on the curve represents a solution to the equation and, therefore, if we know the _x_ coordinate we can calculate the _y_ coordinate by solving the equation y^2^ mod p = (x^3^ + 7) mod p. That allows us to store only the _x_ coordinate of the public key point, omitting the _y_ coordinate and reducing the size of the key and the space required to store it by 256 bits. An almost 50% reduction in size in every transaction adds up to a lot of data saved over time!
-
-Whereas uncompressed public keys have a prefix of +04+, compressed public keys start with either a +02+ or a +03+ prefix. Let's look at why there are two possible prefixes: because the left side of the equation is __y__^2^, the solution for _y_ is a square root, which can have a positive or negative value. Visually, this means that the resulting _y_ coordinate can be above or below the x-axis. As you can see from the graph of the elliptic curve in <<ecc-curve>>, the curve is symmetric, meaning it is reflected like a mirror by the x-axis. So, while we can omit the _y_ coordinate we have to store the _sign_ of _y_ (positive or negative); or in other words, we have to remember if it was above or below the x-axis because each of those options represents a different point and a different public key. When calculating the elliptic curve in binary arithmetic on the finite field of prime order p, the _y_ coordinate is either even or odd, which corresponds to the positive/negative sign as explained earlier. Therefore, to distinguish between the two possible values of _y_, we store a compressed public key with the prefix +02+ if the _y_ is even, and +03+ if it is odd, allowing the software to correctly deduce the _y_ coordinate from the _x_ coordinate and uncompress the public key to the full coordinates of the point. Public key compression is illustrated in <<pubkey_compression>>.
-
-Here's the same public key generated previously, shown as a compressed public key stored in 264 bits (66 hex digits) with the prefix +03+ indicating the _y_ coordinate is odd:
-
-----
-K = 03F028892BAD7ED57D2FB57BF33081D5CFCF6F9ED3D3D7F159C2E2FFF579DC341A
-----
-
-This compressed public key corresponds to the same private key, meaning it is generated from the same private key. However, it looks different from the uncompressed public key. More importantly, if we convert this compressed public key to a Bitcoin address using the double-hash function (+RIPEMD160(SHA256(K))+) it will produce a _different_ Bitcoin address. This can be confusing, because it means that a single private key can produce a public key expressed in two different formats (compressed and uncompressed) that produce two different Bitcoin addresses. However, the private key is identical for both Bitcoin addresses.
-
-[[pubkey_compression]]
-[role="smallerseventy"]
-.Public key compression
-image::images/mbc2_0407.png["pubkey_compression"]
-
-Compressed public keys are gradually becoming the default across Bitcoin clients, which is having a significant impact on reducing the size of transactions and therefore the blockchain. However, not all clients support compressed public keys yet. Newer clients that support compressed public keys have to account for transactions from older clients that do not support compressed public keys. This is especially important when a wallet application is importing private keys from another bitcoin wallet application, because the new wallet needs to scan the blockchain to find transactions corresponding to these imported keys. Which Bitcoin addresses should the bitcoin wallet scan for? The Bitcoin addresses produced by uncompressed public keys, or the Bitcoin addresses produced by compressed public keys? Both are valid Bitcoin addresses, and can be signed for by the private key, but they are different addresses!
-
-To resolve this issue, when private keys are exported from a wallet, the WIF that is used to represent them is implemented differently in newer bitcoin wallets, to indicate that these private keys have been used to produce _compressed_ public keys and therefore _compressed_ Bitcoin addresses. This allows the importing wallet to distinguish between private keys originating from older or newer wallets and search the blockchain for transactions with Bitcoin addresses corresponding to the uncompressed, or the compressed, public keys, respectively. Let's look at how this works in more detail, in the next section.
-
-[[comp_priv]]
-===== Compressed private keys
-
-((("public and private keys", "compressed private keys")))Ironically, the term "compressed private key" is a misnomer, because when a private key is exported as WIF-compressed it is actually one byte _longer_ than an "uncompressed" private key. That is because the private key has an added one-byte suffix (shown as 01 in hex in <<table_4-4>>), which signifies that the private key is from a newer wallet and should only be used to produce compressed public keys. Private keys are not themselves compressed and cannot be compressed. The term "compressed private key" really means "private key from which only compressed public keys should be derived," whereas "uncompressed private key" really means "private key from which only uncompressed public keys should be derived." You should only refer to the export format as "WIF-compressed" or "WIF" and not refer to the private key itself as "compressed" to avoid further confusion.
-
-<<table_4-4>> shows the same key, encoded in WIF and WIF-compressed formats.
-
-[[table_4-4]]
-.Example: Same key, different formats
-[options="header"]
-|=======
-|Format | Private key
-| Hex | 1E99423A4ED27608A15A2616A2B0E9E52CED330AC530EDCC32C8FFC6A526AEDD
-| WIF | 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
-| Hex-compressed | 1E99423A4ED27608A15A2616A2B0E9E52CED330AC530EDCC32C8FFC6A526AEDD01
-| WIF-compressed | KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ
-|=======
-
-Notice that the hex-compressed private key format has one extra byte at the end (01 in hex). While the Base58Check version prefix is the same (0x80) for both WIF and WIF-compressed formats, the addition of one byte on the end of the number causes the first character of the Base58 encoding to change from a 5 to either a _K_ or _L_. Think of this as the Base58 equivalent of the decimal encoding difference between the number 100 and the number 99. While 100 is one digit longer than 99, it also has a prefix of 1 instead of a prefix of 9. As the length changes, it affects the prefix. In Base58, the prefix 5 changes to a _K_ or _L_ as the length of the number increases by one byte.
-
-Remember, these formats are _not_ used interchangeably. In a newer wallet that implements compressed public keys, the private keys will only ever be exported as WIF-compressed (with a _K_ or _L_ prefix). If the wallet is an older implementation and does not use compressed public keys, the private keys will only ever be exported as WIF (with a 5 prefix). The goal here is to signal to the wallet importing these private keys whether it must search the blockchain for compressed or uncompressed public keys and addresses.
-
-If a bitcoin wallet is able to implement compressed public keys, it will use those in all transactions. The private keys in the wallet will be used to derive the public key points on the curve, which will be compressed. The compressed public keys will be used to produce Bitcoin addresses and those will be used in transactions. When exporting private keys from a new wallet that implements compressed public keys, the WIF is modified, with the addition of a one-byte suffix +01+ to the private key. The resulting Base58Check-encoded private key is called a "compressed WIF" and starts with the letter _K_ or _L_, instead of starting with "5" as is the case with WIF-encoded (noncompressed) keys from older wallets.
-
-
-[TIP]
-====
-"Compressed private keys" is a misnomer! They are not compressed; rather, WIF-compressed signifies that the keys should only be used to derive compressed public keys and their corresponding Bitcoin addresses. Ironically, a "WIF-compressed" encoded private key is one byte longer because it has the added +01+ suffix to distinguish it from an "uncompressed" one.((("", startref="KAaddress04")))
-====
-
-=== Implementing Keys and Addresses in Cpass:[++]
-
-Let's look at the complete process of creating a Bitcoin address, from a private key, to a public key (a point on the elliptic curve), to a double-hashed address, and finally, the Base58Check encoding. The C++ code in <<addr_example>> shows the complete step-by-step process, from private key to Base58Check-encoded Bitcoin address. The code example uses the libbitcoin library introduced in <<alt_libraries>> for some helper functions.
-
-[[addr_example]]
-.Creating a Base58Check-encoded Bitcoin address from a private key
-====
-[role="c_less_space"]
-[source, cpp]
-----
-include::code/addr.cpp[]
-----
-====
-
-The code uses a predefined private key to produce the same Bitcoin address every time it is run, as shown in <<addr_example_run>>.((("", startref="base5804")))((("", startref="Abase5804")))
-
-[[addr_example_run]]
-.Compiling and running the addr code
-====
-[source,bash]
-----
-# Compile the addr.cpp code
-$ g++ -o addr addr.cpp -std=c++11 $(pkg-config --cflags --libs libbitcoin)
-# Run the addr executable
-$ ./addr
-Public key: 0202a406624211f2abbdc68da3df929f938c3399dd79fac1b51b0e4ad1d26a47aa
-Address: 1PRTTaJesdNovgne6Ehcdu1fpEdX7913CK
-----
-====
-
-[TIP]
-====
-The code in <<addr_example_run>> produces a Bitcoin address (+1PRTT...+) from a _compressed_ public key (see <<comp_pub>>). If you used the uncompressed public key instead, it would produce a different Bitcoin address (+14K1y...+).
-====
-
-=== Implementing Keys and Addresses in Python
-
-((("keys and addresses", "implementing in Python", id="KApython04")))((("pybitcointools")))The most comprehensive bitcoin library in Python is https://github.com/vbuterin/pybitcointools[pybitcointools] by Vitalik Buterin. In <<key-to-address_script>>, we use the pybitcointools library (imported as "bitcoin") to generate and display keys and addresses in various formats.
-
-[[key-to-address_script]]
-.Key and address generation and formatting with the pybitcointools library
-====
-[source,python]
-----
-include::code/key-to-address-ecc-example.py[]
-----
-====
-
-<<key-to-address_script_run>> shows  the output from running this code.
-
-[[key-to-address_script_run]]
-.Running key-to-address-ecc-example.py
-====
-++++
-<pre data-type="programlisting">
-$ python key-to-address-ecc-example.py
-Private Key (hex) is:
- 3aba4162c7251c891207b747840551a71939b0de081f85c4e44cf7c13e41daa6
-Private Key (decimal) is:
- 26563230048437957592232553826663696440606756685920117476832299673293013768870
-Private Key (WIF) is:
- 5JG9hT3beGTJuUAmCQEmNaxAuMacCTfXuw1R3FCXig23RQHMr4K
-Private Key Compressed (hex) is:
- 3aba4162c7251c891207b747840551a71939b0de081f85c4e44cf7c13e41daa601
-Private Key (WIF-Compressed) is:
- KyBsPXxTuVD82av65KZkrGrWi5qLMah5SdNq6uftawDbgKa2wv6S
-Public Key (x,y) coordinates is:
- (41637322786646325214887832269588396900663353932545912953362782457239403430124L,
- 16388935128781238405526710466724741593761085120864331449066658622400339362166L)
-Public Key (hex) is:
- 045c0de3b9c8ab18dd04e3511243ec2952002dbfadc864b9628910169d9b9b00ec&#x21b5;
-243bcefdd4347074d44bd7356d6a53c495737dd96295e2a9374bf5f02ebfc176
-Compressed Public Key (hex) is:
- 025c0de3b9c8ab18dd04e3511243ec2952002dbfadc864b9628910169d9b9b00ec
-Bitcoin Address (b58check) is:
- 1thMirt546nngXqyPEz532S8fLwbozud8
-Compressed Bitcoin Address (b58check) is:
- 14cxpo3MBCYYWCgF74SWTdcmxipnGUsPw3
-</pre>
-++++
-====
-
-
-<<ec_math>> is another example, using the Python ECDSA library for the elliptic curve math and without using any specialized bitcoin libraries.
-
-[[ec_math]]
-.A script demonstrating elliptic curve math used for bitcoin keys
-====
-[source, python]
-----
-include::code/ec-math.py[]
-----
-====
-
-<<ec_math_run>> shows the output produced by running this script.
-
-[WARNING]
-====
-<<ec_math>> ((("random numbers", "os.urandom", see="entropy")))((("entropy", "os.urandom", see="random numbers")))((("random numbers", "random number generation")))((("entropy", "random number generation")))uses +os.urandom+, which reflects a cryptographically secure random number generator (CSRNG) provided by the underlying operating system. Caution: Depending on the OS, +os.urandom+ may _not_ be implemented with sufficient security or seeded properly and may _not_ be appropriate for generating production-quality bitcoin keys.((("", startref="KApython04")))
-====
-
-[[ec_math_run]]
-.Installing the Python ECDSA library and running the ec_math.py script
-====
-----
-# Install Python PIP package manager
-$ sudo apt-get install python-pip
-# Install the Python ECDSA library
-$ sudo pip install ecdsa
-# Run the script
-$ python ec-math.py
-Secret:  38090835015954358862481132628887443905906204995912378278060168703580660294000
-EC point: (70048853531867179489857750497606966272382583471322935454624595540007269312627, 105262206478686743191060800263479589329920209527285803935736021686045542353380)
-BTC public key: 029ade3effb0a67d5c8609850d797366af428f4a0d5194cb221d807770a1522873
-----
-====
-
-=== Advanced Keys and Addresses
-
-((("keys and addresses", "advanced forms", id="KAadvanced04")))In the following sections we will look at advanced forms of keys and addresses, such as encrypted private keys, script and multisignature addresses, vanity addresses, and paper wallets.
-
-[[p2sh_addresses]]
-==== Pay-to-Script Hash (P2SH) and Multisig Addresses
-
-((("keys and addresses", "advanced forms", "pay-to-script hash and multisig addresses")))((("Pay-to-Script-Hash (P2SH)", "multisig addresses and")))((("multisig addresses")))((("addresses", "multisig addresses")))As we know, traditional Bitcoin addresses begin with the number “1” and are derived from the public key, which is derived from the private key.  Although anyone can send bitcoin to a “1” address, that bitcoin can only be spent by presenting the corresponding private key signature and public key.
-
-((("bitcoin improvement proposals", "Pay to Script Hash (BIP-16)")))Bitcoin addresses that begin with the number “3” are pay-to-script hash (P2SH) addresses, sometimes erroneously called multisignature or multisig addresses. They designate the beneficiary of a bitcoin transaction as the hash of a script, instead of the owner of a public key. The feature was introduced in January 2012 with BIP-16 (see <<appdxbitcoinimpproposals>>), and is being widely adopted because it provides the opportunity to add functionality to the address itself. Unlike transactions that "send" funds to traditional “1” Bitcoin addresses, also known as a pay-to-public-key-hash (P2PKH), funds sent to “3” addresses require something more than the presentation of one public key and one private key signature as proof of ownership. The requirements are designated at the time the address is created, within the script, and all inputs to this address will be encumbered with the same requirements.
-
-A P2SH address is created from a transaction script, which defines who can spend a transaction output (for more details, see <<p2sh>>). Encoding a P2SH address involves using the same double-hash function as used during creation of a Bitcoin address, only applied on the script instead of the public key:
-
-----
-script hash = RIPEMD160(SHA256(script))
-----
-
-The resulting "script hash" is encoded with Base58Check with a version prefix of 5, which results in an encoded address starting with a +3+. An example of a P2SH address is +3F6i6kwkevjR7AsAd4te2YB2zZyASEm1HM+, which can be derived using the Bitcoin Explorer commands +script-encode+, +sha256+, +ripemd160+, and +base58check-encode+ (see <<appdx_bx>>) as follows:
-
-----
-$ echo \
-'DUP HASH160 [89abcdefabbaabbaabbaabbaabbaabbaabbaabba] EQUALVERIFY CHECKSIG' > script
-$ bx script-encode < script | bx sha256 | bx ripemd160 \
-| bx base58check-encode --version 5
-3F6i6kwkevjR7AsAd4te2YB2zZyASEm1HM
-----
-
-[TIP]
-====
-P2SH is not necessarily the same as a multisignature standard transaction. A P2SH address _most often_ represents a multi-signature script, but it might also represent a script encoding other types of transactions.
-====
-
-===== Multisignature addresses and P2SH
-
-Currently, the most common implementation of the P2SH function is the multi-signature address script. As the name implies, the underlying script requires a minimum number of signatures to prove ownership and therefore spend funds. The bitcoin multi-signature feature is designed to require M signatures (also known as the “threshold”) from a total of N keys, known as an M-of-N multisig, where M is equal to or less than N. For example, Bob the coffee shop owner from <<ch01_intro_what_is_bitcoin>> could use a multisignature address requiring 1-of-2 signatures from a key belonging to him and a key belonging to his spouse, ensuring either of them could sign to spend a transaction output locked to this address. This would be similar to a “joint account” as implemented in traditional banking where either spouse can spend with a single signature. Or Gopesh,((("use cases", "offshore contract services"))) the web designer paid by Bob to create a website, might have a 2-of-3 multisignature address for his business that ensures that no funds can be spent unless at least two of the business partners sign a transaction.
-
-We will explore how to create transactions that spend funds from P2SH (and multi-signature) addresses in <<transactions>>.
-
-==== Vanity Addresses
-
-((("keys and addresses", "advanced forms", "vanity addresses")))((("vanity addresses", id="vanity04")))((("addresses", "vanity addresses", id="Avanity04")))Vanity addresses are valid Bitcoin addresses that contain human-readable messages. For example, +1LoveBPzzD72PUXLzCkYAtGFYmK5vYNR33+ is a valid address that contains the letters forming the word "Love" as the first four Base58 letters. Vanity addresses require generating and testing billions of candidate private keys, until a Bitcoin address with the desired pattern is found. Although there are some optimizations in the vanity generation algorithm, the process essentially involves picking a private key at random, deriving the public key, deriving the Bitcoin address, and checking to see if it matches the desired vanity pattern, repeating billions of times until a match is found.
-
-Once a vanity address matching the desired pattern is found, the private key from which it was derived can be used by the owner to spend bitcoin in exactly the same way as any other address. Vanity addresses are no less or more secure than any other address. They depend on the same Elliptic Curve Cryptography (ECC) and SHA as any other address. You can no more easily find the private key of an address starting with a vanity pattern than you can of any other address.
-
-In <<ch01_intro_what_is_bitcoin>>, we introduced Eugenia, a children's charity director operating in the Philippines. Let's say that Eugenia is organizing a bitcoin fundraising drive and wants to use a vanity Bitcoin address to publicize the fundraising. Eugenia will create a vanity address that starts with "1Kids" to promote the children's charity fundraiser. Let's see how this vanity address will be created and what it means for the security of Eugenia's charity.((("use cases", "charitable donations", startref="eugeniafour")))
-
-===== Generating vanity addresses
-
-It's important to realize that a Bitcoin address is simply a number represented by symbols in the Base58 alphabet. The search for a pattern like "1Kids" can be seen as searching for an address in the range from +1Kids11111111111111111111111111111+ to +1Kidszzzzzzzzzzzzzzzzzzzzzzzzzzzzz+. There are approximately 58^29^ (approximately 1.4 * 10^51^) addresses in that range, all starting with "1Kids." <<table_4-11>> shows the range of addresses that have the prefix 1Kids.
-
-[[table_4-11]]
-.The range of vanity addresses starting with "1Kids"
-|=======
-| *From* | +1Kids11111111111111111111111111111+
-| | +1Kids11111111111111111111111111112+
-| | +1Kids11111111111111111111111111113+
-| | +...+
-| *To* | +1Kidszzzzzzzzzzzzzzzzzzzzzzzzzzzzz+
-|=======
-
-Let's look at the pattern "1Kids" as a number and see how frequently we might find this pattern in a Bitcoin address (see <<table_4-12>>). An average desktop computer PC, without any specialized hardware, can search approximately 100,000 keys per second.
-
-[[table_4-12]]
-.The frequency of a vanity pattern (1KidsCharity) and average search time on a desktop PC
-[options="header"]
-|=======
-| Length | Pattern | Frequency | Average search time
-| 1 | 1K | 1 in 58 keys | < 1 milliseconds
-| 2 | 1Ki| 1 in 3,364 | 50 milliseconds
-| 3 | 1Kid | 1 in 195,000 | < 2 seconds
-| 4 | 1Kids | 1 in 11 million | 1 minute
-| 5 | 1KidsC | 1 in 656 million | 1 hour
-| 6 | 1KidsCh | 1 in 38 billion | 2 days
-| 7 | 1KidsCha | 1 in 2.2 trillion | 3–4 months
-| 8 | 1KidsChar | 1 in 128 trillion | 13–18 years
-| 9 | 1KidsChari | 1 in 7 quadrillion | 800 years
-| 10 | 1KidsCharit | 1 in 400 quadrillion | 46,000 years
-| 11 | 1KidsCharity | 1 in 23 quintillion | 2.5 million years
-|=======
-
-
-As you can see, Eugenia won't be creating the vanity address "1KidsCharity" anytime soon, even if she had access to several thousand computers. Each additional character increases the difficulty by a factor of 58. Patterns with more than seven characters are usually found by specialized hardware, such as custom-built desktops with multiple GPUs. These are often repurposed bitcoin mining "rigs" that are no longer profitable for bitcoin mining but can be used to find vanity addresses. Vanity searches on GPU systems are many orders of magnitude faster than on a general-purpose CPU.
-
-Another way to find a vanity address is to outsource the work to a pool of vanity miners, such as the pool at https://vanitypool.appspot.com[Vanity Pool]. A pool of this type is a service that allows those with GPU hardware to earn bitcoin searching for vanity addresses for others. For a small payment (0.01 bitcoin or approximately $5 at the time of this writing), Eugenia can outsource the search for a seven-character pattern vanity address and get results in a few hours instead of having to run a CPU search for months.
-
-Generating a vanity address is a brute-force exercise: try a random key, check the resulting address to see if it matches the desired pattern, repeat until successful. <<vanity_miner_code>> shows an example of a "vanity miner," a program designed to find vanity addresses, written in C++. The example uses the libbitcoin library, which we introduced in <<alt_libraries>>.
-
-[[vanity_miner_code]]
-.Vanity address miner
-====
-[source,cpp]
-----
-include::code/vanity-miner.cpp[]
-----
-====
-
-[NOTE]
-====
-<<vanity_miner_code>> uses +std::random_device+. Depending on the implementation it may reflect a CSRNG provided by the underlying operating system. In the case of a Unix-like operating system such as Linux, it draws from +/dev/urandom+. The random number generator used here is for demonstration purposes, and it is _not_ appropriate for generating production-quality bitcoin keys as it is not implemented with sufficient security.
-====
-
-The example code must be compiled using a pass:[C++] compiler and linked against the libbitcoin library (which must be first installed on that system). To run the example, run the ++vanity-miner++ executable with no parameters (see <<vanity_miner_run>>) and it will attempt to find a vanity address starting with "1kid."
-
-[[vanity_miner_run]]
-.Compiling and running the vanity-miner example
-====
-[source,bash]
-----
-# Compile the code with g++
-$ g++ -o vanity-miner vanity-miner.cpp $(pkg-config --cflags --libs libbitcoin)
-# Run the example
-$ ./vanity-miner
-Found vanity address! 1KiDzkG4MxmovZryZRj8tK81oQRhbZ46YT
-Secret: 57cc268a05f83a23ac9d930bc8565bac4e277055f4794cbd1a39e5e71c038f3f
-# Run it again for a different result
-$ ./vanity-miner
-Found vanity address! 1Kidxr3wsmMzzouwXibKfwTYs5Pau8TUFn
-Secret: 7f65bbbbe6d8caae74a0c6a0d2d7b5c6663d71b60337299a1a2cf34c04b2a623
-# Use "time" to see how long it takes to find a result
-$ time ./vanity-miner
-Found vanity address! 1KidPWhKgGRQWD5PP5TAnGfDyfWp5yceXM
-Secret: 2a802e7a53d8aa237cd059377b616d2bfcfa4b0140bc85fa008f2d3d4b225349
-
-real	0m8.868s
-user	0m8.828s
-sys	0m0.035s
-----
-====
-
-The example code will take a few seconds to find a match for the three-character pattern "kid," as we can see when we use the +time+ Unix command to measure the execution time. Change the +search+ pattern in the source code and see how much longer it takes for four- or five-character patterns!
-
-===== Vanity address security
-
-((("security", "vanity addresses")))Vanity addresses can be used to enhance _and_ to defeat security measures; they are truly a double-edged sword. Used to improve security, a distinctive address makes it harder for adversaries to substitute their own address and fool your customers into paying them instead of you. Unfortunately, vanity addresses also make it possible for anyone to create an address that _resembles_ any random address, or even another vanity address, thereby fooling your customers.
-
-Eugenia could advertise a randomly generated address (e.g., +1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy+) to which people can send their donations. Or, she could generate a vanity address that starts with 1Kids, to make it more distinctive.
-
-In both cases, one of the risks of using a single fixed address (rather than a separate dynamic address per donor) is that a thief might be able to infiltrate your website and replace it with his own address, thereby diverting donations to himself. If you have advertised your donation address in a number of different places, your users may visually inspect the address before making a payment to ensure it is the same one they saw on your website, on your email, and on your flyer. In the case of a random address like +1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy+, the average user will perhaps inspect the first few characters "1J7mdg" and be satisfied that the address matches. Using a vanity address generator, someone with the intent to steal by substituting a similar-looking address can quickly generate addresses that match the first few characters, as shown in <<table_4-13>>.
-
-[[table_4-13]]
-.Generating vanity addresses to match a random address
-|=======
-| *Original Random Address* | 1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy
-| *Vanity (4-character match)* | 1J7md1QqU4LpctBetHS2ZoyLV5d6dShhEy
-| *Vanity (5-character match)* | 1J7mdgYqyNd4ya3UEcq31Q7sqRMXw2XZ6n
-| *Vanity (6-character match)* | 1J7mdg5WxGENmwyJP9xuGhG5KRzu99BBCX
-|=======
-
-So does a vanity address increase security? If Eugenia generates the vanity address +1Kids33q44erFfpeXrmDSz7zEqG2FesZEN+, users are likely to look at the vanity pattern word _and a few characters beyond_, for example noticing the "1Kids33" part of the address. That would force an attacker to generate a vanity address matching at least six characters (two more), expending an effort that is 3,364 times (58 &#x00D7; 58) higher than the effort Eugenia expended for her 4-character vanity. Essentially, the effort Eugenia expends (or pays a vanity pool for) "pushes" the attacker into having to produce a longer pattern vanity. If Eugenia pays a pool to generate an 8-character vanity address, the attacker would be pushed into the realm of 10 characters, which is infeasible on a personal computer and expensive even with a custom vanity-mining rig or vanity pool. What is affordable for Eugenia becomes unaffordable for the attacker, especially if the potential reward of fraud is not high enough to cover the cost of the vanity address generation.((("", startref="Avanity04")))((("", startref="vanity04")))((("", startref="eugeniafour")))
-
-[[paper_wallets]]
-==== Paper Wallets
-
-((("keys and addresses", "advanced forms", "paper wallets")))((("paper wallets", id="paperw04")))((("wallets", "types of", "paper wallets", id="Wpaper04")))Paper wallets are bitcoin private keys printed on paper. Often the paper wallet also includes the corresponding Bitcoin address for convenience, but this is not necessary because it can be derived from the private key.
-
-[WARNING]
-====
-Paper wallets are an OBSOLETE technology and are dangerous for most users. There are many subtle pitfalls involved in generating them, not least of which the possibility that the generating code is compromised with a "back door". Hundreds of bitcoin have been stolen this way. Paper wallets are shown here for informational purposes only and should not be used for storing bitcoin. Use a BIP-39 mnemonic phrase to backup your keys. Use a hardware wallet to store keys and sign transactions. DO NOT USE PAPER WALLETS.
-====
-
-Paper wallets come in many shapes, sizes, and designs, but at a very basic level are just a key and an address printed on paper. <<table_4-14>> shows the simplest form of a paper wallet.
-
-[[table_4-14]]
-.Simplest form of a paper wallet—a printout of the Bitcoin address and private key
-[options="header"]
-|=======================
-|Public address|Private key (WIF)
-|1424C2F4bC9JidNjjTUZCbUxv6Sa1Mt62x|5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
-|=======================
-
-Paper wallets come in many designs and sizes, with many different features. <<paper_wallet_simple>> shows a sample paper wallet.
-
-[[paper_wallet_simple]]
-.An example of a simple paper wallet
-image::images/mbc2_0408.png[]
-
-
-Some are intended to be given as gifts and have seasonal themes, such as Christmas and New Year's themes. Others are designed for storage in a bank vault or safe with the private key hidden in some way, either with opaque scratch-off stickers, or folded and sealed with tamper-proof adhesive foil.
-
-Other designs feature additional copies of the key and address, in the form of detachable stubs similar to ticket stubs, allowing you to store multiple copies to protect against fire, flood, or other natural disasters.((("", startref="KAadvanced04")))((("", startref="Wpaper04")))((("", startref="paperw04")))
-
-[[paper_wallet_spw]]
-.An example of a paper wallet with additional copies of the keys on a backup "stub"
-image::images/mbc2_0412.png[]

ch05.asciidoc

@@ -1,500 +0,0 @@
-[[ch05_wallets]]
-== Wallets
-
-((("wallets", "defined")))The word "wallet" is used to describe a few different things in bitcoin.
-
-At a high level, a wallet is an application that serves as the primary user interface. The wallet controls access to a user's money, managing keys and addresses, tracking the balance, and creating and signing transactions.
-
-More narrowly, from a programmer's perspective, the word "wallet" refers to the data structure used to store and manage a user's keys.
-
-In this chapter we will look at the second meaning, where wallets are containers for private keys, usually implemented as structured files or simple databases.
-
-=== Wallet Technology Overview
-
-In this section we summarize the various technologies used to construct user-friendly, secure, and flexible bitcoin wallets.
-
-((("wallets", "contents of")))A common misconception about bitcoin is that bitcoin wallets contain bitcoin. In fact, the wallet contains only keys. The "coins" are recorded in the blockchain on the Bitcoin network. Users control the coins on the network by signing transactions with the keys in their wallets. ((("keychains")))In a sense, a bitcoin wallet is a _keychain_.
-
-[TIP]
-====
-Bitcoin wallets contain keys, not coins. Each user has a wallet containing keys. Wallets are really keychains containing pairs of private/public keys (see <<private_public_keys>>). Users sign transactions with the keys, thereby proving they own the transaction outputs (their coins). The coins are stored on the blockchain in the form of transaction outputs (often noted as vout or txout).
-====
-
-((("wallets", "types of", "primary distinctions")))There are two primary types of wallets, distinguished by whether the keys they contain are related to each other or not.
-
-((("JBOK wallets", seealso="wallets")))((("wallets", "types of", "JBOK wallets")))((("nondeterministic wallets", seealso="wallets")))The first type is a _nondeterministic wallet_, where each key is independently generated from a random number. The keys are not related to each other. This type of wallet is also known as a JBOK wallet from the phrase "Just a Bunch Of Keys."
-
-((("deterministic wallets", seealso="wallets")))The second type of wallet is a _deterministic wallet_, where all the keys are derived from a single master key, known as the _seed_. All the keys in this type of wallet are related to each other and can be generated again if one has the original seed. ((("key derivation methods")))There are a number of different _key derivation_ methods used in deterministic wallets. ((("hierarchical deterministic (HD) wallets", seealso="wallets")))The most commonly used derivation method uses a tree-like structure and is known as a _hierarchical deterministic_ or _HD_ wallet.
-
-((("mnemonic code words")))Deterministic wallets are initialized from a random sequence (entropy). To make these easier to use, random sequences are encoded as English words, also known as _mnemonic code words_.
-
-The next few sections introduce each of these technologies at a high level.
-
-[[random_wallet]]
-==== Nondeterministic (Random) Wallets
-
-((("wallets", "types of", "nondeterministic (random) wallets")))In the first bitcoin wallet (now called Bitcoin Core), wallets were collections of randomly generated private keys. For example, the original Bitcoin Core client pregenerates 100 random private keys when first started and generates more keys as needed, using each key only once.  Such wallets are being replaced with deterministic wallets because they are cumbersome to manage, back up, and import. The disadvantage of random keys is that if you generate many of them you must keep copies of all of them, meaning that the wallet must be backed up frequently. Each key must be backed up, or the funds it controls are irrevocably lost if the wallet becomes inaccessible. This conflicts directly with the principle of avoiding address reuse, by using each Bitcoin address for only one transaction. Address reuse reduces privacy by associating multiple transactions and addresses with each other. A Type-0 nondeterministic wallet is a poor choice of wallet, especially if you want to avoid address reuse because it means managing many keys, which creates the need for frequent backups. Although the Bitcoin Core client includes a Type-0 wallet, using this wallet is discouraged by developers of Bitcoin Core. <<Type0_wallet>> shows a nondeterministic wallet, containing a loose collection of random keys.
-
-[TIP]
-====
-The use of nondeterministic wallets is discouraged for anything other than simple tests. They are simply too cumbersome to back up and use. Instead, use an industry-standard&#x2013;based _HD wallet_ with a _mnemonic_ random sequence (entropy, or "initial seed") for backup.
-====
-
-[[Type0_wallet]]
-[role="smallersixty"]
-.Type-0 nondeterministic (random) wallet: a collection of randomly generated keys
-image::images/mbc2_0501.png["Non-Deterministic Wallet"]
-
-==== Deterministic (Seeded) Wallets
-
-((("wallets", "types of", "deterministic (seeded) wallets")))Deterministic, or "seeded," wallets are wallets that contain private keys that are all derived from a common seed, through the use of a one-way hash function. The seed is a randomly generated number that is combined with other data, such as an index number or "chain code" (see <<hd_wallets>>) to derive the private keys. In a deterministic wallet, the seed is sufficient to recover all the derived keys, and therefore a single backup at creation time is sufficient. The seed is also sufficient for a wallet export or import, allowing for easy migration of all the user's keys between different wallet implementations. <<Type1_wallet>> shows a logical diagram of a deterministic wallet.
-
-[[Type1_wallet]]
-[role="smallersixty"]
-.Type-1 deterministic (seeded) wallet: a deterministic sequence of keys derived from a seed
-image::images/mbc2_0502.png["Deterministic Wallet"]
-
-[[hd_wallets]]
-==== HD Wallets (BIP-32/BIP-44)
-
-((("wallets", "types of", "hierarchical deterministic (HD) wallets")))((("hierarchical deterministic (HD) wallets")))((("bitcoin improvement proposals", "Hierarchical Deterministic Wallets (BIP-32/BIP-44)")))Deterministic wallets were developed to make it easy to derive many keys from a single "seed". The most advanced form of deterministic wallets is the HD wallet defined by the BIP-32 standard. HD wallets contain keys derived in a tree structure, such that a parent key can derive a sequence of children keys, each of which can derive a sequence of grandchildren keys, and so on, to an infinite depth. This tree structure is illustrated in <<Type2_wallet>>.
-
-[[Type2_wallet]]
-.Type-2 HD wallet: a tree of keys generated from a single seed
-image::images/mbc2_0503.png["HD wallet"]
-
-HD wallets offer two major advantages over random (nondeterministic) keys. First, the tree structure can be used to express additional organizational meaning, such as when a specific branch of subkeys is used to receive incoming payments and a different branch is used to receive change from outgoing payments. Branches of keys can also be used in corporate settings, allocating different branches to departments, subsidiaries, specific functions, or accounting categories.
-
-The second advantage of HD wallets is that users can create a sequence of public keys without having access to the corresponding private keys. This allows HD wallets to be used on an insecure server or in a receive-only capacity, issuing a different public key for each transaction. The public keys do not need to be preloaded or derived in advance, yet the server doesn't have the private keys that can spend the funds.
-
-==== Seeds and Mnemonic Codes (BIP-39)
-
-((("wallets", "technology of", "seeds and mnemonic codes")))((("mnemonic code words")))((("bitcoin improvement proposals", "Mnemonic Code Words (BIP-39)")))HD wallets are a very powerful mechanism for managing many keys and addresses. They are even more useful if they are combined with a standardized way of creating seeds from a sequence of English words that are easy to transcribe, export, and import across wallets. This is known as a _mnemonic_ and the standard is defined by BIP-39. Today, most bitcoin wallets (as well as wallets for other cryptocurrencies) use this standard and can import and export seeds for backup and recovery using interoperable mnemonics.
-
-Let's look at this from a practical perspective. Which of the following seeds is easier to transcribe, record on paper, read without error, export, and import into another wallet?
-
-.A seed for a deterministic wallet, in hex
-----
-0C1E24E5917779D297E14D45F14E1A1A
-----
-
-.A seed for a deterministic wallet, from a 12-word mnemonic
-----
-army van defense carry jealous true
-garbage claim echo media make crunch
-----
-
-==== Wallet Best Practices
-
-((("wallets", "best practices for")))((("bitcoin improvement proposals", "Multipurpose HD Wallet Structure (BIP-43)")))As bitcoin wallet technology has matured, certain common industry standards have emerged that make bitcoin wallets broadly interoperable, easy to use, secure, and flexible. These common standards are:
-
-* Mnemonic code words, based on BIP-39
-* HD wallets, based on BIP-32
-* Multipurpose HD wallet structure, based on BIP-43
-* Multicurrency and multiaccount wallets, based on BIP-44
-
-These standards may change or may become obsolete by future developments, but for now they form a set of interlocking technologies that have become the de facto wallet standard for bitcoin.
-
-The standards have been adopted by a broad range of software and hardware bitcoin wallets, making all these wallets interoperable. A user can export a mnemonic generated on one of these wallets and import it in another wallet, recovering all transactions, keys, and addresses.
-
-((("hardware wallets")))((("hardware wallets", see="also wallets")))Some example of software wallets supporting these standards include (listed alphabetically) Bluewallet, Breadwallet, Copay, and Multibit HD. Examples of hardware wallets supporting these standards include (listed alphabetically) KeepKey, Ledger, and Trezor.
-
-The following sections examine each of these technologies in detail.
-
-[TIP]
-====
-If you are implementing a bitcoin wallet, it should be built as a HD wallet, with a seed derived from, and encoded as, a mnemonic code for backup, following the BIP-32, BIP-39, BIP-43, and BIP-44 standards, as described in the following sections.
-====
-
-==== Using a Bitcoin Wallet
-
-((("wallets", "using bitcoin wallets")))In <<user-stories>> we introduced Gabriel, ((("use cases", "web store", id="gabrielfive")))an enterprising young teenager in Rio de Janeiro, who is running a simple web store that sells bitcoin-branded t-shirts, coffee mugs, and stickers.
-
-((("wallets", "types of", "hardware wallets")))Gabriel uses a Trezor bitcoin hardware wallet (<<a_trezor_device>>) to securely manage his bitcoin. The Trezor is a simple USB device with two buttons that stores keys (in the form of an HD wallet) and signs transactions. Trezor wallets implement all the industry standards discussed in this chapter, so Gabriel is not reliant on any proprietary technology or single vendor solution.
-
-[[a_trezor_device]]
-.A Trezor device: a bitcoin HD wallet in hardware
-image::images/mbc2_0504.png[alt]
-
-When Gabriel used the Trezor for the first time, the device generated a random sequence (entropy), the associated mnemonic and derived a seed from a built-in hardware random number generator. During this initialization phase, the wallet displayed a numbered sequence of words, one by one, on the screen (see <<trezor_mnemonic_display>>).
-
-[[trezor_mnemonic_display]]
-.Trezor displaying one of the mnemonic words
-image::images/mbc2_0505.png["Trezor wallet display of mnemonic word"]
-
-By writing down this mnemonic, Gabriel created a backup (see <<mnemonic_paper_backup>>) that can be used for recovery in the case of loss or damage to the Trezor device. This mnemonic can be used for recovery in a new Trezor or in any one of the many compatible software or hardware wallets. Note that the sequence of words is important, so mnemonic paper backups have numbered spaces for each word. Gabriel had to carefully record each word in the numbered space to preserve the correct sequence.
-
-[[mnemonic_paper_backup]]
-.Gabriel's paper backup of the mnemonic
-[cols="<1,^50,<1,^50", width="80%"]
-|===
-|*1.*| _army_ |*7.*| _garbage_
-|*2.*| _van_ |*8.*| _claim_
-|*3.*| _defense_ |*9.*| _echo_
-|*4.*| _carry_ |*10.*| _media_
-|*5.*| _jealous_ |*11.*| _make_
-|*6.*| _true_ |*12.*| _crunch_
-|===
-
-[NOTE]
-====
-A 12-word mnemonic is shown in <<mnemonic_paper_backup>>, for simplicity. In fact, most hardware wallets generate a more secure 24-word mnemonic. The mnemonic is used in exactly the same way, regardless of length.
-====
-
-For the first implementation of his web store, Gabriel uses a single Bitcoin address, generated on his Trezor device. This single address is used by all customers for all orders. As we will see, this approach has some drawbacks and can be improved upon with an HD wallet.((("", startref="gabrielfive")))
-
-=== Wallet Technology Details
-
-Let's now examine each of the important industry standards that are used by many bitcoin wallets in detail.
-
-[[mnemonic_code_words]]
-==== Mnemonic Code Words (BIP-39)
-
-((("wallets", "technology of", "mnemonic code words")))((("mnemonic code words", id="mnemonic05")))((("bitcoin improvement proposals", "Mnemonic Code Words (BIP-39)", id="BIP3905")))Mnemonic code words are word sequences that represent (encode) a random number used as a seed to derive a deterministic wallet. The sequence of words is sufficient to re-create the seed and from there re-create the wallet and all the derived keys. A wallet application that implements deterministic wallets with mnemonic words will show the user a sequence of 12 to 24 words when first creating a wallet. That sequence of words is the wallet backup and can be used to recover and re-create all the keys in the same or any compatible wallet application. Mnemonic words make it easier for users to back up wallets because they are easy to read and correctly transcribe, as compared to a random sequence of numbers.
-
-[TIP]
-====
-((("brainwallets")))Mnemonic words are often confused with "brainwallets." They are not the same. The primary difference is that a brainwallet consists of words chosen by the user, whereas mnemonic words are created randomly by the wallet and presented to the user. This important difference makes mnemonic words much more secure, because humans are very poor sources of randomness.
-====
-
-Mnemonic codes are defined in BIP-39 (see <<appdxbitcoinimpproposals>>). Note that BIP-39 is one implementation of a mnemonic code standard. ((("Electrum wallet", seealso="wallets")))There is a different standard, with a different set of words, used by the Electrum wallet and predating BIP-39. BIP-39 was proposed by the company behind the Trezor hardware wallet and is incompatible with Electrum's implementation. However, BIP-39 has now achieved broad industry support across dozens of interoperable implementations and should be considered the de facto industry standard.
-
-BIP-39 defines the creation of a mnemonic code and seed, which we describe here in nine steps. For clarity, the process is split into two parts: steps 1 through 6 are shown in <<generating_mnemonic_words>> and steps 7 through 9 are shown in <<mnemonic_to_seed>>.
-
-[[generating_mnemonic_words]]
-===== Generating mnemonic words
-
-Mnemonic words are generated automatically by the wallet using the standardized process defined in BIP-39. The wallet starts from a source of entropy, adds a checksum, and then maps the entropy to a word list:
-
-1. Create a random sequence (entropy) of 128 to 256 bits.
-2. Create a checksum of the random sequence by taking the first (entropy-length/32) bits of its SHA256 hash.
-3. Add the checksum to the end of the random sequence.
-4. Split the result into 11-bit length segments.
-5. Map each 11-bit value to a word from the predefined dictionary of 2048 words.
-6. The mnemonic code is the sequence of words.
-
-<<generating_entropy_and_encoding>> shows how entropy is used to generate mnemonic words.
-
-[[generating_entropy_and_encoding]]
-[role="smallerseventy"]
-.Generating entropy and encoding as mnemonic words
-image::images/mbc2_0506.png["Generating entropy and encoding as mnemonic words"]
-
-<<table_4-5>> shows the relationship between the size of the entropy data and the length of mnemonic codes in words.
-
-[[table_4-5]]
-.Mnemonic codes: entropy and word length
-[options="header"]
-|=======
-|Entropy (bits) | Checksum (bits) | Entropy *+* checksum (bits) | Mnemonic length (words)
-| 128 | 4 | 132 | 12
-| 160 | 5 | 165 | 15
-| 192 | 6 | 198 | 18
-| 224 | 7 | 231 | 21
-| 256 | 8 | 264 | 24
-|=======
-
-[[mnemonic_to_seed]]
-===== From mnemonic to seed
-
-((("key-stretching function")))((("PBKDF2 function")))The mnemonic words represent entropy with a length of 128 to 256 bits. The entropy is then used to derive a longer (512-bit) seed through the use of the key-stretching function PBKDF2. The seed produced is then used to build a deterministic wallet and derive its keys.
-
-((("salts")))((("passphrases")))The key-stretching function takes two parameters: the mnemonic and a _salt_. The purpose of a salt in a key-stretching function is to make it difficult to build a lookup table enabling a brute-force attack. In the BIP-39 standard, the salt has another purpose&#x2014;it allows the introduction of a passphrase that serves as an additional security factor protecting the seed, as we will describe in more detail in <<mnemonic_passphrase>>.
-
-The process described in steps 7 through 9 continues from the process described previously in <<generating_mnemonic_words>>:
-
-++++
-<ol start="7">
-	<li>The first parameter to the PBKDF2 key-stretching function is the <em>mnemonic</em> produced from step 6.</li>
-	<li>The second parameter to the PBKDF2 key-stretching function is a <em>salt</em>. The salt is composed of the string constant "<code>mnemonic</code>" concatenated with an optional user-supplied passphrase string.</li>
-	<li>PBKDF2 stretches the mnemonic and salt parameters using 2048 rounds of hashing with the HMAC-SHA512 algorithm, producing a 512-bit value as its final output. That 512-bit value is the seed.</li>
-</ol>
-++++
-
-<<fig_5_7>> shows how a mnemonic is used to generate a seed.
-
-[[fig_5_7]]
-.From mnemonic to seed
-image::images/mbc2_0507.png["From mnemonic to seed"]
-
-[TIP]
-====
-The key-stretching function, with its 2048 rounds of hashing, is a very effective protection against brute-force attacks against the mnemonic or the passphrase. It makes it extremely costly (in computation) to try more than a few thousand passphrase and mnemonic combinations, while the number of possible derived seeds is vast (2^512^).
-====
-
-Tables pass:[<a data-type="xref" href="#mnemonic_128_no_pass" data-xrefstyle="select: labelnumber">#mnemonic_128_no_pass</a>], pass:[<a data-type="xref" href="#mnemonic_128_w_pass" data-xrefstyle="select: labelnumber">#mnemonic_128_w_pass</a>], and pass:[<a data-type="xref" href="#mnemonic_256_no_pass" data-xrefstyle="select: labelnumber">#mnemonic_256_no_pass</a>] show some examples of mnemonic codes and the seeds they produce (either with or without a passphrase).
-
-[[mnemonic_128_no_pass]]
-.128-bit entropy mnemonic code, no passphrase, resulting seed
-[cols="h,"]
-|=======
-| *Entropy input (128 bits)*| +0c1e24e5917779d297e14d45f14e1a1a+
-| *Mnemonic (12 words)* | +army van defense carry jealous true garbage claim echo media make crunch+
-| *Passphrase*| (none)
-| *Seed  (512 bits)* | +5b56c417303faa3fcba7e57400e120a0ca83ec5a4fc9ffba757fbe63fbd77a89a1a3be4c67196f57c39+
-+a88b76373733891bfaba16ed27a813ceed498804c0570+
-|=======
-
-[[mnemonic_128_w_pass]]
-.128-bit entropy mnemonic code, with passphrase, resulting seed
-[cols="h,"]
-|=======
-| *Entropy input (128 bits)*| +0c1e24e5917779d297e14d45f14e1a1a+
-| *Mnemonic (12 words)* | +army van defense carry jealous true garbage claim echo media make crunch+
-| *Passphrase*| SuperDuperSecret
-| *Seed  (512 bits)* | +3b5df16df2157104cfdd22830162a5e170c0161653e3afe6c88defeefb0818c793dbb28ab3ab091897d0+
-+715861dc8a18358f80b79d49acf64142ae57037d1d54+
-|=======
-
-
-[[mnemonic_256_no_pass]]
-.256-bit entropy mnemonic code, no passphrase, resulting seed
-[cols="h,"]
-|=======
-| *Entropy input (256 bits)* | +2041546864449caff939d32d574753fe684d3c947c3346713dd8423e74abcf8c+
-| *Mnemonic (24 words)* | +cake apple borrow silk endorse fitness top denial coil riot stay wolf
-luggage oxygen faint major edit measure invite love trap field dilemma oblige+
-| *Passphrase*| (none)
-| *Seed (512 bits)* | +3269bce2674acbd188d4f120072b13b088a0ecf87c6e4cae41657a0bb78f5315b33b3a04356e53d062e5+
-+5f1e0deaa082df8d487381379df848a6ad7e98798404+
-|=======
-
-[TIP]
-====
-Many wallets do not allow for the creation of wallets with more than a 12 word mnemonic phrase. You will notice from the tables above that despite the unique lengths of entropy input, the seed size remains the same (512 bits). From a security perspective, the amount of entropy actually used for the production of HD wallets is roughly 128 bits, which equals 12 words.  Providing more than 12 words produces additional entropy which is unnecessary, and this _unused_ entropy is not used for the derivation of the seed in the way that one might initially suspect. From a usability perspective, 12 words is also easier to write down, back up, and store.
-====
-
-[[mnemonic_passphrase]]
-===== Optional passphrase in BIP-39
-
-((("passphrases")))The BIP-39 standard allows the use of an optional passphrase in the derivation of the seed. If no passphrase is used, the mnemonic is stretched with a salt consisting of the constant string +"mnemonic"+, producing a specific 512-bit seed from any given mnemonic. If a passphrase is used, the stretching function produces a _different_ seed from that same mnemonic. In fact, given a single mnemonic, every possible passphrase leads to a different seed. Essentially, there is no "wrong" passphrase. All passphrases are valid and they all lead to different seeds, forming a vast set of possible uninitialized wallets. The set of possible wallets is so large (2^512^) that there is no practical possibility of brute-forcing or accidentally guessing one that is in use.
-
-[TIP]
-====
-There are no "wrong" passphrases in BIP-39. Every passphrase leads to some wallet, which unless previously used will be empty.
-====
-
-The optional passphrase creates two important features:
-
-* A second factor (something memorized) that makes a mnemonic useless on its own, protecting mnemonic backups from compromise by a thief.
-
-* A form of plausible deniability or "duress wallet," where a chosen passphrase leads to a wallet with a small amount of funds used to distract an attacker from the "real" wallet that contains the majority of funds.
-
-However, it is important to note that the use of a passphrase also introduces the risk of loss:
-
-* If the wallet owner is incapacitated or dead and no one else knows the passphrase, the mnemonic code is useless and all the funds stored in the wallet are lost forever.
-
-* Conversely, if the owner backs up the passphrase in the same place as the mnemonic code, it defeats the purpose of a second factor.
-
-While passphrases are very useful, they should only be used in combination with a carefully planned process for backup and recovery, considering the possibility of surviving the owner and allowing his or her family to recover the cryptocurrency estate.
-
-===== Working with mnemonic codes
-
-BIP-39 is implemented as a library in many different programming languages:
-
-https://github.com/trezor/python-mnemonic[python-mnemonic]:: The reference implementation of the standard by the SatoshiLabs team that proposed BIP-39, in Python
-
-https://github.com/bitcoinjs/bip39[bitcoinjs/bip39]:: An implementation of BIP-39, as part of the popular bitcoinJS framework, in JavaScript
-
-https://github.com/libbitcoin/libbitcoin/blob/master/src/wallet/mnemonic.cpp[libbitcoin/mnemonic]:: An implementation of BIP-39, as part of the popular Libbitcoin framework, in pass:[<span class="keep-together">C++</span>]
-
-==== Creating an HD Wallet from the Seed
-
-((("wallets", "technology of", "creating HD wallets from root seed")))((("root seeds")))((("hierarchical deterministic (HD) wallets")))HD wallets are created from a single _root seed_, which is a 128-, 256-, or 512-bit random number. Most commonly, this seed is generated from a _mnemonic_ as detailed in the previous section.
-
-Every key in the HD wallet is deterministically derived from this root seed, which makes it possible to re-create the entire HD wallet from that seed in any compatible HD wallet. This makes it easy to back up, restore, export, and import HD wallets containing thousands or even millions of keys by simply transferring only the mnemonic that the root seed is derived from.
-
-The process of creating the master keys and master chain code for an HD wallet is shown in <<HDWalletFromSeed>>.
-
-[[HDWalletFromSeed]]
-.Creating master keys and chain code from a root seed
-image::images/mbc2_0509.png["HDWalletFromRootSeed"]
-
-The root seed is input into the HMAC-SHA512 algorithm and the resulting hash is used to create a _master private key_ (m) and a _master chain code_ (c).
-
-The master private key (m) then generates a corresponding master public key (M) using the normal elliptic curve multiplication process +m * G+ that we saw in <<pubkey>>.
-
-The chain code (c) is used to introduce entropy in the function that creates child keys from parent keys, as we will see in the next section.
-
-===== Private child key derivation
-
-((("child key derivation (CKD)")))((("public and private keys", "child key derivation (CKD)")))HD wallets use a _child key derivation_ (CKD) function to derive child keys from parent keys.
-
-The child key derivation functions are based on a one-way hash function that combines:
-
-* A parent private or public key (ECDSA compressed key)
-* A seed called a chain code (256 bits)
-* An index number (32 bits)
-
-The chain code is used to introduce deterministic random data to the process, so that knowing the index and a child key is not sufficient to derive other child keys. Knowing a child key does not make it possible to find its siblings, unless you also have the chain code. The initial chain code seed (at the root of the tree) is made from the seed, while subsequent child chain codes are derived from each parent chain code.
-
-These three items (parent key, chain code, and index) are combined and hashed to generate children keys, as follows.
-
-The parent public key, chain code, and the index number are combined and hashed with the HMAC-SHA512 algorithm to produce a 512-bit hash. This 512-bit hash is split into two 256-bit halves. The right-half 256 bits of the hash output become the chain code for the child. The left-half 256 bits of the hash are added to the parent key to produce the child private key. In <<CKDpriv>>, we see this illustrated with the index set to 0 to produce the "zero" (first by index) child of the parent.
-
-[[CKDpriv]]
-.Extending a parent private key to create a child private key
-image::images/mbc2_0510.png["ChildPrivateDerivation"]
-
-Changing the index allows us to extend the parent and create the other children in the sequence, e.g., Child 0, Child 1, Child 2, etc. Each parent key can have 2,147,483,647 (2^31^) children (2^31^ is half of the entire 2^32^ range available because the other half is reserved for a special type of derivation we will talk about later in this chapter).
-
-Repeating the process one level down the tree, each child can in turn become a parent and create its own children, in an infinite number of generations.
-
-===== Using derived child keys
-
-Child private keys are indistinguishable from nondeterministic (random) keys. Because the derivation function is a one-way function, the child key cannot be used to find the parent key. The child key also cannot be used to find any siblings. If you have the n~th~ child, you cannot find its siblings, such as the n&#x2013;1 child or the n+1 child, or any other children that are part of the sequence. Only the parent key and chain code can derive all the children. Without the child chain code, the child key cannot be used to derive any grandchildren either. You need both the child private key and the child chain code to start a new branch and derive grandchildren.
-
-So what can the child private key be used for on its own? It can be used to make a public key and a Bitcoin address. Then, it can be used to sign transactions to spend anything paid to that address.
-
-[TIP]
-====
-A child private key, the corresponding public key, and the Bitcoin address are all indistinguishable from keys and addresses created randomly. The fact that they are part of a sequence is not visible outside of the HD wallet function that created them. Once created, they operate exactly as "normal" keys.
-====
-
-===== Extended keys
-
-((("public and private keys", "extended keys")))((("extended keys")))As we saw earlier, the key derivation function can be used to create children at any level of the tree, based on the three inputs: a key, a chain code, and the index of the desired child. The two essential ingredients are the key and chain code, and combined these are called an _extended key_. The term "extended key" could also be thought of as "extensible key" because such a key can be used to derive children.
-
-Extended keys are stored and represented simply as the concatenation of the 256-bit key and 256-bit chain code into a 512-bit sequence. There are two types of extended keys. An extended private key is the combination of a private key and chain code and can be used to derive child private keys (and from them, child public keys). An extended public key is a public key and chain code, which can be used to create child public keys (_public only_), as described in <<public_key_derivation>>.
-
-Think of an extended key as the root of a branch in the tree structure of the HD wallet. With the root of the branch, you can derive the rest of the branch. The extended private key can create a complete branch, whereas the extended public key can _only_ create a branch of public keys.
-
-[TIP]
-====
-An extended key consists of a private or public key and chain code. An extended key can create children, generating its own branch in the tree structure. Sharing an extended key gives access to the entire branch.
-====
-
-Extended keys are encoded using Base58Check, to easily export and import between different BIP-32&#x2013;compatible wallets. The Base58Check coding for extended keys uses a special version number that results in the prefix "xprv" and "xpub" when encoded in Base58 characters to make them easily recognizable. Because the extended key is 512 or 513 bits, it is also much longer than other Base58Check-encoded strings we have seen previously.
-
-Here's an example of an extended _private_ key, encoded in Base58Check:
-
-----
-xprv9tyUQV64JT5qs3RSTJkXCWKMyUgoQp7F3hA1xzG6ZGu6u6Q9VMNjGr67Lctvy5P8oyaYAL9CAWrUE9i6GoNMKUga5biW6Hx4tws2six3b9c
-----
-
-Here's the corresponding extended _public_ key, encoded in Base58Check:
-
-----
-xpub67xpozcx8pe95XVuZLHXZeG6XWXHpGq6Qv5cmNfi7cS5mtjJ2tgypeQbBs2UAR6KECeeMVKZBPLrtJunSDMstweyLXhRgPxdp14sk9tJPW9
-----
-
-[[public__child_key_derivation]]
-===== Public child key derivation
-
-((("public and private keys", "public child key derivation")))As mentioned  previously, a very useful characteristic of HD wallets is the ability to derive public child keys from public parent keys, _without_ having the private keys. This gives us two ways to derive a child public key: either from the child private key, or directly from the parent public key.
-
-An extended public key can be used, therefore, to derive all of the _public_ keys (and only the public keys) in that branch of the HD wallet structure.
-
-This shortcut can be used to create very secure public key&#x2013;only deployments where a server or application has a copy of an extended public key and no private keys whatsoever. That kind of deployment can produce an infinite number of public keys and Bitcoin addresses, but cannot spend any of the money sent to those addresses. Meanwhile, on another, more secure server, the extended private key can derive all the corresponding private keys to sign transactions and spend the money.
-
-One common application of this solution is to install an extended public key on a web server that serves an ecommerce application. The web server can use the public key derivation function to create a new Bitcoin address for every transaction (e.g., for a customer shopping cart). The web server will not have any private keys that would be vulnerable to theft. Without HD wallets, the only way to do this is to generate thousands of Bitcoin addresses on a separate secure server and then preload them on the ecommerce server. That approach is cumbersome and requires constant maintenance to ensure that the ecommerce server doesn't "run out" of addresses.
-
-((("cold storage")))((("storage", "cold storage")))((("hardware wallets")))Another common application of this solution is for cold-storage or hardware wallets. In that scenario, the extended private key can be stored on a paper wallet or hardware device (such as a Trezor hardware wallet), while the extended public key can be kept online. The user can create "receive" addresses at will, while the private keys are safely stored offline. To spend the funds, the user can use the extended private key on an offline signing Bitcoin client or sign transactions on the hardware wallet device (e.g., Trezor). <<CKDpub>> illustrates the mechanism for extending a parent public key to derive child public keys.
-
-[[CKDpub]]
-.Extending a parent public key to create a child public key
-image::images/mbc2_0511.png["ChildPublicDerivation"]
-
-===== Hardened child key derivation
-
-((("public and private keys", "hardened child key derivation")))((("hardened derivation")))The ability to derive a branch of public keys from an xpub is very useful, but it comes with a potential risk. Access to an xpub does not give access to child private keys. However, because the xpub contains the chain code, if a child private key is known, or somehow leaked, it can be used with the chain code to derive all the other child private keys. A single leaked child private key, together with a parent chain code, reveals all the private keys of all the children. Worse, the child private key together with a parent chain code can be used to deduce the parent private key.
-
-To counter this risk, HD wallets use an alternative derivation function called _hardened derivation_, which "breaks" the relationship between parent public key and child chain code. The hardened derivation function uses the parent private key to derive the child chain code, instead of the parent public key. This creates a "firewall" in the parent/child sequence, with a chain code that cannot be used to compromise a parent or sibling private key. The hardened derivation function looks almost identical to the normal child private key derivation, except that the parent private key is used as input to the hash function, instead of the parent public key, as shown in the diagram in <<CKDprime>>.
-
-[[CKDprime]]
-.Hardened derivation of a child key; omits the parent public key
-image::images/mbc2_0513.png["ChildHardPrivateDerivation"]
-
-[role="pagebreak-before"]
-When the hardened private derivation function is used, the resulting child private key and chain code are completely different from what would result from the normal derivation function. The resulting "branch" of keys can be used to produce extended public keys that are not vulnerable, because the chain code they contain cannot be exploited to reveal any private keys. Hardened derivation is therefore used to create a "gap" in the tree above the level where extended public keys are used.
-
-In simple terms, if you want to use the convenience of an xpub to derive branches of public keys, without exposing yourself to the risk of a leaked chain code, you should derive it from a hardened parent key, rather than a normal (non-hardened) parent key. As a best practice, the level-1 children of the master keys are always derived through the hardened derivation, to prevent compromise of the master keys.
-
-===== Index numbers for normal and hardened derivation
-
-The index number used in the derivation function is a 32-bit integer. To easily distinguish between keys derived through the normal derivation function versus keys derived through hardened derivation, this index number is split into two ranges. Index numbers between 0 and 2^31^&#x2013;1 (0x0 to 0x7FFFFFFF) are used _only_ for normal derivation. Index numbers between 2^31^ and 2^32^&#x2013;1 (0x80000000 to 0xFFFFFFFF) are used _only_ for hardened derivation. Therefore, if the index number is less than 2^31^, the child is normal, whereas if the index number is equal or above 2^31^, the child is hardened.
-
-To make the index number easier to read and display, the index number for hardened children is displayed starting from zero, but with a prime symbol. The first normal child key is therefore displayed as 0, whereas the first hardened child (index 0x80000000) is displayed as 0++&#x27;++. In sequence then, the second hardened key would have index 0x80000001 and would be displayed as 1++&#x27;++, and so on. When you see an HD wallet index i++&#x27;++, that means 2^31^+i.
-
-===== HD wallet key identifier (path)
-
-((("hierarchical deterministic (HD) wallets")))Keys in an HD wallet are identified using a "path" naming convention, with each level of the tree separated by a slash (/) character (see <<table_4-8>>). Private keys derived from the master private key start with "m." Public keys derived from the master public key start with "M." Therefore, the first child private key of the master private key is m/0. The first child public key is M/0. The second grandchild of the first child is m/0/1, and so on.
-
-The "ancestry" of a key is read from right to left, until you reach the master key from which it was derived. For example, identifier m/x/y/z describes the private key that is the z-th child of the parent private key m/x/y, which is the y-th child of the parent private key m/x, which is the x-th child of the parent master private key m.
-
-[[table_4-8]]
-.HD wallet path examples
-[options="header"]
-|=======
-|HD path | Key described
-| m/0 | The first (0) child private key from the master private key (m)
-| m/0/0 | The first (0) child private key from the first child (m/0)
-| m/0'/0 | The first (0) normal child from the first _hardened_ child (m/0')
-| m/1/0 | The first (0) child private key from the second child (m/1)
-| M/23/17/0/0 | The first (0) child public key from the first child (M/23/17/0) from the 18th child (M/23/17) from the 24th child (M/23)
-|=======
-
-===== Navigating the HD wallet tree structure
-
-The HD wallet tree structure offers tremendous flexibility. Each parent extended key can have 4 billion children: 2 billion normal children and 2 billion hardened children. Each of those children can have another 4 billion children, and so on. The tree can be as deep as you want, with an infinite number of generations. With all that flexibility, however, it becomes quite difficult to navigate this infinite tree. It is especially difficult to transfer HD wallets between implementations, because the possibilities for internal organization into branches and subbranches are endless.
-
-Two BIPs offer a solution to this complexity by creating some proposed standards for the structure of HD wallet trees. BIP-43 proposes the use of the first hardened child index as a special identifier that signifies the "purpose" of the tree structure. Based on BIP-43, an HD wallet should use only one level-1 branch of the tree, with the index number identifying the structure and namespace of the rest of the tree by defining its purpose. For example, an HD wallet using only branch m/i++&#x27;++/ is intended to signify a specific purpose and that purpose is identified by index number "i."
-
-Extending that specification, BIP-44 proposes a multiaccount structure as "purpose" number +44'+ under BIP-43. All HD wallets following the BIP-44 structure are identified by the fact that they only used one branch of the tree: m/44'/.
-
-BIP-44 specifies the structure as consisting of five predefined tree levels:
-
------
-m / purpose' / coin_type' / account' / change / address_index
------
-
-The first-level "purpose" is always set to +44'+. The second-level "coin_type" specifies the type of cryptocurrency coin, allowing for multicurrency HD wallets where each currency has its own subtree under the second level. There are three currencies defined for now: Bitcoin is m/44'/0', Bitcoin Testnet is m/44++&#x27;++/1++&#x27;++, and Litecoin is m/44++&#x27;++/2++&#x27;++.
-
-The third level of the tree is "account," which allows users to subdivide their wallets into separate logical subaccounts, for accounting or organizational purposes. For example, an HD wallet might contain two bitcoin "accounts": m/44++&#x27;++/0++&#x27;++/0++&#x27;++ and m/44++&#x27;++/0++&#x27;++/1++&#x27;++. Each account is the root of its own subtree.
-
-((("keys and addresses", see="also public and private keys")))On the fourth level, "change," an HD wallet has two subtrees, one for creating receiving addresses and one for creating change addresses. Note that whereas the previous levels used hardened derivation, this level uses normal derivation. This is to allow this level of the tree to export extended public keys for use in a nonsecured environment. Usable addresses are derived by the HD wallet as children of the fourth level, making the fifth level of the tree the "address_index." For example, the third receiving address for bitcoin payments in the primary account would be M/44++&#x27;++/0++&#x27;++/0++&#x27;++/0/2. <<table_4-9>> shows a few more examples.
-
-[[table_4-9]]
-.BIP-44 HD wallet structure examples
-[options="header"]
-|=======
-|HD path | Key described
-| M/44++&#x27;++/0++&#x27;++/0++&#x27;++/0/2 | The third receiving public key for the primary bitcoin account
-| M/44++&#x27;++/0++&#x27;++/3++&#x27;++/1/14 | The fifteenth change-address public key for the fourth bitcoin account
-| m/44++&#x27;++/2++&#x27;++/0++&#x27;++/0/1 | The second private key in the Litecoin main account, for signing transactions
-|=======
-
-==== Using an Extended Public Key on a Web Store
-
-((("wallets", "technology of", "using extended public keys on web stores")))Let's see how HD wallets are used by continuing our story with Gabriel's web store.((("use cases", "web store", id="gabrielfivetwo")))
-
-Gabriel first set up his web store as a hobby, based on a simple hosted Wordpress page. His store was quite basic with only a few pages and an order form with a single Bitcoin address.
-
-Gabriel used the first Bitcoin address generated by his Trezor device as the main Bitcoin address for his store. This way, all incoming payments would be paid to an address controlled by his Trezor hardware wallet.
-
-Customers would submit an order using the form and send payment to Gabriel's published Bitcoin address, triggering an email with the order details for Gabriel to process. With just a few orders each week, this system worked well enough.
-
-However, the little web store became quite successful and attracted many orders from the local community. Soon, Gabriel was overwhelmed. With all the orders paying the same address, it became difficult to correctly match orders and transactions, especially when multiple orders for the same amount came in close together.
-
-Gabriel's HD wallet offers a much better solution through the ability to derive public child keys without knowing the private keys. Gabriel can load an extended public key (xpub) on his website, which can be used to derive a unique address for every customer order. Gabriel can spend the funds from his Trezor, but the xpub loaded on the website can only generate addresses and receive funds. This feature of HD wallets is a great security feature. Gabriel's website does not contain any private keys and therefore does not need high levels of security.
-
-To export the xpub, Gabriel uses the Trezor Suite desktop app in conjunction with the Trezor hardware wallet. The Trezor device must be plugged in for the public keys to be exported. Note that hardware wallets will never export private keys&#x2014;those always remain on the device. <<export_xpub>> shows what Gabriel sees in Trezor Suite when exporting the xpub.
-
-[[export_xpub]]
-.Exporting an xpub from a Trezor hardware wallet
-image::images/mbc2_0512.png["Exporting the xpub from the Trezor"]
-
-Gabriel copies the xpub to his web store's bitcoin shop software. He uses _BTCPay Server_, which is an open source web-store for a variety of web hosting and content platforms. BTCPay Server uses the xpub to generate a unique address for every purchase. ((("", startref="gabrielfivetwo")))
-
-===== Account Discovery and Management
-
-Gabriel's business is flourishing. He has provided his extended public key (xpub) to _BTCPay Server_, which is generating unique addresses for customers to his website. Every time a customer to Gabriel's website clicks on the "Checkout" button with a specified payment modality (in this case, bitcoin), _BTCPay Server_ generates a new address for that customer.  More specifically, _BTCPay Server_ iterates on the _address_index_ tree to create a new address to display to the customer, as defined by BIP-44. If the customer decides to switch payment methods or abandon the transaction entirely, this Bitcoin address goes unused and will not be used for another customer right away. 
-
-At a single moment in time, Gabriel's website may have a large volume of outstanding addresses for customers making purchases, some of which may go unused and eventually expire. Once these addresses expire, _BTCPay Server_ will go back to reuse these addresses to fill the gap in _address_index_, but it becomes clear how there can be gaps between the _address_index_ leaves of the hierarchical deterministic tree where the money is actually located.  
-
-Let's say that Gabriel is interested in viewing his total amount of bitcoin earned on a watch-only wallet (one that allows you to view transaction history, but not spend funds) that is separate from BTCPay Server but also conforms to the BIP-44 standard. How should this separate wallet go about searching for funds in this vast hierarchical tree, and when should it stop looking? Most wallets will typically follow an iterative process that utilizes a predefined limit, known as the _gap limit_. If, while searching for used addresses, the wallet doesn't find used addresses in a row beyond this limit number, it will stop searching the address chain. The default gap limit is typically set to 20. This is detailed in [[bip-44]]https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki[BIP-44]. 
-
-[TIP]
-====
-Gap limits explain the phenomenon whereby the importing of a wallet may show an incorrect or zero balance. The funds are not lost, but rather, the wallet importing function has not traversed enough leaves to fully detect funds. Many wallets allow this default gap limit to be changed, and Gabriel may need to increase this limit to allow his wallet to fully import his transaction history. 
-====

ch06.asciidoc

@@ -1,759 +0,0 @@
-[[ch06]]
-[[transactions]]
-== Transactions
-
-[[ch06_intro]]
-=== Introduction
-
-((("transactions", "defined")))((("warnings and cautions", see="also security")))Transactions are the most important part of the Bitcoin system. Everything else in bitcoin is designed to ensure that transactions can be created, propagated on the network, validated, and finally added to the global ledger of transactions (the blockchain). Transactions are data structures that encode the transfer of value between participants in the Bitcoin system. Each transaction is a public entry in bitcoin's blockchain, the global double-entry bookkeeping ledger.
-
-In this chapter we will examine all the various forms of transactions, what they contain, how to create them, how they are verified, and how they become part of the permanent record of all transactions. When we use the term "wallet" in this chapter, we are referring to the software that constructs transactions, not just the database of keys.
-
-[[tx_structure]]
-=== Transactions in Detail
-
-((("use cases", "buying coffee", id="alicesix")))In <<ch02_bitcoin_overview>>, we looked at the transaction Alice used to pay for coffee at Bob's coffee shop using a block explorer (<<alices_transactions_to_bobs_cafe>>).
-
-The block explorer application shows a transaction from Alice's "address" to Bob's "address." This is a much simplified view of what is contained in a transaction. In fact, as we will see in this chapter, much of the information shown is constructed by the block explorer and is not actually in the transaction.
-
-[[alices_transactions_to_bobs_cafe]]
-.Alice's transaction to Bob's Cafe
-image::images/mbc2_0208.png["Alice Coffee Transaction"]
-
-[[transactions_behind_the_scenes]]
-==== Transactions&#x2014;Behind the Scenes
-
-((("transactions", "behind the scenes details of")))Behind the scenes, an actual transaction looks very different from a transaction provided by a typical block explorer. In fact, most of the high-level constructs we see in the various bitcoin application user interfaces _do not actually exist_ in the Bitcoin system.
-
-We can use Bitcoin Core's command-line interface (+getrawtransaction+ and +decoderawtransaction+) to retrieve Alice's "raw" transaction, decode it, and see what it contains. The result looks like this:
-
-[[alice_tx]]
-.Alice's transaction decoded
-[source,json]
-----
-{
-  "version": 1,
-  "locktime": 0,
-  "vin": [
-    {
-      "txid": "7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18",
-      "vout": 0,
-      "scriptSig" : "3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813[ALL] 0484ecc0d46f1918b30928fa0e4ed99f16a0fb4fde0735e7ade8416ab9fe423cc5412336376789d172787ec3457eee41c04f4938de5cc17b4a10fa336a8d752adf",
-      "sequence": 4294967295
-    }
-  ],
-  "vout": [
-    {
-      "value": 0.01500000,
-      "scriptPubKey": "OP_DUP OP_HASH160 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 OP_EQUALVERIFY OP_CHECKSIG"
-    },
-    {
-      "value": 0.08450000,
-      "scriptPubKey": "OP_DUP OP_HASH160 7f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a8 OP_EQUALVERIFY OP_CHECKSIG",
-    }
-  ]
-}
-----
-
-You may notice a few things about this transaction, mostly the things that are missing! Where is Alice's address? Where is Bob's address? Where is the 0.1 input "sent" by Alice? In bitcoin, there are no coins, no senders, no recipients, no balances, no accounts, and no addresses. All those things are constructed at a higher level for the benefit of the user, to make things easier to understand.
-
-You may also notice a lot of strange and indecipherable fields and hexadecimal strings. Don't worry, we will explain each field shown here in detail in this chapter.
-
-[[tx_inputs_outputs]]
-=== Transaction Outputs and Inputs
-
-((("transactions", "outputs and inputs", id="Tout06")))((("outputs and inputs", "outputs defined")))((("unspent transaction outputs (UTXO)")))((("UTXO sets")))((("transactions", "outputs and inputs", "output characteristics")))((("outputs and inputs", "output characteristics")))The fundamental building block of a bitcoin transaction is a  _transaction output_. Transaction outputs are indivisible chunks of bitcoin currency, recorded on the blockchain, and recognized as valid by the entire network. Bitcoin full nodes track all available and spendable outputs, known as _unspent transaction outputs_, or _UTXO_. The collection of all UTXO is known as the _UTXO set_ and currently numbers in the millions of UTXO. The UTXO set grows as new UTXO is created and shrinks when UTXO is consumed. Every transaction represents a change (state transition) in the UTXO set.
-
-((("balances")))When we say that a user's wallet has "received" bitcoin, what we mean is that the wallet has detected on the blockchain an UTXO that can be spent with one of the keys controlled by that wallet. Thus, a user's bitcoin "balance" is the sum of all UTXO that user's wallet can spend and which may be scattered among hundreds of transactions and hundreds of blocks. The concept of a balance is created by the wallet application. The wallet calculates the user's balance by scanning the blockchain and aggregating the value of any UTXO the wallet can spend with the keys it controls. Most wallets maintain a database or use a database service to store a quick reference set of all the UTXO they can spend with the keys they control.
-
-((("spent transaction outputs (STXO)")))<<utxo-stxo>> displays the blockchain in three different moments, as the transaction chain from Joe to Gopesh is being built. Notice how each transaction spends an UTXO that was created in a previous transaction, turning it into a _spent transaction output_, or _STXO_. Since transaction #1 (from Joe to Alice) spends a single UTXO (from Joe) and creates a single UTXO (to Alice), it doesn't modify the UTXO set size. On the other hand, transactions #2 and #3 both create change outputs to the sender, spending a single UTXO and creating two UTXO (the payment and the change output). Therefore, each of them increases the UTXO set size by 1.
-
-[[utxo-stxo]]
-.Transaction chain from Joe to Gopesh being built on the blockchain
-image::images/mbc2_0609.png["Transaction chain from Joe to Gopesh being built on the blockchain"]
-
-((("satoshis")))A transaction output can have an arbitrary (integer) value denominated as a multiple of satoshis.  Just as dollars can be divided down to two decimal places as cents, bitcoin can be divided down to eight decimal places as satoshis. Although an output can have any arbitrary value, once created it is indivisible. This is an important characteristic of outputs that needs to be emphasized: outputs are _discrete_ and _indivisible_ units of value, denominated in integer satoshis. An unspent output can only be consumed in its entirety by a transaction.
-
-((("change, making")))If an UTXO is larger than the desired value of a transaction, it must still be consumed in its entirety and change must be generated in the transaction. In other words, if you have an UTXO worth 20 bitcoin and want to pay only 1 bitcoin, your transaction must consume the entire 20-bitcoin UTXO and produce two outputs: one paying 1 bitcoin to your desired recipient and another paying 19 bitcoin in change back to your wallet. As a result of the indivisible nature of transaction outputs, most bitcoin transactions will have to generate change.
-
-Imagine a shopper buying a $1.50 beverage, reaching into her wallet and trying to find a combination of coins and bank notes to cover the $1.50 cost. The shopper will choose exact change if available e.g. a dollar bill and two quarters (a quarter is $0.25), or a combination of smaller denominations (six quarters), or if necessary, a larger unit such as a $5 note. If she hands too much money, say $5, to the shop owner, she will expect $3.50 change, which she will return to her wallet and have available for future transactions.
-
-Similarly, a bitcoin transaction must be created from a user's UTXO in whatever denominations that user has available. Users cannot cut an UTXO in half any more than they can cut a dollar bill in half and use it as currency. The user's wallet application will typically select from the user's available UTXO to compose an amount greater than or equal to the desired transaction amount.
-
-As with real life, the bitcoin application can use several strategies to satisfy the purchase amount: combining several smaller units, finding exact change, or using a single unit larger than the transaction value and making change. All of this complex assembly of spendable UTXO is done by the user's wallet automatically and is invisible to users. It is only relevant if you are programmatically constructing raw transactions from UTXO.
-
-A transaction consumes previously recorded unspent transaction outputs and creates new transaction outputs that can be consumed by a future transaction. This way, chunks of bitcoin value move forward from owner to owner in a chain of transactions consuming and creating UTXO.
-
-((("transactions", "coinbase transactions")))((("coinbase transactions")))((("mining and consensus", "coinbase transactions")))The exception to the output and input chain is a special type of transaction called the _coinbase_ transaction, which is the first transaction in each block. This transaction is placed there by the "winning" miner and creates brand-new bitcoin payable to that miner as a reward for mining. This special coinbase transaction does not consume UTXO; instead, it has a special type of input called the "coinbase." This is how bitcoin's money supply is created during the mining process, as we will see in <<mining>>.
-
-[TIP]
-====
-What comes first? Inputs or outputs, the chicken or the egg? Strictly speaking, outputs come first because coinbase transactions, which generate new bitcoin, have no inputs and create outputs from nothing.
-====
-
-[[tx_outs]]
-==== Transaction Outputs
-
-((("transactions", "outputs and inputs", "output components")))((("outputs and inputs", "output parts")))Every bitcoin transaction creates outputs, which are recorded on the bitcoin ledger. Almost all of these outputs, with one exception (see <<op_return>>) create spendable chunks of bitcoin called UTXO, which are then recognized by the whole network and available for the owner to spend in a future transaction.
-
-UTXO are tracked by every full-node Bitcoin client in the UTXO set. New transactions consume (spend) one or more of these outputs from the UTXO set.
-
-Transaction outputs consist of two parts:
-
-* An amount of bitcoin, denominated in _satoshis_, the smallest bitcoin unit
-* A cryptographic puzzle that determines the conditions required to spend the output
-
-((("locking scripts")))((("scripting", "locking scripts")))((("witnesses")))((("scriptPubKey")))The cryptographic puzzle is also known as a  _locking script_, a _witness script_, or a +scriptPubKey+.
-
-The transaction scripting language, used in the locking script mentioned previously, is discussed in detail in <<tx_script>>.
-
-Now, let's look at Alice's transaction (shown previously in <<transactions_behind_the_scenes>>) and see if we can identify the outputs. In the JSON encoding, the outputs are in an array (list) named +vout+:
-
-[source,json]
-----
-"vout": [
-  {
-    "value": 0.01500000,
-    "scriptPubKey": "OP_DUP OP_HASH160 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 OP_EQUALVERIFY
-    OP_CHECKSIG"
-  },
-  {
-    "value": 0.08450000,
-    "scriptPubKey": "OP_DUP OP_HASH160 7f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a8 OP_EQUALVERIFY OP_CHECKSIG",
-  }
-]
-----
-
-As you can see, the transaction contains two outputs. Each output is defined by a value and a cryptographic puzzle. In the encoding shown by Bitcoin Core, the value is shown in bitcoin, but in the transaction itself it is recorded as an integer denominated in satoshis. The second part of each output is the cryptographic puzzle that sets the conditions for spending. Bitcoin Core shows this as +scriptPubKey+ and shows us a human-readable representation of the script.
-
-The topic of locking and unlocking UTXO will be discussed later, in <<tx_lock_unlock>>. The scripting language that is used for the script in +scriptPubKey+ is discussed in <<tx_script>>. But before we delve into those topics, we need to understand the overall structure of transaction inputs and outputs.
-
-===== Transaction serialization&#x2014;outputs
-
-((("transactions", "outputs and inputs", "structure of")))((("outputs and inputs", "structure of")))((("serialization", "outputs")))When transactions are transmitted over the network or exchanged between applications, they are _serialized_. Serialization is the process of converting the internal representation of a data structure into a format that can be transmitted one byte at a time, also known as a byte stream. Serialization is most commonly used for encoding data structures for transmission over a network or for storage in a file. The serialization format of a transaction output is shown in <<tx_out_structure>>.
-
-[[tx_out_structure]]
-.Transaction output serialization
-[options="header"]
-|=======
-|Size| Field | Description
-| 8 bytes (little-endian) | Amount  | Bitcoin value in satoshis (10^-8^ bitcoin)
-| 1&#x2013;9 bytes (VarInt) | Locking-Script Size | Locking-Script length in bytes, to follow
-| Variable | Locking-Script | A script defining the conditions needed to spend the output
-|=======
-
-Most bitcoin libraries and frameworks do not store transactions internally as byte-streams, as that would require complex parsing every time you needed to access a single field. For convenience and readability, bitcoin libraries store transactions internally in data structures (usually object-oriented structures).
-
-((("deserialization")))((("parsing")))((("transactions", "parsing")))The process of converting from the byte-stream representation of a transaction to a library's internal representation data structure is called _deserialization_ or _transaction parsing_. The process of converting back to a byte-stream for transmission over the network, for hashing, or for storage on disk is called _serialization_.  Most bitcoin libraries have built-in functions for transaction serialization and deserialization.
-
-See if you can manually decode Alice's transaction from the serialized hexadecimal form, finding some of the elements we saw previously. The section containing the two outputs is highlighted in <<example_6_1>> to help you:
-
-[[example_6_1]]
-.Alice's transaction, serialized and presented in hexadecimal notation
-====
-+0100000001186f9f998a5aa6f048e51dd8419a14d8a0f1a8a2836dd73+
-+4d2804fe65fa35779000000008b483045022100884d142d86652a3f47+
-+ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039+
-+ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813+
-+01410484ecc0d46f1918b30928fa0e4ed99f16a0fb4fde0735e7ade84+
-+16ab9fe423cc5412336376789d172787ec3457eee41c04f4938de5cc1+
-+7b4a10fa336a8d752adfffffffff02+*+60e31600000000001976a914ab6+*
-*+8025513c3dbd2f7b92a94e0581f5d50f654e788acd0ef800000000000+*
-*+1976a9147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a888ac+*
-+00000000+
-====
-
-Here are some hints:
-
-* There are two outputs in the highlighted section, each serialized as shown in <<tx_out_structure>>.
-* The value of 0.015 bitcoin is 1,500,000 satoshis. That's +16 e3 60+ in hexadecimal.
-* In the serialized transaction, the value +16 e3 60+ is encoded in little-endian (least-significant-byte-first) byte order, so it looks like +60 e3 16+.
-* The +scriptPubKey+ length is 25 bytes, which is +19+ in hexadecimal.
-
-[[tx_inputs]]
-==== Transaction Inputs
-
-((("transactions", "outputs and inputs", "input components")))((("outputs and inputs", "input components")))((("unspent transaction outputs (UTXO)")))((("UTXO sets")))Transaction inputs identify (by reference) which UTXO will be consumed and provide proof of ownership through an unlocking script.
-
-To build a transaction, a wallet selects from the UTXO it controls, UTXO with enough value to make the requested payment. Sometimes one UTXO is enough, other times more than one is needed. For each UTXO that will be consumed to make this payment, the wallet creates one input pointing to the UTXO and unlocks it with an unlocking script.
-
-Let's look at the components of an input in greater detail. The first part of an input is a pointer to an UTXO by reference to the transaction hash and an output index, which identifies the specific UTXO in that transaction. The second part is an unlocking script, which the wallet constructs in order to satisfy the spending conditions set in the UTXO. Most often, the unlocking script is a digital signature and public key proving ownership of the bitcoin. However, not all unlocking scripts contain signatures. The third part is a sequence number, which will be discussed later.
-
-Consider our example in <<transactions_behind_the_scenes>>. The transaction inputs are an array (list) called +vin+:
-
-[[vin]]
-.The transaction inputs in Alice's transaction
-[source,json]
-----
-"vin": [
-  {
-    "txid": "7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18",
-    "vout": 0,
-    "scriptSig" : "3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813[ALL] 0484ecc0d46f1918b30928fa0e4ed99f16a0fb4fde0735e7ade8416ab9fe423cc5412336376789d172787ec3457eee41c04f4938de5cc17b4a10fa336a8d752adf",
-    "sequence": 4294967295
-  }
-]
-----
-
-As you can see, there is only one input in the list (because one UTXO contained sufficient value to make this payment). The input contains four elements:
-
-* A ((("transaction IDs (txd)")))transaction ID, referencing the transaction that contains the UTXO being spent
-* An output index (+vout+), identifying which UTXO from that transaction is referenced (first one is zero)
-* A +scriptSig+, which satisfies the conditions placed on the UTXO, unlocking it for spending
-* A sequence number (to be discussed later)
-
-In Alice's transaction, the input points to the transaction ID:
-
-----
-7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18
-----
-
-and output index +0+ (i.e., the first UTXO created by that transaction). The unlocking script is constructed by Alice's wallet by first retrieving the referenced UTXO, examining its locking script, and then using it to build the necessary unlocking script to satisfy it.
-
-Looking just at the input you may have noticed that we don't know anything about this UTXO, other than a reference to the parent transaction containing it. We don't know its value (amount in satoshi), and we don't know the locking script that sets the conditions for spending it. To find this information, we must retrieve the referenced UTXO by retrieving the parent transaction that contains it. Notice that because the value of the input is not explicitly stated, we must also use the referenced UTXO in order to calculate the fees that will be paid in this transaction (see <<tx_fees>>).
-
-It's not just Alice's wallet that needs to retrieve UTXO referenced in the inputs. Once this transaction is broadcast to the network, every validating node will also need to retrieve the UTXO referenced in the transaction inputs in order to validate the transaction.
-
-Transactions on their own seem incomplete because they lack context. They reference UTXO in their inputs but without retrieving that UTXO we cannot know the value of the inputs or their locking conditions. When writing bitcoin software, anytime you decode a transaction with the intent of validating it or counting the fees or checking the unlocking script, your code will first have to retrieve the referenced UTXO from the blockchain in order to build the context implied but not present in the UTXO references of the inputs. For example, to calculate the amount paid in fees, you must know the sum of the values of inputs and outputs. But without retrieving the UTXO referenced in the inputs, you do not know their value. So a seemingly simple operation like counting fees in a single transaction in fact involves multiple steps and data from multiple transactions.
-
-We can use the same sequence of commands with Bitcoin Core as we used when retrieving Alice's transaction (+getrawtransaction+ and +decoderawtransaction+). With that we can get the UTXO referenced in the input from Alice's transaction and take a look:
-
-[[alice_input_tx]]
-.UTXO from the previous transaction, referenced in the input from Alice's transaction
-[source,json]
-----
-"vout": [
-   {
-     "value": 0.10000000,
-     "scriptPubKey": "OP_DUP OP_HASH160 7f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a8 OP_EQUALVERIFY OP_CHECKSIG"
-   }
- ]
-----
-
-We see that this UTXO has a value of 0.1 BTC and that it has a locking script (+scriptPubKey+) that contains "OP_DUP OP_HASH160...".
-
-[TIP]
-====
-To fully understand Alice's transaction we had to retrieve the previous transaction referenced as input. A function that retrieves previous transactions and unspent transaction outputs is very common and exists in almost every bitcoin library and API.
-====
-
-===== Transaction serialization&#x2014;inputs
-
-((("serialization", "inputs")))((("transactions", "outputs and inputs", "input serialization")))((("outputs and inputs", "input serialization")))When transactions are serialized for transmission on the network, their inputs are encoded into a byte stream as shown in <<tx_in_structure>>.
-
-[[tx_in_structure]]
-.Transaction input serialization
-[options="header"]
-|=======
-|Size| Field | Description
-| 32 bytes | Transaction Hash | Pointer to the transaction containing the UTXO to be spent
-| 4 bytes | Output Index | The index number of the UTXO to be spent; first one is 0
-| 1&#x2013;9 bytes (VarInt) | Unlocking-Script Size | Unlocking-Script length in bytes, to follow
-| Variable | Unlocking-Script | A script that fulfills the conditions of the UTXO locking script
-| 4 bytes | Sequence Number | Used for locktime or disabled (0xFFFFFFFF)
-|=======
-
-As with the outputs, let's see if we can find the inputs from Alice's transaction in the serialized format. First, the inputs decoded:
-
-[source,json]
-----
-"vin": [
-  {
-    "txid": "7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18",
-    "vout": 0,
-    "scriptSig" : "3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813[ALL] 0484ecc0d46f1918b30928fa0e4ed99f16a0fb4fde0735e7ade8416ab9fe423cc5412336376789d172787ec3457eee41c04f4938de5cc17b4a10fa336a8d752adf",
-    "sequence": 4294967295
-  }
-],
-----
-
-Now, let's see if we can identify these fields in the serialized hex encoding in <<example_6_2>>:
-
-[[example_6_2]]
-.Alice's transaction, serialized and presented in hexadecimal notation
-====
-+0100000001+*+186f9f998a5aa6f048e51dd8419a14d8a0f1a8a2836dd73+*
-*+4d2804fe65fa35779000000008b483045022100884d142d86652a3f47+*
-*+ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039+*
-*+ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813+*
-*+01410484ecc0d46f1918b30928fa0e4ed99f16a0fb4fde0735e7ade84+*
-*+16ab9fe423cc5412336376789d172787ec3457eee41c04f4938de5cc1+*
-*+7b4a10fa336a8d752adfffffffff+*+0260e31600000000001976a914ab6+
-+8025513c3dbd2f7b92a94e0581f5d50f654e788acd0ef800000000000+
-+1976a9147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a888ac00000+
-+000+
-====
-
-Hints:
-
-* The transaction ID is serialized in reversed byte order, so it starts with (hex) +18+ and ends with +79+
-* The output index is a 4-byte group of zeros, easy to identify
-* The length of the +scriptSig+ is 139 bytes, or +8b+ in hex
-* The sequence number is set to +FFFFFFFF+, again easy to identify((("", startref="alicesix")))
-
-ScriptSig is a specific type of unlocking script that when serialized for transmission on the network, inputs are encoded into a byte stream as shown in <<scriptsig_in_structure>>. The serialization of the signature field is detailed in <<seralization_of_signatures_der>>. The signature field also includes a Signature Hash Type (SIGHASH), which is detailed in <<sighash_types>>.
-
-[[scriptsig_in_structure]]
-.ScriptSig input serialization
-[options="header"]
-|=======
-|Size| Field | Description
-| 1&#x2013;9 bytes (VarInt) | Signature Size | Signature length in bytes, to follow
-| Variable | Signature | A signature that is produced by the user’s wallet from his or her private key, which includes a SIGHASH
-| 1&#x2013;9 bytes (VarInt) | Public Key Size | Public key length in bytes, to follow
-| Variable | Public Key | The public key, unhashed
-|=======
-
-[[tx_fees]]
-==== Transaction Fees
-
-((("transactions", "outputs and inputs", "transaction fees")))((("fees", "transaction fees")))((("mining and consensus", "rewards and fees")))Most transactions include transaction fees, which compensate the bitcoin miners for securing the network. Fees also serve as a security mechanism themselves, by making it economically infeasible for attackers to flood the network with transactions. Mining and the fees and rewards collected by miners are discussed in more detail in <<mining>>.
-
-This section examines how transaction fees are included in a typical transaction. Most wallets calculate and include transaction fees automatically. However, if you are constructing transactions programmatically, or using a command-line interface, you must manually account for and include these fees.
-
-Transaction fees serve as an incentive to include (mine) a transaction into the next block and also as a disincentive against abuse of the system by imposing a small cost on every transaction. Transaction fees are collected by the miner who mines the block that records the transaction on the blockchain.
-
-Transaction fees are calculated based on the size of the transaction in kilobytes, not the value of the transaction in bitcoin. Overall, transaction fees are set based on market forces within the Bitcoin network. Miners prioritize transactions based on many different criteria, including fees, and might even process transactions for free under certain circumstances. Transaction fees affect the processing priority, meaning that a transaction with sufficient fees is likely to be included in the next block mined, whereas a transaction with insufficient or no fees might be delayed, processed on a best-effort basis after a few blocks, or not processed at all. Transaction fees are not mandatory, and transactions without fees might be processed eventually; however, including transaction fees encourages priority processing.
-
-Over time, the way transaction fees are calculated and the effect they have on transaction prioritization has evolved. At first, transaction fees were fixed and constant across the network. Gradually, the fee structure relaxed and may be influenced by market forces, based on network capacity and transaction volume. Since at least the beginning of 2016, capacity limits in bitcoin have created competition between transactions, resulting in higher fees and effectively making free transactions a thing of the past. Zero fee or very low fee transactions rarely get mined and sometimes will not even be propagated across the network.
-
-((("fees", "fee relay policies")))((("minrelaytxfee option")))In Bitcoin Core, fee relay policies are set by the +minrelaytxfee+ option. The current default +minrelaytxfee+ is 0.00001 bitcoin or a hundredth of a millibitcoin per kilobyte. Therefore, by default, transactions with a fee less than 0.00001 bitcoin are treated as free and are only relayed if there is space in the mempool; otherwise, they are dropped. Bitcoin nodes can override the default fee relay policy by adjusting the value of +minrelaytxfee+.
-
-((("dynamic fees")))((("fees", "dynamic fees")))Any bitcoin service that creates transactions, including wallets, exchanges, retail applications, etc., _must_ implement dynamic fees. Dynamic fees can be implemented through a third-party fee estimation service or with a built-in fee estimation algorithm. If you're unsure, begin with a third-party service and as you gain experience design and implement your own algorithm if you wish to remove the third-party dependency.
-
-Fee estimation algorithms calculate the appropriate fee, based on capacity and the fees offered by "competing" transactions. These algorithms range from simplistic (average or median fee in the last block) to sophisticated (statistical analysis). They estimate the necessary fee (in satoshis per byte) that will give a transaction a high probability of being selected and included within a certain number of blocks. Most services offer users the option of choosing high, medium, or low priority fees. High priority means users pay higher fees but the transaction is likely to be included in the next block. Medium and low priority means users pay lower transaction fees but the transactions may take much longer to confirm.
-
-((("bitcoinfees (third-party service)")))Many wallet applications use third-party services for fee calculations. One popular service is https://bitcoinfees.earn.com/[_https://bitcoinfees.earn.com/_], which provides an API and a visual chart showing the fee in satoshi/byte for different priorities.
-
-[TIP]
-====
-((("static fees")))((("fees", "static fees")))Static fees are no longer viable on the Bitcoin network. Wallets that set static fees will produce a poor user experience as transactions will often get "stuck" and remain unconfirmed. Users who don't understand bitcoin transactions and fees are dismayed by "stuck" transactions because they think they've lost their money.
-====
-
-The chart in <<bitcoinfeesearncom>> shows the real-time estimate of fees in 10 satoshi/byte increments and the expected confirmation time (in minutes and number of blocks) for transactions with fees in each range. For each fee range (e.g., 61&#x2013;70 satoshi/byte), two horizontal bars show the number of unconfirmed transactions (1405) and total number of transactions in the past 24 hours (102,975), with fees in that range. Based on the graph, the recommended high-priority fee at this time was 80 satoshi/byte, a fee likely to result in the transaction being mined in the very next block (zero block delay). For perspective, the median transaction size is 226 bytes, so the recommended fee for this transaction size would be 18,080 satoshis (0.00018080 BTC).
-
-The fee estimation data can be retrieved via a simple HTTP REST API, at https://bitcoinfees.earn.com/api/v1/fees/recommended[https://bitcoinfees.earn.com/api/v1/fees/recommended]. For example, on the command line using the +curl+ command:
-
-.Using the fee estimation API
-----
-$ curl https://bitcoinfees.earn.com/api/v1/fees/recommended
-
-{"fastestFee":80,"halfHourFee":80,"hourFee":60}
-----
-
-The API returns a JSON object with the current fee estimate for fastest confirmation (+fastestFee+), confirmation within three blocks (+halfHourFee+) and six blocks (+hourFee+), in satoshi per byte.
-
-[[bitcoinfeesearncom]]
-.Fee estimation service bitcoinfees.earn.com
-image::images/mbc2_0602.png[Fee Estimation Service bitcoinfees.earn.com]
-
-==== Adding Fees to Transactions
-
-The data structure of transactions does not have a field for fees. Instead, fees are implied as the difference between the sum of inputs and the sum of outputs. Any excess amount that remains after all outputs have been deducted from all inputs is the fee that is collected by the miners:
-
-[[tx_fee_equation]]
-.Transaction fees are implied, as the excess of inputs minus outputs:
-----
-Fees = Sum(Inputs) – Sum(Outputs)
-----
-
-This is a somewhat confusing element of transactions and an important point to understand, because if you are constructing your own transactions you must ensure you do not inadvertently include a very large fee by underspending the inputs. That means that you must account for all inputs, if necessary by creating change, or you will end up giving the miners a very big tip!
-
-For example, if you consume a 20-bitcoin UTXO to make a 1-bitcoin payment, you must include a 19-bitcoin change output back to your wallet. Otherwise, the 19-bitcoin "leftover" will be counted as a transaction fee and will be collected by the miner who mines your transaction in a block. Although you will receive priority processing and make a miner very happy, this is probably not what you intended.
-
-[WARNING]
-====
-((("warnings and cautions", "change outputs")))If you forget to add a change output in a manually constructed transaction, you will be paying the change as a transaction fee. Saying "Keep the change!" to the miner might not be what you really intended.
-====
-
-((("use cases", "buying coffee")))Let's see how this works in practice, by looking at Alice's coffee purchase again. Alice wants to spend 0.015 bitcoin to pay for coffee. To ensure this transaction is processed promptly, she will want to include a transaction fee, say 0.0005. That will mean that the total cost of the transaction will be 0.0155. Her wallet must therefore source a set of UTXO that adds up to 0.0155 bitcoin or more and, if necessary, create change. Let's say her wallet has a 0.1-bitcoin UTXO available. It will therefore need to consume this UTXO, create one output to Bob's Cafe for 0.015, and a second output with 0.0845 bitcoin in change back to her own wallet, leaving 0.0005 bitcoin unallocated, as an implicit fee for the transaction.
-
-((("use cases", "charitable donations")))((("charitable donations")))Now let's look at a different scenario. Eugenia, our children's charity director in the Philippines, has completed a fundraiser to purchase schoolbooks for the children. She received several thousand small donations from people all around the world, totaling 50 bitcoin, so her wallet is full of very small payments (UTXO). Now she wants to purchase hundreds of schoolbooks from a local publisher, paying in bitcoin.
-
-As Eugenia's wallet application tries to construct a single larger payment transaction, it must source from the available UTXO set, which is composed of many smaller amounts. That means that the resulting transaction will source from more than a hundred small-value UTXO as inputs and only one output, paying the book publisher. A transaction with that many inputs will be larger than one kilobyte, perhaps several kilobytes in size. As a result, it will require a much higher fee than the median-sized transaction.
-
-Eugenia's wallet application will calculate the appropriate fee by measuring the size of the transaction and multiplying that by the per-kilobyte fee. Many wallets will overpay fees for larger transactions to ensure the transaction is processed promptly. The higher fee is not because Eugenia is spending more money, but because her transaction is more complex and larger in size--the fee is independent of the transaction's bitcoin value.((("", startref="Tout06")))
-
-[[tx_script]]
-[role="pagebreak-before less_space_h1"]
-=== Transaction Scripts and Script Language
-
-((("transactions", "scripts and Script language", id="Tsript06")))((("scripting", "transactions and", id="Stransact06")))The bitcoin transaction script language, called _Script_, is a Forth-like reverse-polish notation stack-based execution language. If that sounds like gibberish, you probably haven't studied 1960s programming languages, but that's ok&#x2014;we will explain it all in this chapter. Both the locking script placed on an UTXO and the unlocking script are written in this scripting language. When a transaction is validated, the unlocking script in each input is executed alongside the corresponding locking script to see if it satisfies the spending condition.
-
-Script is a very simple language that was designed to be limited in scope and executable on a range of hardware, perhaps as simple as an embedded device. It requires minimal processing and cannot do many of the fancy things modern programming languages can do. For its use in validating programmable money, this is a deliberate security feature.
-
-((("Pay-to-Public-Key-Hash (P2PKH)")))Today, most transactions processed through the Bitcoin network have the form "Payment to Bob's Bitcoin address" and are based on a script called a Pay-to-Public-Key-Hash script.  However, bitcoin transactions are not limited to the "Payment to Bob's Bitcoin address" script. In fact, locking scripts can be written to express a vast variety of complex conditions. In order to understand these more complex scripts, we must first understand the basics of transaction scripts and script language.
-
-In this section, we will demonstrate the basic components of the bitcoin transaction scripting language and show how it can be used to express simple conditions for spending and how those conditions can be satisfied by unlocking scripts.
-
-[TIP]
-====
-((("programmable money")))Bitcoin transaction validation is not based on a static pattern, but instead is achieved through the execution of a scripting language. This language allows for a nearly infinite variety of conditions to be expressed. This is how bitcoin gets the power of "programmable money".
-====
-
-
-==== Turing Incompleteness
-
-((("Turing incompleteness")))The bitcoin transaction script language contains many operators, but is deliberately limited in one important way--there are no loops or complex flow control capabilities other than conditional flow control. This ensures that the language is not _Turing Complete_, meaning that scripts have limited complexity and predictable execution times. Script is not a general-purpose language. ((("denial-of-service attacks")))((("denial-of-service attacks", see="also security")))((("security", "denial-of-service attacks")))These limitations ensure that the language cannot be used to create an infinite loop or other form of "logic bomb" that could be embedded in a transaction in a way that causes a denial-of-service attack against the Bitcoin network. Remember, every transaction is validated by every full node on the Bitcoin network. A limited language prevents the transaction validation mechanism from being used as a vulnerability.
-
-==== Stateless Verification
-
-((("stateless verification")))The bitcoin transaction script language is stateless, in that there is no state prior to execution of the script, or state saved after execution of the script. Therefore, all the information needed to execute a script is contained within the script. A script will predictably execute the same way on any system. If your system verifies a script, you can be sure that every other system in the Bitcoin network will also verify the script, meaning that a valid transaction is valid for everyone and everyone knows this. This predictability of outcomes is an essential benefit of the Bitcoin system.
-
-[[tx_lock_unlock]]
-==== Script Construction (Lock + Unlock)
-
-Bitcoin's transaction validation engine relies on two types of scripts to validate transactions: a locking script and an unlocking script.
-
-((("locking scripts")))((("unlocking scripts")))((("scripting", "locking scripts")))A locking script is a spending condition placed on an output: it specifies the conditions that must be met to spend the output in the future. ((("scriptPubKey")))Historically, the locking script was called a _scriptPubKey_, because it usually contained a public key or Bitcoin address (public key hash). In this book we refer to it as a "locking script" to acknowledge the much broader range of possibilities of this scripting technology. In most bitcoin applications, what we refer to as a locking script will appear in the source code as +scriptPubKey+. ((("witnesses")))((("cryptographic puzzles")))You will also see the locking script referred to as a _witness script_ (see <<segwit>>) or more generally as a _cryptographic puzzle_. These terms all mean the same thing, at different levels of abstraction.
-
-An unlocking script is a script that "solves," or satisfies, the conditions placed on an output by a locking script and allows the output to be spent. Unlocking scripts are part of every transaction input. Most of the time they contain a digital signature produced by the user's wallet from his or her private key. ((("scriptSig")))Historically, the unlocking script was called _scriptSig_, because it usually contained a digital signature. In most bitcoin applications, the source code refers to the unlocking script as +scriptSig+. You will also see the unlocking script referred to as a _witness_ (see <<segwit>>). In this book, we refer to it as an "unlocking script" to acknowledge the much broader range of locking script requirements, because not all unlocking scripts must contain signatures.
-
-Every bitcoin validating node will validate transactions by executing the locking and unlocking scripts together. Each input contains an unlocking script and refers to a previously existing UTXO. The validation software will copy the unlocking script, retrieve the UTXO referenced by the input, and copy the locking script from that UTXO. The unlocking and locking script are then executed in sequence. The input is valid if the unlocking script satisfies the locking script conditions (see <<script_exec>>). All the inputs are validated independently, as part of the overall validation of the transaction.
-
-Note that the UTXO is permanently recorded in the blockchain, and therefore is invariable and is unaffected by failed attempts to spend it by reference in a new transaction. Only a valid transaction that correctly satisfies the conditions of the output results in the output being considered as "spent" and removed from the set of unspent transaction outputs (UTXO set).
-
-<<scriptSig_and_scriptPubKey>> is an example of the unlocking and locking scripts for the most common type of bitcoin transaction (a payment to a public key hash), showing the combined script resulting from the concatenation of the unlocking and locking scripts prior to script validation.
-
-[[scriptSig_and_scriptPubKey]]
-.Combining scriptSig and scriptPubKey to evaluate a transaction script
-image::images/mbc2_0603.png["scriptSig_and_scriptPubKey"]
-
-===== The script execution stack
-
-Bitcoin's scripting language is called a stack-based language because it uses a data structure called a _stack_. A stack is a very simple data structure that can be visualized as a stack of cards. A stack allows two operations: push and pop. Push adds an item on top of the stack. Pop removes the top item from the stack. Operations on a stack can only act on the topmost item on the stack. A stack data structure is also called a Last-In-First-Out, or "LIFO" queue.
-
-The scripting language executes the script by processing each item from left to right. Numbers (data constants) are pushed onto the stack. Operators push or pop one or more parameters from the stack, act on them, and might push a result onto the stack. For example, +OP_ADD+ will pop two items from the stack, add them, and push the resulting sum onto the stack.
-
-Conditional operators evaluate a condition, producing a boolean result of TRUE or FALSE. For example, +OP_EQUAL+ pops two items from the stack and pushes TRUE (TRUE is represented by the number 1) if they are equal or FALSE (represented by zero) if they are not equal. Bitcoin transaction scripts usually contain a conditional operator, so that they can produce the TRUE result that signifies a valid transaction.
-
-===== A simple script
-
-Now let's apply what we've learned about scripts and stacks to some simple examples.
-
-In <<simplemath_script>>, the script +2 3 OP_ADD 5 OP_EQUAL+ demonstrates the arithmetic addition operator +OP_ADD+, adding two numbers and putting the result on the stack, followed by the conditional operator +OP_EQUAL+, which checks that the resulting sum is equal to +5+. For brevity, the +OP_+ prefix is omitted in the step-by-step example. For more details on the available script operators and functions, see <<tx_script_ops>>.
-
-Although most locking scripts refer to a public key hash (essentially, a Bitcoin address), thereby requiring proof of ownership to spend the funds, the script does not have to be that complex. Any combination of locking and unlocking scripts that results in a TRUE value is valid. The simple arithmetic we used as an example of the scripting language is also a valid locking script that can be used to lock a transaction output.
-
-Use part of the arithmetic example script as the locking script:
-
-----
-3 OP_ADD 5 OP_EQUAL
-----
-
-which can be satisfied by a transaction containing an input with the unlocking script:
-
-----
-2
-----
-
-The validation software combines the locking and unlocking scripts and the resulting script is:
-
-----
-2 3 OP_ADD 5 OP_EQUAL
-----
-
-As we saw in the step-by-step example in <<simplemath_script>>, when this script is executed, the result is +OP_TRUE+, making the transaction valid. Not only is this a valid transaction output locking script, but the resulting UTXO could be spent by anyone with the arithmetic skills to know that the number 2 satisfies the script.
-
-[TIP]
-====
-((("transactions", "valid and invalid")))Transactions are valid if the top result on the stack is +TRUE+ (noted as ++&#x7b;0x01&#x7d;++), any other nonzero value, not +OP_0+, or if the stack is empty after script execution. Transactions are invalid if the top value on the stack is +FALSE+ (a zero-length empty value, noted as ++&#x7b;&#x7d;++) or if script execution is halted explicitly by an operator, such as +OP_VERIFY+, +OP_RETURN+, or a conditional terminator such as +OP_ENDIF+. See <<tx_script_ops>> for details.
-====
-
-[[simplemath_script]]
-.Bitcoin's script validation doing simple math
-image::images/mbc2_0604.png["TxScriptSimpleMathExample"]
-
-[role="pagebreak-before"]
-The following is a slightly more complex script, which calculates ++2 + 7 - 3 + 1++. Notice that when the script contains several operators in a row, the stack allows the results of one operator to be acted upon by the next operator:
-
-----
-2 7 OP_ADD 3 OP_SUB 1 OP_ADD 7 OP_EQUAL
-----
-
-Try validating the preceding script yourself using pencil and paper. When the script execution ends, you should be left with the value +TRUE+ on the stack.
-
-[[script_exec]]
-===== Separate execution of unlocking and locking scripts
-
-((("security", "locking and unlocking scripts")))In the original Bitcoin client, the unlocking and locking scripts were concatenated and executed in sequence. For security reasons, this was changed in 2010, because of a vulnerability that allowed a malformed unlocking script to push data onto the stack and corrupt the locking script. In the current implementation, the scripts are executed separately with the stack transferred between the two executions, as described next.
-
-First, the unlocking script is executed, using the stack execution engine. If the unlocking script is executed without errors (e.g., it has no "dangling" pointers left over), the main stack is copied and the locking script is executed. If the result of executing the locking script with the stack data copied from the unlocking script is "TRUE," the unlocking script has succeeded in resolving the conditions imposed by the locking script and, therefore, the input is a valid authorization to spend the UTXO. If any result other than "TRUE" remains after execution of the combined script, the input is invalid because it has failed to satisfy the spending conditions placed on the UTXO.
-
-
-[[p2pkh]]
-==== Pay-to-Public-Key-Hash (P2PKH)
-
-((("Pay-to-Public-Key-Hash (P2PKH)")))The vast majority of transactions processed on the Bitcoin network spend outputs locked with a Pay-to-Public-Key-Hash or "P2PKH" script. These outputs contain a locking script that locks the output to a public key hash, more commonly known as a Bitcoin address. An output locked by a P2PKH script can be unlocked (spent) by presenting a public key and a digital signature created by the corresponding private key (see <<digital_sigs>>).
-
-((("use cases", "buying coffee")))For example, let's look at Alice's payment to Bob's Cafe again. Alice made a payment of 0.015 bitcoin to the cafe's Bitcoin address. That transaction output would have a locking script of the form:
-
-----
-OP_DUP OP_HASH160 <Cafe Public Key Hash> OP_EQUALVERIFY OP_CHECKSIG
-----
-
-The +Cafe Public Key Hash+ is equivalent to the Bitcoin address of the cafe, without the Base58Check encoding. Most applications would show the _public key hash_ in hexadecimal encoding and not the familiar Bitcoin address Base58Check format that begins with a "1."
-
-The preceding locking script can be satisfied with an unlocking script of the form:
-
-----
-<Cafe Signature> <Cafe Public Key>
-----
-
-The two scripts together would form the following combined validation script:
-
-----
-<Cafe Signature> <Cafe Public Key> OP_DUP OP_HASH160
-<Cafe Public Key Hash> OP_EQUALVERIFY OP_CHECKSIG
-----
-
-When executed, this combined script will evaluate to TRUE if, and only if, the unlocking script matches the conditions set by the locking script. In other words, the result will be TRUE if the unlocking script has a valid signature from the cafe's private key that corresponds to the public key hash set as an encumbrance.
-
-Figures pass:[<a data-type="xref" href="#P2PubKHash1" data-xrefstyle="select: labelnumber">#P2PubKHash1</a>] and pass:[<a data-type="xref" href="#P2PubKHash2" data-xrefstyle="select: labelnumber">#P2PubKHash2</a>] show (in two parts) a step-by-step execution of the combined script, which will prove this is a valid transaction.((("", startref="Tsript06")))((("", startref="Stransact06")))
-
-[[P2PubKHash1]]
-.Evaluating a script for a P2PKH transaction (part 1 of 2)
-image::images/mbc2_0605.png["Tx_Script_P2PubKeyHash_1"]
-
-[[P2PubKHash2]]
-.Evaluating a script for a P2PKH transaction (part 2 of 2)
-image::images/mbc2_0606.png["Tx_Script_P2PubKeyHash_2"]
-
-[[digital_sigs]]
-=== Digital Signatures (ECDSA)
-
-((("transactions", "digital signatures and", id="Tdigsig06")))So far, we have not delved into any detail about "digital signatures." In this section we look at how digital signatures work and how they can present proof of ownership of a private key without revealing that private key.
-
-((("digital signatures", "algorithm used")))((("Elliptic Curve Digital Signature Algorithm (ECDSA)")))The digital signature algorithm used in bitcoin is the _Elliptic Curve Digital Signature Algorithm_, or _ECDSA_. ECDSA is the algorithm used for digital signatures based on elliptic curve private/public key pairs, as described in <<elliptic_curve>>. ECDSA is used by the script functions +OP_CHECKSIG+, +OP_CHECKSIGVERIFY+, +OP_CHECKMULTISIG+, and +OP_CHECKMULTISIGVERIFY+. Any time you see those in a locking script, the unlocking script must contain an ECDSA signature.
-
-((("digital signatures", "purposes of")))A digital signature serves three purposes in bitcoin. First, the signature proves that the owner of the private key, who is by implication the owner of the funds, has _authorized_ the spending of those funds. Secondly, the proof of authorization is _undeniable_ (nonrepudiation). Thirdly, the signature proves that the transaction (or specific parts of the transaction) have not and _cannot be modified_ by anyone after it has been signed.
-
-Note that each transaction input is signed independently. This is critical, as neither the signatures nor the inputs have to belong to or be applied by the same "owners." In fact, a specific transaction scheme called "CoinJoin" uses this fact to create multi-party transactions for privacy.
-
-[NOTE]
-====
-Each transaction input and any signature it may contain is _completely_ independent of any other input or signature. Multiple parties can collaborate to construct transactions and sign only one input each.
-====
-
-[[digital_signature_definition]]
-.Wikipedia's Definition of a "Digital Signature"
-****
-((("digital signatures", "defined")))A digital signature is a mathematical scheme for demonstrating the authenticity of a digital message or documents. A valid digital signature gives a recipient reason to believe that the message was created by a known sender (authentication), that the sender cannot deny having sent the message (nonrepudiation), and that the message was not altered in transit (integrity).
-
-_Source: https://en.wikipedia.org/wiki/Digital_signature_
-****
-
-==== How Digital Signatures Work
-
-((("digital signatures", "how they work")))A digital signature is a _mathematical scheme_ that consists of two parts. The first part is an algorithm for creating a signature, using a private key (the signing key), from a message (the transaction). The second part is an algorithm that allows anyone to verify the signature, given also the message and a public key.
-
-===== Creating a digital signature
-
-In bitcoin's implementation of the ECDSA algorithm, the "message" being signed is the transaction, or more accurately a hash of a specific subset of the data in the transaction (see <<sighash_types>>). The signing key is the user's private key. The result is the signature:
-
-latexmath:[\(Sig = F_{sig}(F_{hash}(m), dA)\)]
-
-where:
-
-* _dA_ is the signing private key
-* _m_ is the transaction (or parts of it)
-* _F_~_hash_~ is the hashing function
-* _F_~_sig_~ is the signing algorithm
-* _Sig_ is the resulting signature
-
-More details on the mathematics of ECDSA can be found in <<ecdsa_math>>.
-
-The function _F_~_sig_~ produces a signature +Sig+ that is composed of two values, commonly referred to as +R+ and +S+:
-
-----
-Sig = (R, S)
-----
-
-((("Distinguished Encoding Rules (DER)")))Now that the two values +R+ and +S+ have been calculated, they are serialized into a byte-stream using an international standard encoding scheme called the _Distinguished Encoding Rules_, or _DER_.
-
-[[seralization_of_signatures_der]]
-===== Serialization of signatures (DER)
-
-Let's look at the transaction Alice ((("use cases", "buying coffee", id="alicesixtwo")))created again. In the transaction input there is an unlocking script that contains the following DER-encoded signature from Alice's wallet:
-
-----
-3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e381301
-----
-
-That signature is a serialized byte-stream of the +R+ and +S+ values produced by Alice's wallet to prove she owns the private key authorized to spend that output. The serialization format consists of nine elements as follows:
-
-* +0x30+&#x2014;indicating the start of a DER sequence
-* +0x45+&#x2014;the length of the sequence (69 bytes)
-  * +0x02+&#x2014;an integer value follows
-  * +0x21+&#x2014;the length of the integer (33 bytes)
-  * +R+&#x2014;++00884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb++
-  * +0x02+&#x2014;another integer follows
-  * +0x20+&#x2014;the length of the integer (32 bytes)
-  * +S+&#x2014;++4b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813++
-* A suffix (+0x01+) indicating the type of hash used (+SIGHASH_ALL+)
-
-See if you can decode Alice's serialized (DER-encoded) signature using this list. The important numbers are +R+ and +S+; the rest of the data is part of the DER encoding scheme.
-
-==== Verifying the Signature
-
-((("digital signatures", "verifying")))To verify the signature, one must have the signature (+R+ and +S+), the serialized transaction, and the public key (that corresponds to the private key used to create the signature). Essentially, verification of a signature means "Only the owner of the private key that generated this public key could have produced this signature on this transaction."
-
-The signature verification algorithm takes the message (a hash of the transaction or parts of it), the signer's public key and the signature (+R+ and +S+ values), and returns TRUE if the signature is valid for this message and public key.
-
-[[sighash_types]]
-==== Signature Hash Types (SIGHASH)
-
-((("digital signatures", "signature hash types")))((("commitment")))Digital signatures are applied to messages, which in the case of bitcoin, are the transactions themselves. The signature implies a _commitment_ by the signer to specific transaction data. In the simplest form, the signature applies to the entire transaction, thereby committing all the inputs, outputs, and other transaction fields. However, a signature can commit to only a subset of the data in a transaction, which is useful for a number of scenarios as we will see in this section.
-
-((("SIGHASH flags")))Bitcoin signatures have a way of indicating which part of a transaction's data is included in the hash signed by the private key using a +SIGHASH+ flag. The +SIGHASH+ flag is a single byte that is appended to the signature. Every signature has a +SIGHASH+ flag and the flag can be different from input to input. A transaction with three signed inputs may have three signatures with different +SIGHASH+ flags, each signature signing (committing) different parts of the transaction.
-
-Remember, each input may contain a signature in its unlocking script. As a result, a transaction that contains several inputs may have signatures with different +SIGHASH+ flags that commit different parts of the transaction in each of the inputs. Note also that bitcoin transactions may contain inputs from different "owners," who may sign only one input in a partially constructed (and invalid) transaction, collaborating with others to gather all the necessary signatures to make a valid transaction. Many of the +SIGHASH+ flag types only make sense if you think of multiple participants collaborating outside the Bitcoin network and updating a partially signed transaction.
-
-[role="pagebreak-before"]
-There are three +SIGHASH+ flags: +ALL+, +NONE+, and +SINGLE+, as shown in <<sighash_types_and_their>>.
-
-[[sighash_types_and_their]]
-.SIGHASH types and their meanings
-[options="header"]
-|=======================
-|+SIGHASH+ flag| Value | Description
-| +ALL+ | 0x01 | Signature applies to all inputs and outputs
-| +NONE+ | 0x02 | Signature applies to all inputs, none of the outputs
-| +SINGLE+ | 0x03 | Signature applies to all inputs but only the one output with the same index number as the signed input
-|=======================
-
-In addition, there is a modifier flag +SIGHASH_ANYONECANPAY+, which can be combined with each of the preceding flags. When +ANYONECANPAY+ is set, only one input is signed, leaving the rest (and their sequence numbers) open for modification. The +ANYONECANPAY+ has the value +0x80+ and is applied by bitwise OR, resulting in the combined flags as shown in <<sighash_types_with_modifiers>>.
-
-[[sighash_types_with_modifiers]]
-.SIGHASH types with modifiers and their meanings
-[options="header"]
-|=======================
-|SIGHASH flag| Value | Description
-| ALL\|ANYONECANPAY | 0x81 | Signature applies to one input and all outputs
-| NONE\|ANYONECANPAY | 0x82 | Signature applies to one input, none of the outputs
-| SINGLE\|ANYONECANPAY | 0x83 | Signature applies to one input and the output with the same index number
-|=======================
-
-These flag combinations are summarized in <<sighash_combinations>>.
-
-[[sighash_combinations]]
-.Summary of different sighash combinations
-image::images/sighash_combinations.png["Summary of different SIGHASH flag combinations"]
-
-The way +SIGHASH+ flags are applied during signing and verification is that a copy of the transaction is made and certain fields within are truncated (set to zero length and emptied). The resulting transaction is serialized. The +SIGHASH+ flag is added to the end of the serialized transaction and the result is hashed. The hash itself is the "message" that is signed. Depending on which +SIGHASH+ flag is used, different parts of the transaction are truncated. The resulting hash depends on different subsets of the data in the transaction. By including the +SIGHASH+ as the last step before hashing, the signature commits the +SIGHASH+ type as well, so it can't be changed (e.g., by a miner).
-
-[NOTE]
-====
-All +SIGHASH+ types sign the transaction +nLocktime+ field (see <<transaction_locktime_nlocktime>>). In addition, the +SIGHASH+ type itself is appended to the transaction before it is signed, so that it can't be modified once signed.
-====
-
-In the example of Alice's transaction (see the list in <<seralization_of_signatures_der>>), we saw that the last part of the DER-encoded signature was +01+, which is the +SIGHASH_ALL+ flag. This locks the transaction data, so Alice's signature is committing the state of all inputs and outputs. This is the most common signature form.
-
-Let's look at some of the other +SIGHASH+ types and how they can be used in practice:
-
-+ALL|ANYONECANPAY+ :: ((("charitable donations")))((("use cases", "charitable donations")))This construction can be used to make a "crowdfunding&#x201d;-style transaction. Someone attempting to raise funds can construct a transaction with a single output. The single output pays the "goal" amount to the fundraiser. Such a transaction is obviously not valid, as it has no inputs. However, others can now amend it by adding an input of their own, as a donation. They sign their own input with +ALL|ANYONECANPAY+. Unless enough inputs are gathered to reach the value of the output, the transaction is invalid. Each donation is a "pledge," which cannot be collected by the fundraiser until the entire goal amount is raised.
-
-+NONE+ :: This construction can be used to create a "bearer check" or "blank check" of a specific amount. It commits to the input, but allows the output locking script to be changed. Anyone can write their own Bitcoin address into the output locking script and redeem the transaction. However, the output value itself is locked by the signature.
-
-+NONE|ANYONECANPAY+ :: This construction can be used to build a "dust collector." Users who have tiny UTXO in their wallets can't spend these because the cost in fees exceeds the value of the dust. With this type of signature, the dust UTXO can be donated for anyone to aggregate and spend whenever they want.
-
-((("Bitmask Sighash Modes")))There are some proposals to modify or expand the +SIGHASH+ system. One such proposal is _Bitmask Sighash Modes_ by Blockstream's Glenn Willen, as part of the Elements project. This aims to create a flexible replacement for +SIGHASH+ types that allows "arbitrary, miner-rewritable bitmasks of inputs and outputs" that can express "more complex contractual precommitment schemes, such as signed offers with change in a distributed asset exchange."
-
-[NOTE]
-====
-You will not see +SIGHASH+ flags presented as an option in a user's wallet application. With few exceptions, wallets construct P2PKH scripts and sign with +SIGHASH_ALL+ flags. To use a different +SIGHASH+ flag, you would have to write software to construct and sign transactions. More importantly, +SIGHASH+ flags can be used by special-purpose bitcoin applications that enable novel uses.
-====
-
-[[ecdsa_math]]
-==== ECDSA Math
-
-((("Elliptic Curve Digital Signature Algorithm (ECDSA)")))As mentioned previously, signatures are created by a mathematical function _F_~_sig_~ that produces a signature composed of two values _R_ and _S_. In this section we look at the function _F_~_sig_~ in more detail.
-
-((("public and private keys", "key pairs", "ephemeral")))The signature algorithm first generates an _ephemeral_ (temporary) private public key pair. This temporary key pair is used in the calculation of the _R_ and _S_ values, after a transformation involving the signing private key and the transaction hash.
-
-The temporary key pair is based on a random number _k_, which is used as the temporary private key. From _k_, we generate the corresponding temporary public key _P_ (calculated as _P = k*G_, in the same way bitcoin public keys are derived; see <<pubkey>>). The _R_ value of the digital signature is then the x coordinate of the ephemeral public key _P_.
-
-From there, the algorithm calculates the _S_ value of the signature, such that:
-
-_S_ = __k__^-1^ (__Hash__(__m__) + __dA__ * __R__) _mod n_
-
-where:
-
-* _k_ is the ephemeral private key
-* _R_ is the x coordinate of the ephemeral public key
-* _dA_ is the signing private key
-* _m_ is the transaction data
-* _n_ is the prime order of the elliptic curve
-
-Verification is the inverse of the signature generation function, using the _R_, _S_ values and the public key to calculate a value _P_, which is a point on the elliptic curve (the ephemeral public key used in signature creation):
-
-_P_ = __S__^-1^ * __Hash__(__m__) * _G_ + __S__^-1^ * _R_ * _Qa_
-
-where:
-
-* _R_ and _S_ are the signature values
-* _Qa_ is Alice's public key
-* _m_ is the transaction data that was signed
-* _G_ is the elliptic curve generator point
-
-If the x coordinate of the calculated point _P_ is equal to _R_, then the verifier can conclude that the signature is valid.
-
-Note that in verifying the signature, the private key is neither known nor revealed.
-
-[TIP]
-====
-ECDSA is necessarily a fairly complicated piece of math; a full explanation is beyond the scope of this book. A number of great guides online take you through it step by step: search for "ECDSA explained" or try this one: https://bit.ly/2r0HhGB[].
-====
-
-==== The Importance of Randomness in Signatures
-
-((("digital signatures", "randomness in")))As we saw in <<ecdsa_math>>, the signature generation algorithm uses a random key _k_, as the basis for an ephemeral private/public key pair. The value of _k_ is not important, _as long as it is random_. If the same value _k_ is used to produce two signatures on different messages (transactions), then the signing _private key_ can be calculated by anyone. Reuse of the same value for _k_ in a signature algorithm leads to exposure of the private key!
-
-[WARNING]
-====
-((("warnings and cautions", "digital signatures")))If the same value _k_ is used in the signing algorithm on two different transactions, the private key can be calculated and exposed to the world!
-====
-
-This is not just a theoretical possibility. We have seen this issue lead to exposure of private keys in a few different implementations of transaction-signing algorithms in bitcoin. People have had funds stolen because of inadvertent reuse of a _k_ value. The most common reason for reuse of a _k_ value is an improperly initialized random-number generator.
-
-((("random numbers", "random number generation")))((("entropy", "random number generation")))((("deterministic initialization")))To avoid this vulnerability, the industry best practice is to not generate _k_ with a random-number generator seeded with entropy, but instead to use a deterministic-random process seeded with the transaction data itself. This ensures that each transaction produces a different _k_. The industry-standard algorithm for deterministic initialization of _k_ is defined in https://tools.ietf.org/html/rfc6979[RFC 6979], published by the Internet Engineering Task Force.
-
-If you are implementing an algorithm to sign transactions in bitcoin, you _must_ use RFC 6979 or a similarly deterministic-random algorithm to ensure you generate a different _k_ for each transaction.((("", startref="Tdigsig06")))
-
-=== Bitcoin Addresses, Balances, and Other Abstractions
-
-((("transactions", "higher-level abstractions", id="Thigher06")))We began this chapter with the discovery that transactions look very different "behind the scenes" than how they are presented in wallets, blockchain explorers, and other user-facing applications. Many of the simplistic and familiar concepts from the earlier chapters, such as Bitcoin addresses and balances, seem to be absent from the transaction structure. We saw that transactions don't contain Bitcoin addresses, per se, but instead operate through scripts that lock and unlock discrete values of bitcoin. Balances are not present anywhere in this system and yet every wallet application prominently displays the balance of the user's wallet.
-
-Now that we have explored what is actually included in a bitcoin transaction, we can examine how the higher-level abstractions are derived from the seemingly primitive components of the transaction.
-
-Let's look again at how Alice's transaction was presented on a popular block explorer (<<alice_transaction_to_bobs_cafe>>).
-
-[[alice_transaction_to_bobs_cafe]]
-.Alice's transaction to Bob's Cafe
-image::images/mbc2_0208.png["Alice Coffee Transaction"]
-
-On the left side of the transaction, the blockchain explorer shows Alice's Bitcoin address as the "sender." In fact, this information is not in the transaction itself. When the blockchain explorer references the transaction it also references the previous transaction associated with the input and extracts the first output from that older transaction. Within that output is a locking script that locks the UTXO to Alice's public key hash (a P2PKH script). The blockchain explorer extracted the public key hash and encoded it using Base58Check encoding to produce and display the Bitcoin address that represents that public key.
-
-Similarly, on the right side, the blockchain explorer shows the two outputs; the first to Bob's Bitcoin address and the second to Alice's Bitcoin address (as change). Once again, to create these Bitcoin addresses, the blockchain explorer extracted the locking script from each output, recognized it as a P2PKH script, and extracted the public-key-hash from within. Finally, the blockchain explorer reencoded each public key hash with Base58Check to produce and display the Bitcoin addresses.
-
-If you were to click on Bob's Bitcoin address, the blockchain explorer would show you the view in <<the_balance_of_bobs_bitcoin_address>>.
-
-[[the_balance_of_bobs_bitcoin_address]]
-.The balance of Bob's Bitcoin address
-image::images/mbc2_0608.png["The balance of Bob's Bitcoin address"]
-
-The blockchain explorer displays the balance of Bob's Bitcoin address. But nowhere in the Bitcoin system is there a concept of a "balance." Rather, the values displayed here are constructed by the blockchain explorer as follows.
-
-To construct the "Total Received" amount, the blockchain explorer first will decode the Base58Check encoding of the Bitcoin address to retrieve the 160-bit hash of Bob's public key that is encoded within the address. Then, the blockchain explorer will search through the database of transactions, looking for outputs with P2PKH locking scripts that contain Bob's public key hash. By summing up the value of all the outputs, the blockchain explorer can produce the total value received.
-
-Constructing the current balance (displayed as "Final Balance") requires a bit more work. The blockchain explorer keeps a separate database of the outputs that are currently unspent, the UTXO set. To maintain this database, the blockchain explorer must monitor the Bitcoin network, add newly created UTXO, and remove spent UTXO, in real time, as they appear in unconfirmed transactions. This is a complicated process that depends on keeping track of transactions as they propagate, as well as maintaining consensus with the Bitcoin network to ensure that the correct chain is followed. Sometimes, the blockchain explorer goes out of sync and its perspective of the UTXO set is incomplete or incorrect.
-
-From the UTXO set, the blockchain explorer sums up the value of all unspent outputs referencing Bob's public key hash and produces the "Final Balance" number shown to the user.
-
-In order to produce this one image, with these two "balances," the blockchain explorer has to index and search through dozens, hundreds, or even hundreds of thousands of transactions.
-
-In summary, the information presented to users through wallet applications, blockchain explorers, and other bitcoin user interfaces is often composed of higher-level abstractions that are derived by searching many different transactions, inspecting their content, and manipulating the data contained within them. By presenting this simplistic view of bitcoin transactions that resemble bank checks from one sender to one recipient, these applications have to abstract a lot of underlying details. They mostly focus on the common types of transactions: P2PKH with SIGHASH_ALL signatures on every input. Thus, while bitcoin applications can present more than 80% of all transactions in an easy-to-read manner, they are sometimes stumped by transactions that deviate from the norm. Transactions that contain more complex locking scripts, or different SIGHASH flags, or many inputs and outputs, demonstrate the simplicity and weakness of these abstractions.
-
-Every day, hundreds of transactions that do not contain P2PKH outputs are confirmed on the blockchain. The blockchain explorers often present these with red warning messages saying they cannot decode an address.
-
-As we will see in the next chapter, these are not necessarily strange transactions. They are transactions that contain more complex locking scripts than the common P2PKH. We will learn how to decode and understand more complex scripts and the applications they support next.((("", startref="Thigher06")))((("", startref="alicesixtwo")))

ch08.asciidoc

@@ -1,336 +0,0 @@
-[[bitcoin_network_ch08]]
-== The Bitcoin Network
-
-=== Peer-to-Peer Network Architecture
-
-((("Bitcoin network", "peer-to-peer architecture")))((("peer-to-peer (P2P)")))Bitcoin is structured as a peer-to-peer network architecture on top of the internet. The term peer-to-peer, or P2P, means that the computers that participate in the network are peers to each other, that they are all equal, that there are no "special" nodes, and that all nodes share the burden of providing network services. The network nodes interconnect in a mesh network with a "flat" topology. There is no server, no centralized service, and no hierarchy within the network. Nodes in a P2P network both provide and consume services at the same time with reciprocity acting as the incentive for participation. P2P networks are inherently resilient, decentralized, and open. A preeminent example of a P2P network architecture was the early internet itself, where nodes on the IP network were equal. Today's internet architecture is more hierarchical, but the Internet Protocol still retains its flat-topology essence. Beyond bitcoin, the largest and most successful application of P2P technologies is file sharing, with Napster as the pioneer and BitTorrent as the most recent evolution of the architecture.
-
-Bitcoin's P2P network architecture is much more than a topology choice. Bitcoin is a P2P digital cash system by design, and the network architecture is both a reflection and a foundation of that core characteristic. Decentralization of control is a core design principle that can only be achieved and maintained by a flat, decentralized P2P consensus network.
-
-((("Bitcoin network", "defined")))The term "Bitcoin network" refers to the collection of nodes running the bitcoin P2P protocol. In addition to the bitcoin P2P protocol, there are other protocols such as Stratum that are used for mining and lightweight or mobile wallets. These additional protocols are provided by gateway routing servers that access the Bitcoin network using the bitcoin P2P protocol and then extend that network to nodes running other protocols. For example, Stratum servers connect Stratum mining nodes via the Stratum protocol to the main Bitcoin network and bridge the Stratum protocol to the bitcoin P2P protocol. We use the term "extended Bitcoin network" to refer to the overall network that includes the bitcoin P2P protocol, pool-mining protocols, the Stratum protocol, and any other related protocols connecting the components of the Bitcoin system.
-
-=== Node Types and Roles
-
-((("Bitcoin network", "node types and roles", id="BNnode08")))((("Bitcoin nodes", "types and roles", id="BNtype08")))Although nodes in the bitcoin P2P network are equal, they may take on different roles depending on the functionality they are supporting. A Bitcoin node is a collection of functions: routing, the blockchain database, mining, and wallet services. A full node with all four of these functions is shown in <<full_node_reference>>.
-
-[[full_node_reference]]
-[role="smallerfifty"]
-.A Bitcoin network node with all four functions: wallet, miner, full blockchain database, and network routing
-image::images/mbc2_0801.png["FullNodeReferenceClient_Small"]
-
-All nodes include the routing function to participate in the network and might include other functionality. All nodes validate and propagate transactions and blocks, and discover and maintain connections to peers. In the full-node example in <<full_node_reference>>, the routing function is indicated by a circle named "Network Routing Node" or with the letter "N."
-
-((("full-node clients")))Some nodes, called full nodes, also maintain a complete and up-to-date copy of the blockchain. Full nodes can autonomously and authoritatively verify any transaction without external reference. ((("simplified-payment-verification (SPV)")))Some nodes maintain only a subset of the blockchain and verify transactions using a method called _simplified payment verification_, or SPV. ((("lightweight clients")))These nodes are known as SPV nodes or lightweight nodes. In the full-node example in the figure, the full-node blockchain database function is indicated by a circle called "Full Blockchain" or the letter "B." In <<bitcoin_network>>, SPV nodes are drawn without the "B" circle, showing that they do not have a full copy of the blockchain.
-
-((("Bitcoin nodes", "mining nodes")))((("mining and consensus", "mining nodes")))((("Proof-of-Work algorithm")))((("mining and consensus", "Proof-of-Work algorithm")))Mining nodes compete to create new blocks by running specialized hardware to solve the Proof-of-Work algorithm. Some mining nodes are also full nodes, maintaining a full copy of the blockchain, while others are lightweight nodes participating in pool mining and depending on a pool server to maintain a full node. The mining function is shown in the full node as a circle called "Miner" or the letter "M."
-
-User wallets might be part of a full node, as is usually the case with desktop Bitcoin clients. Increasingly, many user wallets, especially those running on resource-constrained devices such as smartphones, are SPV nodes. The wallet function is shown in <<full_node_reference>> as a circle called "Wallet" or the letter "W."
-
-In addition to the main node types on the bitcoin P2P protocol, there are servers and nodes running other protocols, such as specialized mining pool protocols and lightweight client-access protocols.
-
-<<node_type_ledgend>> shows the most common node types on the extended Bitcoin network.
-
-=== The Extended Bitcoin Network
-
-((("", startref="BNnode08")))((("", startref="BNtype08")))((("Bitcoin network", "extended network activities")))The main Bitcoin network, running the bitcoin P2P protocol, consists of between 5,000 and 8,000 listening nodes running various versions of the bitcoin reference client (Bitcoin Core) and a few hundred nodes running various other implementations of the bitcoin P2P protocol, such as Bitcoin Classic, Bitcoin Unlimited, BitcoinJ, Libbitcoin, btcd, and bcoin. A small percentage of the nodes on the bitcoin P2P network are also mining nodes, competing in the mining process, validating transactions, and creating new blocks. Various large companies interface with the Bitcoin network by running full-node clients based on the Bitcoin Core client, with full copies of the blockchain and a network node, but without mining or wallet functions. These nodes act as network edge routers, allowing various other services (exchanges, wallets, block explorers, merchant payment processing) to be built on top.
-
-The extended Bitcoin network includes the network running the bitcoin P2P protocol, described earlier, as well as nodes running specialized protocols. Attached to the main bitcoin P2P network are a number of pool servers and protocol gateways that connect nodes running other protocols. These other protocol nodes are mostly pool mining nodes (see <<mining>>) and lightweight wallet clients, which do not carry a full copy of the blockchain.
-
-<<bitcoin_network>> shows the extended Bitcoin network with the various types of nodes, gateway servers, edge routers, and wallet clients and the various protocols they use to connect to each other.
-
-[[node_type_ledgend]]
-.Different types of nodes on the extended Bitcoin network
-image::images/mbc2_0802.png["BitcoinNodeTypes"]
-
-[[bitcoin_network]]
-.The extended Bitcoin network showing various node types, gateways, and protocols
-image::images/mbc2_0803.png["BitcoinNetwork"]
-
-=== Bitcoin Relay Networks
-
-((("Bitcoin network", "Bitcoin Relay Networks")))((("relay networks")))While the bitcoin P2P network serves the general needs of a broad variety of node types, it exhibits too high network latency for the specialized needs of bitcoin mining nodes.
-
-((("propagation", "relay networks and")))Bitcoin miners are engaged in a time-sensitive competition to solve the Proof-of-Work problem and extend the blockchain (see <<mining>>). While participating in this competition, bitcoin miners must minimize the time between the propagation of a winning block and the beginning of the next round of competition. In mining, network latency is directly related to profit margins.
-
-A _Bitcoin Relay Network_ is a network that attempts to minimize the latency in the transmission of blocks between miners. The original https://www.bitcoinrelaynetwork.org[Bitcoin Relay Network] was created by core developer Matt Corallo in 2015 to enable fast synchronization of blocks between miners with very low latency. The network consisted of several specialized nodes hosted on the Amazon Web Services infrastructure around the world and served to connect the majority of miners and mining pools.
-
-((("Fast Internet Bitcoin Relay Engine (FIBRE)")))((("Compact Block optimization")))The original Bitcoin Relay Network was replaced in 2016 with the introduction of the _Fast Internet Bitcoin Relay Engine_ or https://bitcoinfibre.org[_FIBRE_], also created by core developer Matt Corallo. FIBRE is a UDP-based relay network that relays blocks within a network of nodes. FIBRE implements _compact block_ optimization to further reduce the amount of data transmitted and the network latency.
-
-Relay networks are not replacements for bitcoin's P2P network. Instead they are overlay networks that provide additional connectivity between nodes with specialized needs. Like freeways are not replacements for rural roads, but rather shortcuts between two points with heavy traffic, you still need small roads to connect to the freeways.
-
-=== Network Discovery
-
-((("Bitcoin network", "extended network discovery", id="BNextend08")))((("Bitcoin nodes", "network discovery", id="BNodiscover08")))When a new node boots up, it must discover other Bitcoin nodes on the network in order to participate. To start this process, a new node must discover at least one existing node on the network and connect to it. The geographic location of other nodes is irrelevant; the Bitcoin network topology is not geographically defined. Therefore, any existing Bitcoin nodes can be selected at random.
-
-To connect to a known peer, nodes establish a TCP connection, usually to port 8333 (the port generally known as the one used by bitcoin), or an alternative port if one is provided. Upon establishing a connection, the node will start a "handshake" (see <<network_handshake>>) by transmitting a  +version+ message, which contains basic identifying information, including:
-
-+nVersion+:: The bitcoin P2P protocol version the client "speaks" (e.g., 70002)
-+nLocalServices+:: A list of local services supported by the node, currently just +NODE_NETWORK+
-+nTime+:: The current time
-+addrYou+:: The IP address of the remote node as seen from this node
-+addrMe+:: The IP address of the local node, as discovered by the local node
-+subver+:: A sub-version showing the type of software running on this node (e.g., pass:[<span class="keep-together"><code>/Satoshi:0.9.2.1/</code></span>])
-+BestHeight+:: The block height of this node's blockchain
-
-(See https://bit.ly/1qlsC7w[GitHub] for an example of the +version+ network message.)
-
-The +version+ message is always the first message sent by any peer to another peer. The local peer receiving a +version+ message will examine the remote peer's reported +nVersion+ and decide if the remote peer is compatible. If the remote peer is compatible, the local peer will acknowledge the +version+ message and establish a connection by sending a +verack+ message.
-
-How does a new node find peers? The first method is to query DNS using a number of "DNS seeds," which are DNS servers that provide a list of IP addresses of Bitcoin nodes. Some of those DNS seeds provide a static list of IP addresses of stable bitcoin listening nodes. Some of the DNS seeds are custom implementations of BIND (Berkeley Internet Name Daemon) that return a random subset from a list of Bitcoin node addresses collected by a crawler or a long-running Bitcoin node.  The Bitcoin Core client contains the names of nine different DNS seeds. The diversity of ownership and diversity of implementation of the different DNS seeds offers a high level of reliability for the initial bootstrapping process. In the Bitcoin Core client, the option to use the DNS seeds is controlled by the option switch +-dnsseed+ (set to 1 by default, to use the DNS seed).
-
-Alternatively, a bootstrapping node that knows nothing of the network must be given the IP address of at least one Bitcoin node, after which it can establish connections through further introductions. The command-line argument +-seednode+ can be used to connect to one node just for introductions using it as a seed. After the initial seed node is used to form introductions, the client will disconnect from it and use the newly discovered peers.
-
-[[network_handshake]]
-.The initial handshake between peers
-image::images/mbc2_0804.png["NetworkHandshake"]
-
-Once one or more connections are established, the new node will send an +addr+ message containing its own IP address to its neighbors. The neighbors will, in turn, forward the +addr+ message to their neighbors, ensuring that the newly connected node becomes well known and better connected. Additionally, the newly connected node can send +getaddr+ to the neighbors, asking them to return a list of IP addresses of other peers. That way, a node can find peers to connect to and advertise its existence on the network for other nodes to find it. <<address_propagation>> ((("propagation", "address propagation and discovery")))shows the address discovery protocol.
-
-
-[[address_propagation]]
-.Address propagation and discovery
-image::images/mbc2_0805.png["AddressPropagation"]
-
-A node must connect to a few different peers in order to establish diverse paths into the Bitcoin network. Paths are not persistent—nodes come and go—and so the node must continue to discover new nodes as it loses old connections as well as assist other nodes when they bootstrap. Only one connection is needed to bootstrap, because the first node can offer introductions to its peer nodes and those peers can offer further introductions. It's also unnecessary and wasteful of network resources to connect to more than a handful of nodes. After bootstrapping, a node will remember its most recent successful peer connections, so that if it is rebooted it can quickly reestablish connections with its former peer network. If none of the former peers respond to its connection request, the node can use the seed nodes to bootstrap again.
-
-On a node running the Bitcoin Core client, you can list the peer connections with the command +getpeerinfo+:
-
-[source,bash]
-----
-$ bitcoin-cli getpeerinfo
-----
-[source,json]
-----
-[
-    {
-        "addr" : "85.213.199.39:8333",
-        "services" : "00000001",
-        "lastsend" : 1405634126,
-        "lastrecv" : 1405634127,
-        "bytessent" : 23487651,
-        "bytesrecv" : 138679099,
-        "conntime" : 1405021768,
-        "pingtime" : 0.00000000,
-        "version" : 70002,
-        "subver" : "/Satoshi:0.9.2.1/",
-        "inbound" : false,
-        "startingheight" : 310131,
-        "banscore" : 0,
-        "syncnode" : true
-    },
-    {
-        "addr" : "58.23.244.20:8333",
-        "services" : "00000001",
-        "lastsend" : 1405634127,
-        "lastrecv" : 1405634124,
-        "bytessent" : 4460918,
-        "bytesrecv" : 8903575,
-        "conntime" : 1405559628,
-        "pingtime" : 0.00000000,
-        "version" : 70001,
-        "subver" : "/Satoshi:0.8.6/",
-        "inbound" : false,
-        "startingheight" : 311074,
-        "banscore" : 0,
-        "syncnode" : false
-    }
-]
-----
-
-To override the automatic management of peers and to specify a list of IP addresses, users can provide the option +-connect=<IPAddress>+ and specify one or more IP addresses. If this option is used, the node will only connect to the selected IP addresses, instead of discovering and maintaining the peer connections automatically.
-
-If there is no traffic on a connection, nodes will periodically send a message to maintain the connection. If a node has not communicated on a connection for more than 90 minutes, it is assumed to be disconnected and a new peer will be sought. Thus, the network dynamically adjusts to transient nodes and network problems, and can organically grow and shrink as needed without any central control.((("", startref="BNextend08")))((("", startref="BNodiscover08")))
-
-=== Full Nodes
-
-((("Bitcoin network", "full nodes")))((("full-node clients")))((("blockchain (the)", "full blockchain nodes")))Full nodes are nodes that maintain a full blockchain with all transactions. More accurately, they probably should be called "full blockchain nodes." In the early years of bitcoin, all nodes were full nodes and currently the Bitcoin Core client is a full blockchain node. In the past two years, however, new forms of Bitcoin clients have been introduced that do not maintain a full blockchain but run as lightweight clients. We'll examine these in more detail in the next section.
-
-((("blocks", "genesis block")))((("genesis block")))((("blockchain (the)", "genesis block")))Full blockchain nodes maintain a complete and up-to-date copy of the Bitcoin blockchain with all the transactions, which they independently build and verify, starting with the very first block (genesis block) and building up to the latest known block in the network. A full blockchain node can independently and authoritatively verify any transaction without recourse or reliance on any other node or source of information. The full blockchain node relies on the network to receive updates about new blocks of transactions, which it then verifies and incorporates into its local copy of the blockchain.
-
-((("Bitcoin nodes", "full nodes")))Running a full blockchain node gives you the pure bitcoin experience: independent verification of all transactions without the need to rely on, or trust, any other systems. It's easy to tell if you're running a full node because it requires more than one hundred gigabytes of persistent storage (disk space) to store the full blockchain. If you need a lot of disk and it takes two to three days to sync to the network, you are running a full node. That is the price of complete independence and freedom from central authority.
-
-((("Satoshi client")))There are a few alternative implementations of full blockchain Bitcoin clients, built using different programming languages and software architectures. However, the most common implementation is the reference client Bitcoin Core, also known as the Satoshi client. More than 75% of the nodes on the Bitcoin network run various versions of Bitcoin Core. It is identified as "Satoshi" in the sub-version string sent in the +version+ message and shown by the command +getpeerinfo+ as we saw earlier; for example, +/Satoshi:0.8.6/+.
-
-=== Exchanging "Inventory"
-
-((("Bitcoin network", "syncing the blockchain")))The first thing a full node will do once it connects to peers is try to construct a complete blockchain. If it is a brand-new node and has no blockchain at all, it only knows one block, the genesis block, which is statically embedded in the client software. Starting with block #0 (the genesis block), the new node will have to download hundreds of thousands of blocks to synchronize with the network and reestablish the full blockchain.
-
-((("blockchain (the)", "syncing the blockchain")))((("syncing")))The process of syncing the blockchain starts with the +version+ message, because that contains +BestHeight+, a node's current blockchain height (number of blocks). A node will see the +version+ messages from its peers, know how many blocks they each have, and be able to compare to how many blocks it has in its own blockchain. Peered nodes will exchange a +getblocks+ message that contains the hash (fingerprint) of the top block on their local blockchain. One of the peers will be able to identify the received hash as belonging to a block that is not at the top, but rather belongs to an older block, thus deducing that its own local blockchain is longer than its peer's.
-
-The peer that has the longer blockchain has more blocks than the other node and can identify which blocks the other node needs in order to "catch up." It will identify the first 500 blocks to share and transmit their hashes using an +inv+ (inventory) message. The node missing these blocks will then retrieve them, by issuing a series of +getdata+ messages requesting the full block data and identifying the requested blocks using the hashes from the +inv+ message.
-
-Let's assume, for example, that a node only has the genesis block. It will then receive an +inv+ message from its peers containing the hashes of the next 500 blocks in the chain. It will start requesting blocks from all of its connected peers, spreading the load and ensuring that it doesn't overwhelm any peer with requests. The node keeps track of how many blocks are "in transit" per peer connection, meaning blocks that it has requested but not received, checking that it does not exceed a limit (+MAX_BLOCKS_IN_TRANSIT_PER_PEER+). This way, if it needs a lot of blocks, it will only request new ones as previous requests are fulfilled, allowing the peers to control the pace of updates and not overwhelm the network. As each block is received, it is added to the blockchain, as we will see in <<blockchain>>. As the local blockchain is gradually built up, more blocks are requested and received, and the process continues until the node catches up to the rest of the network.
-
-This process of comparing the local blockchain with the peers and retrieving any missing blocks happens any time a node goes offline for any period of time. Whether a node has been offline for a few minutes and is missing a few blocks, or a month and is missing a few thousand blocks, it starts by sending +getblocks+, gets an +inv+ response, and starts downloading the missing blocks. <<inventory_synchronization>> shows the inventory and block propagation protocol.
-
-[[inventory_synchronization]]
-[role="smallerfifty"]
-.Node synchronizing the blockchain by retrieving blocks from a peer
-image::images/mbc2_0806.png["InventorySynchronization"]
-
-[[spv_nodes]]
-=== Simplified Payment Verification (SPV) Nodes
-
-((("Bitcoin network", "SPV nodes", id="BNspvnodes08")))((("Bitcoin nodes", "SPV nodes", id="BNospv08")))((("simplified-payment-verification (SPV)", id="simple08")))Not all nodes have the ability to store the full blockchain. Many Bitcoin clients are designed to run on space- and power-constrained devices, such as smartphones, tablets, or embedded systems. For such devices, a _simplified payment verification_ (SPV) method is used to allow them to operate without storing the full blockchain. These types of clients are called SPV clients or lightweight clients. As bitcoin adoption surges, the SPV node is becoming the most common form of Bitcoin node, especially for bitcoin wallets.
-
-SPV nodes download only the block headers and do not download the transactions included in each block. The resulting chain of blocks, without transactions, is 1,000 times smaller than the full blockchain. SPV nodes cannot construct a full picture of all the UTXOs that are available for spending because they do not know about all the transactions on the network. SPV nodes verify transactions using a slightly different method that relies on peers to provide partial views of relevant parts of the blockchain on demand.
-
-As an analogy, a full node is like a tourist in a strange city, equipped with a detailed map of every street and every address. By comparison, an SPV node is like a tourist in a strange city asking random strangers for turn-by-turn directions while knowing only one main avenue. Although both tourists can verify the existence of a street by visiting it, the tourist without a map doesn't know what lies down any of the side streets and doesn't know what other streets exist. Positioned in front of 23 Church Street, the tourist without a map cannot know if there are a dozen other "23 Church Street" addresses in the city and whether this is the right one. The mapless tourist's best chance is to ask enough people and hope some of them are not trying to mug him.
-
-SPV verifies transactions by reference to their _depth_ in the blockchain instead of their _height_. Whereas a full blockchain node will construct a fully verified chain of thousands of blocks and transactions reaching down the blockchain (back in time) all the way to the genesis block, an SPV node will verify the chain of all blocks (but not all transactions) and link that chain to the transaction of interest.
-
-For example, when examining a transaction in block 300,000, a full node links all 300,000 blocks down to the genesis block and builds a full database of UTXO, establishing the validity of the transaction by confirming that the UTXO remains unspent. An SPV node cannot validate whether the UTXO is unspent. Instead, the SPV node will establish a link between the transaction and the block that contains it, using a _merkle path_ (see <<merkle_trees>>). Then, the SPV node waits until it sees the six blocks 300,001 through 300,006 piled on top of the block containing the transaction and verifies it by establishing its depth under blocks 300,006 to 300,001. The fact that other nodes on the network accepted block 300,000 and then did the necessary work to produce six more blocks on top of it is proof, by proxy, that the transaction was not a double-spend.
-
-An SPV node cannot be persuaded that a transaction exists in a block when the transaction does not in fact exist. The SPV node establishes the existence of a transaction in a block by requesting a merkle path proof and by validating the Proof-of-Work in the chain of blocks. However, a transaction's existence can be "hidden" from an SPV node. An SPV node can definitely prove that a transaction exists but cannot verify that a transaction, such as a double-spend of the same UTXO, doesn't exist because it doesn't have a record of all transactions. This vulnerability can be used in a denial-of-service attack or for a double-spending attack against SPV nodes. To defend against this, an SPV node needs to connect randomly to several nodes, to increase the probability that it is in contact with at least one honest node. This need to randomly connect means that SPV nodes also are vulnerable to network partitioning attacks or Sybil attacks, where they are connected to fake nodes or fake networks and do not have access to honest nodes or the real Bitcoin network.
-
-For most practical purposes, well-connected SPV nodes are secure enough, striking a balance between resource needs, practicality, and security. For infallible security, however, nothing beats running a full blockchain node.
-
-[TIP]
-====
-A full blockchain node verifies a transaction by checking the entire chain of thousands of blocks below it in order to guarantee that the UTXO is not spent, whereas an SPV node checks how deep the block is buried by a handful of blocks above it.
-====
-
-To get the block headers, SPV nodes use a +getheaders+ message instead of +getblocks+. The responding peer will send up to 2,000 block headers using a single +headers+ message. The process is otherwise the same as that used by a full node to retrieve full blocks. SPV nodes also set a filter on the connection to peers, to filter the stream of future blocks and transactions sent by the peers. Any transactions of interest are retrieved using a +getdata+ request. The peer generates a +tx+ message containing the transactions, in response. <<spv_synchronization>> shows the synchronization of block headers.
-
-Because SPV nodes need to retrieve specific transactions in order to selectively verify them, they also create a privacy risk. Unlike full blockchain nodes, which collect all transactions within each block, the SPV node's requests for specific data can inadvertently reveal the addresses in their wallet. For example, a third party monitoring a network could keep track of all the transactions requested by a wallet on an SPV node and use those to associate Bitcoin addresses with the user of that wallet, destroying the user's privacy.
-
-[[spv_synchronization]]
-.SPV node synchronizing the block headers
-image::images/mbc2_0807.png["SPVSynchronization"]
-
-Shortly after the introduction of SPV/lightweight nodes, bitcoin developers added a feature called _bloom filters_ to address the privacy risks of SPV nodes. Bloom filters allow SPV nodes to receive a subset of the transactions without revealing precisely which addresses they are interested in, through a filtering mechanism that uses probabilities rather than fixed patterns.((("", startref="BNspvnodes08")))((("", startref="simple08")))
-
-[[bloom_filters]]
-=== Bloom Filters
-
-((("Bitcoin network", "bloom filters", id="BNebloom08")))((("bloom filters", id="bloom08")))((("privacy, maintaining", id="privacy08")))((("security", "maintaining privacy", id="Sprivacy08")))A bloom filter is a probabilistic search filter that offers an efficient way to express a search pattern while protecting privacy. They are used by SPV nodes to ask their peers for transactions matching a specific pattern, without revealing exactly which addresses, keys, or transactions they are searching for.
-
-In our previous analogy, a tourist without a map is asking for directions to a specific address, "23 Church St." If she asks strangers for directions to this street, she inadvertently reveals her destination. A bloom filter is like asking, "Are there any streets in this neighborhood whose name ends in R-C-H?" A question like that reveals slightly less about the desired destination than asking for "23 Church St." Using this technique, a tourist could specify the desired address in more detail such as "ending in U-R-C-H" or less detail as "ending in H." By varying the precision of the search, the tourist reveals more or less information, at the expense of getting more or less specific results. If she asks a less specific pattern, she gets a lot more possible addresses and better privacy, but many of the results are irrelevant. If she asks for a very specific pattern, she gets fewer results but loses privacy.
-
-Bloom filters serve this function by allowing an SPV node to specify a search pattern for transactions that can be tuned toward precision or privacy. A more specific bloom filter will produce accurate results, but at the expense of revealing what patterns the SPV node is interested in, thus revealing the addresses owned by the user's wallet. A less specific bloom filter will produce more data about more transactions, many irrelevant to the node, but will allow the node to maintain better privacy.
-
-==== How Bloom Filters Work
-
-Bloom filters are implemented as a variable-size array of N binary digits (a bit field) and a variable number of M hash functions. The hash functions are designed to always produce an output that is between 1 and N, corresponding to the array of binary digits. The hash functions are generated deterministically, so that any node implementing a bloom filter will always use the same hash functions and get the same results for a specific input. By choosing different length (N) bloom filters and a different number (M) of hash functions, the bloom filter can be tuned, varying the level of accuracy and therefore privacy.
-
-In <<bloom1>>, we use a very small array of 16 bits and a set of three hash functions to demonstrate how bloom filters work.
-
-[[bloom1]]
-.An example of a simplistic bloom filter, with a 16-bit field and three hash functions
-image::images/mbc2_0808.png["Bloom1"]
-
-The bloom filter is initialized so that the array of bits is all zeros. To add a pattern to the bloom filter, the pattern is hashed by each hash function in turn. Applying the first hash function to the input results in a number between 1 and N. The corresponding bit in the array (indexed from 1 to N) is found and set to +1+, thereby recording the output of the hash function. Then, the next hash function is used to set another bit and so on. Once all M hash functions have been applied, the search pattern will be "recorded" in the bloom filter as M bits that have been changed from +0+ to +1+.
-
-<<bloom2>> is an example of adding a pattern "A" to the simple bloom filter shown in <<bloom1>>.
-
-Adding a second pattern is as simple as repeating this process. The pattern is hashed by each hash function in turn and the result is recorded by setting the bits to +1+. Note that as a bloom filter is filled with more patterns, a hash function result might coincide with a bit that is already set to +1+, in which case the bit is not changed. In essence, as more patterns record on overlapping bits, the bloom filter starts to become saturated with more bits set to +1+ and the accuracy of the filter decreases. This is why the filter is a probabilistic data structure—it gets less accurate as more patterns are added. The accuracy depends on the number of patterns added versus the size of the bit array (N) and number of hash functions (M). A larger bit array and more hash functions can record more patterns with higher accuracy. A smaller bit array or fewer hash functions will record fewer patterns and produce less accuracy.
-
-[[bloom2]]
-.Adding a pattern "A" to our simple bloom filter
-image::images/mbc2_0809.png["Bloom2"]
-
-<<bloom3>> is an example of adding a second pattern "B" to the simple bloom filter.
-
-[[bloom3]]
-[role="smallereighty"]
-.Adding a second pattern "B" to our simple bloom filter
-image::images/mbc2_0810.png["Bloom3"]
-
-To test if a pattern is part of a bloom filter, the pattern is hashed by each hash function and the resulting bit pattern is tested against the bit array. If all the bits indexed by the hash functions are set to +1+, then the pattern is _probably_ recorded in the bloom filter. Because the bits may be set because of overlap from multiple patterns, the answer is not certain, but is rather probabilistic. In simple terms, a bloom filter positive match is a "Maybe, Yes."
-
-<<bloom4>> is an example of testing the existence of pattern "X" in the simple bloom filter. The corresponding bits are set to +1+, so the pattern is probably a match.
-
-[[bloom4]]
-[role="smallereighty"]
-.Testing the existence of pattern "X" in the bloom filter. The result is a probabilistic positive match, meaning "Maybe."
-image::images/mbc2_0811.png["Bloom4"]
-
-On the contrary, if a pattern is tested against the bloom filter and any one of the bits is set to +0+, this proves that the pattern was not recorded in the bloom filter. A negative result is not a probability, it is a certainty. In simple terms, a negative match on a bloom filter is a "Definitely Not!"
-
-<<bloom5>> is an example of testing the existence of pattern "Y" in the simple bloom filter. One of the corresponding bits is set to +0+, so the pattern is definitely not a match.
-
-[[bloom5]]
-.Testing the existence of pattern "Y" in the bloom filter. The result is a definitive negative match, meaning "Definitely Not!"
-image::images/mbc2_0812.png[]
-
-=== How SPV Nodes Use Bloom Filters
-
-Bloom filters are used to filter the transactions (and blocks containing them) that an SPV node receives from its peers, selecting only transactions of interest to the SPV node without revealing which addresses or keys it is interested in.
-
-((("transaction IDs (txid)")))An SPV node will initialize a bloom filter as "empty"; in that state the bloom filter will not match any patterns. The SPV node will then make a list of all the addresses, keys, and hashes that it is interested in. It will do this by extracting the public key hash and script hash and transaction IDs from any UTXO controlled by its wallet. The SPV node then adds each of these to the bloom filter, so that the bloom filter will "match" if these patterns are present in a transaction, without revealing the patterns themselves.
-
-((("Bitcoin nodes", "full nodes")))The SPV node will then send a +filterload+ message to the peer, containing the bloom filter to use on the connection. On the peer, bloom filters are checked against each incoming transaction. The full node checks several parts of the transaction against the bloom filter, looking for a match including:
-
-* The transaction ID
-* The data components from the locking scripts of each of the transaction outputs (every key and hash in the script)
-* Each of the transaction inputs
-* Each of the input signature data components (or witness scripts)
-
-By checking against all these components, bloom filters can be used to match public key hashes, scripts, +OP_RETURN+ values, public keys in signatures, or any future component of a smart contract or complex script.
-
-After a filter is established, the peer will then test each transaction's output against the bloom filter. Only transactions that match the filter are sent to the node.
-
-In response to a +getdata+ message from the node, peers will send a +merkleblock+ message that contains only block headers for blocks matching the filter and a merkle path (see <<merkle_trees>>) for each matching transaction. The peer will then also send +tx+ messages containing the transactions matched by the filter.
-
-As the full node sends transactions to the SPV node, the SPV node discards any false positives and uses the correctly matched transactions to update its UTXO set and wallet balance. As it updates its own view of the UTXO set, it also modifies the bloom filter to match any future transactions referencing the UTXO it just found. The full node then uses the new bloom filter to match new transactions and the whole process repeats.
-
-The node setting the bloom filter can interactively add patterns to the filter by sending a +filteradd+ message. To clear the bloom filter, the node can send a +filterclear+ message. Because it is not possible to remove a pattern from a bloom filter, a node has to clear and resend a new bloom filter if a pattern is no longer desired.
-
-The network protocol and bloom filter mechanism for SPV nodes is defined in https://bit.ly/1x6qCiO[BIP-37 (Peer Services)].((("", startref="BNebloom08")))((("", startref="bloom08")))
-
-
-=== SPV Nodes and Privacy
-
-Nodes that implement SPV have weaker privacy than a full node. A full node receives all transactions and therefore reveals no information about whether it is using some address in its wallet. An SPV node receives a filtered list of transactions related to the addresses that are in its wallet. As a result, it reduces the privacy of the owner.
-
-Bloom filters are a way to reduce the loss of privacy. Without them, an SPV node would have to explicitly list the addresses it was interested in, creating a serious breach of privacy. However, even with bloom filters, an adversary monitoring the traffic of an SPV client or connected to it directly as a node in the P2P network can collect enough information over time to learn the addresses in the wallet of the SPV client.
-
-=== Encrypted and Authenticated Connections
-
-((("Bitcoin network", "encrypted connections")))((("encryption")))((("authentication")))Most new users of bitcoin assume that the network communications of a Bitcoin node are encrypted. In fact, the original implementation of bitcoin communicates entirely in the clear. While this is not a major privacy concern for full nodes, it is a big problem for SPV nodes.
-
-As a way to increase the privacy and security of the bitcoin P2P network, there are two solutions that provide encryption of the communications: _Tor Transport_ and _P2P Authentication and Encryption_ with BIP-150/151.
-
-==== Tor Transport
-
-((("Tor network")))((("The Onion Routing network (Tor)")))Tor, which stands for _The Onion Routing network_, is a software project and network that offers encryption and encapsulation of data through randomized network paths that offer anonymity, untraceability and privacy.
-
-Bitcoin Core offers several configuration options that allow you to run a Bitcoin node with its traffic transported over the Tor network. In addition, Bitcoin Core can also offer a Tor hidden service allowing other Tor nodes to connect to your node directly over Tor.
-
-As of Bitcoin Core version 0.12, a node will offer a hidden Tor service automatically if it is able to connect to a local Tor service. If you have Tor installed and the Bitcoin Core process runs as a user with adequate permissions to access the Tor authentication cookie, it should work automatically. Use the +debug+ flag to turn on Bitcoin Core's debugging for the Tor service like this:
-
-----
-$ bitcoind --daemon --debug=tor
-----
-
-You should see "tor: ADD_ONION successful" in the logs, indicating that Bitcoin Core has added a hidden service to the Tor network.
-
-You can find more instructions on running Bitcoin Core as a Tor hidden service in the Bitcoin Core documentation (_docs/tor.md_) and various online tutorials.
-
-==== Peer-to-Peer Authentication and Encryption
-
-((("Peer-to-Peer authentication and encryption")))((("bitcoin improvement proposals", "Peer Authentication (BIP-150)")))((("bitcoin improvement proposals", "Peer-to-Peer Communication Encryption (BIP-151)")))Two Bitcoin Improvement Proposals, BIP-150 and BIP-151, add support for P2P authentication and encryption in the bitcoin P2P network. These two BIPs define optional services that may be offered by compatible Bitcoin nodes. BIP-151 enables negotiated encryption for all communications between two nodes that support BIP-151. BIP-150 offers optional peer authentication that allows nodes to authenticate each other's identity using ECDSA and private keys. BIP-150 requires that prior to authentication the two nodes have established encrypted communications as per BIP-151.
-
-As of February 2021, BIP-150 and BIP-151 are not implemented in Bitcoin Core. However, the two proposals have been implemented by at least one alternative Bitcoin client named bcoin.
-
-BIP-150 and BIP-151 allow users to run SPV clients that connect to a trusted full node, using encryption and authentication to protect the privacy of the SPV client.
-
-Additionally, authentication can be used to create networks of trusted Bitcoin nodes and prevent Man-in-the-Middle attacks. Finally, P2P encryption, if deployed broadly, would strengthen the resistance of bitcoin to traffic analysis and privacy-eroding surveillance, especially in totalitarian countries where internet use is heavily controlled and monitored.
-
-((("", startref="BNospv08")))((("", startref="privacy08")))((("", startref="Sprivacy08")))The standard is defined in https://github.com/bitcoin/bips/blob/master/bip-0150.mediawiki[BIP-150 (Peer Authentication)] and https://github.com/bitcoin/bips/blob/master/bip-0151.mediawiki[BIP-151 (Peer-to-Peer Communication Encryption)].
-
-=== Transaction Pools
-
-((("Bitcoin network", "transaction pools")))((("transaction pools")))((("memory pools (mempools)")))Almost every node on the Bitcoin network maintains a temporary list of unconfirmed transactions called the _memory pool_, _mempool_, or _transaction pool_. Nodes use this pool to keep track of transactions that are known to the network but are not yet included in the blockchain. For example, a wallet node will use the transaction pool to track incoming payments to the user's wallet that have been received on the network but are not yet confirmed.
-
-As transactions are received and verified, they are added to the transaction pool and relayed to the neighboring nodes to propagate on the network.
-
-((("orphan pools")))((("transactions", "orphaned")))Some node implementations also maintain a separate pool of orphaned transactions. If a transaction's inputs refer to a transaction that is not yet known, such as a missing parent, the orphan transaction will be stored temporarily in the orphan pool until the parent transaction arrives.
-
-When a transaction is added to the transaction pool, the orphan pool is checked for any orphans that reference this transaction's outputs (its children). Any matching orphans are then validated. If valid, they are removed from the orphan pool and added to the transaction pool, completing the chain that started with the parent transaction. In light of the newly added transaction, which is no longer an orphan, the process is repeated recursively looking for any further descendants, until no more descendants are found. Through this process, the arrival of a parent transaction triggers a cascade reconstruction of an entire chain of interdependent transactions by re-uniting the orphans with their parents all the way down the chain.
-
-Both the transaction pool and orphan pool (where implemented) are stored in local memory and are not saved on persistent storage; rather, they are dynamically populated from incoming network messages. When a node starts, both pools are empty and are gradually populated with new transactions received on the network.
-
-Some implementations of the Bitcoin client also maintain an UTXO database or pool, which is the set of all unspent outputs on the blockchain. Bitcoin Core users will find it in the +chainstate/+ folder of their client's data directory. Although the name "UTXO pool" sounds similar to the transaction pool, it represents a different set of data. Unlike the transaction and orphan pools, the UTXO pool is not initialized empty but instead contains millions of entries of unspent transaction outputs, everything that is unspent from all the way back to the genesis block. The UTXO pool may be housed in local memory or as an indexed database table on persistent storage.
-
-Whereas the transaction and orphan pools represent a single node's local perspective and might vary significantly from node to node depending upon when the node was started or restarted, the UTXO pool represents the emergent consensus of the network and therefore will vary little between nodes. Furthermore, the transaction and orphan pools only contain unconfirmed transactions, while the UTXO pool only contains confirmed outputs.

ch09.asciidoc

@@ -1,392 +0,0 @@
-[[blockchain]]
-== The Blockchain
-
-=== Introduction
-
-((("blockchain (the)", "overview of")))The blockchain data structure is an ordered, back-linked list of blocks of transactions. The blockchain can be stored as a flat file, or in a simple database. The Bitcoin Core client stores the blockchain metadata using Google's LevelDB database. Blocks are linked "back," each referring to the previous block in the chain. ((("blocks", "block height")))The blockchain is often visualized as a vertical stack, with blocks layered on top of each other and the first block serving as the foundation of the stack. The visualization of blocks stacked on top of each other results in the use of terms such as "height" to refer to the distance from the first block, and "top" or "tip" to refer to the most recently added block.
-
-((("blocks", "block hash")))((("blocks", "genesis block")))((("blocks", "parent blocks")))((("genesis block")))((("parent blocks")))Each block within the blockchain is identified by a hash, generated using the SHA256 cryptographic hash algorithm on the header of the block. Each block also references a previous block, known as the _parent_ block, through the "previous block hash" field in the block header. In other words, each block contains the hash of its parent inside its own header. The sequence of hashes linking each block to its parent creates a chain going back all the way to the first block ever created, known as the _genesis block_.
-
-Although a block has just one parent, it can temporarily have multiple children. Each of the children refers to the same block as its parent and contains the same (parent) hash in the "previous block hash" field. Multiple children arise during a blockchain "fork," a temporary situation that occurs when different blocks are discovered almost simultaneously by different miners (see <<forks>>). Eventually, only one child block becomes part of the blockchain and the "fork" is resolved. Even though a block may have more than one child, each block can have only one parent. This is because a block has one single "previous block hash" field referencing its single parent.
-
-The "previous block hash" field is inside the block header and thereby affects the _current_ block's hash. The child's own identity changes if the parent's identity changes. When the parent is modified in any way, the parent's hash changes. The parent's changed hash necessitates a change in the "previous block hash" pointer of the child. This in turn causes the child's hash to change, which requires a change in the pointer of the grandchild, which in turn changes the grandchild, and so on. This cascade effect ensures that once a block has many generations following it, it cannot be changed without forcing a recalculation of all subsequent blocks. Because such a recalculation would require enormous computation (and therefore energy consumption), the existence of a long chain of blocks makes the blockchain's deep history immutable, which is a key feature of bitcoin's security.
-
-One way to think about the blockchain is like layers in a geological formation, or glacier core sample. The surface layers might change with the seasons, or even be blown away before they have time to settle. But once you go a few inches deep, geological layers become more and more stable. By the time you look a few hundred feet down, you are looking at a snapshot of the past that has remained undisturbed for millions of years. In the blockchain, the most recent few blocks might be revised if there is a chain recalculation due to a fork. The top six blocks are like a few inches of topsoil. But once you go more deeply into the blockchain, beyond six blocks, blocks are less and less likely to change. ((("transactions", "coinbase transactions")))((("coinbase transactions")))After 100 blocks back, there is so much stability that the coinbase transaction—the transaction containing newly mined bitcoin—can be spent. A few thousand blocks back (a month) and the blockchain is settled history, for all practical purposes. While the protocol always allows a chain to be undone by a longer chain and while the possibility of any block being reversed always exists, the probability of such an event decreases as time passes until it becomes infinitesimal.
-
-=== Structure of a Block
-
-((("blocks", "structure of")))((("blockchain (the)", "block structure")))A block is a container data structure that aggregates transactions for inclusion in the public ledger, the blockchain. The block is made of a header, containing metadata, followed by a long list of transactions that make up the bulk of its size. The block header is 80 bytes, whereas the average transaction is at least 400 bytes and the average block contains more than 1900 transactions. A complete block, with all transactions, is therefore 10,000 times larger than the block header. <<block_structure1>> describes the structure of a block.
-
-[[block_structure1]]
-[role="pagebreak-before"]
-.The structure of a block
-[options="header"]
-|=======
-|Size| Field | Description
-| 4 bytes | Block Size | The size of the block, in bytes, following this field
-| 80 bytes | Block Header | Several fields form the block header
-| 1&#x2013;9 bytes (VarInt) | Transaction Counter | How many transactions follow
-| Variable | Transactions | The transactions recorded in this block
-|=======
-
-[[block_header]]
-=== Block Header
-
-((("blocks", "headers")))((("blockchain (the)", "block headers")))((("headers")))The block header consists of three sets of block metadata. First, there is a reference to a previous block hash, which connects this block to the previous block in the blockchain. The second set of metadata, namely the _difficulty_, _timestamp_, and _nonce_, relate to the mining competition, as detailed in <<mining>>. The third piece of metadata is the merkle tree root, a data structure used to efficiently summarize all the transactions in the block. <<block_header_structure_ch09>> describes the structure of a block header.
-
-
-[[block_header_structure_ch09]]
-.The structure of the block header
-[options="header"]
-|=======
-|Size| Field | Description
-| 4 bytes | Version | A version number to track software/protocol upgrades
-| 32 bytes | Previous Block Hash | A reference to the hash of the previous (parent) block in the chain
-| 32 bytes | Merkle Root | A hash of the root of the merkle tree of this block's transactions
-| 4 bytes | Timestamp | The approximate creation time of this block (in seconds elapsed since Unix Epoch)
-| 4 bytes | Difficulty Target | The Proof-of-Work algorithm difficulty target for this block
-| 4 bytes | Nonce | A counter used for the Proof-of-Work algorithm
-|=======
-
-The nonce, difficulty target, and timestamp are used in the mining process and will be discussed in more detail in <<mining>>.
-
-[[block_hash]]
-=== Block Identifiers: Block Header Hash and Block Height
-
-((("blockchain (the)", "block identifiers")))((("blocks", "block height")))((("blocks", "block hash")))The primary identifier of a block is its cryptographic hash, a digital fingerprint, made by hashing the block header twice through the SHA256 algorithm. The resulting 32-byte hash is called the _block hash_ but is more accurately the _block header hash_, pass:[<span role="keep-together">because only the block header is used to compute it. For example,</span>] +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+ is the block hash of the first bitcoin block ever created. The block hash identifies a block uniquely and unambiguously and can be independently derived by any node by simply double hashing the block header with the SHA256 algorithm.
-
-Note that the block hash is not actually included inside the block's data structure, neither when the block is transmitted on the network, nor when it is stored on a node's persistence storage as part of the blockchain. Instead, the block's hash is computed by each node as the block is received from the network. The block hash might be stored in a separate database table as part of the block's metadata, to facilitate indexing and faster retrieval of blocks from disk.
-
-A second way to identify a block is by its position in the blockchain, called the pass:[<span role="keep-together"><em>block height</em>. The first block ever created is at block height 0 (zero) and is the</span>] pass:[<span role="keep-together">same block that was previously referenced by the following block hash</span>] +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+. A block can thus be identified in two ways: by referencing the block hash or by referencing the block height. Each subsequent block added "on top" of that first block is one position "higher" in the blockchain, like boxes stacked one on top of the other.
-
-In addition, the term _current block height_ indicates the size of the blockchain in blocks at any given time. For example, the current block height on March 1, 2021 was approximately 672,722, meaning there were 672,722 blocks stacked on top of the first block created in January 2009.
-
-Unlike the block hash, the block height is not a unique identifier. Although a single block will always have a specific and invariant block height, the reverse is not true—the block height does not always identify a single block. Two or more blocks might have the same block height, competing for the same position in the blockchain. This scenario is discussed in detail in the section <<forks>>. The block height is also not a part of the block's data structure; it is not stored within the block. Each node dynamically identifies a block's position (height) in the blockchain when it is received from the Bitcoin network. The block height might also be stored as metadata in an indexed database table for faster retrieval.
-
-[TIP]
-====
-A block's _block hash_ always identifies a single block uniquely. A block also always has a specific _block height_. However, it is not always the case that a specific block height can identify a single block. Rather, two or more blocks might compete for a single position in the blockchain.
-====
-
-=== The Genesis Block
-
-((("blocks", "genesis block")))((("blockchain (the)", "genesis block")))The first block in the blockchain is called the genesis block and was created in 2009. It is the common ancestor of all the blocks in the blockchain, meaning that if you start at any block and follow the chain backward in time, you will eventually arrive at the genesis block.
-
-Every node always starts with a blockchain of at least one block because the genesis block is statically encoded within the Bitcoin client software, such that it cannot be altered. Every node always "knows" the genesis block's hash and structure, the fixed time it was created, and even the single transaction within. Thus, every node has the starting point for the blockchain, a secure "root" from which to build a trusted blockchain.
-
-See the statically encoded genesis block inside the Bitcoin Core client, in https://bit.ly/1x6rcwP[_chainparams.cpp_].
-
-The following identifier hash belongs to the genesis block:
-
-----
-000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
-----
-
-You can search for that block hash in any block explorer website, such as _Blockchain.com_, and you will find a page describing the contents of this block, with a URL containing that hash:
-
-https://www.blockchain.com/btc/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
-
-Using the Bitcoin Core reference client on the command line:
-
-----
-$ bitcoin-cli getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
-----
-[source,json]
-----
-{
-    "hash" : "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
-    "confirmations" : 308321,
-    "size" : 285,
-    "height" : 0,
-    "version" : 1,
-    "merkleroot" : "4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b",
-    "tx" : [
-        "4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b"
-    ],
-    "time" : 1231006505,
-    "nonce" : 2083236893,
-    "bits" : "1d00ffff",
-    "difficulty" : 1.00000000,
-    "nextblockhash" : "00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048"
-}
-----
-
-The genesis block contains a hidden message within it. The coinbase transaction input contains the text "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks." This message was intended to offer proof of the earliest date this block was created, by referencing the headline of the British newspaper _The Times_. It also serves as a tongue-in-cheek reminder of the importance of an independent monetary system, with bitcoin's launch occurring at the same time as an unprecedented worldwide monetary crisis. The message was embedded in the first block by Satoshi Nakamoto, bitcoin's creator.
-
-=== Linking Blocks in the Blockchain
-
-((("blocks", "linking blocks in the blockchain")))((("blockchain (the)", "linking blocks in the blockchain")))Bitcoin full nodes maintain a local copy of the blockchain, starting at the genesis block. The local copy of the blockchain is constantly updated as new blocks are found and used to extend the chain. As a node receives incoming blocks from the network, it will validate these blocks and then link them to the existing blockchain. To establish a link, a node will examine the incoming block header and look for the "previous block hash."
-
-Let's assume, for example, that a node has 277,314 blocks in the local copy of the blockchain. The last block the node knows about is block 277,314, with a block header hash of:
-
-----
-00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249
-----
-
-The Bitcoin node then receives a new block from the network, which it parses as follows:
-
-[source,json]
-----
-{
-    "size" : 43560,
-    "version" : 2,
-    "previousblockhash" :
-        "00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249",
-    "merkleroot" :
-        "5e049f4030e0ab2debb92378f53c0a6e09548aea083f3ab25e1d94ea1155e29d",
-    "time" : 1388185038,
-    "difficulty" : 1180923195.25802612,
-    "nonce" : 4215469401,
-    "tx" : [
-        "257e7497fb8bc68421eb2c7b699dbab234831600e7352f0d9e6522c7cf3f6c77",
-
- #[... many more transactions omitted ...]
-
-        "05cfd38f6ae6aa83674cc99e4d75a1458c165b7ab84725eda41d018a09176634"
-    ]
-}
-----
-
-Looking at this new block, the node finds the +previousblockhash+ field, which contains the hash of its parent block. It is a hash known to the node, that of the last block on the chain at height 277,314. Therefore, this new block is a child of the last block on the chain and extends the existing blockchain. The node adds this new block to the end of the chain, making the blockchain longer with a new height of 277,315. <<chain_of_blocks>> shows the chain of three blocks, linked by references in the +previousblockhash+ field.
-
-
-[[merkle_trees]]
-=== Merkle Trees
-
-((("merkle trees", id="merkle09")))((("blockchain (the)", "merkle trees", id="BCTmerkle09")))Each block in the Bitcoin blockchain contains a summary of all the transactions in the block using a _merkle tree_.
-
-((("binary hash trees", see="merkle trees")))A _merkle tree_, also known as a _binary hash tree_, is a data structure used for efficiently summarizing and verifying the integrity of large sets of data. Merkle trees are binary trees containing cryptographic hashes. The term "tree" is used in computer science to describe a branching data structure, but these trees are usually displayed upside down with the "root" at the top and the "leaves" at the bottom of a diagram, as you will see in the examples that follow.
-
-[[chain_of_blocks]]
-[role="smallerfourtyfive"]
-.Blocks linked in a chain by reference to the previous block header hash
-image::images/mbc2_0901.png[]
-
-Merkle trees are used in bitcoin to summarize all the transactions in a block, producing an overall digital fingerprint of the entire set of transactions, providing a very efficient process to verify whether a transaction is included in a block. A merkle tree is constructed by recursively hashing pairs of nodes until there is only one hash, called the _root_, or _merkle root_. The cryptographic hash algorithm used in bitcoin's merkle trees is SHA256 applied twice, also known as double-SHA256.
-
-When N data elements are hashed and summarized in a merkle tree, you can check to see if any one data element is included in the tree with at most +2*log~2~(N)+ calculations, making this a very efficient data structure.
-
-The merkle tree is constructed bottom-up. In the following example, we start with four transactions, A, B, C, and D, which form the _leaves_ of the merkle tree, as shown in <<simple_merkle>>. The transactions are not stored in the merkle tree; rather, their data is hashed and the resulting hash is stored in each leaf node as H~A~, H~B~, H~C~, and H~D~:
-
-++++
-<pre data-type="codelisting">
-H<sub>A</sub> = SHA256(SHA256(Transaction A))
-</pre>
-++++
-
-Consecutive pairs of leaf nodes are then summarized in a parent node, by concatenating the two hashes and hashing them together. For example, to construct the parent node H~AB~, the two 32-byte hashes of the children are concatenated to create a 64-byte string. That string is then double-hashed to produce the parent node's hash:
-
-++++
-<pre data-type="codelisting">
-H<sub>AB</sub> = SHA256(SHA256(H<sub>A</sub> + H<sub>B</sub>))
-</pre>
-++++
-
-The process continues until there is only one node at the top, the node known as the merkle root. That 32-byte hash is stored in the block header and summarizes all the data in all four transactions. <<simple_merkle>> shows how the root is calculated by pair-wise hashes of the nodes.
-
-[[simple_merkle]]
-.Calculating the nodes in a merkle tree
-image::images/mbc2_0902.png["merkle_tree"]
-
-((("balanced trees")))Because the merkle tree is a binary tree, it needs an even number of leaf nodes. If there is an odd number of transactions to summarize, the last transaction hash will be duplicated to create an even number of leaf nodes, also known as a _balanced tree_. This is shown in <<merkle_tree_odd>>, where transaction C is duplicated.
-
-[[merkle_tree_odd]]
-.Duplicating one data element achieves an even number of data elements
-image::images/mbc2_0903.png["merkle_tree_odd"]
-
-The same method for constructing a tree from four transactions can be generalized to construct trees of any size. In bitcoin it is common to have several hundred to more than a thousand transactions in a single block, which are summarized in exactly the same way, producing just 32 bytes of data as the single merkle root. In <<merkle_tree_large>>, you will see a tree built from 16 transactions. Note that although the root looks bigger than the leaf nodes in the diagram, it is the exact same size, just 32 bytes. Whether there is one transaction or a hundred thousand transactions in the block, the merkle root always summarizes them into 32 bytes.
-
-((("authentication paths")))To prove that a specific transaction is included in a block, a node only needs to produce +log~2~(N)+ 32-byte hashes, constituting an _authentication path_ or _merkle path_ connecting the specific transaction to the root of the tree. This is especially important as the number of transactions increases, because the base-2 logarithm of the number of transactions increases much more slowly. This allows Bitcoin nodes to efficiently produce paths of 10 or 12 hashes (320–384 bytes), which can provide proof of a single transaction out of more than a thousand transactions in a megabyte-sized block.
-
-[[merkle_tree_large]]
-.A merkle tree summarizing many data elements
-image::images/mbc2_0904.png["merkle_tree_large"]
-
-In <<merkle_tree_path>>, a node can prove that a transaction K is included in the block by producing a merkle path that is only four 32-byte hashes long (128 bytes total). The path consists of the four hashes (shown with a shaded background in <<merkle_tree_path>>) H~L~, H~IJ~, H~MNOP~, and H~ABCDEFGH~. With those four hashes provided as an authentication path, any node can prove that H~K~ (with a black background at the bottom of the diagram) is included in the merkle root by computing four additional pair-wise hashes H~KL~, H~IJKL~, H~IJKLMNOP~, and the merkle tree root (outlined in a dashed line in the diagram).
-
-[[merkle_tree_path]]
-.A merkle path used to prove inclusion of a data element
-image::images/mbc2_0905.png["merkle_tree_path"]
-
-
-The code in <<merkle_example>> demonstrates the process of creating a merkle tree from the leaf-node hashes up to the root, using the libbitcoin library for some helper functions.
-
-[[merkle_example]]
-[role="pagebreak-before"]
-.Building a merkle tree
-====
-[source, cpp]
-----
-include::code/merkle.cpp[]
-----
-====
-
-<<merkle_example_run>> shows the result of compiling and running the merkle code.
-
-[[merkle_example_run]]
-.Compiling and running the merkle example code
-====
-[source,bash]
-----
-# Compile the merkle.cpp code
-$ g++ -o merkle merkle.cpp $(pkg-config --cflags --libs libbitcoin)
-# Run the merkle executable
-$ ./merkle
-Current merkle hash list:
-  32650049a0418e4380db0af81788635d8b65424d397170b8499cdc28c4d27006
-  30861db96905c8dc8b99398ca1cd5bd5b84ac3264a4e1b3e65afa1bcee7540c4
-
-Current merkle hash list:
-  d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
-
-Result: d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
-
-----
-====
-
-The efficiency of merkle trees becomes obvious as the scale increases. <<block_structure2>> shows the amount of data that needs to be exchanged as a merkle path to prove that a transaction is part of a block.
-
-[[block_structure2]]
-.Merkle tree efficiency
-[options="header"]
-|=======
-|Number of transactions| Approx. size of block | Path size (hashes) | Path size (bytes)
-| 16 transactions | 4 kilobytes | 4 hashes | 128 bytes
-| 512 transactions | 128 kilobytes | 9 hashes | 288 bytes
-| 2048 transactions | 512 kilobytes | 11 hashes | 352 bytes
-| 65,535 transactions | 16 megabytes | 16 hashes | 512 bytes
-|=======
-
-As you can see from the table, while the block size increases rapidly, from 4 KB with 16 transactions to a block size of 16 MB to fit 65,535 transactions, the merkle path required to prove the inclusion of a transaction increases much more slowly, from 128 bytes to only 512 bytes. With merkle trees, a node can download just the block headers (80 bytes per block) and still be able to identify a transaction's inclusion in a block by retrieving a small merkle path from a full node, without storing or transmitting the vast majority of the blockchain, which might be several gigabytes in size. Nodes that do not maintain a full blockchain, called simplified payment verification (SPV) nodes, use merkle paths to verify transactions without downloading full blocks.
-
-=== Merkle Trees and Simplified Payment Verification (SPV)
-
-((("simplified-payment-verification (SPV)")))((("Bitcoin nodes", "SPV nodes")))Merkle trees are used extensively by SPV nodes. SPV nodes don't have all transactions and do not download full blocks, just block headers. In order to verify that a transaction is included in a block, without having to download all the transactions in the block, they use an authentication path, or merkle path.
-
-Consider, for example, an SPV node that is interested in incoming payments to an address contained in its wallet. The SPV node will establish a bloom filter (see <<bloom_filters>>) on its connections to peers to limit the transactions received to only those containing addresses of interest. When a peer sees a transaction that matches the bloom filter, it will send that block using a +merkleblock+ message. The +merkleblock+ message contains the block header as well as a merkle path that links the transaction of interest to the merkle root in the block. The SPV node can use this merkle path to connect the transaction to the block and verify that the transaction is included in the block. The SPV node also uses the block header to link the block to the rest of the blockchain. The combination of these two links, between the transaction and block, and between the block and blockchain, proves that the transaction is recorded in the blockchain. All in all, the SPV node will have received less than a kilobyte of data for the block header and merkle path, an amount of data that is more than a thousand times less than a full block (about 1 megabyte currently).((("", startref="BCTmerkle09")))((("", startref="merkle09")))
-
-=== Bitcoin's Test Blockchains
-
-((("blockchain (the)", "test blockchains", id="BCTtest09")))((("mainnet", seealso="blockchain (the)")))You might be surprised to learn that there is more than one Bitcoin blockchain. The "main" Bitcoin blockchain, the one created by Satoshi Nakamoto on January 3rd, 2009, the one with the genesis block we studied in this chapter, is called _mainnet_.  There are other Bitcoin blockchains that are used for testing purposes: at this time _testnet_, _segnet_, and _regtest_. Let's look at each in turn.((("testnet", id="testnet09")))
-
-
-==== Testnet&#x2014;Bitcoin's Testing Playground
-
-Testnet is the name of the test blockchain, network, and currency that is used for testing purposes. The testnet is a fully featured live P2P network, with wallets, test bitcoin (testnet coins), mining, and all the other features of mainnet.  There are really only two differences: testnet coins are meant to be worthless and mining difficulty should be low enough that anyone can mine testnet coins relatively easily (keeping them worthless).
-
-Any software development that is intended for production use on bitcoin's mainnet should first be tested on testnet with test coins. This protects both the developers from monetary losses due to bugs and the network from unintended behavior due to bugs.
-
-Keeping the coins worthless and the mining easy, however, is not easy. Despite pleas from developers, some people use advanced mining equipment (GPUs and ASICs) to mine on testnet. This increases the difficulty, makes it impossible to mine with a CPU, and eventually makes it difficult enough to get test coins that people start valuing them, so they're not worthless. As a result, every now and then, the testnet has to be scrapped and restarted from a new genesis block, resetting the difficulty.
-
-The current testnet is called _testnet3_, the third iteration of testnet, restarted in February 2011 to reset the difficulty from the previous testnet.
-
-Keep in mind that testnet3 is a large blockchain, in excess of 25 GB in 2021. It will take a day or so to sync fully and use up resources on your computer. Not as much as mainnet, but not exactly "lightweight" either. One good way to run a testnet node is as a virtual machine image (e.g., VirtualBox, Docker, Cloud Server, etc.) dedicated for that purpose.
-
-===== Using testnet
-
-Bitcoin Core, like almost all other bitcoin software, has full support for operation on testnet instead of mainnet. All of Bitcoin Core's functions work on testnet, including the wallet, mining testnet coins, and syncing a full testnet node.
-
-To start Bitcoin Core on testnet instead of mainnet you use the +testnet+ switch:
-
-----
-$ bitcoind -testnet
-----
-
-In the logs you should see that bitcoind is building a new blockchain in the +testnet3+ subdirectory of the default bitcoind directory:
-
-----
-bitcoind: Using data directory /home/username/.bitcoin/testnet3
-----
-
-To connect to bitcoind, you use the +bitcoin-cli+ command-line tool, but you must also switch it to testnet mode:
-
-----
-$ bitcoin-cli -testnet getblockchaininfo
-{
-  "chain": "test",
-  "blocks": 1088,
-  "headers": 139999,
-  "bestblockhash": "0000000063d29909d475a1c4ba26da64b368e56cce5d925097bf3a2084370128",
-  "difficulty": 1,
-  "mediantime": 1337966158,
-  "verificationprogress": 0.001644065914099759,
-  "chainwork": "0000000000000000000000000000000000000000000000000000044104410441",
-  "pruned": false,
-  "softforks": [
-
-  [...]
-----
-
-You can also run on testnet3 with other full-node implementations, such as +btcd+ (written in Go) and +bcoin+ (written in JavaScript), to experiment and learn in other programming languages and frameworks.
-
-In 2021, testnet3 supports all the features of mainnet, including Segregated Witness (see <<segwit>>). Therefore, testnet3 can also be used to test Segregated Witness features.((("", startref="testnet09")))
-
-==== Segnet&#x2014;The Segregated Witness Testnet
-
-
-((("segnet")))In 2016, a special-purpose testnet was launched to aid in development and testing of Segregated Witness (aka segwit; see <<segwit>>). This test blockchain is called +segnet+ and can be joined by running a special version (branch) of Bitcoin Core.
-
-((("segwit (Segregated Witness)")))Since segwit was added to testnet3, it is no longer necessary to use segnet for testing of segwit features.
-
-In the future it is likely we will see other testnet blockchains that are specifically designed to test a single feature or major architectural change, like segnet.
-
-==== Regtest&#x2014;The Local Blockchain
-
-((("regtest (Regression Testing)")))Regtest, which stands for "Regression Testing," is a Bitcoin Core feature that allows you to create a local blockchain for testing purposes. Unlike testnet3, which is a public and shared test blockchain, the regtest blockchains are intended to be run as closed systems for local testing. You launch a regtest blockchain from scratch, creating a local genesis block. You may add other nodes to the network, or run it with a single node only to test the Bitcoin Core software.
-
-To start Bitcoin Core in regtest mode, you use the +regtest+ flag:
-
-----
-$ bitcoind -regtest
-----
-
-Just like with testnet, Bitcoin Core will initialize a new blockchain under the _regtest_ subdirectory of your bitcoind default directory:
-
-----
-bitcoind: Using data directory /home/username/.bitcoin/regtest
-----
-
-To use the command-line tool, you need to specify the +regtest+ flag too. Let's try the +getblockchaininfo+ command to inspect the regtest blockchain:
-
-----
-$ bitcoin-cli -regtest getblockchaininfo
-{
-  "chain": "regtest",
-  "blocks": 0,
-  "headers": 0,
-  "bestblockhash": "0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206",
-  "difficulty": 4.656542373906925e-10,
-  "mediantime": 1296688602,
-  "verificationprogress": 1,
-  "chainwork": "0000000000000000000000000000000000000000000000000000000000000002",
-  "pruned": false,
-  [...]
-----
-
-As you can see, there are no blocks yet. Let's mine some (500 blocks) and earn the reward:
-
-----
-$ bitcoin-cli -regtest generate 500
-[
-  "7afed70259f22c2bf11e406cb12ed5c0657b6e16a6477a9f8b28e2046b5ba1ca",
-  "1aca2f154a80a9863a9aac4c72047a6d3f385c4eec5441a4aafa6acaa1dada14",
-  "4334ecf6fb022f30fbd764c3ee778fabbd53b4a4d1950eae8a91f1f5158ed2d1",
-  "5f951d34065efeaf64e54e91d00b260294fcdfc7f05dbb5599aec84b957a7766",
-  "43744b5e77c1dfece9d05ab5f0e6796ebe627303163547e69e27f55d0f2b9353",
-   [...]
-  "6c31585a48d4fc2b3fd25521f4515b18aefb59d0def82bd9c2185c4ecb754327"
-]
-----
-
-It will only take a few seconds to mine all these blocks, which certainly makes it easy for testing. If you check your wallet balance, you will see that you earned reward for the first 400 blocks (coinbase rewards must be 100 blocks deep before you can spend them):
-
-----
-$ bitcoin-cli -regtest getbalance
-12462.50000000
-----
-
-=== Using Test Blockchains for Development
-
-((("development environment", "test blockchains and")))Bitcoin's various blockchains (+regtest+, +segnet+, +testnet3+, +mainnet+) offer a range of testing environments for bitcoin development. Use the test blockchains whether you are developing for Bitcoin Core, or another full-node consensus client; an application such as a wallet, exchange, ecommerce site; or even developing novel smart contracts and complex scripts.
-
-You can use the test blockchains to establish a development pipeline. Test your code locally on a +regtest+ as you develop it. Once you are ready to try it on a public network, switch to +testnet+ to expose your code to a more dynamic environment with more diversity of code and applications. Finally, once you are confident your code works as expected, switch to +mainnet+ to deploy it in production. As you make changes, improvements, bug fixes, etc., start the pipeline again, deploying each change first on +regtest+, then on +testnet+, and finally into production.((("", startref="BCTtest09")))

ch09_fees.adoc

@@ -0,0 +1,685 @@
+[[tx_fees]]
+== Transaction Fees
+
+++++
+<p class="fix_tracking">
+The digital signature we saw Alice create in <a data-type="xref" href="#c_signatures">#c_signatures</a> only
+proves that she knows her private key and that she committed to a
+transaction that pays Bob.  She can create another signature that
+instead commits to a transaction paying Carol—a transaction that spends
+the same output (bitcoins) that she used to pay Bob.  Those two
+transactions are now <em>conflicting transactions</em> because only one
+transaction spending a particular output can be included in the valid
+blockchain with the most proof of work—the blockchain that full nodes
+use to determine which keys control which bitcoins.
+</p>
+++++
+
+To((("conflicting transactions")))((("transactions", "conflicts in"))) protect himself against conflicting transactions, it would be wise
+for Bob to wait until the transaction from Alice is included in the
+blockchain to a sufficient depth before he considers the money he
+received as his to spend (see <<confirmations>>).
+For Alice's transaction to be included in the
+blockchain, it must be included in a _block_ of transactions.  There are
+a limited number of((("blocks", "transactions in")))((("transactions", "in blocks", secondary-sortas="blocks"))) blocks produced in a given amount of time, and each
+block only has a limited amount of space.  Only the miner who creates
+that block gets to choose which transactions to include.  Miners may
+select transactions by any criteria they want, including refusing to
+include any transactions at all.
+
+++++
+<div data-type="note">
+<p class="fix_tracking"> When we say "transactions" in this chapter, we refer to every
+transaction in a block except for the first transaction.  The first
+transaction in a block is a <em>coinbase transaction</em>, described in
+<a data-type="xref" href="#coinbase_transactions">#coinbase_transactions</a>, which allows the miner of the block to
+collect their reward for producing the block.  Unlike other
+transactions, a coinbase transaction doesn't spend the output of a
+previous transaction and is also an exception to several other rules
+that apply to other transactions.  Coinbase transactions don't pay
+transaction fees, don't need to be fee bumped, aren't subject to
+transaction pinning, and are largely uninteresting to the following
+discussion about fees—so we're going to ignore them in this chapter.
+</p>
+</div>
+++++
+
+
+The criterion that almost all miners use to select which transactions to
+include in their blocks is to maximize their revenue.  Bitcoin was
+specifically designed to accommodate this by providing a mechanism that
+allows a transaction to give money to the miner who includes that
+transaction in a block.  We call that mechanism _transaction fees_,
+although it's not a fee in the usual sense of that word.  It's not an
+amount set by the protocol or by any particular miner--it's much more
+like a bid in an auction.  The good being purchased is the portion of
+limited space in a block that a transaction will consume.  Miners choose
+the set of transactions whose bids will allow them to earn the greatest
+revenue.
+
+In this chapter, we'll explore various aspects of those
+bids--transaction fees--and how they influence the creation and
+management of Bitcoin transactions.
+
+=== Who Pays the Transaction Fee?
+
+Most ((("transaction fees", "responsibility for", id="fees-responsibility")))((("payments", "transaction fees", see="transaction fees")))((("fees", see="transaction fees")))payment systems involve some sort of fee for transacting, but
+often this fee is hidden from typical buyers.  For example, a merchant
+may advertise the same item for the same price whether you pay with cash
+or a credit card even though their payment processor may charge them
+a higher fee for credit transactions than their bank charges them for
+cash deposits.
+
+In Bitcoin, every spend of bitcoins must be authenticated (typically
+with a signature), so it's not possible for a transaction to pay a fee
+without the permission of the spender.  It is possible for the receiver
+of a transaction to pay a fee in a different transaction--and we'll see
+that in use later--but if we want a single transaction to pay its own
+fee, that fee needs to be something agreed upon by the spender.  It
+can't be hidden.
+
+Bitcoin transactions are designed so that it doesn't take any extra
+space in a transaction for a spender to commit to the fee it pays.  That
+means that, even though it's possible to pay the fee in a different
+transaction, it's most efficient (and thus cheapest) to pay the fee in a
+single transaction.
+
+In Bitcoin,
+the fee is a bid and the amount paid contributes to determining how long
+it will take the transaction to confirm.  Both spenders and receivers of
+a payment typically have an interest in having it confirming quickly, so
+normally allowing only spenders to choose fees can sometimes be a
+problem; we'll look at a solution to that problem in <<cpfp>>.  However,
+in many common payment flows, the parties with the highest desire to see a
+transaction confirm quickly--that is, the parties who would be the most
+willing to pay higher fees--are the spenders.
+
+For those reasons, both technical and practical, it is customary in
+Bitcoin for spenders to pay transaction fees.  There are exceptions,
+such as for merchants that accept unconfirmed transactions and in
+protocols that don't immediately broadcast [.keep-together]#transactions# after they are
+signed (preventing the spender from being able to choose an appropriate
+fee for the current market).  We'll explore those exceptions((("transaction fees", "responsibility for", startref="fees-responsibility"))) later.
+
+=== Fees and Fee Rates
+
+Each ((("transaction fees", "fee rates", id="fees-rates")))((("fee rates", id="fee-rate")))transaction only pays a single fee--it doesn't matter how large the
+transaction is.  However, the larger transactions become, the fewer of
+them a miner will be able to fit in a block.  For that reason, miners
+evaluate transactions the same way you might comparison shop between
+several equivalent items at the market: they divide the price by the
+quantity.
+
+Whereas you might divide the cost of several different bags of rice by
+each bag's weight to find the lowest price per weight (best deal), miners
+divide the fee of a transaction by its size (also called its weight) to
+find the highest fee per weight (most revenue).  In Bitcoin, we use the
+term _fee rate_ for a transaction's size divided by weight.  Due to
+changes in Bitcoin over the years, fee rate can be expressed in
+different units:
+
+- BTC/Bytes (a legacy unit rarely used anymore)
+- BTC/Kilobytes (a legacy unit rarely used anymore)
+- BTC/Vbytes (rarely used)
+- BTC/Kilo-vbyte (used mainly in Bitcoin Core)
+- Satoshi/Vbyte (most commonly used today)
+- Satoshi/Weight (also commonly used today)
+
+We recommend either the sat/vbyte or sat/weight units for displaying
+fee rates.
+
+[WARNING]
+====
+Be careful ((("absurd fees")))((("excessive fees")))((("transaction fees", "overpaying")))((("overpaying transaction fees")))accepting input for fee rates.  If a user copies and pastes a
+fee rate printed in one denominator into a field using a different
+denominator, they could overpay fees by 1,000 times.  If they instead
+switch the numerator, they could theoretically overpay by 100,000,000
+times.  Wallets should make it hard for the user to pay an excessive
+fee rate and may want to prompt the user to confirm any fee rate that was
+not generated by the wallet itself using a trusted data source.
+
+An excessive fee, also called an _absurd fee_, is any fee rate that's
+significantly higher than the amount that fee rate estimators currently
+expect is necessary to get a transaction confirmed in the next block.
+Note that wallets should not entirely prevent users from choosing an
+excessive fee rate--they should only make using such a fee rate hard to do
+by accident.  There are legitimate reasons for users to overpay fees on
+rare occasions.
+====
+
+=== Estimating Appropriate Fee Rates
+
+We've ((("estimating fee rates", id="estimate-fee-rate")))established that you can pay a lower fee rate if you're willing to
+wait longer for your transaction to be confirmed, with the exception
+that paying too low of a fee rate could result in your transaction never
+confirming.  Because fee rates are bids in an open auction for block
+space, it's not possible to perfectly predict what fee rate you need to
+pay to get your transaction confirmed by a certain time.  However, we
+can generate a rough estimate based on what fee rates other transactions
+have paid in the recent past.
+
+A full node can record three pieces of information about each
+transactions it sees: the time (block height) when it first received
+that transaction, the block height when that transaction was confirmed,
+and the fee rate paid by that transaction.  By grouping together
+transactions that arrived at similar heights, were confirmed at similar
+heights, and which paid similar fees, we can calculate how many blocks it
+took to confirm transactions paying a certain fee rate.  We can then
+assume that a transaction paying a similar fee rate now will take a
+similar number of blocks to confirm.  Bitcoin Core includes a fee rate
+estimator that uses these principles, which can be called using the
+`estimatesmartfee` RPC with a parameter specifying how many blocks
+you're willing to wait before the transaction is highly likely to
+confirm (for example, 144 blocks is about 1 day):
+
+----
+$ bitcoin-cli -named estimatesmartfee conf_target=144
+{
+  "feerate": 0.00006570,
+  "blocks": 144
+}
+----
+
+Many web-based services also provide fee estimation as an API.  For a
+current list, see https://oreil.ly/TB6IN.
+
+As mentioned, fee rate estimation can never be perfect.  One common
+problem is that the fundamental demand might change, adjusting the
+equilibrium and either increasing prices (fees) to new heights or
+decreasing them toward the minimum.
+If fee rates go down, then a transaction
+that previously paid a normal fee rate might now be paying a high fee
+rate and it will be confirmed earlier than expected.  There's no way to
+lower the fee rate on a transaction you've already sent, so you're stuck
+paying a higher fee rate.  But, when fee rates go up, there's a need for
+methods to be able to increase the fee rates on those transactions,
+which is called _fee bumping_.  There are two commonly used types of fee
+bumping in Bitcoin, replace by fee (RBF) and child pays for ((("fee rates", startref="fee-rate")))((("transaction fees", "fee rates", startref="fees-rates")))parent
+(CPFP).
+
+[[rbf]]
+=== Replace By Fee (RBF) Fee Bumping
+
+To((("transaction fees", "fee bumping", "RBF (replace by fee)", id="transaction-fees-bump-rbf")))((("fee bumping", "RBF (replace by fee)", id="fee-bump-rbf")))((("RBF (replace by fee) fee bumping", id="rbf-ch9"))) increase the fee of a transaction using RBF fee bumping, you create
+a conflicting version of the transaction that pays a higher fee.  Two
+or more transactions are considered((("conflicting transactions")))((("transactions", "conflicts in"))) to be _conflicting transactions_ if
+only one of them can be included in a valid blockchain, forcing a miner
+to choose only one of them.  Conflicts occur when two or more transactions
+each try to spend one of the same UTXOs, i.e., they each include an input
+that has the same outpoint (reference to the output of a previous
+transaction).
+
+To prevent someone from consuming large amounts of bandwidth by creating
+an unlimited number of conflicting transactions and sending them through
+the network of relaying full nodes, Bitcoin Core and other full nodes
+that support transaction replacement require each replacement
+transaction to pay a higher fee rate than the transaction being
+replaced.  Bitcoin Core also currently requires the replacement
+transaction to pay a higher total fee than the original transaction, but
+this requirement has undesired side effects and developers have been
+looking for ways to remove it at the time of writing.
+
+Bitcoin Core((("Bitcoin Core", "RBF variants", id="bitcoin-core-rbf"))) currently supports two variations of RBF:
+
+Opt-in RBF::
+  An unconfirmed transaction can signal to miners and full nodes that
+  the creator of the transaction wants to allow it to be replaced by a
+  higher fee rate version.  This signal and the rules for using it
+  are specified in BIP125.  As of this writing, this has been enabled by
+  default in Bitcoin Core for several years.
+
+Full RBF::
+  Any unconfirmed transaction can be replaced by a higher fee rate
+  version.  As of this writing, this can be optionally enabled in
+  Bitcoin Core (but it is disabled by default).
+
+.Why Are There Two Variants of RBF?
+****
+The reason for the two different versions of RBF is that full RBF has
+been controversial.  Early versions of Bitcoin allowed transaction
+replacement, but this behavior was disabled for several releases.  During
+that time, a miner or full node using the software now called Bitcoin
+Core would not replace the first version of an unconfirmed transaction
+they received with any different version.  Some merchants came to expect
+this behavior: they assumed that any valid unconfirmed transaction that
+paid an appropriate fee rate would eventually become a confirmed
+transaction, so they provided their goods or services shortly after
+receiving such an unconfirmed transaction.
+
+However, there's no way for the Bitcoin protocol to guarantee that any
+unconfirmed transaction will eventually be confirmed.  As mentioned
+earlier in this chapter, every miner gets to choose for themselves which
+transactions they will try to confirm--including which versions of those
+transactions.  Bitcoin Core is open source software, so anyone with a
+copy of its source code can add (or remove) transaction replacement.
+Even if Bitcoin Core wasn't open source, Bitcoin is an open protocol
+that can be reimplemented from scratch by a sufficiently competent
+programmer, allowing the reimplementor to include or not include
+transaction replacement.
+
+Transaction replacement breaks the assumption of some merchants that
+every reasonable unconfirmed transaction will eventually be confirmed.
+An alternative version of a transaction can pay the same outputs as the
+original, but it isn't required to pay any of those outputs.  If the
+first version of an unconfirmed transaction pays a merchant, the second
+version might not pay them.  If the merchant provided goods or services
+based on the first version, but the second version gets confirmed, then
+the merchant will not receive payment for its costs.
+
+Some merchants, and people supporting them, requested that transaction
+replacement not be reenabled in Bitcoin Core.  Other people pointed out
+that transaction replacement provides benefits, including the ability to
+fee bump transactions that initially paid too low of a fee rate.
+
+Eventually, developers working on Bitcoin Core implemented a compromise:
+instead of allowing every unconfirmed transaction to be replaced (full
+RBF), they only programmed Bitcoin Core to allow replacement of
+transactions that signaled they wanted to allow replacement (opt-in RBF).
+Merchants can check the transactions they receive for the opt-in
+signal and treat those transactions differently than those without the
+signal.
+
+This doesn't change the fundamental concern: anyone can still alter
+their copy of Bitcoin Core, or create a reimplementation, to allow full
+RBF--and some developers even did this, but seemingly few people used
+their software.
+
+After several years, developers working on Bitcoin Core changed the
+compromise slightly.  In addition to keeping opt-in RBF by default, they
+added an option that allows users to enable full RBF.  If enough mining
+hash rate and relaying full nodes enable this option, it will be
+possible for any unconfirmed transaction to eventually be replaced by a
+version paying a higher fee rate.  As of this writing, it's not clear
+whether or not that has happened ((("Bitcoin Core", "RBF variants", startref="bitcoin-core-rbf")))yet.
+****
+
+As a user, if you plan to use RBF fee bumping, you will first need to
+choose a wallet that supports it, such as one of the wallets listed as
+having "Sending support" on
+https://oreil.ly/IhMzx.
+
+As a developer, if you plan to implement RBF fee bumping, you will first
+need to decide whether to perform opt-in RBF or full RBF.  At the time
+of writing, opt-in RBF is the only method that's sure to work.  Even if
+full RBF becomes reliable, there will likely be several years where
+replacements of opt-in transactions get confirmed slightly faster than
+full-RBF replacements.  If you choose opt-in RBF, your wallet will need
+to implement the signaling specified in BIP125, which is a simple
+modification to any one of the sequence fields in a transaction (see
+<<sequence>>).  If you choose full RBF, you don't need to include any
+signaling in your transactions.  Everything else related to RBF is the
+same for both approaches.
+
+When you need to fee bump a transaction, you will simply create a new
+transaction that spends at least one of the same UTXOs as the original
+transaction you want to replace.  You will likely want to keep the
+same outputs in the transaction that pay the receiver (or receivers).
+You may pay the increased fee by reducing the value of your change
+output or by adding additional inputs to the transaction.  Developers
+should provide users with a fee-bumping interface that does all of this
+work for them and simply asks them (or suggests to them) how much the
+fee rate should be increased.
+
+[WARNING]
+====
+Be very careful when creating more than one replacement of the same
+transaction.  You must ensure than all versions of the transactions
+conflict with each other.  If they aren't all conflicts, it may be
+possible for multiple separate transactions to confirm, leading you to
+overpay the receivers.  For example:
+
+- Transaction version 0 includes input _A_.
+- Transaction version 1 includes inputs _A_ and _B_ (e.g., you had to add
+  input _B_ to pay the extra fees)
+- Transaction version 2 includes inputs _B_ and _C_ (e.g., you had to add input
+  _C_ to pay the extra fees but _C_ was large enough that you no longer
+  need input _A_).
+
+In this scenario, any miner who saved version 0 of the transaction
+will be able to confirm both it and version 2 of the transaction.  If
+both versions pay the same receivers, they'll be paid twice (and the
+miner will receive transaction fees from two separate transactions).
+
+A simple method to avoid this problem is to ensure the replacement
+transaction always includes all of the same inputs as the previous
+version of the transaction.
+====
+
+The advantage of RBF fee bumping over other types of fee bumping is that
+it can be very efficient at using block space.  Often, a replacement
+transaction is the same size as the transaction it replaces.  Even when
+it's larger, it's often the same size as the transaction the user would
+have created if they had paid the increased fee rate in the first place.
+
+The fundamental disadvantage of RBF fee bumping is that it can normally
+only be performed by the creator of the transaction--the person or
+people who were required to provide signatures or other authentication
+data for the transaction.  An exception to this is transactions that
+were designed to allow additional inputs to be added by using sighash
+flags (see <<sighash_types>>), but that presents its own challenges.  In
+general, if you're the receiver of an unconfirmed transaction and you
+want to make it confirm faster (or at all), you can't use an RBF fee
+bump; you need some other method.
+
+There are additional problems ((("transaction fees", "fee bumping", "RBF (replace by fee)", startref="transaction-fees-bump-rbf")))((("fee bumping", "RBF (replace by fee)", startref="fee-bump-rbf")))((("RBF (replace by fee) fee bumping", startref="rbf-ch9")))with RBF that we'll explore in <<transaction_pinning>>.
+
+[[cpfp]]
+=== Child Pays for Parent (CPFP) Fee Bumping
+
+Anyone ((("transaction fees", "fee bumping", "CPFP (child pays for parent)", id="transaction-fees-bump-cpfp")))((("fee bumping", "CPFP (child pays for parent)", id="fee-bump-cpfp")))((("CPFP (child pays for parent) fee bumping", id="cpfp-ch9")))who receives the output of an unconfirmed transaction can
+incentivize miners to confirm that transaction by spending that output.
+The transaction you want to get confirmed is called the _parent
+transaction_.  A transaction that spends an output of the parent
+transaction is called a _child transaction_.
+
+As we learned in <<outpoints>>, every input in a confirmed transaction
+must reference the unspent output of a transaction that appears earlier
+in the blockchain (whether earlier in the same block or in a previous
+block).  That means a miner who wants to confirm a child transaction
+must also ensure that its parent transaction is confirmed.  If the
+parent transaction hasn't been confirmed yet but the child transaction
+pays a high enough fee, the miner can consider whether it would be
+profitable to confirm both of them in the same block.
+
+To evaluate the profitability of mining both a parent and child
+transaction, the miner looks at them as a _package of transactions_ with
+an aggregate size and aggregate fees, from which the fees can be divided
+by the size to calculate((("package fee rate"))) a _package fee rate_.  The miner can then sort
+all of the individual transactions and transaction packages they know
+about by fee rate and include the highest-revenue ones in the block
+they're attempting to mine, up to the maximum size (weight) allowed to
+be included in a block.  To find even more packages that might be
+profitable to mine, the miner can evaluate packages across multiple
+generations (e.g., an unconfirmed parent transaction being combined with
+both its child and grandchild).  This is ((("ancestor fee rate mining")))((("CPFP (child pays for parent) fee bumping", "ancestor fee rate mining")))called _ancestor fee rate
+mining_.
+
+Bitcoin Core has implemented ancestor fee rate mining for many years,
+and it's believed that almost all miners use it at the time of writing.
+That means it's practical for wallets to use this feature to fee bump an
+incoming transaction by using a child transaction to pay for its parent
+(CPFP).
+
+CPFP has several advantages over RBF.  Anyone who receives an output
+from a transaction can use CPFP--that includes both the receivers of
+payments and the spender (if the spender included a change output).  It
+also doesn't require replacing the original transaction, which makes it
+less disruptive to some merchants than RBF.
+
+The primary disadvantage of CPFP compared to RBF is that CPFP typically
+uses more block space.  In RBF, a fee bump transaction is often the same
+size as the transaction it replaces.  In CPFP, a fee bump adds a whole
+separate transaction.  Using extra block space requires paying extra
+fees beyond the cost of the fee bump.
+
+There are several challenges with CPFP, some of which we'll explore in
+<<transaction_pinning>>.  One other problem that we
+specifically need to mention is the minimum relay fee rate problem,
+which is addressed by ((("transaction fees", "fee bumping", "CPFP (child pays for parent)", startref="transaction-fees-bump-cpfp")))((("fee bumping", "CPFP (child pays for parent)", startref="fee-bump-cpfp")))((("CPFP (child pays for parent) fee bumping", startref="cpfp-ch9")))package relay.
+
+=== Package Relay
+
+Early versions((("transaction fees", "package relay", id="transaction-fee-package-relay")))((("package relay", id="package-relay"))) of Bitcoin Core didn't place any limits on the number of
+unconfirmed transactions they stored for later relay and mining in their
+mempools (see <<mempool>>).  Of course, computers have physical limits, whether
+it's the memory (RAM) or disk space--it's not possible for a full node
+to store an unlimited number of unconfirmed transactions.  Later
+versions of Bitcoin Core limited the size of the mempool to hold about
+one day's worth of transactions, storing only the transactions or packages
+with the highest fee rate.
+
+That works extremely well for most things, but it creates a dependency
+problem.  In order to calculate the fee rate for a transaction package,
+we need both the parent and descendant transactions--but if the parent
+transaction doesn't pay a high enough fee rate, it won't be kept in a
+node's mempool.  If a node receives a child transaction without having
+access to its parent, it can't do anything with that transaction.
+
+The solution to this problem is the ability to relay transactions as a
+package, called _package relay_, allowing the receiving node to evaluate
+the fee rate of the entire package before operating on any individual
+transaction.  As of this writing, developers working on Bitcoin Core
+have made significant progress on implementing package relay, and a
+limited early version of it may be available by the time this book is
+published.
+
+Package relay is especially important for protocols based on
+time-sensitive presigned transactions, such as Lightning Network (LN).  In
+non-cooperative cases, some presigned transactions can't be fee bumped
+using RBF, forcing them to depend on CPFP.  In those protocols, some
+transactions may also be created long before they need to be broadcast,
+making it effectively impossible to estimate an appropriate fee rate.
+If a presigned transaction pays a fee rate below the amount necessary to
+get into a node's mempool, there's no way to fee bump it with a child.
+If that prevents the transaction from confirming in time, an honest user
+might lose money.  Package relay is the solution for this critical
+problem.
+
+[[transaction_pinning]]
+=== Transaction Pinning
+
+++++
+<p class="fix_tracking">
+Although both RBF and CPFP fee bumping work in the basic cases we
+described, there are rules related to both
+methods that are designed to prevent denial-of-service attacks on miners
+and relaying full nodes.  An unfortunate side effect of those rules
+is that they can sometimes prevent someone from being able to use fee
+bumping.  Making it impossible or difficult to fee bump a transaction is
+called <em>transaction pinning</em>.</p>
+++++
+
+One((("transaction fees", "fee bumping", "transaction pinning", id="transaction-fee-bump-pin")))((("fee bumping", "transaction pinning", id="fee-bump-pin")))((("transaction pinning", id="transaction-pin")))((("RBF (replace by fee) fee bumping", "transaction pinning", id="rbf-pin")))((("CPFP (child pays for parent) fee bumping", "transaction pinning", id="cpfp-pin"))) of the major denial of service concerns revolves around the effect of
+transaction relationships.  Whenever the output of a transaction is
+spent, that transaction's identifier (txid) is referenced by the child
+transaction.  However, when a transaction is replaced, the replacement
+has a different txid.  If that replacement transaction gets confirmed,
+none of its descendants can be included in the same blockchain.  It's
+possible to re-create and re-sign the descendant transactions, but that's
+not guaranteed to happen.  This has related but divergent implications
+for RBF and CPFP:
+
+- In the context of RBF, when Bitcoin Core accepts a replacement
+  transaction, it keeps things simple by forgetting about the original
+  transaction and all descendant transactions that depended on that
+  original.  To ensure that it's more profitable for miners to accept
+  replacements, Bitcoin Core only accepts a replacement transaction if it
+  pays more fees than all the transactions that will be forgotten.
++
+The downside of this approach is that Alice can create a small
+transaction that pays Bob.  Bob can then use his output to create a
+large child transaction.  If Alice then wants to replace her original
+transaction, she needs to pay a fee that's larger than what both she and
+Bob originally paid.  For example, if Alice's original transaction was
+about 100 vbytes and Bob's transaction was about 100,000 vbytes, and
+they both used the same fee rate, Alice now needs to pay more than 1,000
+times as much as she originally paid in order to RBF fee bump her
+transaction.
+
+- In the context of CPFP, any time the node considers including a
+  package in a block, it must remove the transactions in that package
+  from any other package it wants to consider for the same block.  For
+  example, if a child transaction pays for 25 ancestors, and each of
+  those ancestors has 25 other children, then including the package in
+  the block requires updating approximately 625 packages (25^2^).
+  Similarly, if a transaction with 25 descendants is removed from a
+  node's mempool (such as for being included in a block), and each of
+  those descendants has 25 other ancestors, another 625 packages need to
+  be updated.  Each time we double our parameter (e.g., from 25 to 50),
+  we quadruple the amount of work our node needs to perform.
++
+Additionally, a transaction and all of its descendants is not
+  useful to keep in a mempool long term if an alternative version of
+  that transaction is mined--none of those transactions can now be
+  confirmed unless there's a rare blockchain reorganization.  Bitcoin
+  Core will remove from its mempool every transaction that can no longer
+  be confirmed on the current blockchain.  At it's worst, that can
+  waste an enormous amount of your node's bandwidth and possibly be used
+  to prevent transactions from propagating correctly.
++
+To prevent these problems, and other related
+  problems, Bitcoin Core limits a parent transaction to having a maximum
+  of 25 ancestors or descendants in its mempool and limits the
+  total size of all those transactions to 100,000 vbytes.  The downside
+  of this approach is that users are prevented from creating CPFP fee
+  bumps if a transaction already has too many descendants (or if it and
+  its descendants are too large).
+
+Transaction pinning can happen by accident, but it also represents a
+serious vulnerability for multiparty time-sensitive protocols such as
+LN.  If your counterparty can prevent one of your
+transactions from confirming by a deadline, they may be able to steal
+money from you.
+
+Protocol developers have been working on mitigating problems with
+transaction pinning for several years.  One partial solution is
+described in <<cpfp_carve_out>>.  Several other solutions have been
+proposed, and at least one solution is being actively ((("transaction fees", "fee bumping", "transaction pinning", startref="transaction-fee-bump-pin")))((("fee bumping", "transaction pinning", startref="fee-bump-pin")))((("transaction pinning", startref="transaction-pin")))((("RBF (replace by fee) fee bumping", "transaction pinning", startref="rbf-pin")))((("CPFP (child pays for parent) fee bumping", "transaction pinning", startref="cpfp-pin")))((("transaction fees", "fee bumping", "CPFP carve outs", id="transaction-fee-bump-carveout")))((("fee bumping", "CPFP carve outs", id="fee-bump-carveout")))((("carve outs (CPFP)", id="carveout")))((("CPFP (child pays for parent) fee bumping", "carve outs", id="cpfp-carveout")))developed as of
+this writing&mdash;https://oreil.ly/300dv[ephemeral anchors].
+
+[[cpfp_carve_out]]
+=== CPFP Carve Out and Anchor Outputs
+
+++++
+<p class="fix_tracking2">
+In 2018, developers working on LN had a problem.
+Their protocol uses transactions that require signatures from two
+different parties.  Neither party wants to trust the other, so they sign
+transactions at a point in the protocol when trust isn't needed,
+allowing either of them to broadcast one of those transactions at a
+later time when the other party may not want to (or be able to) fulfill
+its obligations.  The problem with this approach is that the
+transactions might need to be broadcast at an unknown time, far in the future, beyond any
+reasonable ability to estimate an appropriate fee rate for the
+transactions.</p>
+++++
+
+In theory, the developers could have designed their transactions to
+allow fee bumping with either RBF (using special sighash flags) or CPFP,
+but both of those protocols are vulnerable to transaction pinning.
+Given that the involved transactions are time sensitive, allowing a
+counterparty to use transaction pinning to delay confirmation of a
+transaction can easily lead to a repeatable exploit that malicious
+parties could use to steal money from honest parties.
+
+LN developer Matt Corallo proposed a solution: give the rules for CPFP
+fee bumping a special exception, called _CPFP carve out_.  The normal
+rules for CPFP forbid the inclusion of an additional descendant if it
+would cause a parent transaction to have 26 or more descendants or if it
+would cause a parent and all of its descendants to exceed 100,000 vbytes
+in size.  Under the rules of CPFP carve out, a single additional
+transaction up to 1,000 vbytes in size can be added to a package even if
+it would exceed the other limits as long as it is a direct child of an
+unconfirmed transaction with no unconfirmed ancestors.
+
+++++
+<p class="fix_tracking">
+For example, Bob and Mallory both co-sign a transaction with two
+outputs, one to each of them.  Mallory broadcasts that transaction and
+uses her output to attach either 25 child transactions or any smaller
+number of child transactions equaling 100,000 vbytes in size.  Without
+carve-out, Bob would be unable to attach another child transaction to
+his output for CPFP fee bumping.  With carve-out, he can spend one of
+the two outputs in the transaction, the one that belongs to him, as long
+as his child transaction is less than 1,000 vbytes in size (which should
+be more than enough space).</p>
+++++
+
+It's not allowed to use CPFP carve-out more than once, so it only works
+for two-party protocols.  There have been proposals to extend it to
+protocols involving more participants, but there hasn't been much demand
+for that and developers are focused on building more generic solutions
+to transaction pinning attacks.
+
+As of this writing, most popular LN implementations use a transaction
+template called _anchor outputs_, which is designed to be used ((("anchor outputs (CPFP)")))((("transaction fees", "fee bumping", "CPFP carve outs", startref="transaction-fee-bump-carveout")))((("fee bumping", "CPFP carve outs", startref="fee-bump-carveout")))((("carve outs (CPFP)", startref="carveout")))((("CPFP (child pays for parent) fee bumping", "carve outs", startref="cpfp-carveout")))with CPFP
+carve out.
+
+=== Adding Fees to Transactions
+
+The data((("transaction fees", "change outputs and")))((("change output", "transaction fees and")))((("outputs", "transaction fees and")))((("inputs", "transaction fees and"))) structure of transactions does not have a field for fees.
+Instead, fees are implied as the difference between the sum of inputs
+and the sum of outputs. Any excess amount that remains after all outputs
+have been deducted from all inputs is the fee that is collected by the
+miners:
+
+[latexmath]
+++++
+\begin{equation}
+{Fees = Sum(Inputs) - Sum(Outputs)}
+\end{equation}
+++++
+
+
+This is a somewhat confusing element of transactions and an important
+point to understand because if you are constructing your own
+transactions, you must ensure you do not inadvertently include a very
+large fee by underspending the inputs. That means that you must account
+for all inputs, if necessary, by creating change, or you will end up
+giving the miners a very big tip!
+
+For example, if you spend a 20-bitcoin UTXO to make a 1-bitcoin
+payment, you must include a 19-bitcoin change output back to your
+wallet. Otherwise, the 19-bitcoin "leftover" will be counted as a
+transaction fee and will be collected by the miner who mines your
+transaction in a block. Although you will receive priority processing
+and make a miner very happy, this is probably not what you intended.
+
+[WARNING]
+====
+If you forget to add a
+change output in a manually constructed transaction, you will be paying
+the change as a transaction fee. "Keep the change!" might not be what
+you intended.
+====
+
+[[fee_sniping]]
+=== Timelock Defense Against Fee Sniping
+
+Fee sniping ((("transaction fees", "fee sniping", id="transaction-fee-sniping")))((("fee sniping", id="fee-snipe")))((("timelocks", "fee sniping and", id="timelock-fee-snipe")))((("lock time", "fee sniping and", id="lock-time-fee-snipe")))is a theoretical
+attack scenario where miners attempting to rewrite past blocks "snipe"
+higher-fee transactions from future blocks to maximize their
+[.keep-together]#profitability.#
+
+For example, let's say the highest block in existence is block
+#100,000. If instead of attempting to mine block #100,001 to extend the
+chain, some miners attempt to remine  #100,000. These miners can choose
+to include any valid transaction (that hasn't been mined yet) in their
+candidate block  #100,000. They don't have to remine the block with the
+same transactions. In fact, they have the incentive to select the most
+profitable (highest fee per kB) transactions to include in their block.
+They can include any transactions that were in the "old" block
+#100,000, as well as any transactions from the current mempool.
+Essentially they have the option to pull transactions from the "present"
+into the rewritten "past" when they re-create block  #100,000.
+
+Today, this attack is not very lucrative because the block subsidy is much
+higher than total fees per block. But at some point in the future,
+transaction fees will be the majority of the reward (or even the
+entirety of the reward). At that time, this scenario becomes inevitable.
+
+Several wallets discourage fee sniping by creating transactions with a
+lock time that limits those transactions to being included in the next
+block or any later block. In our
+scenario, our wallet would set lock time to 100,001 on any
+transaction it created. Under normal circumstances, this lock time has
+no effect&#x2014;the transactions could only be included in block
+#100,001 anyway; it's the next block.
+
+But under a reorganization attack, the miners would not be able to pull
+high-fee transactions from the mempool because all those transactions
+would be timelocked to block #100,001. They can only remine  #100,000
+with whatever transactions were valid at that time, essentially gaining
+no new fees.
+
+This does not entirely prevent fee sniping, but it does make it less
+profitable in some cases and can help preserve the stability of the
+Bitcoin network as the block subsidy declines.  We recommend all wallets
+implement anti-fee sniping when it doesn't interfere with the wallet's
+other uses of the lock time field.
+
+As Bitcoin continues to mature, and as the subsidy continues to decline,
+fees become more and more important to Bitcoin users, both in their
+day-to-day use for getting transactions confirmed quickly and in
+providing an incentive for miners to continue securing Bitcoin
+transactions with new proof of work.

ch11.asciidoc

@@ -1,78 +0,0 @@
-[[ch11]]
-== Bitcoin Security
-
-Securing bitcoin is challenging because bitcoin is not an abstract reference to value, like a balance in a bank account. Bitcoin is very much like digital cash or gold. You've probably heard the expression, "Possession is nine-tenths of the law." Well, in bitcoin, possession is ten-tenths of the law. Possession of the keys to unlock the bitcoin is equivalent to possession of cash or a chunk of precious metal. You can lose it, misplace it, have it stolen, or accidentally give the wrong amount to someone. In every one of these cases, users have no recourse, just as if they dropped cash on a public sidewalk.
-
-However, bitcoin has capabilities that cash, gold, and bank accounts do not. A bitcoin wallet, containing your keys, can be backed up like any file. It can be stored in multiple copies, even printed on paper for hard-copy backup. You can't "back up" cash, gold, or bank accounts. Bitcoin is different enough from anything that has come before that we need to think about bitcoin security in a novel way too.
-
-=== Security Principles
-
-((("security", "security principles", id="Sprinc11")))((("decentralized systems", "security of")))The core principle in bitcoin is decentralization and it has important implications for security. A centralized model, such as a traditional bank or payment network, depends on access control and vetting to keep bad actors out of the system. By comparison, a decentralized system like bitcoin pushes the responsibility and control to the users. Because security of the network is based on Proof-of-Work, not access control, the network can be open and no encryption is required for bitcoin traffic.
-
-On a traditional payment network, such as a credit card system, the payment is open-ended because it contains the user's private identifier (the credit card number). After the initial charge, anyone with access to the identifier can "pull" funds and charge the owner again and again. Thus, the payment network has to be secured end-to-end with encryption and must ensure that no eavesdroppers or intermediaries can compromise the payment traffic, in transit or when it is stored (at rest). If a bad actor gains access to the system, he can compromise current transactions _and_ payment tokens that can be used to create new transactions. Worse, when customer data is compromised, the customers are exposed to identity theft and must take action to prevent fraudulent use of the compromised accounts.
-
-Bitcoin is dramatically different. A bitcoin transaction authorizes only a specific value to a specific recipient and cannot be forged or modified. It does not reveal any private information, such as the identities of the parties, and cannot be used to authorize additional payments. Therefore, a bitcoin payment network does not need to be encrypted or protected from eavesdropping. In fact, you can broadcast bitcoin transactions over an open public channel, such as unsecured WiFi or Bluetooth, with no loss of security.
-
-Bitcoin's decentralized security model puts a lot of power in the hands of the users. With that power comes responsibility for maintaining the secrecy of the keys. For most users that is not easy to do, especially on general-purpose computing devices such as internet-connected smartphones or laptops. Although bitcoin's decentralized model prevents the type of mass compromise seen with credit cards, many users are not able to adequately secure their keys and get hacked, one by one.
-
-
-==== Developing Bitcoin Systems Securely
-
-((("decentralized systems", "bitcoin as")))The most important principle for bitcoin developers is decentralization. Most developers will be familiar with centralized security models and might be tempted to apply these models to their bitcoin applications, with disastrous results.
-
-Bitcoin's security relies on decentralized control over keys and on independent transaction validation by miners. If you want to leverage bitcoin's security, you need to ensure that you remain within the bitcoin security model. In simple terms: don't take control of keys away from users and don't take transactions off the blockchain.
-
-For example, many early bitcoin exchanges concentrated all user funds in a single "hot" wallet with keys stored on a single server. Such a design removes control from users and centralizes control over keys in a single system. Many such systems have been hacked, with disastrous consequences for their customers.
-
-((("transactions", "off blockchain")))((("off-blockchain transactions")))Another common mistake is to take transactions "off blockchain" in a misguided effort to reduce transaction fees or accelerate transaction processing. An "off blockchain" system will record transactions on an internal, centralized ledger and only occasionally synchronize them to the Bitcoin blockchain. This practice, again, substitutes decentralized bitcoin security with a proprietary and centralized approach. When transactions are off blockchain, improperly secured centralized ledgers can be falsified, diverting funds and depleting reserves, unnoticed.
-
-Unless you are prepared to invest heavily in operational security, multiple layers of access control, and audits (as the traditional banks do) you should think very carefully before taking funds outside of bitcoin's decentralized security context. Even if you have the funds and discipline to implement a robust security model, such a design merely replicates the fragile model of traditional financial networks, plagued by identity theft, corruption, and embezzlement. To take advantage of bitcoin's unique decentralized security model, you have to avoid the temptation of centralized architectures that might feel familiar but ultimately subvert bitcoin's security.
-
-==== The Root of Trust
-
-((("root of trust concept")))Traditional security architecture is based upon a concept called the _root of trust_, which is a trusted core used as the foundation for the security of the overall system or application. Security architecture is developed around the root of trust as a series of concentric circles, like layers in an onion, extending trust outward from the center. Each layer builds upon the more-trusted inner layer using access controls, digital signatures, encryption, and other security primitives. As software systems become more complex, they are more likely to contain bugs, which make them vulnerable to security compromise. As a result, the more complex a software system becomes, the harder it is to secure. The root of trust concept ensures that most of the trust is placed within the least complex part of the system, and therefore least vulnerable, parts of the system, while more complex software is layered around it. This security architecture is repeated at different scales, first establishing a root of trust within the hardware of a single system, then extending that root of trust through the operating system to higher-level system services, and finally across many servers layered in concentric circles of diminishing trust.
-
-((("mining and consensus", "security and consensus")))Bitcoin security architecture is different. In bitcoin, the consensus system creates a trusted public ledger that is completely decentralized. A correctly validated blockchain uses the genesis block as the root of trust, building a chain of trust up to the current block. Bitcoin systems can and should use the blockchain as their root of trust. When designing a complex bitcoin application that consists of services on many different systems, you should carefully examine the security architecture in order to ascertain where trust is being placed. Ultimately, the only thing that should be explicitly trusted is a fully validated blockchain. If your application explicitly or implicitly vests trust in anything but the blockchain, that should be a source of concern because it introduces vulnerability. A good method to evaluate the security architecture of your application is to consider each individual component and evaluate a hypothetical scenario where that component is completely compromised and under the control of a malicious actor. Take each component of your application, in turn, and assess the impacts on the overall security if that component is compromised. If your application is no longer secure when components are compromised, that shows you have misplaced trust in those components. A bitcoin application without vulnerabilities should be vulnerable only to a compromise of the bitcoin consensus mechanism, meaning that its root of trust is based on the strongest part of the bitcoin security architecture.
-
-The numerous examples of hacked bitcoin exchanges serve to underscore this point because their security architecture and design fails even under the most casual scrutiny. These centralized implementations had invested trust explicitly in numerous components outside the Bitcoin blockchain, such as hot wallets, centralized ledger databases, vulnerable encryption keys, and similar schemes.((("", startref="Sprinc11")))
-
-
-=== User Security Best Practices
-
-((("security", "user security best practices", id="Suser11")))((("use cases", "user security best practices", id="UCsecurity11")))Humans have used physical security controls for thousands of years. By comparison, our experience with digital security is less than 50 years old. Modern general-purpose operating systems are not very secure and not particularly suited to storing digital money. Our computers are constantly exposed to external threats via always-on internet connections. They run thousands of software components from hundreds of authors, often with unconstrained access to the user's files. A single piece of rogue software, among the many thousands installed on your computer, can compromise your keyboard and files, stealing any bitcoin stored in wallet applications. The level of computer maintenance required to keep a computer virus-free and trojan-free is beyond the skill level of all but a tiny minority of computer users.
-
-Despite decades of research and advancements in information security, digital assets are still woefully vulnerable to a determined adversary. Even the most highly protected and restricted systems, in financial services companies, intelligence agencies, and defense contractors, are frequently breached. Bitcoin creates digital assets that have intrinsic value and can be stolen and diverted to new owners instantly and irrevocably. This creates a massive incentive for hackers. Until now, hackers had to convert identity information or account tokens—such as credit cards and bank accounts—into value after compromising them. Despite the difficulty of fencing and laundering financial information, we have seen ever-escalating thefts. Bitcoin escalates this problem because it doesn't need to be fenced or laundered; it is intrinsic value within a digital asset.
-
-Fortunately, bitcoin also creates the incentives to improve computer security. Whereas previously the risk of computer compromise was vague and indirect, bitcoin makes these risks clear and obvious. Holding bitcoin on a computer serves to focus the user's mind on the need for improved computer security. As a direct result of the proliferation and increased adoption of bitcoin and other digital currencies, we have seen an escalation in both hacking techniques and security solutions. In simple terms, hackers now have a very juicy target and users have a clear incentive to defend themselves.
-
-Over the past three years, as a direct result of bitcoin adoption, we have seen tremendous innovation in the realm of information security in the form of hardware encryption, key storage and hardware wallets, multisignature technology, and digital escrow. In the following sections we will examine various best practices for practical user security.
-
-==== Physical Bitcoin Storage
-
-((("storage", "physical bitcoin storage")))((("paper wallets")))((("wallets", "types of", "paper wallets")))((("paper wallets", see="also wallets")))Because most users are far more comfortable with physical security than information security, a very effective method for protecting bitcoin is to convert them into physical form. Bitcoin keys are nothing more than long numbers. This means that they can be stored in a physical form, such as printed on paper or etched on a metal coin. Securing the keys then becomes as simple as physically securing the printed copy of the bitcoin keys. A set of bitcoin keys that is printed on paper is called a "paper wallet," and there are many free tools that can be used to create them. I personally keep the vast majority of my bitcoin (99% or more) stored on paper wallets, encrypted with BIP-38, with multiple copies locked in safes. ((("cold storage")))((("storage", "cold storage")))Keeping bitcoin offline is called _cold storage_ and it is one of the most effective security techniques. A cold storage system is one where the keys are generated on an offline system (one never connected to the internet) and stored offline either on paper or on digital media, such as a USB memory stick.
-
-==== Hardware Wallets
-
-((("wallets", "types of", "hardware wallets")))((("hardware wallets")))In the long term, bitcoin security increasingly will take the form of hardware tamper-proof wallets. Unlike a smartphone or desktop computer, a bitcoin hardware wallet has just one purpose: to hold bitcoin securely. Without general-purpose software to compromise and with limited interfaces, hardware wallets can deliver an almost foolproof level of security to nonexpert users. I expect to see hardware wallets become the predominant method of bitcoin storage. For an example of such a hardware wallet, see the https://trezor.io/[Trezor].
-
-==== Balancing Risk
-
-((("risk, balancing and diversifying", seealso="security")))Although most users are rightly concerned about bitcoin theft, there is an even bigger risk. Data files get lost all the time. If they contain bitcoin, the loss is much more painful. In the effort to secure their bitcoin wallets, users must be very careful not to go too far and end up losing the bitcoin. In July 2011, a well-known bitcoin awareness and education project lost almost 7,000 bitcoin. In their effort to prevent theft, the owners had implemented a complex series of encrypted backups. In the end they accidentally lost the encryption keys, making the backups worthless and losing a fortune. Like hiding money by burying it in the desert, if you secure your bitcoin too well you might not be able to find it again.
-
-==== Diversifying Risk
-
-Would you carry your entire net worth in cash in your wallet? Most people would consider that reckless, yet bitcoin users often keep all their bitcoin in a single wallet. Instead, users should spread the risk among multiple and diverse bitcoin wallets. Prudent users will keep only a small fraction, perhaps less than 5%, of their bitcoin in an online or mobile wallet as "pocket change." The rest should be split between a few different storage mechanisms, such as a desktop wallet and offline (cold storage).
-
-==== Multisig and Governance
-
-((("multisig addresses")))((("addresses", "multisig addresses")))Whenever a company or individual stores large amounts of bitcoin, they should consider using a multisignature Bitcoin address. Multisignature addresses secure funds by requiring a minimum number of signatures to make a payment. The signing keys should be stored in a number of different locations and under the control of different people. In a corporate environment, for example, the keys should be generated independently and held by several company executives, to ensure no single person can compromise the funds. Multisignature addresses can also offer redundancy, where a single person holds several keys that are stored in different locations.
-
-==== Survivability
-
-((("survivability")))((("digital asset executors")))((("passwords", "survivability and")))((("security", "passwords")))One important security consideration that is often overlooked is availability, especially in the context of incapacity or death of the key holder. Bitcoin users are told to use complex passwords and keep their keys secure and private, not sharing them with anyone. Unfortunately, that practice makes it almost impossible for the user's family to recover any funds if the user is not available to unlock them. In most cases, in fact, the families of bitcoin users might be completely unaware of the existence of the bitcoin funds.
-
-If you have a lot of bitcoin, you should consider sharing access details with a trusted relative or lawyer. A more complex survivability scheme can be set up with multi-signature access and estate planning through a lawyer specialized as a "digital asset executor."((("", startref="Suser11")))((("", startref="UCsecurity11")))
-
-=== Conclusion
-
-Bitcoin is a completely new, unprecedented, and complex technology. Over time we will develop better security tools and practices that are easier to use by nonexperts. For now, bitcoin users can use many of the tips discussed here to enjoy a secure and trouble-free bitcoin experience.

ch11_blockchain.adoc

@@ -0,0 +1,851 @@
+[[blockchain]]
+== The Blockchain
+
+The ((("blockchain", "explained", id="blockchain-explain")))blockchain is the history of every confirmed Bitcoin transaction.
+It's what allows every full node to independently determine what keys and
+scripts control which bitcoins.  In this chapter, we'll look at the
+structure of the blockchain and see how it uses cryptographic
+commitments and other clever tricks to make every part of it easy for
+full nodes (and sometimes lightweight clients) to validate.
+
+The blockchain data structure is
+an ordered, back-linked list of blocks of transactions.  The blockchain
+can be stored as a flat file or in a simple database.
+Blocks are linked "back," each referring to the previous block in the
+chain. The blockchain is often visualized
+as a vertical stack, with blocks layered on top of each other and the
+first block serving as the foundation of the stack. The visualization of
+blocks stacked on top of each other results in the use of terms such as
+"height" to refer to the distance from the first block, and "top" or
+"tip" to refer to the most recently added block.
+
+Each block
+within the blockchain is identified by a hash, generated using the
+SHA256 cryptographic hash algorithm on the header of the block. Each
+block also commits to the previous block, known as ((("parent blocks")))the _parent_ block,
+through the "previous block hash" field in the block header.
+The sequence of hashes linking each block to its parent creates a chain
+going back all the way to the first block ever created, known ((("genesis block")))as the
+_genesis block_.
+
+Although a block has just one parent, it can have ((("child blocks")))multiple
+children. Each of the children commits to the same parent block.
+Multiple children arise during a blockchain "fork," a temporary
+situation that can occur when different blocks are discovered almost
+simultaneously by different miners (see <<forks>>). Eventually only one
+child block becomes part of the blockchain accepted by all full nodes, and the "fork" is resolved.
+
+The "previous block hash" field is inside the block header and thereby
+affects the _current_ block's hash.
+Any change to a parent block
+requires a child block's hash to change, which requires a change in the
+pointer of the grandchild, which in turn changes the grandchild, and so
+on. This sequence ensures that, once a block has many generations
+following it, it cannot be changed without forcing a recalculation of
+all subsequent blocks. Because such a recalculation would require
+enormous computation (and therefore energy consumption), the existence
+of a long chain of blocks makes the blockchain's deep history impractical to change,
+which is a key feature of Bitcoin's security.
+
+One way to think about the blockchain is like layers in a geological
+formation, or glacier core sample. The surface layers might change with
+the seasons, or even be blown away before they have time to settle. But
+once you go a few inches deep, geological layers become more and more
+stable. By the time you look a few hundred feet down, you are looking at
+a snapshot of the past that has remained undisturbed for millions of
+years. In the blockchain, the most recent few blocks might be revised if
+there is a chain reorganization due to a fork. The top six blocks are
+like a few inches of topsoil. But once you go more deeply into the
+blockchain, beyond six blocks, blocks are less and less likely to
+change. After 100 blocks back there is so much stability that
+the coinbase transaction--the transaction containing the reward in
+bitcoin for creating a new block--can be spent.
+While the
+protocol always allows a chain to be undone by a longer chain and while
+the possibility of any block being reversed always exists, the
+probability of such an event decreases as time passes until it ((("blockchain", "explained", startref="blockchain-explain")))becomes
+infinitesimal.
+
+=== Structure of a Block
+
+A block ((("blocks", "structure of")))is a container data structure that aggregates
+transactions for inclusion in the blockchain. The
+block is made of a header, containing metadata, followed by a long list
+of transactions that make up the bulk of its size. The block header is
+80 bytes, whereas the total size of all transactions in a block can be
+up to about 4,000,000 bytes.  A complete block,
+with all transactions, can therefore be almost 50,000 times larger than the block
+header. <<block_structure1>> describes how Bitcoin Core stores the structure of a block.
+
+++++
+<table id="block_structure1">
+<caption>The structure of a block</caption>
+<thead>
+<tr>
+<th>Size</th>
+<th>Field</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><p>4 bytes</p></td>
+<td><p>Block Size</p></td>
+<td><p>The size of the block, in bytes, following this field</p></td>
+</tr>
+<tr>
+<td><p>80 bytes</p></td>
+<td><p>Block Header</p></td>
+<td><p>Several fields form the block header</p></td>
+</tr>
+<tr>
+<td><p>1–3 bytes (compactSize)</p></td>
+<td><p>Transaction Counter</p></td>
+<td><p>How many transactions follow</p></td>
+</tr>
+<tr>
+<td><p>Variable</p></td>
+<td><p>Transactions</p></td>
+<td><p>The transactions recorded in this block</p></td>
+</tr>
+</tbody>
+</table>
+++++
+
+
+[[block_header]]
+=== Block Header
+
+The ((("blocks", "block header")))((("block header")))block header consists of
+block metadata as shown in <<block_header_structure_ch09>>.
+
+++++
+<table id="block_header_structure_ch09">
+<caption>The structure of the block header</caption>
+<thead>
+<tr>
+<th>Size</th>
+<th>Field</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><p>4 bytes</p></td>
+<td><p>Version</p></td>
+<td><p>Originally a version field; its use has evolved over time</p></td>
+</tr>
+<tr>
+<td><p>32 bytes</p></td>
+<td><p>Previous Block Hash</p></td>
+<td><p>A hash of the previous (parent) block in the chain</p></td>
+</tr>
+<tr>
+<td><p>32 bytes</p></td>
+<td><p>Merkle Root</p></td>
+<td><p>The root hash of the merkle tree of this block’s transactions</p></td>
+</tr>
+<tr>
+<td><p>4 bytes</p></td>
+<td><p>Timestamp</p></td>
+<td><p>The approximate creation time of this block (Unix epoch time)</p></td>
+</tr>
+<tr>
+<td><p>4 bytes</p></td>
+<td><p>Target</p></td>
+<td><p>A compact encoding of the proof-of-work target for this block</p></td>
+</tr>
+<tr>
+<td><p>4 bytes</p></td>
+<td><p>Nonce</p></td>
+<td><p>Arbitrary data used for the proof-of-work algorithm</p></td>
+</tr>
+</tbody>
+</table>
+++++
+
+The nonce, target, and timestamp are used in the mining
+process and will be discussed in more detail in <<mining>>.
+
+[[block_hash]]
+=== Block Identifiers: Block Header Hash and Block Height
+
+The ((("blocks", "identifiers", id="block-identify")))((("block header hash", id="block-header-hash")))((("block height", id="block-height")))primary identifier of a block
+is its cryptographic hash, a commitment made by hashing the
+block header twice through the SHA256 algorithm. The resulting 32-byte
+hash is called the _block hash_ but is more accurately the _block header
+hash_, pass:[<span class="keep-together">because only the block header is
+used to compute it. For example,</span>]
++000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+ is
+the block hash of the first block on Bitcoin's blockchain. The block hash
+identifies a block uniquely and unambiguously and can be independently
+derived by any node by simply hashing the block header.
+
+Note that the block hash is not actually included inside the block's
+data structure.
+Instead, the block's hash is computed by each node as the
+block is received from the network. The block hash might be stored in a
+separate database table as part of the block's metadata, to facilitate
+indexing and faster retrieval of blocks from disk.
+
+A second way to identify a block is by its position in the blockchain,
+called the pass:[<span class="keep-together"><em>block height</em>. The
+genesis block is at block height 0 (zero) and is the</span>]
+pass:[<span class="keep-together">same block that was previously
+referenced by the following block hash</span>]
++000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+. A
+block can thus be identified in two ways: by referencing the block hash
+or by referencing the block height. Each subsequent block added "on top"
+of that first block is one position "higher" in the blockchain, like
+boxes stacked one on top of the other. The block height 800,000 was
+reached during the writing of this book in mid-2023, meaning there were
+800,000 blocks stacked on top of the first block created in January
+2009.
+
+Unlike the block hash, the block height is not a unique identifier.
+Although a single block will always have a specific and invariant block
+height, the reverse is not true—the block height does not always
+identify a single block. Two or more blocks might have the same block
+height, competing for the same position in the blockchain. This scenario
+is discussed in detail in the section <<forks>>.  In early blocks, the block height was
+also not a part of the block's data structure; it was not stored within
+the block. Each node dynamically identified a block's position (height)
+in the blockchain when it was received from the Bitcoin network.  A
+later protocol change (BIP34) began including the block height in the
+coinbase transaction, although its purpose was to ensure each block had
+a different coinbase transaction.  Nodes still need to dynamically
+identify a block's height in order to validate the coinbase field.  The
+block height might also be stored as metadata in an indexed database
+table for faster retrieval.
+
+[TIP]
+====
+A block's _block hash_ always identifies a single block uniquely. A
+block also always has a specific _block height_. However, it is not
+always the case that a specific block height identifies a single
+block. Rather, two or more blocks might compete for a single position((("blocks", "identifiers", startref="block-identify")))((("block header hash", startref="block-header-hash")))((("block height", startref="block-height"))) in
+the blockchain.
+====
+
+=== The Genesis Block
+
+The first block((("blockchain", "genesis block", id="blockchain-genesis")))((("genesis block", id="genesis-block")))((("Bitcoin Core", "genesis block", id="bitcoin-core-genesis"))) in the blockchain is called the _genesis block_
+and was created in 2009. It is the common ancestor of all the blocks in
+the blockchain, meaning that if you start at any block and follow the
+chain backward in time, you will eventually arrive at the genesis block.
+
+Every node always starts with a blockchain of at least one block because
+the genesis block is statically encoded within Bitcoin Core,
+such that it cannot be altered. Every node always "knows" the
+genesis block's hash and structure, the fixed time it was created, and
+even the single transaction within. Thus, every node has the starting
+point for the blockchain, a secure "root" from which to build a trusted
+blockchain.
+
+See the statically encoded genesis block inside the Bitcoin Core client
+in https://oreil.ly/FqPW5[_chainparams.cpp_].
+
+The following identifier hash belongs to the genesis block:
+
+----
+000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
+----
+
+You can search for that block hash in almost any block explorer website, such
+as _blockstream.info_, and you will find a page describing the contents
+of this block, with a URL containing that hash:
+
+[quote]
+____
+https://blockstream.info/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
+____
+
+Alternatively, you can get the block using Bitcoin Core on the command line:
+
+----
+$ bitcoin-cli getblock \
+  000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
+----
+[source,json]
+----
+{
+  "hash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
+  "confirmations": 790496,
+  "height": 0,
+  "version": 1,
+  "versionHex": "00000001",
+  "merkleroot": "4a5e1e4baab89f3a32518a88c3[...]76673e2cc77ab2127b7afdeda33b",
+  "time": 1231006505,
+  "mediantime": 1231006505,
+  "nonce": 2083236893,
+  "bits": "1d00ffff",
+  "difficulty": 1,
+  "chainwork": "[...]000000000000000000000000000000000000000000000100010001",
+  "nTx": 1,
+  "nextblockhash": "00000000839a8e6886ab5951d7[...]fc90947ee320161bbf18eb6048",
+  "strippedsize": 285,
+  "size": 285,
+  "weight": 1140,
+  "tx": [
+    "4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b"
+  ]
+}
+----
+
+The genesis block contains a message within it. The coinbase
+transaction input contains the text "The Times 03/Jan/2009 Chancellor on
+brink of second bailout for banks." This message was intended to offer
+proof of the earliest date this block could have been created, by referencing the
+headline of the British newspaper _The Times_. It also serves as a
+tongue-in-cheek reminder of the importance of an independent monetary
+system, with Bitcoin's launch occurring at the same time as an
+unprecedented worldwide monetary crisis. The message was embedded in the
+first block by Satoshi Nakamoto, Bitcoin's ((("Nakamoto, Satoshi")))((("blockchain", "genesis block", startref="blockchain-genesis")))((("genesis block", startref="genesis-block")))((("Bitcoin Core", "genesis block", startref="bitcoin-core-genesis")))creator.
+
+=== Linking Blocks in the Blockchain
+
+Bitcoin((("blockchain", "linking blocks", id="blockchain-link")))((("blocks", "linking in blockchain", id="block-link")))((("linking blocks in blockchain", id="link-block"))) full nodes validate every
+block in the blockchain after the genesis block. Their local view of
+the blockchain is constantly updated as new blocks are found and used to
+extend the chain. As a node receives incoming blocks from the network,
+it will validate these blocks and then link them to its view of the existing
+blockchain. To establish a link, a node will examine the incoming block
+header and look for the "previous block hash."
+
+[role="less_space pagebreak-before"]
+Let's assume, for example, that a node has 277,314 blocks in the local
+copy of the blockchain. The last block the node knows about is block
+277,314, with a block header hash of:
+
+----
+00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249
+----
+
+The Bitcoin node then receives a new block from the network, which it
+parses as follows:
+
+[source,json]
+----
+{
+    "size" : 43560,
+    "version" : 2,
+    "previousblockhash" :
+        "00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249",
+    "merkleroot" :
+        "5e049f4030e0ab2debb92378f53c0a6e09548aea083f3ab25e1d94ea1155e29d",
+    "time" : 1388185038,
+    "difficulty" : 1180923195.25802612,
+    "nonce" : 4215469401,
+    "tx" : [
+        "257e7497fb8bc68421eb2c7b699dbab234831600e7352f0d9e6522c7cf3f6c77",
+        "[... many more transactions omitted ...]",
+        "05cfd38f6ae6aa83674cc99e4d75a1458c165b7ab84725eda41d018a09176634"
+    ]
+}
+----
+
+Looking at this new block, the node finds the +previousblockhash+ field,
+which contains the hash of its parent block. It is a hash known to the
+node, that of the last block on the chain at height 277,314. Therefore,
+this new block is a child of the last block on the chain and extends the
+existing blockchain. The node adds this new block to the end of the
+chain, making the blockchain longer with a new height of 277,315.
+<<chain_of_blocks>> shows the chain of three blocks, linked by
+references in((("blockchain", "linking blocks", startref="blockchain-link")))((("blocks", "linking in blockchain", startref="block-link")))((("linking blocks in blockchain", startref="link-block"))) the +previousblockhash+ field.
+
+[[chain_of_blocks]]
+.Blocks linked in a chain by each referencing the previous block header hash.
+image::images/mbc3_1101.png[]
+
+[[merkle_trees]]
+=== Merkle Trees
+
+Each block((("blockchain", "merkle trees", id="blockchain-merkle")))((("merkle trees", id="merkle-tree-explain"))) in the Bitcoin blockchain contains
+a summary of all the transactions in the block using a _merkle tree_.
+
+A _merkle tree_, also known
+as a _binary hash tree_, is ((("binary hash trees")))a data structure used for efficiently
+summarizing and verifying the integrity of large sets of data. Merkle
+trees are binary trees containing cryptographic hashes. The term "tree"
+is used in computer science to describe a branching data structure, but
+these trees are usually displayed upside down with the "root" at the top
+and the "leaves" at the bottom of a diagram, as you will see in the
+examples that follow.
+
+Merkle trees are used in Bitcoin to summarize all the transactions in a
+block, producing an overall commitment to the entire set of
+transactions and permitting a very efficient process to verify whether a
+transaction is included in a block. A merkle tree is constructed by
+recursively hashing pairs of elements until there is only one hash, called
+the _root_, or _merkle root_. The cryptographic hash algorithm used in
+Bitcoin's merkle trees is SHA256 applied twice, also known as
+double-SHA256.
+
+When N data elements are hashed and summarized in a merkle tree, you can
+check to see if any one data element is included in the tree with 
+about +log~2~(N)+ calculations, making this a very efficient data
+structure.
+
+The merkle tree is constructed bottom-up. In the following example, we
+start with four transactions, A, B, C, and D, which form the _leaves_ of
+the merkle tree, as shown in <<simple_merkle>>. The transactions are not
+stored in the merkle tree; rather, their data is hashed and the
+resulting hash is stored in each leaf node as H~A~, H~B~, H~C~, and
+H~D~:
+
+++++
+<pre data-type="codelisting">
+H<sub>A</sub> = SHA256(SHA256(Transaction A))
+</pre>
+++++
+
+Consecutive pairs of leaf nodes are then summarized in a parent node by
+concatenating the two hashes and hashing them together. For example, to
+construct the parent node H~AB~, the two 32-byte hashes of the children
+are concatenated to create a 64-byte string. That string is then
+double-hashed to produce the parent node's hash:
+
+++++
+<pre data-type="codelisting">
+H<sub>AB</sub> = SHA256(SHA256(H<sub>A</sub> || H<sub>B</sub>))
+</pre>
+++++
+
+The process continues until there is only one node at the top, the node
+known as the merkle root. That 32-byte hash is stored in the block
+header and summarizes all the data in all four transactions.
+<<simple_merkle>> shows how the root is calculated by pair-wise hashes
+of the nodes.
+
+[[simple_merkle]]
+.Calculating the nodes in a merkle tree.
+image::images/mbc3_1102.png["merkle_tree"]
+
+Because the merkle tree is a binary tree, it needs
+an even number of leaf nodes. If there are an odd number of transactions
+to summarize, the last transaction hash will be duplicated to create an
+even number of leaf nodes, also known ((("balanced merkle trees")))as a _balanced tree_. This is
+shown in <<merkle_tree_odd>>, where transaction C is duplicated.
+Similarly, if there are an odd number of hashes to process at any level,
+the last hash is duplicated.
+
+[[merkle_tree_odd]]
+.Duplicating one data element achieves an even number of data elements.
+image::images/mbc3_1103.png["merkle_tree_odd"]
+
+.A Design Flaw in Bitcoin's Merkle Tree
+****
+An extended comment in Bitcoin Core's source code, reproduced here with slight revisions, describes a
+significant problem in the design of Bitcoin's duplication of odd
+elements in its merkle tree:
+
+
+____
+WARNING! If you're reading this because you're learning about crypto
+and/or designing a new system that will use merkle trees, keep in mind
+that the following merkle tree algorithm has a serious flaw related to
+duplicate txids, resulting in a vulnerability (CVE-2012-2459).
+
+The reason is that if the number of hashes in the list at a given level
+is odd, the last one is duplicated before computing the next level (which
+is unusual in merkle trees). This results in certain sequences of
+transactions leading to the same merkle root. For example, the two
+trees in <<cve_tree>>:
+
+[[cve_tree]]
+[role="width-90"]
+.Two Bitcoin-style merkle trees with the same root but a different number of leaves.
+image::images/mbc3_1104.png["Two Bitcoin-style merkle trees with the same root but a different number of leaves"]
+
+The transaction lists [1,2,3,4,5,6] and [1,2,3,4,5,6,5,6] (where 5 and
+6 are repeated) result in the same root hash A (because the hash of both
+of (F) and (F,F) is C).
+
+The vulnerability results from being able to send a block with such a
+transaction list, with the same merkle root, and the same block hash as
+the original without duplication, resulting in failed validation. If the
+receiving node proceeds to mark that block as permanently invalid
+however, it will fail to accept further unmodified (and thus potentially
+valid) versions of the same block. We defend against this by detecting
+the case where we would hash two identical hashes at the end of the list
+together, and treating that identically to the block having an invalid
+merkle root. Assuming no double-SHA256 collisions, this will detect all
+known ways of changing the transactions without affecting the merkle
+root.
+
+++++
+<p data-type="attribution">Bitcoin Core <em>src/consensus/merkle.cpp</em></p>
+++++
+____
+
+****
+
+[role="less_space pagebreak-before"]
+The same method for constructing a tree from four transactions can be
+generalized to construct trees of any size. In Bitcoin it is common to
+have several thousand transactions in a single
+block, which are summarized in exactly the same way, producing just 32
+bytes of data as the single merkle root. In <<merkle_tree_large>>, you
+will see a tree built from 16 transactions. Note that although the root
+looks bigger than the leaf nodes in the diagram, it is the exact same
+size, just 32 bytes. Whether there is one transaction or ten
+thousand transactions in the block, the merkle root always summarizes
+them into 32 bytes.
+
+To prove that a specific transaction is
+included in a block, a node only needs to produce approximately +log~2~(N)+ 32-byte
+hashes, constituting ((("authentication path")))((("merkle path")))an _authentication path_ or _merkle path_
+connecting the specific transaction to the root of the tree. This is
+especially important as the number of transactions increases because
+the base-2 logarithm of the number of transactions increases much more
+slowly. This allows Bitcoin nodes to efficiently produce paths of 10 or
+12 hashes (320–384 bytes), which can provide proof of a single
+transaction out of more than a thousand transactions in a multimegabyte
+block.
+
+[[merkle_tree_large]]
+.A merkle tree summarizing many data elements.
+image::images/mbc3_1105.png["merkle_tree_large"]
+
+In <<merkle_tree_path>>, a node can prove that a transaction K is
+included in the block by producing a merkle path that is only four
+32-byte hashes long (128 bytes total). The path consists of the four
+hashes (shown with a shaded background) H~L~,
+H~IJ~, H~MNOP~, and H~ABCDEFGH~. With those four hashes provided as an
+authentication path, any node can prove that H~K~ (with a black
+background at the bottom of the diagram) is included in the merkle root
+by computing four additional pair-wise hashes H~KL~, H~IJKL~,
+H~IJKLMNOP~, and the merkle tree root (outlined in a dashed line in the
+diagram).
+
+[[merkle_tree_path]]
+.A merkle path used to prove inclusion of a data element.
+image::images/mbc3_1106.png["merkle_tree_path"]
+
+The efficiency of merkle trees becomes obvious as the scale increases.
+The largest possible block can hold almost 16,000 transactions in 4,000,000
+bytes, but proving any particular one of those 16,000 transactions
+is a part of that block only requires a copy of the transaction, a copy
+of the 80-byte block header, and 448 bytes for the merkle proof.  That
+makes the largest possible proof almost 10,000 times smaller than the
+largest possible Bitcoin block.
+
+=== Merkle Trees and Lightweight Clients
+
+Merkle trees are ((("Bitcoin network", "lightweight clients", "merkle trees and")))((("lightweight clients", "merkle trees and")))used extensively by lightweight clients. Lightweight clients don't
+have all transactions and do not download full blocks, just block
+headers. In order to verify that a transaction is included in a block,
+without having to download all the transactions in the block, they use
+a merkle path.
+
+Consider, for example, a lightweight client that is interested in incoming
+payments to an address contained in its wallet. The lightweight client will
+establish a bloom filter (see <<bloom_filters>>) on its connections to
+peers to limit the transactions received to only those containing
+addresses of interest. When a peer sees a transaction that matches the
+bloom filter, it will send that block using a +merkleblock+ message. The
++merkleblock+ message contains the block header as well as a merkle path
+that links the transaction of interest to the merkle root in the block.
+The lightweight client can use this merkle path to connect the transaction to the
+block header and verify that the transaction is included in the block. The lightweight
+client also uses the block header to link the block to the rest of the
+blockchain. The combination of these two links, between the transaction
+and block and between the block and blockchain, proves that the
+transaction is recorded in the blockchain. All in all, the lightweight client will
+have received less than a kilobyte of data for the block header and
+merkle path, an amount of data that is more than a thousand times less
+than a full block (about 2 MB ((("blockchain", "merkle trees", startref="blockchain-merkle")))((("merkle trees", startref="merkle-tree-explain")))currently).
+
+=== Bitcoin's Test Blockchains
+
+You might be
+surprised to learn that there is more than one blockchain used with Bitcoin. The
+"main" Bitcoin blockchain, the one created by Satoshi Nakamoto on
+January 3rd, 2009, the one with the genesis block we studied in this
+chapter, is ((("mainnet")))called _mainnet_.  There are other Bitcoin blockchains that
+are used for testing purposes: at this time _testnet_, _signet_, and
+_regtest_. Let's look at each in turn.
+
+==== Testnet: Bitcoin's Testing Playground
+
+Testnet is ((("blockchain", "test blockchains", "testnet", id="blockchain-test-testnet")))((("test blockchains", "testnet", id="test-block-testnet")))((("testnet", id="testnet")))the name of the test blockchain, network, and currency that
+is used for testing purposes. The testnet is a fully featured live P2P
+network, with wallets, test bitcoins (testnet coins), mining, and all
+the other features of mainnet.  The most important difference is that
+testnet coins are meant to be worthless.
+
+Any software development that is intended for production use on
+Bitcoin's mainnet can first be tested on testnet with test coins.
+This protects both the developers from monetary losses due to bugs and
+the network from unintended behavior due to bugs.
+
+The current testnet is called _testnet3_, the third iteration of
+testnet, restarted in February 2011 to reset the difficulty from the
+previous testnet.  Testnet3 is a large blockchain, in excess of 30 GB in
+2023. It will take a while to sync fully and use up resources
+on your computer. Not as much as mainnet, but not exactly "lightweight"
+either.
+
+[TIP]
+====
+Testnet and the other test blockchains described in this book don't use
+the same address prefixes as mainnet addresses to prevent someone from
+accidentally sending real bitcoins to a test address.  Mainnet addresses
+begin with +1+, +3+, or +bc1+.  Addresses for the test networks
+mentioned in this book begin with +m+, +n+, or +tb1+.  Other test
+networks, or new protocols being developed on test networks, may use
+other address prefixes or alterations.
+====
+
+===== Using testnet
+
+Bitcoin Core, like((("Bitcoin Core", "testnet"))) many other Bitcoin programs, has full support
+for operation on testnet as an alternative mainnet. All of Bitcoin Core's
+functions work on testnet, including the wallet, mining testnet coins,
+and syncing a full testnet node.
+
+[role="less_space pagebreak-before"]
+To start Bitcoin Core on testnet instead of mainnet you use the
++testnet+ switch:
+
+----
+$ bitcoind -testnet
+----
+
+In the logs you should see that bitcoind is building a new blockchain in
+the +testnet3+ subdirectory of the default bitcoind directory:
+
+----
+bitcoind: Using data directory /home/username/.bitcoin/testnet3
+----
+
+To connect to bitcoind, you use the +bitcoin-cli+ command-line tool, but
+you must also switch it to testnet mode:
+
+----
+$ bitcoin-cli -testnet getblockchaininfo
+{
+  "chain": "test",
+  "blocks": 1088,
+  "headers": 139999,
+  "bestblockhash": "0000000063d29909d475a1c[...]368e56cce5d925097bf3a2084370128",
+  "difficulty": 1,
+  "mediantime": 1337966158,
+  "verificationprogress": 0.001644065914099759,
+  "chainwork": "[...]000000000000000000000000000000000000000000044104410441",
+  "pruned": false,
+  "softforks": [
+
+  [...]
+----
+
+You can also run on testnet3 with other full-node implementations, such
+as +btcd+ (written in Go) and +bcoin+ (written in JavaScript), to
+experiment and learn in other programming languages and frameworks.
+
+Testnet3 supports all the features of mainnet, including
+segregated witness v0 and v1 (see <<segwit>> and <<taproot>>). Therefore, testnet3 can also be
+used to test segregated witness features.
+
+===== Problems with testnet
+
+Testnet doesn't just use the same data structures as Bitcoin, it also
+uses almost exactly the same proof-of-work security mechanism as
+Bitcoin.  The notable differences for testnet are that its minimum
+difficulty is half that of Bitcoin and that it's allowed to include a
+block at the minimum difficulty if that block's timestamp is more than
+20 minutes after the previous block.
+
+Unfortunately, Bitcoin's PoW security mechanism was designed to depend
+on economic incentives--incentives which don't exist in a test
+blockchain that is forbidden from having value.  On mainnet, miners are
+incentivized to include user transactions in their blocks because those
+transactions pay fees.  On testnet, transactions still contain something
+called fees, but those fees don't have any economic value.  That means
+the only incentive for a testnet miner to include transactions is
+because they want to help users and developers to test their software.
+
+Alas, people who like to disrupt systems often feel a stronger
+incentive, at least in the short term.  Because PoW mining is designed
+to be permissionless, anyone can mine, whether their intention is good
+or not.  That means disruptive miners can create many blocks in a row on
+testnet without including any user transactions.  When those attacks
+happen, testnet becomes unusable for users and ((("blockchain", "test blockchains", "testnet", startref="blockchain-test-testnet")))((("test blockchains", "testnet", startref="test-block-testnet")))((("testnet", startref="testnet")))developers.
+
+==== Signet: The Proof of Authority Testnet
+
+There's no ((("blockchain", "test blockchains", "signet", id="blockchain-test-signet")))((("test blockchains", "signet", id="test-block-signet")))((("signet", id="signet")))known way for a system dependent on permissionless PoW to
+provide a highly usable blockchain without introducing economic
+incentives, so Bitcoin protocol developers began considering
+alternatives.  The primary goal was to preserve as much of the structure of
+Bitcoin as possible so that software could run on a testnet with minimal
+changes--but to also provide an environment that would remain useful.
+A secondary goal was to produce a reusable design that would allow
+developers of new software to easily create their own test networks.
+
+The solution implemented in Bitcoin Core and other software is called
+_signet_, as defined by BIP325.  A signet is a test network where each
+block must contain proof (such as a signature) that the creation of that
+block was sanctioned by a trusted authority.
+
+Whereas mining in Bitcoin is permissionless--anyone can do it--mining on
+signet is fully permissioned.  Only those with permission can do it.
+This would be a completely unacceptable change to Bitcoin's mainnet--no
+one would use that software--but it's reasonable on a testnet where coins have
+no value and the only purpose is testing software and systems.
+
+BIP325 signets are designed to make it very easy to create your own.  If
+you disagree with how someone else is running their signet, you can
+start your own signet and connect your software to it.
+
+===== The default signet and custom signets
+
+Bitcoin Core supports((("Bitcoin Core", "signet")))((("default signet")))((("custom signets"))) a default signet, which we believe to be the most
+widely used signet at the time of writing.  It is currently operated by
+two contributors to that project.  If you start Bitcoin Core with the
++signet+ parameter and no other signet-related parameters, this is the
+signet you will be using.
+
+As of this writing, the default signet has about 150,000 blocks and is
+about a gigabyte in size.  It supports all of the same features as
+Bitcoin's mainnet and is also used for testing proposed upgrades through
+the Bitcoin Inquisition project, which is a software fork of Bitcoin
+Core that's only designed to run on signet.
+
+If you want to use a different signet, called a _custom signet_, you
+will need to know the script used to determine when a block is
+authorized, called((("challenge script"))) the _challenge_ script.  This is a standard Bitcoin
+script, so it can use features such as multisig to allow multiple people
+to authorize blocks.  You may also need to connect to a seed node that
+will provide you with the addresses of peers on the custom signet.  For
+example:
+
+----
+bitcoind -signet -signetchallenge=0123...cdef -signetseednode=example.com:1234
+----
+
+As of this writing, we generally recommend that the public testing of
+mining software occur on testnet3 and that all other public testing of
+Bitcoin software occur on the default signet.
+
+To interact with your chosen signet, you can use the +-signet+ parameter
+with +bitcoin-cli+, similar to how you used testnet.  For((("blockchain", "test blockchains", "signet", startref="blockchain-test-signet")))((("test blockchains", "signet", startref="test-block-signet")))((("signet", startref="signet"))) example:
+
+----
+$ bitcoin-cli -signet getblockchaininfo
+{
+  "chain": "signet",
+  "blocks": 143619,
+  "headers": 143619,
+  "bestblockhash": "000000c46cb3505ddd296537[...]ad1c5768e2908439382447572a93",
+  "difficulty": 0.003020638517858618,
+  "time": 1684530244,
+  "mediantime": 1684526116,
+  "verificationprogress": 0.999997961940662,
+  "initialblockdownload": false,
+  "chainwork": "[...]000000000000000000000000000000000000000000019ab37d2194",
+  "size_on_disk": 769525915,
+  "pruned": false,
+  "warnings": ""
+}
+----
+
+==== Regtest: The Local Blockchain
+
+Regtest, ((("blockchain", "test blockchains", "regtest", id="blockchain-test-regtest")))((("test blockchains", "regtest", id="test-block-regtest")))((("regtest", id="regtest")))which stands for
+"Regression Testing," is a Bitcoin Core feature that allows you to
+create a local blockchain for testing purposes. Unlike signet and testnet3, which
+are public and shared test blockchains, the regtest blockchains are
+intended to be run as closed systems for local testing. You launch a
+regtest blockchain from scratch. You may
+add other nodes to the network or run it with a single node only to
+test the Bitcoin Core software.
+
+To start ((("Bitcoin Core", "regtest")))Bitcoin Core in regtest mode, you use the +regtest+ flag:
+
+----
+$ bitcoind -regtest
+----
+
+Just like with testnet, Bitcoin Core will initialize a new blockchain
+under the _regtest_ subdirectory of your bitcoind default directory:
+
+----
+bitcoind: Using data directory /home/username/.bitcoin/regtest
+----
+
+[role="less_space pagebreak-before"]
+To use the command-line tool, you need to specify the +regtest+ flag
+too. Let's try the +getblockchaininfo+ command to inspect the regtest
+blockchain:
+
+----
+$ bitcoin-cli -regtest getblockchaininfo
+{
+  "chain": "regtest",
+  "blocks": 0,
+  "headers": 0,
+  "bestblockhash": "0f9188f13cb7b2c71f2a335e3[...]b436012afca590b1a11466e2206",
+  "difficulty": 4.656542373906925e-10,
+  "mediantime": 1296688602,
+  "verificationprogress": 1,
+  "chainwork": "[...]000000000000000000000000000000000000000000000000000002",
+  "pruned": false,
+  [...]
+----
+
+As you can see, there are no blocks yet. Let's create a default wallet,
+get an address, and then mine some (500 blocks) to earn the reward:
+
+----
+$ bitcoin-cli -regtest createwallet ""
+
+$ bitcoin-cli -regtest getnewaddress
+bcrt1qwvfhw8pf79kw6tvpmtxyxwcfnd2t4e8v6qfv4a
+
+$ bitcoin-cli -regtest generatetoaddress 500 \
+  bcrt1qwvfhw8pf79kw6tvpmtxyxwcfnd2t4e8v6qfv4a
+[
+  "3153518205e4630d2800a4cb65b9d2691ac68eea99afa7fd36289cb266b9c2c0",
+  "621330dd5bdabcc03582b0e49993702a8d4c41df60f729cc81d94b6e3a5b1556",
+  "32d3d83538ba128be3ba7f9dbb8d1ef03e1b536f65e8701893f70dcc1fe2dbf2",
+  ...,
+  "32d55180d010ffebabf1c3231e1666e9eeed02c905195f2568c987c2751623c7"
+]
+----
+
+It will only take a few seconds to mine all these blocks, which
+certainly makes it easy for testing. If you check your wallet balance,
+you will see that you earned the rewards for the first 400 blocks (coinbase
+rewards must be 100 blocks deep before you can ((("blockchain", "test blockchains", "regtest", startref="blockchain-test-regtest")))((("test blockchains", "regtest", startref="test-block-regtest")))((("regtest", startref="regtest")))spend them):
+
+----
+$ bitcoin-cli -regtest getbalance
+12462.50000000
+----
+
+=== Using Test Blockchains for Development
+
+Bitcoin's ((("blockchain", "test blockchains", "development usage")))((("test blockchains", "development usage")))various
+blockchains (regtest, signet, testnet3, mainnet) offer a range
+of testing environments for bitcoin development. Use the test
+blockchains whether you are developing for Bitcoin Core or another
+full-node consensus client; developing an application such as a wallet, exchange,
+ecommerce site; or even developing novel smart contracts and complex
+scripts).
+
+You can use the test blockchains to establish a development pipeline.
+Test your code locally on a regtest as you develop it. Once you are
+ready to try it on a public network, switch to signet or testnet to expose your
+code to a more dynamic environment with more diversity of code and
+applications. Finally, once you are confident your code works as
+expected, switch to mainnet to deploy it in production. As you make
+changes, improvements, bug fixes, etc., start the pipeline again,
+deploying each change first on regtest, then on signet or testnet, and finally
+into production.
+
+Now that we know what data the blockchain contains and how cryptographic
+commitments securely tie the various parts together, we will look at the
+special commitment that both provide computational security and
+ensure no block can be changed without invalidating all other blocks
+built on top of it: Bitcoin's mining function.

ch12.asciidoc

@@ -1,409 +0,0 @@
-[[ch12]]
-== Blockchain Applications
-
-Let's now build on our understanding of bitcoin by looking at it as an _application platform_. Nowadays, many people use the term "blockchain" to refer to any application platform that shares the design principles of bitcoin. The term is often misused and applied to many things that fail to deliver the primary features that bitcoin's blockchain delivers.
-
-In this chapter we will look at the features offered by the Bitcoin blockchain, as an application platform. We will consider the application building _primitives_, which form the building blocks of any blockchain application. We will look at several important applications that use these primitives, such as payment (state) channels and routed payment channels (Lightning Network).
-
-=== Introduction
-
-((("blockchain applications", "benefits of Bitcoin system")))The Bitcoin system was designed as a decentralized currency and payment system. However, most of its functionality is derived from much lower-level constructs that can be used for much broader applications. Bitcoin wasn't built with components such as accounts, users, balances, and payments. Instead, it uses a transactional scripting language with low-level cryptographic functions, as we saw in <<transactions>>. Just as the higher-level concepts of accounts, balances, and payments can be derived from these basic primitives, so can many other complex applications. Thus, the Bitcoin blockchain can become an application platform offering trust services to applications, such as smart contracts, far surpassing the original purpose of digital currency and payments.
-
-=== Building Blocks (Primitives)
-
-((("blockchain applications", "building blocks for (primitives)")))((("primitives")))When operating correctly and over the long term, the Bitcoin system offers certain guarantees, which can be used as building blocks to create applications. These include:
-
-No Double-Spend:: The most fundamental guarantee of bitcoin's decentralized consensus algorithm ensures that no UTXO can be spent twice.
-
-Immutability:: Once a transaction is recorded in the blockchain and sufficient work has been added with subsequent blocks, the transaction's data becomes immutable. Immutability is underwritten by energy, as rewriting the blockchain requires the expenditure of energy to produce Proof-of-Work. The energy required and therefore the degree of immutability increases with the amount of work committed on top of the block containing a transaction.
-
-Neutrality:: The decentralized Bitcoin network propagates valid transactions regardless of the origin or content of those transactions. This means that anyone can create a valid transaction with sufficient fees and trust they will be able to transmit that transaction and have it included in the blockchain at any time.
-
-Secure Timestamping:: The consensus rules reject any block whose timestamp is too far in the past or future. This ensures that timestamps on blocks can be trusted. The timestamp on a block implies an unspent-before guarantee for the inputs of all included transactions.
-
-Authorization:: Digital signatures, validated in a decentralized network, offer authorization guarantees. Scripts that contain a requirement for a digital signature cannot be executed without authorization by the holder of the private key implied in the script.
-
-Auditability:: All transactions are public and can be audited. All transactions and blocks can be linked back in an unbroken chain to the genesis block.
-
-Accounting:: In any transaction (except the coinbase transaction) the value of inputs is equal to the value of outputs plus fees. It is not possible to create or destroy bitcoin value in a transaction. The outputs cannot exceed the inputs.
-
-Nonexpiration:: A valid transaction does not expire. If it is valid today, it will be valid in the near future, as long as the inputs remain unspent and the consensus rules do not change.
-
-Integrity:: A bitcoin transaction signed with +SIGHASH_ALL+ or parts of a transaction signed by another +SIGHASH+ type cannot be modified without invalidating the signature, thus invalidating the transaction itself.
-
-Transaction Atomicity:: Bitcoin transactions are atomic. They are either valid and confirmed (mined) or not. Partial transactions cannot be mined and there is no interim state for a transaction. At any point in time a transaction is either mined, or not.
-
-Discrete (Indivisible) Units of Value:: Transaction outputs are discrete and indivisible units of value. They can either be spent or unspent, in full. They cannot be divided or partially spent.
-
-Quorum of Control:: Multisignature constraints in scripts impose a quorum of authorization, predefined in the multisignature scheme. The M-of-N requirement is enforced by the consensus rules.
-
-Timelock/Aging:: Any script clause containing a relative or absolute timelock can only be executed after its age exceeds the time specified.
-
-Replication:: The decentralized storage of the blockchain ensures that when a transaction is mined, after sufficient confirmations, it is replicated across the network and becomes durable and resilient to power loss, data loss, etc.
-
-Forgery Protection:: A transaction can only spend existing, validated outputs. It is not possible to create or counterfeit value.
-
-Consistency:: In the absence of miner partitions, blocks that are recorded in the blockchain are subject to reorganization or disagreement with exponentially decreasing likelihood, based on the depth at which they are recorded. Once deeply recorded, the computation and energy required to change makes change practically infeasible.
-
-Recording External State:: A transaction can commit a data value, via +OP_RETURN+, representing a state transition in an external state machine.
-
-Predictable Issuance:: Less than 21 million bitcoin will be issued, at a predictable rate.
-
-The list of building blocks is not complete and more are added with each new feature introduced into bitcoin.
-
-=== Applications from Building Blocks
-
-((("blockchain applications", "examples of")))The building blocks offered by bitcoin are elements of a trust platform that can be used to compose applications. Here are some examples of applications that exist today and the building blocks they use:
-
-Proof-of-Existence (Digital Notary):: ((("digital notary services")))((("Proof of Existence")))Immutability + Timestamp + Durability. A digital fingerprint can be committed with a transaction to the blockchain, proving that a document existed (Timestamp) at the time it was recorded. The fingerprint cannot be modified ex-post-facto (Immutability) and the proof will be stored permanently (Durability).
-
-Kickstarter (Lighthouse):: Consistency + Atomicity + Integrity. If you sign one input and the output (Integrity) of a fundraiser transaction, others can contribute to the fundraiser but it cannot be spent (Atomicity) until the goal (output value) is funded (Consistency).
-
-Payment Channels:: ((("payment (state) channels", "building blocks (primitives) used in")))Quorum of Control + Timelock + No Double Spend + Nonexpiration + Censorship Resistance + Authorization. A multisig 2-of-2 (Quorum) with a timelock (Timelock) used as the "settlement" transaction of a payment channel can be held (Nonexpiration) and spent at any time (Censorship Resistance) by either party (Authorization). The two parties can then create commitment transactions that double-spend (No Double-Spend) the settlement on a shorter timelock (Timelock).
-
-=== Counterparty
-
-((("blockchain applications", "Counterparty")))((("Counterparty")))((("smart contracts")))((("Ethereum Virtual Machine (EVM)")))((("extrinsic asset management")))((("virtual asset management")))Counterparty is a protocol layer built on top of bitcoin. The Counterparty protocol offers the ability to create and trade virtual assets and tokens. In addition, Counterparty offers a decentralized exchange for assets. Counterparty is also implementing smart contracts, based on the Ethereum Virtual Machine (EVM).
-
-Counterparty embeds metadata in bitcoin transactions, using the +OP_RETURN+ opcode or 1-of-N multisignature addresses that encode metadata in the place of public keys. Using these mechanisms, Counterparty implements a protocol layer encoded in bitcoin transactions. The additional protocol layer can be interpreted by applications that are Counterparty-aware, such as wallets and blockchain explorers, or any application built using the Counterparty libraries.
-
-((("digital ownership")))Counterparty can be used as a platform for other applications and services, in turn. For example, Tokenly is a platform built on top of Counterparty that allows content creators, artists, and companies to issue tokens that express digital ownership and can be used to rent, access, trade, or shop for content, products, and services. Other applications leveraging Counterparty include games (Spells of Genesis) and grid computing projects (Folding Coin).
-
-More details about Counterparty can be found at https://counterparty.io. The open source project can be found at https://github.com/CounterpartyXCP[].
-
-[[state_channels]]
-=== Payment Channels and State Channels
-
-_Payment channels_ ((("blockchain applications", "payment (state) channels", id="BCApayment12")))((("payment (state) channels", "defined")))are a trustless mechanism for exchanging bitcoin transactions between two parties, outside of the Bitcoin blockchain. These transactions, which would be valid if settled on the Bitcoin blockchain, are held off-chain instead, acting as _promissory notes_ for eventual batch settlement. Because the transactions are not settled, they can be exchanged without the usual settlement latency, allowing extremely high transaction throughput, low (submillisecond) latency, and fine (satoshi-level) granularity.
-
-Actually, the term _channel_ is a metaphor. State channels are virtual constructs represented by the exchange of state between two parties, outside of the blockchain. There are no "channels" per se and the underlying data transport mechanism is not the channel. We use the term channel to represent the relationship and shared state between two parties, outside of the blockchain.
-
-((("payment (state) channels", "concept of")))To further explain this concept, think of a TCP stream. From the perspective of higher-level protocols it is a "socket" connecting two applications across the internet. But if you look at the network traffic, a TCP stream is just a virtual channel over IP packets. Each endpoint of the TCP stream sequences and assembles IP packets to create the illusion of a stream of bytes. Underneath, it's all disconnected packets. Similarly, a payment channel is just a series of transactions. If properly sequenced and connected, they create redeemable obligations that you can trust even though you don't trust the other side of the channel.
-
-In this section we will look at various forms of payment channels. First, we will examine the mechanisms used to construct a one-way (unidirectional) payment channel for a metered micropayment service, such as streaming video. Then, we will expand on this mechanism and introduce bidirectional payment channels. Finally, we will look at how bidirectional channels can be connected end-to-end to form multihop channels in a routed network, first proposed under the name _Lightning Network_.
-
-Payment channels are part of the broader concept of a _state channel_, which represents an off-chain alteration of state, secured by eventual settlement in a blockchain. A payment channel is a state channel where the state being altered is the balance of a virtual currency.
-
-==== State Channels&#x2014;Basic Concepts and Terminology
-
-((("payment (state) channels", "terminology")))A state channel is established between two parties, through a transaction that locks a shared state on the blockchain. This is called the _funding transaction_ or _anchor transaction_. This single transaction must be transmitted to the network and mined to establish the channel. In the example of a payment channel, the locked state is the initial balance (in currency) of the channel.
-
-The two parties then exchange signed transactions, called _commitment transactions_, that alter the initial state. These transactions are valid transactions in that they _could_ be submitted for settlement by either party, but instead are held off-chain by each party pending the channel closure. State updates can be created as fast as each party can create, sign, and transmit a transaction to the other party. In practice this means that thousands of transactions per second can be exchanged.
-
-When exchanging commitment transactions the two parties also invalidate the previous states, so that the most up-to-date commitment transaction is always the only one that can be redeemed. This prevents either party from cheating by unilaterally closing the channel with an expired prior state that is more favorable to them than the current state. We will examine the various mechanisms that can be used to invalidate prior state in the rest of this chapter.
-
-Finally, the channel can be closed either cooperatively, by submitting a final _settlement transaction_ to the blockchain, or unilaterally, by either party submitting the last commitment transaction to the blockchain. A unilateral close option is needed in case one of the parties unexpectedly disconnects. The settlement transaction represents the final state of the channel and is settled on the blockchain.
-
-In the entire lifetime of the channel, only two transactions need to be submitted for mining on the blockchain: the funding and settlement transactions. In between these two states, the two parties can exchange any number of commitment transactions that are never seen by anyone else, nor submitted to the blockchain.
-
-<<payment_channel>> illustrates a payment channel between Bob and Alice, showing the funding, commitment, and settlement transactions.((("use cases", "buying coffee", startref="alicetwelve")))
-
-[[payment_channel]]
-.A payment channel between Bob and Alice, showing the funding, commitment, and settlement transactions
-image::images/mbc2_1201.png["A payment channel between Bob and Alice, showing the funding, commitment, and settlement transactions"]
-
-==== Simple Payment Channel Example
-
-((("payment (state) channels", "example of", id="PSCexample12")))To explain state channels, we start with a very simple example. We demonstrate a one-way channel, meaning that value is flowing in one direction only. We will also start with the naive assumption that no one is trying to cheat, to keep things simple. Once we have the basic channel idea explained, we will then look at what it takes to make it trustless so that neither party _can_ cheat, even if they are trying to.
-
-For this example we will assume two participants: Emma and Fabian. Fabian offers a video streaming service that is billed by the second using a micropayment channel. Fabian charges 0.01 millibit (0.00001 BTC) per second of video, equivalent to 36 millibits (0.036 BTC) per hour of video. Emma is a user who purchases this streaming video service from Fabian. <<emma_fabian_streaming_video>> shows Emma buying the video streaming service from Fabian using a payment channel.
-
-[[emma_fabian_streaming_video]]
-.Emma purchases streaming video from Fabian with a payment channel, paying for each second of video
-image::images/mbc2_1202.png["Emma purchases streaming video from Fabian with a payment channel, paying for each second of video"]
-
-In this example, Fabian and Emma are using special software that handles both the payment channel and the video streaming. Emma is running the software in her browser, Fabian is running it on a server. The software includes basic bitcoin wallet functionality and can create and sign bitcoin transactions. Both the concept and the term "payment channel" are completely hidden from the users. What they see is video that is paid for by the second.
-
-To set up the payment channel, Emma and Fabian establish a 2-of-2 multisignature address, with each of them holding one of the keys. From Emma's perspective, the software in her browser presents a QR code with a P2SH address (starting with "3"), and asks her to submit a "deposit" for up to 1 hour of video. The address is then funded by Emma. Emma's transaction, paying to the multisignature address, is the funding or anchor transaction for the payment channel.
-
-For this example, let's say that Emma funds the channel with 36 millibits (0.036 BTC). This will allow Emma to consume _up to_ 1 hour of streaming video. The funding transaction in this case sets the maximum amount that can be transmitted in this channel, setting the _channel capacity_.
-
-The funding transaction consumes one or more inputs from Emma's wallet, sourcing the funds. It creates one output with a value of 36 millibits paid to the multisignature 2-of-2 address controlled jointly between Emma and Fabian. It may have additional outputs for change back to Emma's wallet.
-
-Once the funding transaction is confirmed, Emma can start streaming video. Emma's software creates and signs a commitment transaction that changes the channel balance to credit 0.01 millibit to Fabian's address and refund 35.99 millibits back to Emma. The transaction signed by Emma consumes the 36 millibits output created by the funding transaction and creates two outputs: one for her refund, the other for Fabian's payment. The transaction is only partially signed&#x2014;it requires two signatures (2-of-2), but only has Emma's signature. When Fabian's server receives this transaction, it adds the second signature (for the 2-of-2 input) and returns it to Emma together with 1 second worth of video. Now both parties have a fully signed commitment transaction that either can redeem, representing the correct up-to-date balance of the channel. Neither party broadcasts this transaction to the network.
-
-In the next round, Emma's software creates and signs another commitment transaction (commitment #2) that consumes the _same_ 2-of-2 output from the funding transaction. The second commitment transaction allocates one output of 0.02 millibits to Fabian's address and one output of 35.98 millibits back to Emma's address. This new transaction is payment for two cumulative seconds of video. Fabian's software signs and returns the second commitment transaction, together with another second of video.
-
-In this way, Emma's software continues to send commitment transactions to Fabian's server in exchange for streaming video. The balance of the channel gradually accumulates in favor of Fabian, as Emma consumes more seconds of video. Let's say Emma watches 600 seconds (10 minutes) of video, creating and signing 600 commitment transactions. The last commitment transaction (#600) will have two outputs, splitting the balance of the channel, 6 millibits to Fabian and 30 millibits to Emma.
-
-Finally, Emma selects "Stop" to stop streaming video. Either Fabian or Emma can now transmit the final state transaction for settlement. This last transaction is the _settlement transaction_ and pays Fabian for all the video Emma consumed, refunding the remainder of the funding transaction to Emma.
-
-<<video_payment_channel>> shows the channel between Emma and Fabian and the commitment transactions that update the balance of the channel.
-
-In the end, only two transactions are recorded on the blockchain: the funding transaction that established the channel and a settlement transaction that allocated the final balance correctly between the two participants.((("", startref="PSCexample12")))
-
-[[video_payment_channel]]
-.Emma's payment channel with Fabian, showing the commitment transactions that update the balance of the channel
-image::images/mbc2_1203.png["Emma's payment channel with Fabian, showing the commitment transactions that update the balance of the channel"]
-
-==== Making Trustless Channels
-
-((("payment (state) channels", "making trustless channels", id="PSCtrust12")))The channel we just described works, but only if both parties cooperate, without any failures or attempts to cheat. Let's look at some of the scenarios that break this channel and see what is needed to fix those:
-
-* Once the funding transaction happens, Emma needs Fabian's signature to get any money back. If Fabian disappears, Emma's funds are locked in a 2-of-2 and effectively lost. This channel, as constructed, leads to a loss of funds if one of the parties disconnects before there is at least one commitment transaction signed by both parties.
-
-* While the channel is running, Emma can take any of the commitment transactions Fabian has countersigned and transmit one to the blockchain. Why pay for 600 seconds of video, if she can transmit commitment transaction #1 and only pay for 1 second of video? The channel fails because Emma can cheat by broadcasting a prior commitment that is in her favor.
-
-Both of these problems can be solved with timelocks&#x2014;let's look at how we could use transaction-level timelocks (+nLocktime+).
-
-Emma cannot risk funding a 2-of-2 multisig unless she has a guaranteed refund. To solve this problem, Emma constructs the funding and refund transactions at the same time. She signs the funding transaction but doesn't transmit it to anyone. Emma transmits only the refund transaction to Fabian and obtains his signature.
-
-The refund transaction acts as the first commitment transaction and its timelock establishes the upper bound for the channel's life. In this case, Emma could set the +nLocktime+ to 30 days or 4320 blocks into the future. All subsequent commitment transactions must have a shorter timelock, so that they can be redeemed before the refund transaction.
-
-Now that Emma has a fully signed refund transaction, she can confidently transmit the signed funding transaction knowing that she can eventually, after the timelock expires, redeem the refund transaction even if Fabian disappears.
-
-Every commitment transaction the parties exchange during the life of the channel will be timelocked into the future. But the delay will be slightly shorter for each commitment so the most recent commitment can be redeemed before the prior commitment it invalidates. Because of the nLockTime, neither party can successfully propagate any of the commitment transactions until their timelock expires. If all goes well, they will cooperate and close the channel gracefully with a settlement transaction, making it unnecessary to transmit an intermediate commitment transaction. If not, the most recent commitment transaction can be propagated to settle the account and invalidate all prior commitment transactions.
-
-For example, if commitment transaction #1 is timelocked to 4320 blocks in the future, then commitment transaction #2 is timelocked to 4319 blocks in the future. Commitment transaction #600 can be spent 600 blocks before commitment transaction #1 becomes valid.
-
-<<timelocked_commitments>> shows each commitment transaction setting a shorter timelock, allowing it to be spent before the previous commitments become valid.
-
-[[timelocked_commitments]]
-.Each commitment sets a shorter timelock, allowing it to be spent before the previous commitments become valid
-image::images/mbc2_1204.png["Each commitment sets a shorter timelock, allowing it to be spent before the previous commitments become valid"]
-
-Each subsequent commitment transaction must have a shorter timelock so that it may be broadcast before its predecessors and before the refund transaction. The ability to broadcast a commitment earlier ensures it will be able to spend the funding output and preclude any other commitment transaction from being redeemed by spending the output. The guarantees offered by the Bitcoin blockchain, preventing double-spends and enforcing timelocks, effectively allow each commitment transaction to invalidate its predecessors.
-
-State channels use timelocks to enforce smart contracts across a time dimension. In this example we saw how the time dimension guarantees that the most recent commitment transaction becomes valid before any earlier commitments. Thus, the most recent commitment transaction can be transmitted, spending the inputs and invalidating prior commitment transactions. The enforcement of smart contracts with absolute timelocks protects against cheating by one of the parties. This implementation needs nothing more than absolute transaction-level timelocks (+nLocktime+). Next, we will see how script-level timelocks, +CHECKLOCKTIMEVERIFY+ and +CHECKSEQUENCEVERIFY+, can be used to construct more flexible, useful, and sophisticated state channels.
-
-The first form of unidirectional payment channel was demonstrated as a prototype video streaming application in 2015 by an Argentinian team of developers.
-
-Timelocks are not the only way to invalidate prior commitment transactions. In the next sections we will see how a revocation key can be used to achieve the same result. Timelocks are effective but they have two distinct disadvantages. By establishing a maximum timelock when the channel is first opened, they limit the lifetime of the channel. Worse, they force channel implementations to strike a balance between allowing long-lived channels and forcing one of the participants to wait a very long time for a refund in case of premature closure. For example, if you allow the channel to remain open for 30 days, by setting the refund timelock to 30 days, if one of the parties disappears immediately the other party must wait 30 days for a refund. The more distant the endpoint, the more distant the refund.
-
-The second problem is that since each subsequent commitment transaction must decrement the timelock, there is an explicit limit on the number of commitment transactions that can be exchanged between the parties. For example, a 30-day channel, setting a timelock of 4320 blocks into the future, can only accommodate 4320 intermediate commitment transactions before it must be closed. There is a danger in setting the timelock commitment transaction interval at 1 block. By setting the timelock interval between commitment transactions to 1 block, a developer is creating a very high burden for the channel participants who have to be vigilant, remain online and watching, and be ready to transmit the right commitment transaction at any time.
-
-Now that we understand how timelocks can be used to invalidate prior commitments, we can see the difference between closing the channel cooperatively and closing it unilaterally by broadcasting a commitment transaction. All commitment transactions are timelocked, therefore broadcasting a commitment transaction will always involve waiting until the timelock has expired. But if the two parties agree on what the final balance is and know they both hold commitment transactions that will eventually make that balance a reality, they can construct a settlement transaction without a timelock representing that same balance. In a cooperative close, either party takes the most recent commitment transaction and builds a settlement transaction that is identical in every way except that it omits the timelock. Both parties can sign this settlement transaction knowing there is no way to cheat and get a more favorable balance. By cooperatively signing and transmitting the settlement transaction they can close the channel and redeem their balance immediately. Worst case, one of the parties can be petty, refuse to cooperate, and force the other party to do a unilateral close with the most recent commitment transaction. But if they do that, they have to wait for their funds too.((("", startref="PSCtrust12")))
-
-==== Asymmetric Revocable Commitments
-
-((("payment (state) channels", "asymmetric revocable commitments", id="PSCaymetric12")))A better way to handle the prior commitment states is to explicitly revoke them. However, this is not easy to achieve. A key characteristic of bitcoin is that once a transaction is valid, it remains valid and does not expire. The only way to cancel a transaction is by double-spending its inputs with another transaction before it is mined. That's why we used timelocks in the simple payment channel example above to ensure that more recent commitments could be spent before older commitments were valid. However, sequencing commitments in time creates a number of constraints that make payment channels difficult to use.
-
-Even though a transaction cannot be canceled, it can be constructed in such a way as to make it undesirable to use. The way we do that is by giving each party a _revocation key_ that can be used to punish the other party if they try to cheat. This mechanism for revoking prior commitment transactions was first proposed as part of the Lightning Network.
-
-To explain revocation keys, we will construct a more complex payment channel between two exchanges run by Hitesh and Irene. Hitesh and Irene run bitcoin exchanges in India and the USA, respectively. Customers of Hitesh's Indian exchange often send payments to customers of Irene's USA exchange and vice versa. Currently, these transactions occur on the Bitcoin blockchain, but this means paying fees and waiting several blocks for confirmations. Setting up a payment channel between the exchanges will significantly reduce the cost and accelerate the transaction flow.
-
-Hitesh and Irene start the channel by collaboratively constructing a funding transaction, each funding the channel with 5 bitcoin.
-The initial balance is 5 bitcoin for Hitesh and 5 bitcoin for Irene. The funding transaction locks the channel state in a 2-of-2 multisig, just like in the example of a simple channel.
-
-The funding transaction may have one or more inputs from Hitesh (adding up to 5 bitcoin or more), and one or more inputs from Irene (adding up to 5 bitcoin or more). The inputs have to slightly exceed the channel capacity in order to cover the transaction fees. The transaction has one output that locks the 10 total bitcoin to a 2-of-2 multisig address controlled by both Hitesh and Irene. The funding transaction may also have one or more outputs returning change to Hitesh and Irene if their inputs exceeded their intended channel contribution. This is a single transaction with inputs offered and signed by two parties. It has to be constructed in collaboration and signed by each party before it is transmitted.
-
-Now, instead of creating a single commitment transaction that both parties sign, Hitesh and Irene create two different commitment transactions that are _asymmetric_.
-
-Hitesh has a commitment transaction with two outputs. The first output pays Irene the 5 bitcoin she is owed _immediately_. The second output pays Hitesh the 5 bitcoin he is owed, but only after a timelock of 1000 blocks. The transaction outputs look like this:
-
-----
-Input: 2-of-2 funding output, signed by Irene
-
-Output 0 <5 bitcoin>:
-    <Irene's Public Key> CHECKSIG
-
-Output 1 <5 bitcoin>:
-    <1000 blocks>
-    CHECKSEQUENCEVERIFY
-    DROP
-    <Hitesh's Public Key> CHECKSIG
-----
-
-Irene has a different commitment transaction with two outputs. The first output pays Hitesh the 5 bitcoin he is owed immediately. The second output pays Irene the 5 bitcoin she is owed but only after a timelock of 1000 blocks. The commitment transaction Irene holds (signed by Hitesh) looks like this:
-
-----
-Input: 2-of-2 funding output, signed by Hitesh
-
-Output 0 <5 bitcoin>:
-    <Hitesh's Public Key> CHECKSIG
-
-Output 1 <5 bitcoin>:
-    <1000 blocks>
-    CHECKSEQUENCEVERIFY
-    DROP
-    <Irene's Public Key> CHECKSIG
-----
-
-This way, each party has a commitment transaction, spending the 2-of-2 funding output. This input is signed by the _other_ party. At any time the party holding the transaction can also sign (completing the 2-of-2) and broadcast. However, if they broadcast the commitment transaction, it pays the other party immediately whereas they have to wait for a timelock to expire. By imposing a delay on the redemption of one of the outputs, we put each party at a slight disadvantage when they choose to unilaterally broadcast a commitment transaction. But a time delay alone isn't enough to encourage fair conduct.
-
-<<asymmetric_commitments>> shows two asymmetric commitment transactions, where the output paying the holder of the commitment is delayed.
-
-[[asymmetric_commitments]]
-.Two asymmetric commitment transactions with delayed payment for the party holding the transaction
-image::images/mbc2_1205.png["Two asymmetric commitment transactions with delayed payment for the party holding the transaction"]
-
-Now we introduce the final element of this scheme: a revocation key that prevents a cheater from broadcasting an expired commitment. The revocation key allows the wronged party to punish the cheater by taking the entire balance of the channel.
-
-The revocation key is composed of two secrets, each half generated independently by each channel participant. It is similar to a 2-of-2 multisig, but constructed using elliptic curve arithmetic, so that both parties know the revocation public key but each party knows only half the revocation secret key.
-
-In each round, both parties reveal their half of the revocation secret to the other party, thereby giving the other party (who now has both halves) the means to claim the penalty output if this revoked transaction is ever broadcast.
-
-Each of the commitment transactions has a "delayed" output. The redemption script for that output allows one party to redeem it after 1000 blocks, _or_ the other party to redeem it if they have a revocation key, penalizing transmission of a revoked commitment.
-
-So when Hitesh creates a commitment transaction for Irene to sign, he makes the second output payable to himself after 1000 blocks, or to the revocation public key (of which he only knows half the secret). Hitesh constructs this transaction. He will only reveal his half of the revocation secret to Irene when he is ready to move to a new channel state and wants to revoke this commitment.
-
-The second output's script looks like this:
-
-----
-Output 0 <5 bitcoin>:
-    <Irene's Public Key> CHECKSIG
-
-Output 1 <5 bitcoin>:
-IF
-    # Revocation penalty output
-    <Revocation Public Key>
-ELSE
-    <1000 blocks>
-    CHECKSEQUENCEVERIFY
-    DROP
-    <Hitesh's Public Key>
-ENDIF
-CHECKSIG
-----
-
-Irene can confidently sign this transaction, since if transmitted it will immediately pay her what she is owed. Hitesh holds the transaction, but knows that if he transmits it in a unilateral channel closing, he will have to wait 1000 blocks to get paid.
-
-When the channel is advanced to the next state, Hitesh has to _revoke_ this commitment transaction before Irene agrees to sign the next commitment transaction. To do that, all he has to do is send his half of the _revocation key_ to Irene. Once Irene has both halves of the revocation secret key for this commitment, she can sign the next commitment with confidence. She knows that if Hitesh tries to cheat by publishing the prior commitment, she can use the revocation key to redeem Hitesh's delayed output. _If Hitesh cheats, Irene gets BOTH outputs_. Meanwhile, Hitesh only has half the revocation secret for that revocation public key and can't redeem the output until 1000 blocks. Irene will be able to redeem the output and punish Hitesh before the 1000 blocks have elapsed.
-
-The revocation protocol is bilateral, meaning that in each round, as the channel state is advanced, the two parties exchange new commitments, exchange revocation secrets for the previous commitments, and sign each other's new commitment transactions. As they accept a new state, they make the prior state impossible to use, by giving each other the necessary revocation secrets to punish any cheating.
-
-Let's look at an example of how it works. One of Irene's customers wants to send 2 bitcoin to one of Hitesh's customers. To transmit 2 bitcoin across the channel, Hitesh and Irene must advance the channel state to reflect the new balance. They will commit to a new state (state number 2) where the channel's 10 bitcoin are split, 7 bitcoin to Hitesh and 3 bitcoin to Irene. To advance the state of the channel, they will each create new commitment transactions reflecting the new channel balance.
-
-As before, these commitment transactions are asymmetric so that the commitment transaction each party holds forces them to wait if they redeem it. Crucially, before signing new commitment transactions, they must first exchange revocation keys to invalidate the prior commitment. In this particular case, Hitesh's interests are aligned with the real state of the channel and therefore he has no reason to broadcast a prior state. However, for Irene, state number 1 leaves her with a higher balance than state 2. When Irene gives Hitesh the revocation key for her prior commitment transaction (state number 1) she is effectively revoking her ability to profit from regressing the channel to a prior state because with the revocation key, Hitesh can redeem both outputs of the prior commitment transaction without delay. Meaning if Irene broadcasts the prior state, Hitesh can exercise his right to take all of the outputs.
-
-Importantly, the revocation doesn't happen automatically. While Hitesh has the ability to punish Irene for cheating, he has to watch the blockchain diligently for signs of cheating. If he sees a prior commitment transaction broadcast, he has 1000 blocks to take action and use the revocation key to thwart Irene's cheating and punish her by taking the entire balance, all 10 bitcoin.
-
-Asymmetric revocable commitments with relative time locks (+CSV+) are a much better way to implement payment channels and a very significant innovation in this technology. With this construct, the channel can remain open indefinitely and can have billions of intermediate commitment transactions. In prototype implementations of Lightning Network, the commitment state is identified by a 48-bit index, allowing more than 281 trillion (2.8 x 10^14^) state transitions in any single channel!((("", startref="PSCaymetric12")))
-
-==== Hash Time Lock Contracts (HTLC)
-
-((("Hash Time Lock Contracts (HTLC)")))((("payment (state) channels", "Hash Time Lock Contracts (HTLC)")))Payment channels can be further extended with a special type of smart contract that allows the participants to commit funds to a redeemable secret, with an expiration time. This feature is called a _Hash Time Lock Contract_, or _HTLC_, and is used in both bidirectional and routed payment channels.
-
-Let's first explain the "hash" part of the HTLC. To create an HTLC, the intended recipient of the payment will first create a secret +R+. They then calculate the hash of this secret +H+:
-
-----
-H = Hash(R)
-----
-
-This produces a hash +H+ that can be included in an output's locking script. Whoever knows the secret can use it to redeem the output. The secret +R+ is also referred to as a _preimage_ to the hash function. The preimage is just the data that is used as input to a hash function.
-
-The second part of an HTLC is the "time lock" component. If the secret is not revealed, the payer of the HTLC can get a "refund" after some time. This is achieved with an absolute time lock using +CHECKLOCKTIMEVERIFY+.
-
-The script implementing an HTLC might look like this:
-
-----
-IF
-    # Payment if you have the secret R
-    HASH160 <H> EQUALVERIFY
-ELSE
-    # Refund after timeout.
-    <locktime> CHECKLOCKTIMEVERIFY DROP
-    <Payer Public Key> CHECKSIG
-ENDIF
-----
-
-Anyone who knows the secret +R+, which when hashed equals to +H+, can redeem this output by exercising the first clause of the +IF+ flow.
-
-If the secret is not revealed and the HTLC claimed, after a certain number of blocks the payer can claim a refund using the second clause in the +IF+ flow.
-
-This is a basic implementation of an HTLC. This type of HTLC can be redeemed by _anyone_ who has the secret +R+. An HTLC can take many different forms with slight variations to the script. For example, adding a +CHECKSIG+ operator and a public key in the first clause restricts redemption of the hash to a named recipient, who must also know the secret +R+.((("", startref="BCApayment12")))
-
-[[lightning_network]]
-=== Routed Payment Channels (Lightning Network)
-
-((("blockchain applications", "routed payment channels", seealso="Lightning Network", id="BCAlightning12")))((("routed payment channels", see="Lightning Network")))((("Lightning Network", "defined")))The Lightning Network is a proposed routed network of bidirectional payment channels connected end-to-end. A network like this can allow any participant to route a payment from channel to channel without trusting any of the intermediaries. The Lightning Network was https://lightning.network/lightning-network-paper.pdf[first described by Joseph Poon and Thadeus Dryja in February 2015], building on the concept of payment channels as proposed and elaborated upon by many others.
-
-"Lightning Network" refers to a specific design for a routed payment channel network, which has now been implemented by at least five different open source teams. ((("Basics of Lightning Technology (BOLT)")))The independent implementations are coordinated by a set of interoperability standards described in the https://bit.ly/2rBHeoL[_Basics of Lightning Technology (BOLT)_ paper].
-
-Prototype implementations of the Lightning Network have been released by several teams.
-
-The Lightning Network is one possible way of implementing routed payment channels. There are several other designs that aim to achieve similar goals, such as Teechan and Tumblebit.
-
-==== Basic Lightning Network Example
-
-((("Lightning Network", "basic example")))Let's see how this works.
-
-In this example, we have five participants: Alice, Bob, Carol, Diana, and Eric. These five participants have opened payment channels with each other, in pairs. Alice has a payment channel with Bob. Bob is connected to Carol, Carol to Diana, and Diana to Eric. For simplicity let's assume each channel is funded with 2 bitcoin by each participant, for a total capacity of 4 bitcoin in each channel.
-
-<<lightning_network_fig>> shows five participants in a Lightning Network, connected by bidirectional payment channels that can be linked to make a payment from Alice to Eric (<<lightning_network>>).
-
-[[lightning_network_fig]]
-.A series of bidirectional payment channels linked to form a Lightning Network that can route a payment from Alice to Eric
-image::images/mbc2_1206.png["A series of bi-directional payment channels linked to form a Lightning Network"]
-
-Alice wants to pay Eric 1 bitcoin. However, Alice is not connected to Eric by a payment channel. Creating a payment channel requires a funding transaction, which must be committed to the Bitcoin blockchain. Alice does not want to open a new payment channel and commit more of her funds. Is there a way to pay Eric, indirectly?
-
-<<ln_payment_process>> shows the step-by-step process of routing a payment from Alice to Eric, through a series of HTLC commitments on the payment channels connecting the participants.
-
-[[ln_payment_process]]
-.Step-by-step payment routing through a Lightning Network
-image::images/mbc2_1207.png["Step-by-step payment routing through a Lightning Network"]
-
-Alice is running a Lightning Network (LN) node that is keeping track of her payment channel to Bob and has the ability to discover routes between payment channels. Alice's LN node also has the ability to connect over the internet to Eric's LN node. Eric's LN node creates a secret +R+ using a random number generator. Eric's node does not reveal this secret to anyone. Instead, Eric's node calculates a hash +H+ of the secret +R+ and transmits this hash to Alice's node (see <<ln_payment_process>> step 1).
-
-Now Alice's LN node constructs a route between Alice's LN node and Eric's LN node. The routing algorithm used will be examined in more detail later, but for now let's assume that Alice's node can find an efficient route.
-
-Alice's node then constructs an HTLC, payable to the hash +H+, with a 10-block refund timeout (current block + 10), for an amount of 1.003 bitcoin (see <<ln_payment_process>> step 2). The extra 0.003 will be used to compensate the intermediate nodes for their participation in this payment route. Alice offers this HTLC to Bob, deducting 1.003 bitcoin from her channel balance with Bob and committing it to the HTLC. The HTLC has the following meaning: _"Alice is committing 1.003 of her channel balance to be paid to Bob if Bob knows the secret, or refunded back to Alice's balance if 10 blocks elapse."_ The channel balance between Alice and Bob is now expressed by commitment transactions with three outputs: 2 bitcoin balance to Bob, 0.997 bitcoin balance to Alice, 1.003 bitcoin committed in Alice's HTLC. Alice's balance is reduced by the amount committed to the HTLC.
-
-Bob now has a commitment that if he is able to get the secret +R+ within the next 10 blocks, he can claim the 1.003 locked by Alice. With this commitment in hand, Bob's node constructs an HTLC on his payment channel with Carol. Bob's HTLC commits 1.002 bitcoin to hash +H+ for 9 blocks, which Carol can redeem if she has secret +R+ (see <<ln_payment_process>> step 3). Bob knows that if Carol can claim his HTLC, she has to produce +R+. If Bob has +R+ in nine blocks, he can use it to claim Alice's HTLC to him. He also makes 0.001 bitcoin for committing his channel balance for nine blocks. If Carol is unable to claim his HTLC and he is unable to claim Alice's HTLC, everything reverts back to the prior channel balances and no one is at a loss. The channel balance between Bob and Carol is now: 2 to Carol, 0.998 to Bob, 1.002 committed by Bob to the HTLC.
-
-Carol now has a commitment that if she gets +R+ within the next nine blocks, she can claim 1.002 bitcoin locked by Bob. Now she can make an HTLC commitment on her channel with Diana. She commits an HTLC of 1.001 bitcoin to hash +H+, for eight blocks, which Diana can redeem if she has secret +R+ (see <<ln_payment_process>> step 4). From Carol's perspective, if this works she is 0.001 bitcoin better off and if it doesn't she loses nothing. Her HTLC to Diana is only viable if +R+ is revealed, at which point she can claim the HTLC from Bob. The channel balance between Carol and Diana is now: 2 to Diana, 0.999 to Carol, 1.001 committed by Carol to the HTLC.
-
-Finally, Diana can offer an HTLC to Eric, committing 1 bitcoin for seven blocks to hash +H+ (see <<ln_payment_process>> step 5). The channel balance between Diana and Eric is now: 2 to Eric, 1 to Diana, 1 committed by Diana to the HTLC.
-
-However, at this hop in the route, Eric _has_ secret +R+. He can therefore claim the HTLC offered by Diana. He sends +R+ to Diana and claims the 1 bitcoin, adding it to his channel balance (see <<ln_payment_process>> step 6). The channel balance is now: 1 to Diana, 3 to Eric.
-
-Now, Diana has secret +R+. Therefore, she can now claim the HTLC from Carol. Diana transmits +R+ to Carol and adds the 1.001 bitcoin to her channel balance (see <<ln_payment_process>> step 7). Now the channel balance between Carol and Diana is: 0.999 to Carol, 3.001 to Diana. Diana has "earned" 0.001 for participating in this payment route.
-
-Flowing back through the route, the secret +R+ allows each participant to claim the outstanding HTLCs. Carol claims 1.002 from Bob, setting the balance on their channel to: 0.998 to Bob, 3.002 to Carol (see <<ln_payment_process>> step 8). Finally, Bob claims the HTLC from Alice (see <<ln_payment_process>> step 9). Their channel balance is updated as: 0.997 to Alice, 3.003 to Bob.
-
-Alice has paid Eric 1 bitcoin without opening a channel to Eric. None of the intermediate parties in the payment route had to trust each other. For the short-term commitment of their funds in the channel they are able to earn a small fee, with the only risk being a small delay in refund if the channel was closed or the routed payment failed.
-
-==== Lightning Network Transport and Routing
-
-((("Lightning Network", "transport and routing")))All communications between LN nodes are encrypted point-to-point. In addition, nodes have a long-term public key that they use as an identifier and to authenticate each other.
-
-Whenever a node wishes to send a payment to another node, it must first construct a _path_ through the network by connecting payment channels with sufficient capacity. Nodes advertise routing information, including what channels they have open, how much capacity each channel has, and what fees they charge to route payments. The routing information can be shared in a variety of ways and different routing protocols are likely to emerge as Lightning Network technology advances. Some Lightning Network implementations use the IRC protocol as a convenient mechanism for nodes to announce routing information. Another implementation of route discovery uses a P2P model where nodes propagate channel announcements to their peers, in a "flooding" model, similar to how bitcoin propagates transactions. Future plans include a proposal called https://bit.ly/2r5TACm[Flare], which is a hybrid routing model with local node "neighborhoods" and longer-range beacon nodes.
-
-In our previous example, Alice's node uses one of these route discovery mechanisms to find one or more paths connecting her node to Eric's node. Once Alice's node has constructed a path, she will initialize that path through the network, by propagating a series of encrypted and nested instructions to connect each of the adjacent payment channels.
-
-Importantly, this path is only known to Alice's node. All other participants in the payment route see only the adjacent nodes. From Carol's perspective, this looks like a payment from Bob to Diana. Carol does not know that Bob is actually relaying a payment from Alice. She also doesn't know that Diana will be relaying a payment to Eric.
-
-This is a critical feature of the Lightning Network, because it ensures privacy of payments and makes it very difficult to apply surveillance, censorship, or blacklists. But how does Alice establish this payment path, without revealing anything to the intermediary nodes?
-
-The Lightning Network implements an onion-routed protocol based on a scheme called https://bit.ly/2q6ZDrP[Sphinx]. This routing protocol ensures that a payment sender can construct and communicate a path through the Lightning Network such that:
-
-* Intermediate nodes can verify and decrypt their portion of route information and find the next hop.
-
-* Other than the previous and next hops, they cannot learn about any other nodes that are part of the path.
-
-* They cannot identify the length of the payment path, or their own position in that path.
-
-* Each part of the path is encrypted in such a way that a network-level attacker cannot associate the packets from different parts of the path to each other.
-
-* Unlike Tor (an onion-routed anonymization protocol on the internet), there are no "exit nodes" that can be placed under surveillance. The payments do not need to be transmitted to the Bitcoin blockchain; the nodes just update channel balances.
-
-Using this onion-routed protocol, Alice wraps each element of the path in a layer of encryption, starting with the end and working backward. She encrypts a message to Eric with Eric's public key. This message is wrapped in a message encrypted to Diana, identifying Eric as the next recipient. The message to Diana is wrapped in a message encrypted to Carol's public key and identifying Diana as the next recipient. The message to Carol is encrypted to Bob's key. Thus, Alice has constructed this encrypted multilayer "onion" of messages. She sends this to Bob, who can only decrypt and unwrap the outer layer. Inside, Bob finds a message addressed to Carol that he can forward to Carol but cannot decipher himself. Following the path, the messages get forwarded, decrypted, forwarded, etc., all the way to Eric. Each participant knows only the previous and next node in each hop.((("", startref="alicetwelve")))
-
-Each element of the path contains information on the HTLC that must be extended to the next hop, the amount that is being sent, the fee to include, and the CLTV locktime (in blocks) expiration of the HTLC. As the route information propagates, the nodes make HTLC commitments forward to the next hop.
-
-At this point, you might be wondering how it is possible that the nodes do not know the length of the path and their position in that path. After all, they receive a message and forward it to the next hop. Doesn't it get shorter, allowing them to deduce the path size and their position? To prevent this, the path is always fixed at 20 hops and padded with random data. Each node sees the next hop and a fixed-length encrypted message to forward. Only the final recipient sees that there is no next hop. To everyone else it seems as if there are always 20 more hops to go.
-
-==== Lightning Network Benefits
-
-((("Lightning Network", "benefits of")))A Lightning Network is a second-layer routing technology. It can be applied to any blockchain that supports some basic capabilities, such as multisignature transactions, timelocks, and basic smart contracts.
-
-If a Lightning Network is layered on top of the Bitcoin network, the Bitcoin network can gain a significant increase in capacity, privacy, granularity, and speed, without sacrificing the principles of trustless operation without intermediaries:
-
-Privacy:: Lightning Network payments are much more private than payments on the Bitcoin blockchain, as they are not public. While participants in a route can see payments propagated across their channels, they do not know the sender or recipient.
-
-Fungibility:: A Lightning Network makes it much more difficult to apply surveillance and blacklists on bitcoin, increasing the fungibility of the currency.
-
-Speed:: Bitcoin transactions using Lightning Network are settled in milliseconds, rather than minutes, as HTLCs are cleared without committing transactions to a block.
-
-Granularity:: A Lightning Network can enable payments at least as small as the bitcoin "dust" limit, perhaps even smaller. Some proposals allow for subsatoshi increments.
-
-Capacity:: A Lightning Network increases the capacity of the Bitcoin system by several orders of magnitude. There is no practical upper bound to the number of payments per second that can be routed over a Lightning Network, as it depends only on the capacity and speed of each node.
-
-Trustless Operation:: A Lightning Network uses bitcoin transactions between nodes that operate as peers without trusting each other. Thus, a Lightning Network preserves the principles of the Bitcoin system, while expanding its operating parameters significantly.
-
-Of course, as mentioned previously, the Lightning Network protocol is not the only way to implement routed payment channels. Other proposed systems include Tumblebit and Teechan. At this time, however, the Lightning Network has already been deployed on testnet. Several different teams have developed competing implementations of LN and are working toward a common interoperability standard (called BOLT). It is likely that Lightning Network will be the first routed payment channel network to be deployed in production.((("", startref="BCAlightning12")))
-
-=== Conclusion
-
-We have examined just a few of the emerging applications that can be built using the Bitcoin blockchain as a trust platform. These applications expand the scope of bitcoin beyond payments and beyond financial instruments, to encompass many other applications where trust is critical. By decentralizing the basis of trust, the Bitcoin blockchain is a platform that will spawn many revolutionary applications in a wide variety of industries.

ch13_security.adoc

@@ -0,0 +1,291 @@
+[[ch11]]
+== Bitcoin Security
+
+Securing your bitcoins is challenging because bitcoins are
+are not like a balance in a bank account. Your bitcoins are very
+much like digital cash or gold. You've probably heard the expression,
+"Possession is nine-tenths of the law." Well, in Bitcoin, possession is
+ten-tenths of the law. Possession of the keys to spend certain bitcoins is
+equivalent to possession of cash or a chunk of precious metal. You can
+lose it, misplace it, have it stolen, or accidentally give the wrong
+amount to someone. In every one of these cases, users have no recourse
+within the protocol, just as if they dropped cash on a public sidewalk.
+
+However, the Bitcoin system has capabilities that cash, gold, and bank accounts do
+not. A Bitcoin wallet, containing your keys, can be backed up like any
+file. It can be stored in multiple copies, even printed on paper for
+hard-copy backup. You can't "back up" cash, gold, or bank accounts.
+Bitcoin is different enough from anything that has come before that we
+need to think about securing our bitcoins in a novel way too.
+
+=== Security Principles
+
+The ((("Bitcoin", "security", "principles of", id="bitcoin-security-principle")))((("security", "principles of", id="security-principle")))((("decentralized consensus", "as security principle", secondary-sortas="security principle", id="decentral-consensus-principle")))core principle in Bitcoin is
+decentralization and it has important implications for security. A
+centralized model, such as a traditional bank or payment network,
+depends on access control and vetting to keep bad actors out of the
+system. By comparison, a decentralized system like Bitcoin pushes the
+responsibility and control to the users. Because the security of the network
+is based on independent verification, the network can be open
+and no encryption is required for Bitcoin traffic (although encryption
+can still be useful).
+
+On a traditional payment network, such as a credit card system, the
+payment is open-ended because it contains the user's private identifier
+(the credit card number). After the initial charge, anyone with access
+to the identifier can "pull" funds and charge the owner again and again.
+Thus, the payment network has to be secured end-to-end with encryption
+and must ensure that no eavesdroppers or intermediaries can compromise
+the payment traffic in transit or when it is stored (at rest). If a bad
+actor gains access to the system, he can compromise current transactions
+_and_ payment tokens that can be used to create new transactions. Worse,
+when customer data is compromised, the customers are exposed to identity
+theft and must take action to prevent fraudulent use of the compromised
+accounts.
+
+Bitcoin is dramatically different. A Bitcoin transaction authorizes only
+a specific value to a specific recipient and cannot be forged.
+It does not reveal any private information, such as the
+identities of the parties, and cannot be used to authorize additional
+payments. Therefore, a Bitcoin payment network does not need to be
+encrypted or protected from eavesdropping. In fact, you can broadcast
+Bitcoin transactions over an open public channel, such as unsecured WiFi
+or Bluetooth, with no loss of security.
+
+Bitcoin's decentralized security model puts a lot of power in the hands
+of the users. With that power comes responsibility for maintaining the
+secrecy of their keys. For most users that is not easy to do, especially
+on general-purpose computing devices such as internet-connected
+smartphones or laptops. Although Bitcoin's decentralized model prevents
+the type of mass compromise seen with credit cards, many users are not
+able to adequately secure their keys and get hacked, one by one.
+
+==== Developing Bitcoin Systems Securely
+
+A critical principle
+for Bitcoin developers is decentralization. Most developers will be
+familiar with centralized security models and might be tempted to apply
+these models to their Bitcoin applications, with disastrous results.
+
+Bitcoin's security relies on decentralized control over keys and on
+independent transaction validation by users. If you want to leverage
+Bitcoin's security, you need to ensure that you remain within the
+Bitcoin security model. In simple terms: don't take control of keys away
+from users and don't outsource validation.
+
+For example, many early Bitcoin exchanges concentrated all user funds in
+a single "hot" wallet with keys stored on a single server. Such a design
+removes control from users and centralizes control over keys in a single
+system. Many such systems have been hacked, with disastrous consequences
+for their customers.
+
+Unless you are prepared to invest heavily in operational security,
+multiple layers of access control, and audits (as the traditional banks
+do), you should think very carefully before taking funds outside of
+Bitcoin's decentralized security context. Even if you have the funds and
+discipline to implement a robust security model, such a design merely
+replicates the fragile model of traditional financial networks, plagued
+by identity theft, corruption, and embezzlement. To take advantage of
+Bitcoin's unique decentralized security model, you have to avoid the
+temptation of centralized architectures that might feel familiar but
+ultimately subvert Bitcoin's ((("decentralized consensus", "as security principle", secondary-sortas="security principle", startref="decentral-consensus-principle")))security.
+
+==== The Root of Trust
+
+Traditional ((("root of trust", id="root-trust")))security architecture is based
+upon a concept called the _root of trust_, which is a trusted core used
+as the foundation for the security of the overall system or application.
+Security architecture is developed around the root of trust as a series
+of concentric circles, like layers in an onion, extending trust outward
+from the center. Each layer builds upon the more-trusted inner layer
+using access controls, digital signatures, encryption, and other
+security primitives. As software systems become more complex, they are
+more likely to contain bugs, which make them vulnerable to security
+compromise. As a result, the more complex a software system becomes, the
+harder it is to secure. The root of trust concept ensures that most of
+the trust is placed within the least complex part of the system, and
+therefore the least vulnerable parts of the system, while more complex
+software is layered around it. This security architecture is repeated at
+different scales, first establishing a root of trust within the hardware
+of a single system, then extending that root of trust through the
+operating system to higher-level system services, and finally across
+many servers layered in concentric circles of diminishing trust.
+
+Bitcoin security
+architecture is different. In Bitcoin, the consensus system creates a
+trusted blockchain that is completely decentralized. A correctly
+validated blockchain uses the genesis block as the root of trust,
+building a chain of trust up to the current block. Bitcoin systems can
+and should use the blockchain as their root of trust. When designing a
+complex Bitcoin application that consists of services on many different
+systems, you should carefully examine the security architecture in order
+to ascertain where trust is being placed. Ultimately, the only thing
+that should be explicitly trusted is a fully validated blockchain. If
+your application explicitly or implicitly vests trust in anything but
+the blockchain, that should be a source of concern because it introduces
+vulnerability. A good method to evaluate the security architecture of
+your application is to consider each individual component and evaluate a
+hypothetical scenario where that component is completely compromised and
+under the control of a malicious actor. Take each component of your
+application, in turn, and assess the impacts on the overall security if
+that component is compromised. If your application is no longer secure
+when components are compromised, that shows you have misplaced trust in
+those components. A Bitcoin application without vulnerabilities should
+be vulnerable only to a compromise of the Bitcoin consensus mechanism,
+meaning that its root of trust is based on the strongest part of the
+Bitcoin security architecture.
+
+The numerous examples of hacked Bitcoin exchanges serve to underscore
+this point because their security architecture and design fails even
+under the most casual scrutiny. These centralized implementations had
+invested trust explicitly in numerous components outside the Bitcoin
+blockchain, such as hot wallets, centralized databases,
+vulnerable encryption keys, and ((("Bitcoin", "security", "principles of", startref="bitcoin-security-principle")))((("security", "principles of", startref="security-principle")))((("root of trust", startref="root-trust")))similar schemes.
+
+=== User Security Best Practices
+
+Humans ((("Bitcoin", "security", "best practices", id="bitcoin-security-best-practice")))((("security", "best practices", id="security-best-practice")))((("best practices, security", id="best-practice-security")))have
+used physical security controls for thousands of years. By comparison,
+our experience with digital security is less than 50 years old. Modern
+general-purpose operating systems are not very secure and not
+particularly suited to storing digital money. Our computers are
+constantly exposed to external threats via always-on internet
+connections. They run thousands of software components from hundreds of
+authors, often with unconstrained access to the user's files. A single
+piece of rogue software, among the many thousands installed on your
+computer, can compromise your keyboard and files, stealing any bitcoins
+stored in wallet applications. The level of computer maintenance
+required to keep a computer virus-free and trojan-free is beyond the
+skill level of all but a tiny minority of computer users.
+
+Despite decades of research and advancements in information security,
+digital assets are still woefully vulnerable to a determined adversary.
+Even the most highly protected and restricted systems, in financial
+services companies, intelligence agencies, and defense contractors, are
+frequently breached. Bitcoin creates digital assets that have intrinsic
+value and can be stolen and diverted to new owners instantly and
+irrevocably. This creates a massive incentive for hackers. Until now,
+hackers had to convert identity information or account tokens—such as
+credit cards and bank accounts—into value after compromising them.
+Despite the difficulty of fencing and laundering financial information,
+we have seen ever-escalating thefts. Bitcoin escalates this problem
+because it doesn't need to be fenced or laundered; bitcoins are valuable
+by themselves.
+
+Bitcoin also creates the incentives to improve computer
+security. Whereas previously the risk of computer compromise was vague
+and indirect, Bitcoin makes these risks clear and obvious. Holding
+bitcoins on a computer serves to focus the user's mind on the need for
+improved computer security. As a direct result of the proliferation and
+increased adoption of Bitcoin and other digital currencies, we have seen
+an escalation in both hacking techniques and security solutions. In
+simple terms, hackers now have a very juicy target and users have a
+clear incentive to defend themselves.
+
+Over the past three years, as a direct result of Bitcoin adoption, we
+have seen tremendous innovation in the realm of information security in
+the form of hardware encryption, key storage and hardware signing devices,
+multisignature technology, and digital escrow. In the following sections
+we will examine various best practices for practical user security.
+
+==== Physical Bitcoin Storage
+
+Because most ((("bitcoins", "physical storage")))((("physical bitcoin storage")))((("storing bitcoins", id="storing-bitcoin")))users are far more
+comfortable with physical security than information security, a very
+effective method for protecting bitcoins is to convert them into physical
+form. Bitcoin keys, and the seeds used to create them, are nothing more than long numbers. This means that
+they can be stored in a physical form, such as printed on paper or
+etched on a metal plate. Securing the keys then becomes as simple as
+physically securing a printed copy of the key seed. A seed
+that is printed on paper is called a "paper backup," and
+many wallets can create them.
+Keeping bitcoins
+offline is ((("cold storage")))called _cold storage_ and it is one of the most effective
+security techniques. A cold storage system is one where the keys are
+generated on an offline system (one never connected to the internet) and
+stored offline either on paper or on digital media, such as a USB memory
+stick.
+
+==== Hardware Signing Devices
+
+In the ((("hardware signing devices")))long term, Bitcoin security may increasingly take the
+form of tamper-proof hardware signing devices. Unlike a smartphone or desktop
+computer, a Bitcoin hardware signing device only needs to hold keys and
+use them to generate signatures.  Without general-purpose software to
+compromise and
+with limited interfaces, hardware signing devices can deliver strong
+security to nonexpert users. Hardware
+signing devices may become the predominant method of storing bitcoins.
+
+==== Ensuring Your Access
+
+Although
+most users ((("backing up", "importance of")))are rightly concerned about theft of their bitcoins, there is an even
+bigger risk. Data files get lost all the time. If they contain Bitcoin keys,
+the loss is much more painful. In the effort to secure their Bitcoin
+wallets, users must be very careful not to go too far and end up losing
+their bitcoins. In July 2011, a well-known Bitcoin awareness and education
+project lost almost 7,000 bitcoin. In their effort to prevent theft, the
+owners had implemented a complex series of encrypted backups. In the end
+they accidentally lost the encryption keys, making the backups worthless
+and losing a fortune. Like hiding money by burying it in the desert, if
+you secure your bitcoins too well you might not be able to find them again.
+
+[WARNING]
+====
+To spend bitcoins, you may((("wallets", "recovery codes")))((("recovery codes"))) need to back up more than just your private
+keys or the BIP32 seed used to derive them.  This is especially the case
+when multisignatures or complex scripts are being used.  Most output
+scripts commit to the actual conditions that must be fulfilled to spend
+the bitcoins in that output, and it's not possible to fulfill that
+commitment unless your wallet software can reveal those conditions to
+the network.  Wallet recovery codes must include this information.  For
+more details, see <<ch05_wallets>>.
+====
+
+==== Diversifying Risk
+
+Would you((("risk diversification")))((("storing bitcoins", startref="storing-bitcoin"))) carry your entire net worth in cash in your wallet? Most
+people would consider that reckless, yet Bitcoin users often keep all
+their bitcoins using a single wallet application. Instead, users should spread the risk
+among multiple and diverse Bitcoin applications. Prudent users will keep only
+a small fraction, perhaps less than 5%, of their bitcoins in an online or
+mobile wallet as "pocket change." The rest should be split between a few
+different storage mechanisms, such as a desktop wallet and offline (cold
+storage).
+
+==== Multisig and Governance
+
+Whenever a ((("multisignature addresses")))((("addresses", "multisignature")))company or individual stores large amounts of
+bitcoins, they should consider using a multisignature Bitcoin address.
+Multisignature addresses secure funds by requiring more than one
+signature to make a payment. The signing keys should be stored in a
+number of different locations and under the control of different people.
+In a corporate environment, for example, the keys should be generated
+independently and held by several company executives to ensure that no
+single person can compromise the funds. Multisignature addresses can
+also offer redundancy, where a single person holds several keys that are
+stored in different locations.
+
+
+==== Survivability
+
+One important
+security ((("survivability (of bitcoin access)")))((("estate planning")))consideration that is often overlooked is availability,
+especially in the context of incapacity or death of the key holder.
+Bitcoin users are told to use complex passwords and keep their keys
+secure and private, not sharing them with anyone. Unfortunately, that
+practice makes it almost impossible for the user's family to recover any
+funds if the user is not available to unlock them. In most cases, in
+fact, the families of Bitcoin users might be completely unaware of the
+existence of the bitcoin funds.
+
+If you have a lot of bitcoins, you should consider sharing access details
+with a trusted relative or lawyer. A more complex survivability scheme
+can be set up with multisignature access and estate planning through a
+lawyer specialized as a "digital asset executor."
+
+Bitcoin is a complex new technology that is still being explored by developers. Over
+time we will develop better security tools and practices that are easier
+to use by nonexperts. For now, Bitcoin users can use many of the tips
+discussed here to enjoy a secure and trouble-free Bitcoin ((("Bitcoin", "security", "best practices", startref="bitcoin-security-best-practice")))((("security", "best practices", startref="security-best-practice")))((("best practices, security", startref="best-practice-security")))experience.

code/addr.cpp

@@ -1,48 +0,0 @@
-#include <bitcoin/bitcoin.hpp>
-
-int main()
-{
-    // Private secret key string as base16
-    bc::ec_secret decoded;
-    bc::decode_base16(decoded,
-        "038109007313a5807b2eccc082c8c3fbb988a973cacf1a7df9ce725c31b14776");
-
-    bc::wallet::ec_private secret(
-        decoded, bc::wallet::ec_private::mainnet_p2kh);
-
-    // Get public key.
-    bc::wallet::ec_public public_key(secret);
-    std::cout << "Public key: " << public_key.encoded() << std::endl;
-
-    // Create Bitcoin address.
-    // Normally you can use:
-    //    bc::wallet::payment_address payaddr =
-    //        public_key.to_payment_address(
-    //            bc::wallet::ec_public::mainnet_p2kh);
-    //  const std::string address = payaddr.encoded();
-
-    // Compute hash of public key for P2PKH address.
-    bc::data_chunk public_key_data;
-    public_key.to_data(public_key_data);
-    const auto hash = bc::bitcoin_short_hash(public_key_data);
-
-    bc::data_chunk unencoded_address;
-    // Reserve 25 bytes
-    //   [ version:1  ]
-    //   [ hash:20    ]
-    //   [ checksum:4 ]
-    unencoded_address.reserve(25);
-    // Version byte, 0 is normal BTC address (P2PKH).
-    unencoded_address.push_back(0);
-    // Hash data
-    bc::extend_data(unencoded_address, hash);
-    // Checksum is computed by hashing data, and adding 4 bytes from hash.
-    bc::append_checksum(unencoded_address);
-    // Finally we must encode the result in Bitcoin's base58 encoding.
-    assert(unencoded_address.size() == 25);
-    const std::string address = bc::encode_base58(unencoded_address);
-
-    std::cout << "Address: " << address << std::endl;
-    return 0;
-}
-

code/bip-table.py

@@ -1,43 +0,0 @@
-
-# Convert mediawiki list of BIPs to asciidoc table for book appendix
-# Gnarly hack of regex with no error checking - it worked once
-
-import re
-
-regex_num = re.compile("\\|.\\[\\[bip-\\d+.mediawiki\\|(\\d+)\\]\\]")
-regex_altnum = re.compile("\\D+(\\d+)\\D+")
-
-bips = []
-
-f = open('README.mediawiki.txt', 'r')
-
-line = f.readline()
-
-while (line[0] != "|"):
-    line = f.readline()
-
-while (line[1] == '-'):
-    line_num = f.readline()
-    line_layer = f.readline()[2:-1]
-    line_title = f.readline()[2:-1]
-    line_owner = f.readline()[2:-1]
-    line_type = f.readline()[2:-1]
-    line_status = f.readline()[2:-1]
-    line = f.readline()
-    while (line[0] != "|"):
-        line = f.readline()
-
-    num = regex_num.match(line_num)
-    alt_num = regex_altnum.match(line_num)
-    if num:
-        bip_num = num.group(1)
-    elif alt_num:
-        bip_num = alt_num.group(1)
-
-    print("|[[bip-{0}]]https://github.com/bitcoin/bips/blob/master/bip-{0:04d}"
-          ".mediawiki[BIP-{0}] |{1} |{2} |{3} |{4} ".format(int(bip_num),
-                                                            line_title,
-                                                            line_owner,
-                                                            line_type,
-                                                            line_status))
-f.close()

code/ec-math.py

@@ -1,55 +0,0 @@
-import ecdsa
-import os
-
-# secp256k1, http://www.oid-info.com/get/1.3.132.0.10
-_p = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F
-_r = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141
-_b = 0x0000000000000000000000000000000000000000000000000000000000000007
-_a = 0x0000000000000000000000000000000000000000000000000000000000000000
-_Gx = 0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798
-_Gy = 0x483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
-curve_secp256k1 = ecdsa.ellipticcurve.CurveFp(_p, _a, _b)
-generator_secp256k1 = ecdsa.ellipticcurve.Point(curve_secp256k1, _Gx, _Gy, _r)
-oid_secp256k1 = (1, 3, 132, 0, 10)
-SECP256k1 = ecdsa.curves.Curve("SECP256k1", curve_secp256k1,
-                               generator_secp256k1, oid_secp256k1)
-ec_order = _r
-
-curve = curve_secp256k1
-generator = generator_secp256k1
-
-
-def random_secret():
-    # Collect 256 bits of random data from the OS's cryptographically secure
-    # random number generator
-    byte_array = (os.urandom(32)).hex()
-
-    return int(byte_array,16)
-
-def get_point_pubkey(point):
-    if (point.y() % 2) == 1:
-        key = '03' + '%064x' % point.x()
-    else:
-        key = '02' + '%064x' % point.x()
-    return key
-
-
-def get_point_pubkey_uncompressed(point):
-    key = ('04' +
-           '%064x' % point.x() +
-           '%064x' % point.y())
-    return key
-
-# Generate a new private key.
-secret = random_secret()
-print("Secret: ", secret)
-
-# Get the public key point.
-point = secret * generator
-print("Elliptic Curve point:", point)
-
-print("BTC public key:", get_point_pubkey(point))
-
-# Given the point (x, y) we can create the object using:
-point1 = ecdsa.ellipticcurve.Point(curve, point.x(), point.y(), ec_order)
-assert(point1 == point)

code/extract-from-pk-script.go

@@ -1,45 +0,0 @@
-package main
-
-import (
-	"encoding/hex"
-	"fmt"
-	"os"
-
-	"github.com/conformal/btcnet"
-	"github.com/conformal/btcscript"
-)
-
-// go run extract-from-pk-script.go
-
-// This example demonstrates extracting information from a standard public key
-// script.
-
-func main() {
-	scriptHex := "76a914128004ff2fcaf13b2b91eb654b1dc2b674f7ec6188ac"
-
-	ExtractPkScriptAddrs(scriptHex)
-	// Output:
-	// Script Class: pubkeyhash
-	// Addresses: [12gpXQVcCL2qhTNQgyLVdCFG2Qs2px98nV]
-	// Required Signatures: 1
-}
-
-func ExtractPkScriptAddrs(scriptHex string) {
-	script, err := hex.DecodeString(scriptHex)
-	handle(err)
-
-	// Extract and print details from the script.
-	scriptClass, addresses, reqSigs, err := btcscript.ExtractPkScriptAddrs(script, &btcnet.MainNetParams)
-	handle(err)
-
-	fmt.Println("Script Class:", scriptClass)
-	fmt.Println("Addresses:", addresses)
-	fmt.Println("Required Signatures:", reqSigs)
-}
-
-func handle(err error) {
-	if err != nil {
-		fmt.Println(err)
-		os.Exit(1)
-	}
-}

code/get-utxo.py

@@ -1,31 +0,0 @@
-# get unspent outputs from blockchain API
-
-import json
-import requests
-
-# example address
-address = '1Dorian4RoXcnBv9hnQ4Y2C1an6NJ4UrjX'
-
-# The API URL is https://blockchain.info/unspent?active=<address>
-# It returns a JSON object with a list "unspent_outputs", containing UTXO, like this:
-# {	"unspent_outputs":[
-#   {
-#     "tx_hash":"ebadfaa92f1fd29e2fe296eda702c48bd11ffd52313e986e99ddad9084062167",
-#     "tx_index":51919767,
-#     "tx_output_n": 1,
-#     "script":"76a9148c7e252f8d64b0b6e313985915110fcfefcf4a2d88ac",
-#     "value": 8000000,
-#     "value_hex": "7a1200",
-#     "confirmations":28691
-#   },
-# ...
-# ]}
-
-resp = requests.get('https://blockchain.info/unspent?active=%s' % address)
-utxo_set = json.loads(resp.text)["unspent_outputs"]
-
-for utxo in utxo_set:
-    print("%s:%d - %ld Satoshis" % (utxo['tx_hash'], utxo['tx_output_n'],
-                                    utxo['value']))
-    # Or try...
-    # print("{tx_hash}:{tx_output_n} - {value} Satoshis".format(**utxo))

code/hash_example.py

@@ -1,18 +0,0 @@
-# example of iterating a nonce in a hashing algorithm's input
-
-from __future__ import print_function
-import hashlib
-
-text = "I am Satoshi Nakamoto"
-
-# iterate nonce from 0 to 19
-for nonce in range(20):
-
-    # add the nonce to the end of the text
-    input_data = text + str(nonce)
-
-    # calculate the SHA-256 hash of the input (text+nonce)
-    hash_data = hashlib.sha256(input_data.encode()).hexdigest()
-
-    # show the input and hash result
-    print(input_data, '=>', hash_data)

code/key-to-address-ecc-example.py

@@ -1,46 +0,0 @@
-from __future__ import print_function
-import cryptos
-
-# Generate a random private key
-valid_private_key = False
-while not valid_private_key:
-    private_key = cryptos.random_key()
-    decoded_private_key = cryptos.decode_privkey(private_key, 'hex')
-    valid_private_key = 0 < decoded_private_key < cryptos.N
-
-print("Private Key (hex) is: ", private_key)
-print("Private Key (decimal) is: ", decoded_private_key)
-
-# Convert private key to WIF format
-wif_encoded_private_key = cryptos.encode_privkey(decoded_private_key, 'wif')
-print("Private Key (WIF) is: ", wif_encoded_private_key)
-
-# Add suffix "01" to indicate a compressed private key
-compressed_private_key = private_key + '01'
-print("Private Key Compressed (hex) is: ", compressed_private_key)
-
-# Generate a WIF format from the compressed private key (WIF-compressed)
-wif_compressed_private_key = cryptos.encode_privkey(
-    cryptos.decode_privkey(compressed_private_key, 'hex_compressed'), 'wif_compressed')
-print("Private Key (WIF-Compressed) is: ", wif_compressed_private_key)
-
-# Multiply the EC generator point G with the private key to get a public key point
-public_key = cryptos.fast_multiply(cryptos.G, decoded_private_key)
-print("Public Key (x,y) coordinates is:", public_key)
-
-# Encode as hex, prefix 04
-hex_encoded_public_key = cryptos.encode_pubkey(public_key, 'hex')
-print("Public Key (hex) is:", hex_encoded_public_key)
-
-# Compress public key, adjust prefix depending on whether y is even or odd
-(public_key_x, public_key_y) = public_key
-compressed_prefix = '02' if (public_key_y % 2) == 0 else '03'
-hex_compressed_public_key = compressed_prefix + (cryptos.encode(public_key_x, 16).zfill(64))
-print("Compressed Public Key (hex) is:", hex_compressed_public_key)
-
-# Generate Bitcoin address from public key
-print("Bitcoin Address (b58check) is:", cryptos.pubkey_to_address(public_key))
-
-# Generate compressed Bitcoin address from compressed public key
-print("Compressed Bitcoin Address (b58check) is:",
-      cryptos.pubkey_to_address(hex_compressed_public_key))

code/max_money.py

@@ -1,11 +1,12 @@
-# Original block reward for miners was 50 BTC = 50 0000 0000 Satoshis
-start_block_reward = 50 * 10**8
+# Original block reward for miners was 50 BTC
+start_block_reward = 50
 # 210000 is around every 4 years with a 10 minute block interval
 reward_interval = 210000
 
 
 def max_money():
-    current_reward = start_block_reward
+    # 50 BTC = 50 0000 0000 Satoshis
+    current_reward = 50 * 10**8
     total = 0
     while current_reward > 0:
         total += reward_interval * current_reward

code/merkle.cpp

@@ -1,62 +0,0 @@
-#include <bitcoin/bitcoin.hpp>
-
-bc::hash_digest create_merkle(bc::hash_list& merkle)
-{
-    // Stop if hash list is empty.
-    if (merkle.empty())
-        return bc::null_hash;
-    else if (merkle.size() == 1)
-        return merkle[0];
-
-    // While there is more than 1 hash in the list, keep looping...
-    while (merkle.size() > 1)
-    {
-        // If number of hashes is odd, duplicate last hash in the list.
-        if (merkle.size() % 2 != 0)
-            merkle.push_back(merkle.back());
-        // List size is now even.
-        assert(merkle.size() % 2 == 0);
-
-        // New hash list.
-        bc::hash_list new_merkle;
-        // Loop through hashes 2 at a time.
-        for (auto it = merkle.begin(); it != merkle.end(); it += 2)
-        {
-            // Join both current hashes together (concatenate).
-            bc::data_chunk concat_data(bc::hash_size * 2);
-            auto concat = bc::serializer<
-                decltype(concat_data.begin())>(concat_data.begin());
-            concat.write_hash(*it);
-            concat.write_hash(*(it + 1));
-            // Hash both of the hashes.
-            bc::hash_digest new_root = bc::bitcoin_hash(concat_data);
-            // Add this to the new list.
-            new_merkle.push_back(new_root);
-        }
-        // This is the new list.
-        merkle = new_merkle;
-
-        // DEBUG output -------------------------------------
-        std::cout << "Current merkle hash list:" << std::endl;
-        for (const auto& hash: merkle)
-            std::cout << "  " << bc::encode_base16(hash) << std::endl;
-        std::cout << std::endl;
-        // --------------------------------------------------
-    }
-    // Finally we end up with a single item.
-    return merkle[0];
-}
-
-int main()
-{
-    // Replace these hashes with ones from a block to reproduce the same merkle root.
-    bc::hash_list tx_hashes{{
-        bc::hash_literal("0000000000000000000000000000000000000000000000000000000000000000"),
-        bc::hash_literal("0000000000000000000000000000000000000000000000000000000000000011"),
-        bc::hash_literal("0000000000000000000000000000000000000000000000000000000000000022"),
-    }};
-    const bc::hash_digest merkle_root = create_merkle(tx_hashes);
-    std::cout << "Result: " << bc::encode_base16(merkle_root) << std::endl;
-    return 0;
-}
-

code/p2wpkh.js

@@ -1,8 +0,0 @@
-var btc = require('bitcore-lib')
-var oldAddress = btc.Address.fromString("1Ek9S3QNnutPV7GhtzR8Lr8yKPhxnUP8iw") // here's the old address
-var oldHash = oldAddress.hashBuffer
-var segwitP2PKH = Buffer.concat([new Buffer("0014","hex"), oldHash]) // 0x00 + 0x14 (pushdata 20 bytes) + old pubkeyhash
-var p2shHash = btc.crypto.Hash.sha256ripemd160(segwitP2PKH)
-var p2shAddress = btc.Address.fromScriptHash(p2shHash)
-var newAddress = p2shAddress.toString()
-// 36ghjA1KSAB1jDYD2RdiexEcY7r6XjmDQk

code/pay-to-addr-script.go

@@ -1,48 +0,0 @@
-package main
-
-import (
-	"fmt"
-	"os"
-
-	"github.com/conformal/btcnet"
-	"github.com/conformal/btcscript"
-	"github.com/conformal/btcutil"
-)
-
-// This example demonstrates creating a script which pays to a Bitcoin address.
-// It also prints the created script hex and uses the DisasmString function to
-// display the disassembled script.
-
-func main() {
-	addressStr := "12gpXQVcCL2qhTNQgyLVdCFG2Qs2px98nV"
-
-	PayToAddrScript(addressStr)
-	// Output:
-	// Script Hex: 76a914128004ff2fcaf13b2b91eb654b1dc2b674f7ec6188ac
-	// Script Disassembly: OP_DUP OP_HASH160 128004ff2fcaf13b2b91eb654b1dc2b674f7ec61 OP_EQUALVERIFY OP_CHECKSIG
-}
-
-func PayToAddrScript(addressStr string) {
-	// Parse the address to send the coins to into a btcutil.Address
-	// which is useful to ensure the accuracy of the address and determine
-	// the address type.  It is also required for the upcoming call to
-	// PayToAddrScript.
-	address, err := btcutil.DecodeAddress(addressStr, &btcnet.MainNetParams)
-	handle(err)
-
-	// Create a public key script that pays to the address.
-	script, err := btcscript.PayToAddrScript(address)
-	handle(err)
-	fmt.Printf("Script Hex: %x\n", script)
-
-	disasm, err := btcscript.DisasmString(script)
-	handle(err)
-	fmt.Println("Script Disassembly:", disasm)
-}
-
-func handle(err error) {
-	if err != nil {
-		fmt.Println(err)
-		os.Exit(1)
-	}
-}

code/proof-of-work-example.py

@@ -1,64 +0,0 @@
-#!/usr/bin/env python
-# example of proof-of-work algorithm
-
-import hashlib
-import time
-
-try:
-    long        # Python 2
-    xrange
-except NameError:
-    long = int  # Python 3
-    xrange = range
-
-max_nonce = 2 ** 32  # 4 billion
-
-
-def proof_of_work(header, difficulty_bits):
-    # calculate the difficulty target
-    target = 2 ** (256 - difficulty_bits)
-
-    for nonce in xrange(max_nonce):
-        hash_result = hashlib.sha256((str(header) + str(nonce)).encode()).hexdigest()
-
-        # check if this is a valid result, equal to or below the target
-        if long(hash_result, 16) <= target:
-            print("Success with nonce %d" % nonce)
-            print("Hash is %s" % hash_result)
-            return (hash_result, nonce)
-
-    print("Failed after %d (max_nonce) tries" % nonce)
-    return nonce
-
-
-if __name__ == '__main__':
-    nonce = 0
-    hash_result = ''
-
-    # difficulty from 0 to 31 bits
-    for difficulty_bits in xrange(32):
-        difficulty = 2 ** difficulty_bits
-        print("Difficulty: %ld (%d bits)" % (difficulty, difficulty_bits))
-        print("Starting search...")
-
-        # checkpoint the current time
-        start_time = time.time()
-
-        # make a new block which includes the hash from the previous block
-        # we fake a block of transactions - just a string
-        new_block = 'test block with transactions' + hash_result
-
-        # find a valid nonce for the new block
-        (hash_result, nonce) = proof_of_work(new_block, difficulty_bits)
-
-        # checkpoint how long it took to find a result
-        end_time = time.time()
-
-        elapsed_time = end_time - start_time
-        print("Elapsed Time: %.4f seconds" % elapsed_time)
-
-        if elapsed_time > 0:
-
-            # estimate the hashes per second
-            hash_power = float(long(nonce) / elapsed_time)
-            print("Hashing Power: %ld hashes per second" % hash_power)

code/pycoin_example.py

@@ -1,44 +0,0 @@
-#!/usr/bin/env python
-
-from __future__ import print_function
-
-from pycoin.key import Key
-
-from pycoin.key.validate import is_address_valid, is_wif_valid
-from pycoin.services import spendables_for_address
-from pycoin.tx.tx_utils import create_signed_tx
-
-
-def get_address(which):
-    while 1:
-        print("enter the %s address=> " % which, end='')
-        address = input()
-        is_valid = is_address_valid(address)
-        if is_valid:
-            return address
-        print("invalid address, please try again")
-
-
-src_address = get_address("source")
-spendables = spendables_for_address(src_address)
-print(spendables)
-
-while 1:
-    print("enter the WIF for %s=> " % src_address, end='')
-    wif = input()
-    is_valid = is_wif_valid(wif)
-    if is_valid:
-        break
-    print("invalid wif, please try again")
-
-key = Key.from_text(wif)
-if src_address not in (key.address(use_uncompressed=False), key.address(use_uncompressed=True)):
-    print("** WIF doesn't correspond to %s" % src_address)
-print("The secret exponent is %d" % key.secret_exponent())
-
-dst_address = get_address("destination")
-
-tx = create_signed_tx(spendables, payables=[dst_address], wifs=[wif])
-
-print("here is the signed output transaction")
-print(tx.as_hex())

code/rpc_block.py

@@ -3,9 +3,9 @@ from bitcoin.rpc import RawProxy
 p = RawProxy()
 
 # The block height where Alice's transaction was recorded
-blockheight = 277316
+blockheight = 775072
 
-# Get the block hash of block with height 277316
+# Get the block hash of the block at the given height
 blockhash = p.getblockhash(blockheight)
 
 # Retrieve the block by its hash

code/rpc_transaction.py

@@ -3,7 +3,7 @@ from bitcoin.rpc import RawProxy
 p = RawProxy()
 
 # Alice's transaction ID
-txid = "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2"
+txid = "466200308696215bbc949d5141a49a4138ecdfdfaa2a8029c1f9bcecd1f96177"
 
 # First, retrieve the raw transaction in hex
 raw_tx = p.getrawtransaction(txid)
@@ -13,4 +13,4 @@ decoded_tx = p.decoderawtransaction(raw_tx)
 
 # Retrieve each of the outputs from the transaction
 for output in decoded_tx['vout']:
-    print(output['scriptPubKey']['addresses'], output['value'])
+    print(output['scriptPubKey']['address'], output['value'])

code/satoshi-words.cpp

@@ -1,26 +0,0 @@
-/*
-  Display the genesis block message by Satoshi.
-*/
-#include <iostream>
-#include <bitcoin/bitcoin.hpp>
-
-int main()
-{
-    // Create genesis block.
-    bc::chain::block block = bc::chain::block::genesis_mainnet();
-    // Genesis block contains a single coinbase transaction.
-    assert(block.transactions().size() == 1);
-    // Get first transaction in block (coinbase).
-    const bc::chain::transaction& coinbase_tx = block.transactions()[0];
-    // Coinbase tx has a single input.
-    assert(coinbase_tx.inputs().size() == 1);
-    const bc::chain::input& coinbase_input = coinbase_tx.inputs()[0];
-    // Convert the input script to its raw format.
-    const auto prefix = false;
-    const bc::data_chunk& raw_message = coinbase_input.script().to_data(prefix);
-    // Convert this to a std::string.
-    std::string message(raw_message.begin(), raw_message.end());
-    // Display the genesis block message.
-    std::cout << message << std::endl;
-    return 0;
-}

code/select-utxo.py

@@ -1,71 +0,0 @@
-# Selects outputs from a UTXO list using a greedy algorithm.
-
-from sys import argv
-
-try:
-    long        # Python 2
-except NameError:
-    long = int  # Python 3
-
-
-class OutputInfo:
-    def __init__(self, tx_hash, tx_index, value):
-        self.tx_hash = tx_hash
-        self.tx_index = tx_index
-        self.value = value
-
-    def __repr__(self):
-        return "<%s:%s with %s Satoshis>" % (self.tx_hash, self.tx_index,
-                                             self.value)
-
-
-# Select optimal outputs for a send from unspent outputs list.
-# Returns output list and remaining change to be sent to
-# a change address.
-def select_outputs_greedy(unspent, min_value):
-    # Fail if empty.
-    if not unspent:
-        return None
-    # Partition into 2 lists.
-    lessers = [utxo for utxo in unspent if utxo.value < min_value]
-    greaters = [utxo for utxo in unspent if utxo.value >= min_value]
-    key_func = lambda utxo: utxo.value
-    if greaters:
-        # Not-empty. Find the smallest greater.
-        min_greater = min(greaters, key=key_func)
-        change = min_greater.value - min_value
-        return [min_greater], "Change: %d Satoshis" % change
-    # Not found in greaters. Try several lessers instead.
-    # Rearrange them from biggest to smallest. We want to use the least
-    # amount of inputs as possible.
-    lessers.sort(key=key_func, reverse=True)
-    result = []
-    accum = 0
-    for utxo in lessers:
-        result.append(utxo)
-        accum += utxo.value
-        if accum >= min_value:
-            change = accum - min_value
-            return result, "Change: %d Satoshis" % change
-    # No results found.
-    return None, 0
-
-
-def main():
-    unspent = [
-        OutputInfo("ebadfaa92f1fd29e2fe296eda702c48bd11ffd52313e986e99ddad9084062167", 1,  8000000),
-        OutputInfo("6596fd070679de96e405d52b51b8e1d644029108ec4cbfe451454486796a1ecf", 0,  16050000),
-        OutputInfo("b2affea89ff82557c60d635a2a3137b8f88f12ecec85082f7d0a1f82ee203ac4", 0,  10000000),
-        OutputInfo("7dbc497969c7475e45d952c4a872e213fb15d45e5cd3473c386a71a1b0c136a1", 0,  25000000),
-        OutputInfo("55ea01bd7e9afd3d3ab9790199e777d62a0709cf0725e80a7350fdb22d7b8ec6", 17,  5470541),
-        OutputInfo("12b6a7934c1df821945ee9ee3b3326d07ca7a65fd6416ea44ce8c3db0c078c64", 0,  10000000),
-        OutputInfo("7f42eda67921ee92eae5f79bd37c68c9cb859b899ce70dba68c48338857b7818", 0,  16100000),
-    ]
-    target = long(argv[1]) if len(argv) > 1 else 55000000
-    print("For transaction amount %d Satoshis (%f bitcoin) use: " %
-          (target, target / 10.0 ** 8))
-    print(select_outputs_greedy(unspent, target))
-
-
-if __name__ == "__main__":
-    main()

code/vanity-miner.cpp

@@ -1,73 +0,0 @@
-#include <random>
-#include <bitcoin/bitcoin.hpp>
-
-// The string we are searching for
-const std::string search = "1kid";
-
-// Generate a random secret key. A random 32 bytes.
-bc::ec_secret random_secret(std::default_random_engine& engine);
-// Extract the Bitcoin address from an EC secret.
-std::string bitcoin_address(const bc::ec_secret& secret);
-// Case insensitive comparison with the search string.
-bool match_found(const std::string& address);
-
-int main()
-{
-    // random_device on Linux uses "/dev/urandom"
-    // CAUTION: Depending on implementation this RNG may not be secure enough!
-    // Do not use vanity keys generated by this example in production
-    std::random_device random;
-    std::default_random_engine engine(random());
-    
-    // Loop continuously...
-    while (true)
-    {
-        // Generate a random secret.
-        bc::ec_secret secret = random_secret(engine);
-        // Get the address.
-        std::string address = bitcoin_address(secret);
-        // Does it match our search string? (1kid)
-        if (match_found(address))
-        {
-            // Success!
-            std::cout << "Found vanity address! " << address << std::endl;
-            std::cout << "Secret: " << bc::encode_base16(secret) << std::endl;
-            return 0;
-        }
-    }
-    // Should never reach here!
-    return 0;
-}
-
-bc::ec_secret random_secret(std::default_random_engine& engine)
-{
-    // Create new secret...
-    bc::ec_secret secret;
-    // Iterate through every byte setting a random value...
-    for (uint8_t& byte: secret)
-        byte = engine() & 255;
-    // Return result.
-    return secret;
-}
-
-std::string bitcoin_address(const bc::ec_secret& secret)
-{
-    // Convert secret to payment address
-    bc::wallet::ec_private private_key(secret);
-    bc::wallet::payment_address payaddr(private_key);
-    // Return encoded form.
-    return payaddr.encoded();
-}
-
-bool match_found(const std::string& address)
-{
-    auto addr_it = address.begin();
-    // Loop through the search string comparing it to the lower case
-    // character of the supplied address.
-    for (auto it = search.begin(); it != search.end(); ++it, ++addr_it)
-        if (*it != std::tolower(*addr_it))
-            return false;
-    // Reached end of search string, so address matches.
-    return true;
-}
-

code/websocket-example.go

@@ -1,86 +0,0 @@
-package main
-
-import (
-	"io/ioutil"
-	"log"
-	"path/filepath"
-	"time"
-
-	"github.com/conformal/btcrpcclient"
-	"github.com/conformal/btcutil"
-	"github.com/conformal/btcwire"
-)
-
-// This example demonstrates a connection to the Bitcoin network
-// by using websockets via btcd, use of notifications and an rpc
-// call to getblockcount.
-//
-// Install and run btcd:
-// 		$ go get github.com/conformal/btcd/...
-//		$ btcd -u rpcuser -P rpcpass
-//
-// Install btcrpcclient:
-// 		$ go get github.com/conformal/btcrpcclient
-//
-// Run this example:
-// 		$ go run websocket-example.go
-//
-func main() {
-	// Only override the handlers for notifications you care about.
-	// Also note most of these handlers will only be called if you register
-	// for notifications. See the documentation of the btcrpcclient
-	// NotificationHandlers type for more details about each handler.
-	ntfnHandlers := btcrpcclient.NotificationHandlers{
-		OnBlockConnected: func(hash *btcwire.ShaHash, height int32) {
-			log.Printf("Block connected: %v (%d)", hash, height)
-		},
-		OnBlockDisconnected: func(hash *btcwire.ShaHash, height int32) {
-			log.Printf("Block disconnected: %v (%d)", hash, height)
-		},
-	}
-
-	// Connect to local btcd RPC server using websockets.
-	btcdHomeDir := btcutil.AppDataDir("btcd", false)
-	certs, err := ioutil.ReadFile(filepath.Join(btcdHomeDir, "rpc.cert"))
-	if err != nil {
-		log.Fatal(err)
-	}
-	connCfg := &btcrpcclient.ConnConfig{
-		Host:         "localhost:8334",
-		Endpoint:     "ws",
-		User:         "rpcuser",
-		Pass:         "rpcpass",
-		Certificates: certs,
-	}
-	client, err := btcrpcclient.New(connCfg, &ntfnHandlers)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Register for block connect and disconnect notifications.
-	if err := client.NotifyBlocks(); err != nil {
-		log.Fatal(err)
-	}
-	log.Println("NotifyBlocks: Registration Complete")
-
-	// Get the current block count.
-	blockCount, err := client.GetBlockCount()
-	if err != nil {
-		log.Fatal(err)
-	}
-	log.Printf("Block count: %d", blockCount)
-
-	// For this example gracefully shutdown the client after 10 seconds.
-	// Ordinarily when to shutdown the client is highly application
-	// specific.
-	log.Println("Client shutdown in 10 seconds...")
-	time.AfterFunc(time.Second*10, func() {
-		log.Println("Client shutting down...")
-		client.Shutdown()
-		log.Println("Client shutdown complete.")
-	})
-
-	// Wait until the client either shuts down gracefully (or the user
-	// terminates the process with Ctrl+C).
-	client.WaitForShutdown()
-}

colo.html

@@ -1,5 +1,6 @@
-<section id="colophon" data-type="colophon">
-  <h1>Colophon</h1>
+<section id="colophon" data-type="colophon" xmlns="http://www.w3.org/1999/xhtml" class="pagebreak-before">
+
+  <h1 class="less_space">Colophon</h1>
 
 <p>The animal on the cover of <em>Mastering Bitcoin</em> is a leafcutter ant (<em>Atta colombica</em>). The leafcutter ant (a nongeneric name) is a tropical, fungus-growing ant endemic to South and Central America, Mexico, and southern United States. Aside from humans, leafcutter ants form the largest and most complex animal societies on the planet. They are named for the way they chew leaves, which serve as nutrition for their fungal garden.</p>
 
@@ -7,7 +8,7 @@
 
 <p>Once a colony has matured, ants are divided into castes based on size, with each caste performing various functions. There are usually four castes: minims, the smallest workers that tend to the young and fungus gardens; minors, slightly larger than minima, are the first line of defense for the colony and patrol the surrounding terrain and attack enemies; mediae, the general foragers that cut leaves and bring back leaf fragments to the nest; and majors, the largest worker ants that act as soldiers, defending the nest from intruders. Recent research has shown that majors also clear main foraging trails and carry bulky items back to the nest.</p> 
 
-<p>Many of the animals on O'Reilly covers are endangered; all of them are important to the world. To learn more about how you can help, go to <a class="orm:hideurl" href="http://animals.oreilly.com/"><em>animals.oreilly.com</em></a>.</p>
+<p>Many of the animals on O'Reilly covers are endangered; all of them are important to the world. To learn more about how you can help, go to <a class="orm:hideurl" href="https://animals.oreilly.com"><em>animals.oreilly.com</em></a>.</p>
 
-<p>The cover image is from <em>Insects Abroad</em>. The cover fonts are URW Typewriter and Guardian Sans. The text font is Adobe Minion Pro; the heading font is Adobe Myriad Condensed; and the code font is Dalton Maag's Ubuntu Mono.</p>
+<p>The cover illustration is by Karen Montgomery, based on an image from <em>Insects Abroad</em>. The cover fonts are Gilroy Semibold and Guardian Sans. The text font is Adobe Minion Pro; the heading font is Adobe Myriad Condensed; and the code font is Dalton Maag's Ubuntu Mono.</p>
 </section>

copyright.html

@@ -1,57 +1,60 @@
 <section data-type="copyright-page" xmlns="http://www.w3.org/1999/xhtml">
 <h1>Mastering Bitcoin</h1>
 
-<p class="author">by <span class="firstname">Andreas </span> <span class="othername mi">M. </span> <span class="surname">Antonopoulos</span></p>
+<p class="author">by <span class="firstname">Andreas </span> <span class="othername mi">M. </span> <span class="surname">Antonopoulos</span> and <span class="firstname">David </span> <span class="othername mi">A. </span> <span class="surname">Harding</span></p>
 
-<p class="copyright">Copyright © 2017 Andreas M. Antonopoulos, LLC. All rights reserved.</p>
+<p class="copyright">Copyright © 2024 David Harding. All rights reserved.</p>
 
 <p class="printlocation">Printed in the United States of America.</p>
 
 <p class="publisher">Published by <span class="publishername">O'Reilly Media, Inc.</span>, 1005 Gravenstein Highway North, Sebastopol, CA 95472.</p>
 
-<p>O'Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (<a href="http://oreilly.com/safari">http://oreilly.com/safari</a>). For more information, contact our corporate/institutional sales department: 800-998-9938 or <span data-type="email"><em>corporate@oreilly.com</em></span>.</p>
+<p>O'Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (<a href="https://oreilly.com">https://oreilly.com</a>). For more information, contact our corporate/institutional sales department: 800-998-9938 or <span data-type="email"><em>corporate@oreilly.com</em></span>.</p>
 
 <ul class="stafflist">
-	<li><span class="staffrole">Editor:</span> Tim McGovern</li>
-	<li><span class="staffrole">GitHub Editor:</span> Will Binns</li>
-	<li><span class="staffrole">Production Editor:</span> Nicholas Adams</li>
+	<li><span class="staffrole">Acquisitions Editor: </span>Michelle Smith</li>
+	<li><span class="staffrole">Development Editor: </span>Angela Rufino</li>
+	<li><span class="staffrole">Production Editor:</span> Clare Laylock</li>
 	<li><span class="staffrole">Copyeditor:</span> Kim Cofer</li>
-	<li><span class="staffrole">Proofreader:</span> Christina Edwards</li>
-	<li><span class="staffrole">Indexer:</span> Judy McConville</li>
+	<li><span class="staffrole">Proofreader:</span> Heather Walley</li>
+	<li><span class="staffrole">Indexer:</span> nSight, Inc.</li>
 	<li><span class="staffrole">Interior Designer:</span> David Futato</li>
 	<li><span class="staffrole">Cover Designer:</span> Randy Comer</li>
-	<li><span class="staffrole">Illustrator:</span> Rebecca Demarest</li>
+	<li><span class="staffrole">Illustrator:</span> Kate Dullea</li>
 </ul>
 <!--Add additional printedition spans below as needed.-->
 
 <ul class="printings">
+	<li><span class="printedition">December 2014:</span> First Edition</li>
 	<li><span class="printedition">June 2017:</span> Second Edition</li>
+	<li><span class="printedition">November 2023:</span> Third Edition</li>
 </ul>
 <!--Add additional revdate spans below as needed.-->
 
 <div>
-<h1 class="revisions">Revision History for the Second Edition</h1>
+<h1 class="revisions">Revision History for the Third Edition</h1>
 
+<!--ul class="releases">
+	<li><span class="revdate">2023-02-22:</span> First Release</li>
+</ul-->
 <ul class="releases">
-	<li><span class="revdate">2017-06-01:</span> First Release</li>
-	<li><span class="revdate">2017-07-21:</span> Second Release</li>
-	<li><span class="revdate">2018-03-23:</span> Third Release</li>
+	<li><span class="revdate">2023-11-03:</span> First Release</li>
 </ul>
 </div>
 
-<p class="errata">See <a href="http://oreilly.com/catalog/errata.csp?isbn=9781491954386">http://oreilly.com/catalog/errata.csp?isbn=9781491954386</a> for release details.</p>
+<p class="errata">See <a href="https://oreilly.com/catalog/errata.csp?isbn=9781098150099">https://oreilly.com/catalog/errata.csp?isbn=9781098150099</a> for release details.</p>
 
 <div class="legal">
 <p>The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. <em>Mastering Bitcoin</em>, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.</p>
 
-<p>While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. <!--PROD: Uncomment the following sentence if appropriate and add it to the 
+<p>While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. <!--PROD: Uncomment the following sentence if appropriate and add it to the 
         above para:--> <!--This book is not intended as [legal/medical/financial; use the appropriate
         reference] advice. Please consult a qualified professional if you 
         require [legal/medical/financial] advice.--></p>
 </div>
 
 <div class="copyright-bottom">
-<p class="isbn">978-1-491-95438-6</p>
+<p class="isbn">978-1-098-15009-9</p>
 
 <p class="printer">[LSI]</p>
 </div>

cover.html

@@ -1,3 +1,3 @@
 <figure data-type="cover">
 <img src="images/cover.png"/>
-</figure>
+</figure>

dedication.html

@@ -1,7 +1,15 @@
-<section data-type="dedication" xmlns="http://www.w3.org/1999/xhtml">
+<section data-type="dedication" xmlns="http://www.w3.org/1999/xhtml" >
 
 <p>Dedicated to my mum, Theresa (1946&#x2013;2017)</p>
 <p>She taught me to love books and question authority</p>
 <p>Thank you, mum</p>
+<p data-type="attribution">—Andreas</p>
+<br/><br/><br/><br/><br/>
 
+<p>For Amanda</p>
+<p>It wasn't until I met you that I</p>
+<p>actually began living in paradise</p>
+
+
+<p data-type="attribution">—Dave</p>
 </section>