From 6ac0122731c45ca80026e0611ea09b41cbad1dc0 Mon Sep 17 00:00:00 2001 From: "David A. Harding" Date: Wed, 28 Jun 2023 15:19:52 -1000 Subject: [PATCH] CH07-13: edits for arufino feedback (thanks!) --- ch08.asciidoc | 27 ++++++++++++++++++--------- ch09.asciidoc | 27 ++++++++++++++++++++------- ch10.asciidoc | 30 ++++++++++++++++-------------- ch11.asciidoc | 2 -- ch12.asciidoc | 2 -- chapters/fees.adoc | 46 ++++++++++++++++++++++++++-------------------- 6 files changed, 80 insertions(+), 54 deletions(-) diff --git a/ch08.asciidoc b/ch08.asciidoc index e3f272aa..096b5351 100644 --- a/ch08.asciidoc +++ b/ch08.asciidoc @@ -112,7 +112,8 @@ in the blockchain used by all full nodes, and miners only get paid for blocks that are widely accepted. Whichever block has a second block built on top of it first wins (unless -there's another near-tie), which is called a _block-finding race_. +there's another near-tie), which is called a _block-finding race_ and is +illustrated in <>. Block-finding races give the advantage to the largest miners, so they act in opposition to Bitcoin's essential decentralization. To prevent block-finding races and allow miners of any size to participate equally @@ -120,6 +121,7 @@ in the lottery that is Bitcoin mining, it's extremely useful to minimize the time between when one miner announces a new block and when other miners receive that block. +[[mining_race]] .A blockchain fork requiring a mining race image::images/race1.dot.png["Mining race"] @@ -141,8 +143,8 @@ found. For any transaction that isn't found in your local node's mempool, your node can send a request to the peer for a copy. Conversely, if the remote peer believes your node's mempool doesn't have -some of transactions that appear in the block, it can include a copy of -those transaction in the compact block. +some of the transactions that appear in the block, it can include a copy of +those transactions in the compact block. If the remote peer guesses correctly about what transactions your node has in its mempool, and which it does not, it will send a block nearly @@ -159,7 +161,7 @@ each block. ==== There are two modes that Bitcoin Core currently implements for sending -compact blocks: +compact blocks, illustrated in <>: Low-bandwidth mode:: When your node requests that a peer use low-bandwidth mode (the default), @@ -181,13 +183,14 @@ High-bandwidth mode:: Skipping validation before relay allows new blocks to travel across the network with minimal delays at each hop. + -The downside of high-bandwidth node is that your node is likely to +The downside of high-bandwidth mode is that your node is likely to receive redundant information from each high-bandwidth peer it chooses. As of this writing, Bitcoin Core currently only asks three peers to use high-bandwidth mode (and it tries to choose peers that have a history of quickly announcing blocks). // released into the public domain by Nicolas Dorier +[[bip152_illustration]] .BIP152 modes compared (from BIP152) image::images/bip152.png["BIP152"] @@ -199,8 +202,9 @@ relay before compact blocks were implemented. === Private Block Relay Networks -Although compact blocks go a long way towards minimizing the latency -of block propagation, it's possible to minimize latency further. Unlike +Although compact blocks go a long way towards minimizing the time it +takes for blocks to propagate across the network, +it's possible to minimize latency further. Unlike compact blocks, though, the other solutions involve tradeoffs that make them unavailable or unsuitable for the public P2P relay network. For that reason, there has been experimentation with private relay @@ -221,7 +225,7 @@ reconstructed if they are lost. FEC avoids the problem of a compact block (or some parts of it) not arriving due to problems with the underlying network connection. -Those problems frequently occur but we mostly don't notice them +Those problems frequently occur but we don't often notice them because we mostly use protocols that automatically re-request the missing data. However, requesting missing data triples the time to receive it. For example: @@ -938,7 +942,7 @@ smaller of the numbers: |=== If we count the frequency of each difference occurring, we see that the -small differences are much more likely to occur that the large +small differences are much more likely to occur than the large differences: [cols="1,1"] @@ -1234,3 +1238,8 @@ the emergent consensus of the network and therefore will vary little between nodes. Furthermore, the mempool and orphan pools only contain unconfirmed transactions, while the UTXO database only contains confirmed outputs. + +Now that we have a understanding of many of the data types and +structures used by nodes and clients to send data across the Bitcoin +network, it's time to look at the software that's responsible for +keeping the network secure and operational. diff --git a/ch09.asciidoc b/ch09.asciidoc index 0719ef7d..eed61cc9 100644 --- a/ch09.asciidoc +++ b/ch09.asciidoc @@ -1,8 +1,15 @@ [[blockchain]] == The Blockchain +The blockchain is the history of every confirmed Bitcoin transaction. +It's what allows every full node to indepdently determine what keys and +scripts control which bitcoins. In this chapter, we'll look at the +structure of the blockchain and see how it uses cryptographic +commitments and other clever tricks to make every part of it easy for +full nodes (and sometimes light clients) to validate. + ((("blockchain (the)", "overview of")))The blockchain data structure is -an ordered, back-linked list of blocks of transactions. The blockchain +an ordered, back-linked list of blocks of transactions.The blockchain can be stored as a flat file, or in a simple database. Blocks are linked "back," each referring to the previous block in the chain. ((("blocks", "block height")))The blockchain is often visualized @@ -309,6 +316,11 @@ chain, making the blockchain longer with a new height of 277,315. <> shows the chain of three blocks, linked by references in the +previousblockhash+ field. +[[chain_of_blocks]] +[role="smallerfourtyfive"] +.Blocks linked in a chain by reference to the previous block header hash +image::images/mbc2_0901.png[] + [[merkle_trees]] === Merkle Trees @@ -325,11 +337,6 @@ these trees are usually displayed upside down with the "root" at the top and the "leaves" at the bottom of a diagram, as you will see in the examples that follow. -[[chain_of_blocks]] -[role="smallerfourtyfive"] -.Blocks linked in a chain by reference to the previous block header hash -image::images/mbc2_0901.png[] - Merkle trees are used in bitcoin to summarize all the transactions in a block, producing an overall commitment to the entire set of transactions and permitting a very efficient process to verify whether a @@ -418,7 +425,7 @@ image::images/mbc2_0904.png["merkle_tree_large"] In <>, a node can prove that a transaction K is included in the block by producing a merkle path that is only four 32-byte hashes long (128 bytes total). The path consists of the four -hashes (shown with a shaded background in <>) H~L~, +hashes (shown with a shaded background H~L~, H~IJ~, H~MNOP~, and H~ABCDEFGH~. With those four hashes provided as an authentication path, any node can prove that H~K~ (with a black background at the bottom of the diagram) is included in the merkle root @@ -811,3 +818,9 @@ expected, switch to +mainnet+ to deploy it in production. As you make changes, improvements, bug fixes, etc., start the pipeline again, deploying each change first on +regtest+, then on +signet+ or +testnet+, and finally into production.((("", startref="BCTtest09"))) + +Now that we know what data the blockchain contains and how cryptographic +commitments securely tie the various parts together, we will look at the +specical commitment that both provides computational security and +ensures no block can be changed without invalidating all other blocks +built on top of it: Bitcoin's mining function. diff --git a/ch10.asciidoc b/ch10.asciidoc index 621ef7aa..4fb812ad 100644 --- a/ch10.asciidoc +++ b/ch10.asciidoc @@ -23,9 +23,9 @@ implementing the monetary supply. [TIP] ==== -((("decentralized systems", "bitcoin mining and")))The purpose of mining -is not the creation of new bitcoin. That's the incentive system. Mining -is the mechanism by which Bitcoin's _consensus security_ is _decentralized_. +((("decentralized systems", "bitcoin mining and"))) +Mining +is one of the mechanisms by which Bitcoin's _consensus security_ is _decentralized_. ==== Miners record new transactions on the global ledger. A @@ -79,7 +79,7 @@ In this chapter, we will first examine mining as a monetary supply mechanism and then look at the most important function of mining: the decentralized consensus mechanism that underpins Bitcoin's security. -To understand mining and consensus, we will follow Alice's transaction +To understand mining and consensus, we will track Alice's transaction as it is received and added to a block by Jing's mining equipment. Then we will follow the block as it is mined, added to the blockchain, and accepted by the Bitcoin network through the process of emergent @@ -122,7 +122,7 @@ mined and more may be mined in the future, resulting in a lower total issuance of the currency. ==== -In the example code in <>, we calculate the total amount of +In the code in <>, we calculate the total amount of bitcoin that will be issued. [[max_money]] @@ -591,7 +591,7 @@ as listed in <>. | 4 bytes | Nonce | A counter used for the Proof-of-Work algorithm |======= -The version field was originally a integer field and was used in three +The version field was originally an integer field and was used in three upgrades to the Bitcoin network, those defined in BIPs 34, 66, and 65. Each time, the version number was incremented. Later upgrades defined the version field as a bitfield, called _VersionBits_, allowing up to 29 @@ -717,9 +717,7 @@ Python 2.7.1 ---- ==== -<> shows the result of calculating the hash of +"I am -Satoshi Nakamoto"+: -+5d7c7ba21cbbcd75d14800b100252d5b428e5b1213d27c385bc141ca6b47989e+. This +This 256-bit number is the _hash_ or _digest_ of the phrase and depends on every part of the phrase. Adding a single letter, punctuation mark, or any other character will produce a different hash. @@ -1463,6 +1461,11 @@ solutions. Let's say, for example, that the miners building on top of star-triangle-rhombus). They immediately propagate this new block and the entire network sees it as a valid solution as shown in <>. +[[fork4]] +[role="smallereighty"] +.Visualization of a blockchain fork event: a new block extends one fork, reorganizing the network +image::images/mbc2_1005.png["Visualization of a blockchain fork event: a new block extends one fork"] + All nodes that had chosen "triangle" as the winner in the previous round will simply extend the chain one more block. The nodes that chose "upside-down triangle" as the winner, however, will now see two chains: @@ -1484,11 +1487,6 @@ star-triangle-rhombus, with "rhombus" as the last block in the chain. All miners immediately start working on candidate blocks that reference "rhombus" as their parent to extend the star-triangle-rhombus chain. -[[fork4]] -[role="smallereighty"] -.Visualization of a blockchain fork event: a new block extends one fork, reorganizing the network -image::images/mbc2_1005.png["Visualization of a blockchain fork event: a new block extends one fork"] - [[fork5]] [role="smallereighty"] .Visualization of a blockchain fork event: the network reorganizes on a new longest chain @@ -2506,3 +2504,7 @@ forces compromise. Some see this as a weakness of consensus systems. In time, you may come to see it as the system's greatest strength. + +At this point in the book, we've finished talking about the Bitcoin +system itself. What's left are software, tools, and other protocols +built on top of Bitcoin. diff --git a/ch11.asciidoc b/ch11.asciidoc index 3bc40dc4..0a736ab6 100644 --- a/ch11.asciidoc +++ b/ch11.asciidoc @@ -280,8 +280,6 @@ can be set up with multi-signature access and estate planning through a lawyer specialized as a "digital asset executor."((("", startref="Suser11")))((("", startref="UCsecurity11"))) -=== Conclusion - Bitcoin is a completely new, unprecedented, and complex technology. Over time we will develop better security tools and practices that are easier to use by nonexperts. For now, Bitcoin users can use many of the tips diff --git a/ch12.asciidoc b/ch12.asciidoc index 2e7f0b33..8303ff0a 100644 --- a/ch12.asciidoc +++ b/ch12.asciidoc @@ -1253,8 +1253,6 @@ systems include Tumblebit and Teechan. At this time, however, the Lightning Network has already been deployed and has tens of thousands of users. -=== Conclusion - We have examined just a few of the emerging applications that can be built using the Bitcoin blockchain as a trust platform. These applications expand the scope of bitcoin beyond payments and beyond diff --git a/chapters/fees.adoc b/chapters/fees.adoc index 9a384139..3d571fff 100644 --- a/chapters/fees.adoc +++ b/chapters/fees.adoc @@ -85,7 +85,7 @@ minimum transaction relay fee must be a fairly small amount or it would create an incentive for users to begin sending transactions to miners directly, which would be bad for mining decentralization. -When demand exceeds supply, then the only thing that prevents fee costs +When demand exceeds supply, the only thing that prevents fee costs from rising to infinity is that higher prices curtail demand. Each time the price increases, the number of people who want to pay that price to include a transaction in a block decreases. Conversely, when prices @@ -99,12 +99,12 @@ image::../images/fee-negative-feedback.png[Negative feedback loop] In particular, people who want to send a transaction during a period of high fees will try to find an alternative. There are many alternatives -for most reasons people need to send a transaction, such as moving to an +for most reasons why people need to send a transaction, such as moving to an offchain payment system like Lightning Network, using an alternative payment network, or taking care to construct the smallest transaction possible. But it's also possible to take advantage of the negative feedback loop to pay less fees. All you need to do is create a -transaction now which pays a fee that's equal to or higher than what +transaction which pays a fee that's equal to or higher than what miners will accept in the future--even if that price is below what they accept now. As the market of other people adapts to the higher price by lowering demand, and as miners confirm all transactions paying a higher @@ -123,7 +123,7 @@ see this scenario illustrated in <>. [[img_wait_times]] .Wait times for different fee rates -image::../images/fee-patience.png[Wait times for different feerates] +image::../images/fee-patience.png[Wait times for different fee rates] In other words, you don't need to pay peak prices, or switch to another payment method, if you're willing to wait longer for your transaction to @@ -206,28 +206,28 @@ different units: - weight/satoshi (also commonly used today) We recommend either the vByte/sat or weight/sat units for displaying -feerates. +fee rates. [WARNING] ==== -Be careful accepting input for feerates. If a user copy and pastes a -feerate printed in one enumerator into a field using a different +Be careful accepting input for fee rates. If a user copy and pastes a +fee rate printed in one enumerator into a field using a different enumerator, they could overpay fees by 1,000 times. If they instead switch the denominator, they could theoretically overpay by 100,000,000 times. Wallets should make it hard for the user to pay an excessive -feerate and may want to prompt the user to confirm any feerate that was +fee rate and may want to prompt the user to confirm any fee rate that was not generated by the wallet itself using a trusted data source. -An excessive fee, also called an _absurd fee_, is any feerate that's -significantly higher than the amount that feerate estimators currently +An excessive fee, also called an _absurd fee_, is any fee rate that's +significantly higher than the amount that fee rate estimators currently expect is necessary to get a transaction confirmed in the next block. Note that wallets should not entirely prevent users from choosing an -excessive feerate--they should only make using such a feerate hard to do +excessive fee rate--they should only make using such a fee rate hard to do by accident. There are legitimate reasons for users to overpay fees on rare occasions. ==== -=== Estimating appropriate feerates +=== Estimating appropriate fee rates We've established that you can pay a lower fee rate if you're willing to wait longer for your transaction to be confirmed, with the exception @@ -266,7 +266,7 @@ current list, see https://www.lopp.net/bitcoin-information/fee-estimates.html As mentioned, fee rate estimation can never be perfect. One common problem is that the fundamental demand might change, adjusting the equilibrium and either increasing prices (fees) to new heights or -decreasing them towards the minimum, as show in +decreasing them towards the minimum, as shown in <>. If fee rates go down, then a transaction that previously paid a normal fee rate might now be paying a high fee rate and it will be confirmed earlier than expected. There's no way to @@ -491,7 +491,7 @@ separate transaction. Using extra block space requires paying extra fees beyond the the cost of the fee bump. There are several challenges with CPFP, some of which we'll explore in -the <> section. One other problem which we +<>. One other problem which we specifically need to mention is the minimum relay fee rate problem, which is addressed by package relay. @@ -502,7 +502,7 @@ each node's mempool. Of course, computers have physical limits, whether it's the memory (RAM) or disk space--it's not possible for a full node to store an unlimited number of unconfirmed transactions. Later versions of Bitcoin Core limited the size of the mempool to hold about -one day worth of transactions, storing only the transactions or packages +one day's worth of transactions, storing only the transactions or packages with the highest fee rate. That works extremely well for most things, but it creates a dependency @@ -536,7 +536,7 @@ problem. === Transaction Pinning Although both RBF and CPFP fee bumping work in the basic cases we -described in the sections about them, there are rules related to both +described, there are rules related to both methods that are designed to prevent denial of service attacks on miners and relaying full nodes. An unfortunate side effect of those rules is that they can sometimes prevent someone from being able to use fee @@ -566,7 +566,7 @@ large child transaction. If Alice then wants to replace her original transaction, she needs to pay a fee that's larger than what both she and Bob originally paid. For example, if Alice's original transaction was about 100 vbytes and Bob's transaction was about 100,000 vbytes, and -they both used the same feerate, Alice now needs to pay more than 1,000 +they both used the same fee rate, Alice now needs to pay more than 1,000 times as much as she originally paid in order to RBF fee bump her transaction. @@ -593,7 +593,7 @@ Additionally, a transaction and all of its descendants is not + To prevent these problems, and other related problems, Bitcoin Core limits a parent transaction to having a maximum - of 25 ancestors or descendants in its mempool pool and limits the + of 25 ancestors or descendants in its mempool and limits the total size of all those transactions to 100,000 vbytes. The downside of this approach is that users are prevented from creating CPFP fee bumps if a transaction already has too many descendants (or if it and @@ -631,7 +631,7 @@ allow fee bumping with either RBF (using special sighash flags) or CPFP, but both of those protocols are vulnerable to transaction pinning. Given that the involved transactions are time sensitive, allowing a counterparty to use transaction pinning to delay confirmation of a -transaction can easily lead to an repeatable exploit that malicious +transaction can easily lead to a repeatable exploit that malicious parties could use to steal money from honest parties. LN developer Matt Corallo proposed a solution: give the rules for CPFP @@ -735,7 +735,7 @@ transaction it created. Under normal circumstances, this +nLocktime+ has no effect—the transactions could only be included in block #100,001 anyway; it's the next block. -But under a blockchain fork attack, the miners would not be able to pull +But under a reorganization attack, the miners would not be able to pull high-fee transactions from the mempool, because all those transactions would be timelocked to block #100,001. They can only remine #100,000 with whatever transactions were valid at that time, essentially gaining @@ -746,3 +746,9 @@ profitable in some cases and so can help preserve the stability of the Bitcoin network as the block subsidy declines. We recommend all wallets implement anti fee sniping when it doesn't interfere with the wallet's other uses of the nLockTime field. + +As Bitcoin continues to mature, and as the subsidy continues to decline, +fees become more and more important to Bitcoin users, both in their +day-to-day use for getting transactions confirmed quickly and in +providing an incentive for miners to continue securing Bitcoin +transactions with new proof-of-work.