arno/bitcoinbook
1
0
Fork 0
mirror of https://github.com/bitcoinbook/bitcoinbook synced 2024-11-22 16:18:11 +00:00
Code Issues Releases Wiki Activity

part 3 CE

Browse Source
This commit is contained in:
Clare Laylock 2023-10-03 09:25:59 -04:00
parent 11577d434c
commit 311eac3b34
4 changed files with 230 additions and 235 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

217
ch08_signatures.adoc
View File

@ -27,7 +27,7 @@ intact.
Each transaction input and any signatures it may contain is _completely_ Each transaction input and any signatures it may contain is _completely_
independent of any other input or signature. Multiple parties can independent of any other input or signature. Multiple parties can
collaborate to construct transactions and sign only one input each. collaborate to construct transactions and sign only one input each.
Several protocols uses this fact to create multi-party transactions for Several protocols use this fact to create multiparty transactions for
privacy. privacy.
==== ====
@ -44,7 +44,7 @@ signing key). The second part is an algorithm
that allows anyone to verify the signature, given also the message and the corresponding that allows anyone to verify the signature, given also the message and the corresponding
public key. public key.
==== Creating a digital signature ==== Creating a Digital Signature
In Bitcoin's use of digital signature algorithms, the "message" being In Bitcoin's use of digital signature algorithms, the "message" being
signed is the transaction, or more accurately a hash of a specific signed is the transaction, or more accurately a hash of a specific
@ -62,7 +62,7 @@ where:
* _F_~_sig_~ is the signing algorithm * _F_~_sig_~ is the signing algorithm
* _Sig_ is the resulting signature * _Sig_ is the resulting signature
More details on the mathematics of schnorr and ECDSA signatures can be found in <<schnorr_signatures>> You can find more details on the mathematics of schnorr and ECDSA signatures in <<schnorr_signatures>>
and <<ecdsa_signatures>>. and <<ecdsa_signatures>>.
In both schnorr and ECDSA signatures, the function _F_~_sig_~ produces a signature +Sig+ that is composed of In both schnorr and ECDSA signatures, the function _F_~_sig_~ produces a signature +Sig+ that is composed of
@ -70,7 +70,7 @@ two values. There are differences between the two values in the
different algorithms, which we'll explore later. different algorithms, which we'll explore later.
After the two values After the two values
are calculated, they are serialized into a byte-stream. For ECDSA are calculated, they are serialized into a byte stream. For ECDSA
signatures, the encoding uses an international standard encoding scheme signatures, the encoding uses an international standard encoding scheme
called the called the
_Distinguished Encoding Rules_, or _DER_. For schnorr signatures, a _Distinguished Encoding Rules_, or _DER_. For schnorr signatures, a
@ -78,10 +78,7 @@ simpler serialization format is used.
==== Verifying the Signature ==== Verifying the Signature
The signature verification The signature verification algorithm takes the message (a hash of parts of the transaction and related data), the signer's public key and the signature, and returns ++TRUE++ if the signature is valid for this message and public key.
algorithm takes the message (a hash of parts of the transaction and
related data), the signer's public key and the signature, and returns
TRUE if the signature is valid for this message and public key.
To verify the signature, one must have the signature, the serialized To verify the signature, one must have the signature, the serialized
transaction, some data about the output being spent, and the public key transaction, some data about the output being spent, and the public key
@ -127,16 +124,16 @@ There are three +SIGHASH+ flags: +ALL+, +NONE+, and +SINGLE+, as shown
in <<sighash_types_and_their>>. in <<sighash_types_and_their>>.
[[sighash_types_and_their]] [[sighash_types_and_their]]
.SIGHASH types and their meanings .++SIGHASH++ types and their meanings
[options="header"] [options="header"]
|======================= |=======================
|+SIGHASH+ flag| Value | Description |++SIGHASH++ flag| Value | Description
| +ALL+ | 0x01 | Signature applies to all inputs and outputs | ++ALL++ | ++0x01++ | Signature applies to all inputs and outputs
| +NONE+ | 0x02 | Signature applies to all inputs, none of the outputs | ++NONE++ | ++0x02++ | Signature applies to all inputs, none of the outputs
| +SINGLE+ | 0x03 | Signature applies to all inputs but only the one output with the same index number as the signed input | ++SINGLE++ | ++0x03++ | Signature applies to all inputs but only the one output with the same index number as the signed input
|======================= |=======================
In addition, there is a modifier flag +SIGHASH_ANYONECANPAY+, which can In addition, there is a modifier flag, +SIGHASH_ANYONECANPAY+, which can
be combined with each of the preceding flags. When +ANYONECANPAY+ is be combined with each of the preceding flags. When +ANYONECANPAY+ is
set, only one input is signed, leaving the rest (and their sequence set, only one input is signed, leaving the rest (and their sequence
numbers) open for modification. The +ANYONECANPAY+ has the value +0x80+ numbers) open for modification. The +ANYONECANPAY+ has the value +0x80+
@ -144,13 +141,13 @@ and is applied by bitwise OR, resulting in the combined flags as shown
in <<sighash_types_with_modifiers>>. in <<sighash_types_with_modifiers>>.
[[sighash_types_with_modifiers]] [[sighash_types_with_modifiers]]
.SIGHASH types with modifiers and their meanings .++SIGHASH++ types with modifiers and their meanings
[options="header"] [options="header"]
|======================= |=======================
|SIGHASH flag| Value | Description |++SIGHASH++ flag| Value | Description
| ALL\|ANYONECANPAY | 0x81 | Signature applies to one input and all outputs | ++ALL\++|++ANYONECANPAY++ | ++0x81++ | Signature applies to one input and all outputs
| NONE\|ANYONECANPAY | 0x82 | Signature applies to one input, none of the outputs | ++NONE\++|++ANYONECANPAY++ | ++0x82++ | Signature applies to one input, none of the outputs
| SINGLE\|ANYONECANPAY | 0x83 | Signature applies to one input and the output with the same index number | ++SINGLE\++|++ANYONECANPAY++ | ++0x83++ | Signature applies to one input and the output with the same index number
|======================= |=======================
The way +SIGHASH+ flags are applied during signing and verification is The way +SIGHASH+ flags are applied during signing and verification is
@ -195,12 +192,12 @@ By itself, this allows any miner to change
the output destination and claim the funds for themselves, but if other the output destination and claim the funds for themselves, but if other
required signatures in the transaction use +SIGHASH_ALL+ or another type required signatures in the transaction use +SIGHASH_ALL+ or another type
that commits to the output, it allows those spenders to change the that commits to the output, it allows those spenders to change the
destination without allowing any third-parties (like miners) to modify destination without allowing any third parties (like miners) to modify
the outputs. the outputs.
+NONE|ANYONECANPAY+ :: This construction can be used to build a "dust +NONE|ANYONECANPAY+ :: This construction can be used to build a "dust
collector." Users who have tiny UTXOs in their wallets can't spend these collector." Users who have tiny UTXOs in their wallets can't spend these
without the cost in fees exceeding the value of the UTXO, see without the cost in fees exceeding the value of the UTXO; see
<<uneconomical_outputs>>. With this type <<uneconomical_outputs>>. With this type
of signature, the uneconomical UTXOs can be donated for anyone to aggregate and of signature, the uneconomical UTXOs can be donated for anyone to aggregate and
spend whenever they want. spend whenever they want.
@ -212,23 +209,22 @@ new sighash flags. A signature using +SIGHASH_ANYPREVOUT+ would not
commit to an input's outpoint field, allowing it to be used to spend any commit to an input's outpoint field, allowing it to be used to spend any
previous output for a particular witness program. For example, if Alice previous output for a particular witness program. For example, if Alice
receives two outputs for the same amount to the same witness program receives two outputs for the same amount to the same witness program
(e.g. requiring a single signature from her wallet), a (e.g., requiring a single signature from her wallet), a
+SIGHASH_ANYPREVOUT+ signature for spending either one of those outputs +SIGHASH_ANYPREVOUT+ signature for spending either one of those outputs
could be copied and used to spend the other output to the same could be copied and used to spend the other output to the same
destination. destination.
A signature using +SIGHASH_ANYPREVOUTANYSCRIPT+ would not A signature using +SIGHASH_ANYPREVOUTANYSCRIPT+ would not
commit to the outpoint, the amount, the witness program, or the commit to the outpoint, the amount, the witness program, or the
specific leaf in the taproot merkle tree (script tree), allowing it to spend any previous output specific leaf in the taproot merkle tree (script tree), allowing it to spend any previous output that the signature could satisfy. For example, if Alice received two
which the signature could satisfy. For example, if Alice received two outputs for different amounts and different witness programs (e.g., one
outputs for different amounts and different witness programs (e.g. one
requiring a single signature and another requiring her signature plus some requiring a single signature and another requiring her signature plus some
other data), a +SIGHASH_ANYPREVOUTANYSCRIPT+ signature for spending other data), a +SIGHASH_ANYPREVOUTANYSCRIPT+ signature for spending
either one of those outputs could be copied and used to spend the other either one of those outputs could be copied and used to spend the other
output to the same destination (assuming the extra data for the second output to the same destination (assuming the extra data for the second
output was known). output was known).
The main expected use for the two SIGHASH_ANYPREVOUT opcodes is improved The main expected use for the two ++SIGHASH_ANYPREVOUT++ opcodes is improved
payment channels, such as those used in the Lightning Network, although payment channels, such as those used in the Lightning Network, although
several other uses have been described. several other uses have been described.
@ -236,27 +232,27 @@ several other uses have been described.
==== ====
You will not often see +SIGHASH+ flags presented as an option in a user's You will not often see +SIGHASH+ flags presented as an option in a user's
wallet application. Simple wallet applications wallet application. Simple wallet applications
sign with +SIGHASH_ALL+ flags. More sophisticated applications, such as sign with [.keep-together]#+SIGHASH_ALL+# flags. More sophisticated applications, such as
Lightning Network nodes, may use alternative +SIGHASH+ flags, but they Lightning Network nodes, may use alternative +SIGHASH+ flags, but they
use protocols that have been extensively reviewed to understand the use protocols that have been extensively reviewed to understand the
influence of the alternative flags. influence of the alternative flags.
==== ====
[[schnorr_signatures]] [[schnorr_signatures]]
=== Schnorr signatures === Schnorr Signatures
In 1989, Claus Schnorr published a paper describing the signature In 1989, Claus Schnorr published a paper describing the signature
algorithm that's become eponymous with him. The algorithm isn't algorithm that's become eponymous with him. The algorithm isn't
specific to the Elliptic Curve Cryptography (ECC) that Bitcoin and many specific to the elliptic curve cryptography (ECC) that Bitcoin and many
other applications use, although it is perhaps most strongly associated other applications use, although it is perhaps most strongly associated
with ECC today. Schnorr signatures have a number of nice properties: with ECC today. Schnorr signatures have a number of nice properties:
Provable security:: Provable security::
A mathematical proof of the security of schnorr signatures depends on A mathematical proof of the security of schnorr signatures depends on
only the difficulty of solving the Discrete Logarithm Problem (DLP), only the difficulty of solving the Discrete Logarithm Problem (DLP),
particularly for Elliptic Curves (EC) for Bitcoin, and the ability of particularly for elliptic curves (EC) for Bitcoin, and the ability of
a hash function (like the SHA256 function used in Bitcoin) to produce a hash function (like the SHA256 function used in Bitcoin) to produce
unpredictable values, called the Random Oracle Model (ROM). Other unpredictable values, called the random oracle model (ROM). Other
signature algorithms have additional dependencies or require much signature algorithms have additional dependencies or require much
larger public keys or signatures for equivalent security to larger public keys or signatures for equivalent security to
ECC-Schnorr (when the threat is defined as classical computers; other ECC-Schnorr (when the threat is defined as classical computers; other
@ -269,12 +265,12 @@ Linearity::
properties. The first property is that summing together two or more properties. The first property is that summing together two or more
variables and then running a function on that sum will produce the variables and then running a function on that sum will produce the
same value as running the function on each of the variables same value as running the function on each of the variables
independently and then summing together the results, e.g. independently and then summing together the results, e.g.,
+f(x + y + z) == f(x) + f(y) + f(z)+; this property is called +f(x + y + z) == f(x) + f(y) + f(z)+; this property is called
_additivity_. The second property is that multiplying a variable and _additivity_. The second property is that multiplying a variable and
then running a function on that product will produce the same value as then running a function on that product will produce the same value as
running the function on the variable and then multiplying it by the running the function on the variable and then multiplying it by the
same amount, e.g. +f(a * x) == a * f(x)+; this property is called same amount, e.g., +f(a * x) == a * f(x)+; this property is called
_homogeneity of degree 1_. _homogeneity of degree 1_.
+ +
In cryptographic operations, some functions may be private (such In cryptographic operations, some functions may be private (such
@ -285,7 +281,7 @@ In cryptographic operations, some functions may be private (such
specific benefits of linearity in schnorr signatures in specific benefits of linearity in schnorr signatures in
<<schnorr_multisignatures>> and <<schnorr_threshold_signatures>>. <<schnorr_multisignatures>> and <<schnorr_threshold_signatures>>.
Batch Verification:: Batch verification::
When used in a certain way (which Bitcoin does), one consequence of When used in a certain way (which Bitcoin does), one consequence of
schnorr's linearity is that it's relatively straightforward to verify schnorr's linearity is that it's relatively straightforward to verify
more than one schnorr signature at the same time in less time than it more than one schnorr signature at the same time in less time than it
@ -297,17 +293,16 @@ Batch Verification::
Later in this chapter, we'll describe the schnorr signature algorithm Later in this chapter, we'll describe the schnorr signature algorithm
exactly as it's used in Bitcoin, but we're going to start with a exactly as it's used in Bitcoin, but we're going to start with a
simplified version of it and work our way towards the actual protocol in simplified version of it and work our way toward the actual protocol in
stages. stages.
Alice starts by choosing a large random number (+x+), which we call her Alice starts by choosing a large random number (+x+), which we call her
_private key_. She also knows a public point on Bitcoin's Elliptic _private key_. She also knows a public point on Bitcoin's elliptic
Curve (EC) called the Generator (+G+) (see <<public_key_derivation>>). Alice uses EC curve called the Generator (+G+) (see <<public_key_derivation>>). Alice uses EC
multiplication to multiply +G+ by her private key +x+, in which case +x+ multiplication to multiply +G+ by her private key +x+, in which case +x+
is called a _scalar_ because it scales up +G+. The result is +xG+, is called a _scalar_ because it scales up +G+. The result is +xG+,
which we call Alice's _public key_. Alice gives her public key to Bob. which we call Alice's _public key_. Alice gives her public key to Bob.
Even though Bob also knows +G+, the Discrete Logarithm Problem (DLP) Even though Bob also knows +G+, the Discrete Logarithm Problem prevents Bob from being able to divide +xG+ by +G+ to derive Alice's
prevents Bob from being able to divide +xG+ by +G+ to derive Alice's
private key. private key.
At some later time, Bob wants Alice to identify herself by proving At some later time, Bob wants Alice to identify herself by proving
@ -330,7 +325,7 @@ process:
be used to multiply an EC point. be used to multiply an EC point.
3. Alice now has the numbers (scalars) +x+, +k+, and +e+. She combines 3. Alice now has the numbers (scalars) +x+, +k+, and +e+. She combines
them together to produce a final scalar +s+ using the formula: them together to produce a final scalar +s+ using the formula
+s = k + ex+. She gives +s+ to Bob. +s = k + ex+. She gives +s+ to Bob.
4. Bob now knows the scalars +s+ and +e+, but not +x+ or +k+. However, 4. Bob now knows the scalars +s+ and +e+, but not +x+ or +k+. However,
@ -339,11 +334,10 @@ process:
the operation Alice performed: +sG == kG + exG+. If that is equal, the operation Alice performed: +sG == kG + exG+. If that is equal,
then Bob can be sure that Alice knew +x+ when she generated +s+. then Bob can be sure that Alice knew +x+ when she generated +s+.
.Schnorr identity protocol with integers instead of points .Schnorr Identity Protocol with Integers Instead of Points
**** ****
It might be easier to understand the interactive schnorr identity It might be easier to understand the interactive schnorr identity
protocol if we create an insecure oversimplification by substituting each of the values above protocol if we create an insecure oversimplification by substituting each of the preceding values (including +G+) with simple integers instead of points on an elliptic curve.
(including +G+) with simple integers instead of points on a elliptic curve.
For example, we'll use the prime numbers starting with 3: For example, we'll use the prime numbers starting with 3:
Setup: Alice chooses +x=3+ as her private key. She multiplies it by the Setup: Alice chooses +x=3+ as her private key. She multiplies it by the
@ -364,7 +358,7 @@ generator +G=5+ to get her public key +xG=15+. She gives Bob +15+.
Of course, this is an oversimplified example. When working with simple Of course, this is an oversimplified example. When working with simple
integers, we can divide products by the generator +G+ to get the integers, we can divide products by the generator +G+ to get the
underlying scalar, which isn't secure. This is why a critical property underlying scalar, which isn't secure. This is why a critical property
of the Elliptic Curve Cryptography (ECC) used in Bitcoin is that of the elliptic curve cryptography used in Bitcoin is that
multiplication is easy but division by a point on the curve is impractical. Also, with numbers multiplication is easy but division by a point on the curve is impractical. Also, with numbers
this small, finding underlying values (or valid substitutes) through this small, finding underlying values (or valid substitutes) through
brute force is easy; the numbers used in Bitcoin are much larger. brute force is easy; the numbers used in Bitcoin are much larger.
@ -373,7 +367,8 @@ brute force is easy; the numbers used in Bitcoin are much larger.
Let's discuss some of the features of the interactive schnorr Let's discuss some of the features of the interactive schnorr
identity protocol that make it secure: identity protocol that make it secure:
- The nonce (+k+). In step 1, Alice chooses a number that Bob doesn't The nonce (+k+)::
In step 1, Alice chooses a number that Bob doesn't
know and can't guess and gives him the scaled form of that number, know and can't guess and gives him the scaled form of that number,
+kG+. At that point, Bob also already has her public key (+xG+), +kG+. At that point, Bob also already has her public key (+xG+),
which is the scaled form of +x+, her private key. That means when Bob is working on which is the scaled form of +x+, her private key. That means when Bob is working on
@ -387,7 +382,8 @@ identity protocol that make it secure:
be able to leverage that into figuring out Alice's private key. See be able to leverage that into figuring out Alice's private key. See
<<nonce_warning>> for more details. <<nonce_warning>> for more details.
- The challenge scalar (+e+). Bob waits to receive Alice's public nonce The challenge scalar (+e+)::
Bob waits to receive Alice's public nonce
and then proceeds in step 2 to give her a number (the challenge and then proceeds in step 2 to give her a number (the challenge
scalar) that Alice didn't previously know and couldn't have guessed. scalar) that Alice didn't previously know and couldn't have guessed.
It's critical that Bob only give her the challenge scalar after she It's critical that Bob only give her the challenge scalar after she
@ -395,8 +391,8 @@ identity protocol that make it secure:
who didn't know +x+ wanted to impersonate Alice, and Bob accidentally who didn't know +x+ wanted to impersonate Alice, and Bob accidentally
gave them the challenge scalar +e+ before they told him the public gave them the challenge scalar +e+ before they told him the public
nonce +kG+. This allows the impersonator to change parameters on both sides of nonce +kG+. This allows the impersonator to change parameters on both sides of
the equation that Bob will use for verification, +sG == kG + exG+, the equation that Bob will use for verification, +sG == kG + exG+;
specifically they can change both +sG+ and +kG+. Think about a specifically, they can change both +sG+ and +kG+. Think about a
simplified form of that expression: x = y + a. If you can change both simplified form of that expression: x = y + a. If you can change both
+x+ and +y+, you can cancel out +a+ using +x' = (x - a) + a+. Any +x+ and +y+, you can cancel out +a+ using +x' = (x - a) + a+. Any
value you choose for +x+ will now satisfy the equation. For the value you choose for +x+ will now satisfy the equation. For the
@ -426,7 +422,7 @@ node that wants to authenticate her.
A simple technique, known as the Fiat-Shamir transform after its A simple technique, known as the Fiat-Shamir transform after its
discoverers, can turn the schnorr interactive identity protocol discoverers, can turn the schnorr interactive identity protocol
into a non-interactive digital signature scheme. Recall the importance into a noninteractive digital signature scheme. Recall the importance
of steps 1 and 2--including that they be performed in order. Alice must of steps 1 and 2--including that they be performed in order. Alice must
commit to an unpredictable nonce; Bob must give Alice an unpredictable commit to an unpredictable nonce; Bob must give Alice an unpredictable
challenge scalar only after he has received her commitment. Recall also challenge scalar only after he has received her commitment. Recall also
@ -460,9 +456,9 @@ We've now defined a version of the schnorr signature protocol, but
there's one more thing we need to do to address a Bitcoin-specific there's one more thing we need to do to address a Bitcoin-specific
concern. In BIP32 key derivation, as described in concern. In BIP32 key derivation, as described in
<<public_child_key_derivation>>, the algorithm for unhardened derivation <<public_child_key_derivation>>, the algorithm for unhardened derivation
takes a public key and adds to it a non-secret value to produce a takes a public key and adds to it a nonsecret value to produce a
derived public key. That means it's also possible to add that derived public key. That means it's also possible to add that
non-secret value to a valid signature for one key to produce a signature nonsecret value to a valid signature for one key to produce a signature
for a related key. That related signature is valid but it wasn't for a related key. That related signature is valid but it wasn't
authorized by the person possessing the private key, which is a major authorized by the person possessing the private key, which is a major
security failure. To protect BIP32 unhardened derivation and security failure. To protect BIP32 unhardened derivation and
@ -486,13 +482,13 @@ generate a private key from a large random seed value). She uses the
parameters defined in secp256k1 (see <<elliptic_curve>>) to multiply the parameters defined in secp256k1 (see <<elliptic_curve>>) to multiply the
generator +G+ by her scalar +x+, producing +xG+ (her public key). She generator +G+ by her scalar +x+, producing +xG+ (her public key). She
gives her public key to everyone who will later authenticate her Bitcoin gives her public key to everyone who will later authenticate her Bitcoin
transactions (e.g. by having +xG+ included in a transaction output). When transactions (e.g., by having +xG+ included in a transaction output). When
she's ready to spend, she begins generating her signature: she's ready to spend, she begins generating her signature:
1. Alice chooses a large random private nonce +k+ and derives the public 1. Alice chooses a large random private nonce +k+ and derives the public
nonce +kG+. nonce +kG+.
2. She chooses her message +m+ (e.g. transaction data) and generates the 2. She chooses her message +m+ (e.g., transaction data) and generates the
challenge scalar +e = hash(kG || xG || m)+. challenge scalar +e = hash(kG || xG || m)+.
3. She produces the scalar +s = k + ex+. The two values +kG+ and +s+ 3. She produces the scalar +s = k + ex+. The two values +kG+ and +s+
@ -502,19 +498,19 @@ she's ready to spend, she begins generating her signature:
the witness structure of her spending transaction and then relaying that the witness structure of her spending transaction and then relaying that
transaction to full nodes. transaction to full nodes.
4. The verifiers (e.g. full nodes) use +s+ to derive +sG+ and then 4. The verifiers (e.g., full nodes) use +s+ to derive +sG+ and then
verify that +sG == kG + hash(kG || xG || m)*xG+. If the equation is verify that +sG == kG + hash(kG || xG || m)*xG+. If the equation is
valid, Alice proved that she knows her private key +x+ (without valid, Alice proved that she knows her private key +x+ (without
revealing it) and committed to the message +m+ (containing the revealing it) and committed to the message +m+ (containing the
transaction data). transaction data).
==== Serialization of schnorr signatures ==== Serialization of Schnorr Signatures
A schnorr signature consists of two values, +kG+ and +s+. The value A schnorr signature consists of two values, +kG+ and +s+. The value
+kG+ is a point on Bitcoin's elliptic curve (called secp256k1) and so +kG+ is a point on Bitcoin's elliptic curve (called secp256k1) and so
would normally be represented by two 32-byte coordinates, e.g. +(x,y)+. would normally be represented by two 32-byte coordinates, e.g., +(x,y)+.
However, only the x coordinate is needed, so only that value is However, only the _x_ coordinate is needed, so only that value is
included. When you see +kG+ below, note that it's only that point's x included. When you see +kG+ in schnorr signatures for Bitcoin, note that it's only that point's _x_
coordinate. coordinate.
The value +s+ is a scalar (a number meant to multiply other numbers). For The value +s+ is a scalar (a number meant to multiply other numbers). For
@ -522,8 +518,8 @@ Bitcoin's secp256k1 curve, it can never be more than 32 bytes long.
Although both +kG+ and +s+ can sometimes be values that can be Although both +kG+ and +s+ can sometimes be values that can be
represented with fewer than 32 bytes, it's improbable that they'd be represented with fewer than 32 bytes, it's improbable that they'd be
much smaller than 32 bytes, and so they're serialized as two 32 byte much smaller than 32 bytes, and so they're serialized as two 32-byte
values (i.e., values smaller than 32 bytes have leading zeroes). values (i.e., values smaller than 32 bytes have leading zeros).
They're serialized in the order of +kG+ and then +s+, producing exactly They're serialized in the order of +kG+ and then +s+, producing exactly
64 bytes. 64 bytes.
@ -541,7 +537,7 @@ the serialization used for ECDSA signatures described in
<<serialization_of_signatures_der>>. <<serialization_of_signatures_der>>.
[[schnorr_multisignatures]] [[schnorr_multisignatures]]
==== Schnorr-based scriptless multisignatures ==== Schnorr-based Scriptless Multisignatures
In the single-signature schnorr protocol described in <<schnorr_signatures>>, Alice In the single-signature schnorr protocol described in <<schnorr_signatures>>, Alice
uses a signature (+kG+, +s+) to publicly prove her knowledge of her uses a signature (+kG+, +s+) to publicly prove her knowledge of her
@ -560,9 +556,9 @@ that are believed to be secure.
==== ====
Alice and Bob need to derive the public key for +x+, which is +xG+. Alice and Bob need to derive the public key for +x+, which is +xG+.
Since it's possible to use Elliptic Curve (EC) operations to add two EC Since it's possible to use elliptic curve operations to add two EC
points together, they start by Alice deriving +yG+ and Bob deriving points together, they start by Alice deriving +yG+ and Bob deriving
+zG+. Then then add them together to create +xG = yG + zG+. The point +zG+. They then add them together to create +xG = yG + zG+. The point
+xG+ is their _aggregated public key_. To create a signature, they begin the +xG+ is their _aggregated public key_. To create a signature, they begin the
simple multisignature protocol: simple multisignature protocol:
@ -571,8 +567,8 @@ simple multisignature protocol:
public nonce +aG+ and +bG+. Together, they produce an aggregated public nonce +aG+ and +bG+. Together, they produce an aggregated
public nonce +kG = aG + bG+. public nonce +kG = aG + bG+.
2. They agree on the message to sign, +m+ (e.g. a transaction), and 2. They agree on the message to sign, +m+ (e.g., a transaction), and
each generate a copy of the challenge scalar: +e = hash(kG || xG || m)+. each generates a copy of the challenge scalar: +e = hash(kG || xG || m)+.
3. Alice produces the scalar +q = a + ey+. Bob produces the scalar 3. Alice produces the scalar +q = a + ey+. Bob produces the scalar
+r = b + ez+. They add the scalars together to produce +r = b + ez+. They add the scalars together to produce
@ -583,17 +579,17 @@ simple multisignature protocol:
Alice and Bob have proven that they know the sum of their private keys without Alice and Bob have proven that they know the sum of their private keys without
either one of them revealing their private key to the other or anyone either one of them revealing their private key to the other or anyone
else. The protocol can be extended to any number of participants, e.g. else. The protocol can be extended to any number of participants; e.g.,
a million people could prove they knew the sum of their million a million people could prove they knew the sum of their million
different keys. different keys.
The protocol above has several security problems. Most notable is that one The preceding protocol has several security problems. Most notable is that one
party might learn the public keys of the other parties before committing party might learn the public keys of the other parties before committing
to their own public key. For example, Alice generates her public key to their own public key. For example, Alice generates her public key
+yG+ honestly and shares it with Bob. Bob generates his public key +yG+ honestly and shares it with Bob. Bob generates his public key
using +zG - yG+. When their two keys are combined (+yG + zG - yG+), the using +zG - yG+. When their two keys are combined (+yG + zG - yG+), the
positive and negative +yG+ terms cancel out so the public key only represents positive and negative +yG+ terms cancel out so the public key only represents
the private key for +z+, i.e. Bob's private key. Now Bob can create a the private key for +z+, i.e., Bob's private key. Now Bob can create a
valid signature without any assistance from Alice. This is called a valid signature without any assistance from Alice. This is called a
_key cancellation attack_. _key cancellation attack_.
@ -623,14 +619,14 @@ signature parameters. The different nonces must not be related in any
way. For a multisignature, every participant must follow these rules or way. For a multisignature, every participant must follow these rules or
it could compromise the security of other participants. In addition, it could compromise the security of other participants. In addition,
cancellation and other attacks need to be prevented. Different cancellation and other attacks need to be prevented. Different
protocols that accomplish these aims make different tradeoffs, so protocols that accomplish these aims make different trade-offs, so
there's no single multisignature protocol to recommend in all cases. there's no single multisignature protocol to recommend in all cases.
Instead, we'll note three from the MuSig family of protocols: Instead, we'll note three from the MuSig family of protocols:
MuSig:: MuSig::
Also called _MuSig1_, this protocol requires three rounds of Also called _MuSig1_, this protocol requires three rounds of
communication during the signing process, making it similar to the communication during the signing process, making it similar to the
process we described above. MuSig1's greatest advantage is its process we just described. MuSig1's greatest advantage is its
simplicity. simplicity.
MuSig2:: MuSig2::
@ -645,13 +641,13 @@ MuSig-DN::
DN stands for Deterministic Nonce, which eliminates as a concern a DN stands for Deterministic Nonce, which eliminates as a concern a
problem known as the _repeated session attack_. It can't be combined problem known as the _repeated session attack_. It can't be combined
with key exchange and it's significantly more complex to implement with key exchange and it's significantly more complex to implement
than MuSig or Musig2. than MuSig or MuSig2.
For most applications, MuSig2 is the best multisignature protocol For most applications, MuSig2 is the best multisignature protocol
available at the time of writing. available at the time of writing.
[[schnorr_threshold_signatures]] [[schnorr_threshold_signatures]]
==== Schnorr-based scriptless threshold signatures ==== Schnorr-based Scriptless Threshold Signatures
Scriptless multisignature protocols only work for k-of-k signing. Scriptless multisignature protocols only work for k-of-k signing.
Everyone with a partial public key that becomes part of the aggregated Everyone with a partial public key that becomes part of the aggregated
@ -664,10 +660,10 @@ _threshold signature_.
We saw script-based threshold signatures in We saw script-based threshold signatures in
<<multisig>>. But just as <<multisig>>. But just as
scriptless multisignatures save space and increase privacy compared to scriptless multisignatures save space and increase privacy compared to
scripted multisigantures, _scriptless threshold signatures_ save space and scripted multisignatures, _scriptless threshold signatures_ save space and
increase privacy compared to _scripted threshold signatures_. To anyone increase privacy compared to _scripted threshold signatures_. To anyone
not involved in the signing, a _scriptless threshold signatures_ looks not involved in the signing, a _scriptless threshold signature_ looks
like any other signature which could've been created by a single-sig like any other signature that could've been created by a single-sig
user or through a scriptless multisignature protocol. user or through a scriptless multisignature protocol.
Various methods are known for generating scriptless threshold Various methods are known for generating scriptless threshold
@ -685,7 +681,7 @@ on some additional information to each share, called a correction code,
that allows any two of them to recover the number. This scheme is not that allows any two of them to recover the number. This scheme is not
secure because each share gives its holder partial knowledge of Alice's secure because each share gives its holder partial knowledge of Alice's
secret, making it easier for the participant to guess Alice's secret secret, making it easier for the participant to guess Alice's secret
than a non-participant who didn't have a share. than a nonparticipant who didn't have a share.
A secure secret sharing scheme prevents participants from learning A secure secret sharing scheme prevents participants from learning
anything about the secret unless they combine the minimum threshold anything about the secret unless they combine the minimum threshold
@ -697,7 +693,7 @@ after its discoverer, one of the same discoverers of the Fiat-Shamir
transform we saw in <<schnorr_signatures>>. transform we saw in <<schnorr_signatures>>.
In some cryptographic protocols, such as the scriptless threshold signature In some cryptographic protocols, such as the scriptless threshold signature
schemes we're working towards, it's critical for Bob, Carol, and Dan to schemes we're working toward, it's critical for Bob, Carol, and Dan to
know that Alice followed her side of the protocol correctly. They need to know that Alice followed her side of the protocol correctly. They need to
know that the shares she creates all derive from the same secret, that know that the shares she creates all derive from the same secret, that
she used the threshold value she claims, and that she gave each one of she used the threshold value she claims, and that she gave each one of
@ -705,7 +701,7 @@ them a different share. A protocol that can accomplish all of that,
and still be a secure secret sharing scheme, is a _verifiable secret and still be a secure secret sharing scheme, is a _verifiable secret
sharing scheme_. sharing scheme_.
To see how multisignatures and verifiable secret sharing works for To see how multisignatures and verifiable secret sharing work for
Alice, Bob, and Carol, imagine they each wish to receive funds that can Alice, Bob, and Carol, imagine they each wish to receive funds that can
be spent by any two of them. They collaborate as described in be spent by any two of them. They collaborate as described in
<<schnorr_multisignatures>> to produce a regular multisignature public <<schnorr_multisignatures>> to produce a regular multisignature public
@ -726,8 +722,8 @@ partial private key. Alice and Bob also have their private keys,
allowing them to create a scriptless multisignature with all three allowing them to create a scriptless multisignature with all three
necessary keys. necessary keys.
In other words, the scriptless threshold signature scheme described In other words, the scriptless threshold signature scheme just described
above is the same as a scriptless multisignature scheme except that is the same as a scriptless multisignature scheme except that
a threshold number of participants have the ability to reconstruct the a threshold number of participants have the ability to reconstruct the
partial private keys of any other participants who are unable or partial private keys of any other participants who are unable or
unwilling to sign. unwilling to sign.
@ -735,7 +731,8 @@ unwilling to sign.
This does point to a few things to be aware about when considering a This does point to a few things to be aware about when considering a
scriptless threshold signature protocol: scriptless threshold signature protocol:
1. No accountability: because Alice and Bob reconstruct Carol's partial No accountability::
Because Alice and Bob reconstruct Carol's partial
private key, there can be no fundamental difference between a scriptless private key, there can be no fundamental difference between a scriptless
multisignature produced by a process that involved Carol and one that multisignature produced by a process that involved Carol and one that
didn't. Even if Alice, Bob, or Carol claim that they didn't sign, didn't. Even if Alice, Bob, or Carol claim that they didn't sign,
@ -743,15 +740,16 @@ there's no guaranteed way for them to prove that they didn't
help produce the signature. If it's important to know which members of help produce the signature. If it's important to know which members of
the group signed, you will need to use a script. the group signed, you will need to use a script.
2. Manipulation attacks: imagine that Bob tells Alice that Carol is Manipulation attacks::
Imagine that Bob tells Alice that Carol is
unavailable, so they work together to reconstruct Carol's partial unavailable, so they work together to reconstruct Carol's partial
private key. Then Bob tells Carol that Alice is unavailable, so they private key. Then Bob tells Carol that Alice is unavailable, so they
work together to reconstruct Alice's partial private key. Now Bob has work together to reconstruct Alice's partial private key. Now Bob has
his own partial private key plus the keys of Alice and Carol, allowing his own partial private key plus the keys of Alice and Carol, allowing
him to spend the funds himself without their involvement. This attack can him to spend the funds himself without their involvement. This attack can
be addressed if all of the participants agree to only communicate using a be addressed if all of the participants agree to only communicate using a
scheme that allows any one of them to see all of the other's messages, scheme that allows any one of them to see all of the other's messages;
e.g. if Bob tells Alice that Carol is unavailable, Carol is able to see e.g., if Bob tells Alice that Carol is unavailable, Carol is able to see
that message before she begins working with Bob. Other solutions, that message before she begins working with Bob. Other solutions,
possibly more robust solutions, to this problem were being researched at possibly more robust solutions, to this problem were being researched at
the time of writing. the time of writing.
@ -762,7 +760,7 @@ by multiple Bitcoin contributors and we expect peer-reviewed solutions
will become available after the publication of this book. will become available after the publication of this book.
[[ecdsa_signatures]] [[ecdsa_signatures]]
=== ECDSA signatures === ECDSA Signatures
Unfortunately for the future development of Bitcoin and many other Unfortunately for the future development of Bitcoin and many other
applications, Claus Schnorr patented the algorithm he discovered and applications, Claus Schnorr patented the algorithm he discovered and
@ -770,7 +768,7 @@ prevented its use in open standards and open source software for almost
two decades. Cryptographers in the early 1990s who were blocked from two decades. Cryptographers in the early 1990s who were blocked from
using the schnorr signature scheme developed an alternative construction using the schnorr signature scheme developed an alternative construction
called the _Digital Signature Algorithm_ (DSA), with a version adapted called the _Digital Signature Algorithm_ (DSA), with a version adapted
to Elliptic Curves called ECDSA. to elliptic curves called ECDSA.
The ECDSA scheme and standardized parameters for suggested curves it could be used The ECDSA scheme and standardized parameters for suggested curves it could be used
with were widely implemented in cryptographic libraries by the time with were widely implemented in cryptographic libraries by the time
@ -790,20 +788,20 @@ More complex::
Less provable security:: Less provable security::
The interactive schnorr signature identification protocol depends only The interactive schnorr signature identification protocol depends only
on the strength of the Elliptic Curve Discrete Logarithm Problem on the strength of the elliptic curve Discrete Logarithm Problem
(ECDLP). The non-interactive authentication protocol used in Bitcoin (ECDLP). The non-interactive authentication protocol used in Bitcoin
also relies on the Random Oracle Model (ROM). However, ECDSA's extra also relies on the random oracle model (ROM). However, ECDSA's extra
complexity has prevented a complete proof of its security being complexity has prevented a complete proof of its security being
published (to the best of our knowledge). We are not experts in published (to the best of our knowledge). We are not experts in
proving cryptographic algorithms, but it seems unlikely after 30 years proving cryptographic algorithms, but it seems unlikely after 30 years
that ECDSA will be proven to only require the same two assumptions as that ECDSA will be proven to only require the same two assumptions as
schnorr. schnorr.
Non-linear:: Nonlinear::
ECDSA signatures cannot be easily combined to create scriptless ECDSA signatures cannot be easily combined to create scriptless
multisignatures or used in related advanced applications such as multisignatures or used in related advanced applications such as
multi-party signature adaptors. There are workarounds for this multiparty signature adaptors. There are workarounds for this
problem, but they involve additional extra complexity which problem, but they involve additional extra complexity that
significantly slows down operations and which, in some cases, has significantly slows down operations and which, in some cases, has
resulted in software accidentally leaking private keys. resulted in software accidentally leaking private keys.
@ -814,19 +812,18 @@ values are _R_ and _s_.
The signature The signature
algorithm first generates a private nonce (_k_) and derives from it a public algorithm first generates a private nonce (_k_) and derives from it a public
nonce (_K_). The _R_ value of the digital signature is then the x nonce (_K_). The _R_ value of the digital signature is then the _x_
coordinate of the nonce _K_. coordinate of the nonce _K_.
From there, the algorithm calculates the _s_ value of the signature, From there, the algorithm calculates the _s_ value of the signature. Like we did with schnorr signatures, operations involving
such that. Like we did with schnorr signatures, operations involving integers are modulus p:
integers are modulus p.
_s_ = __k__^-1^ (__Hash__(__m__) + __x__ * __R__) _s_ = __k__^-1^ (__Hash__(__m__) + __x__ * __R__)
where: where:
* _k_ is the private nonce * _k_ is the private nonce
* _R_ is the x coordinate of the public nonce * _R_ is the _x_ coordinate of the public nonce
* _x_ is the Alice's private key * _x_ is the Alice's private key
* _m_ is the message (transaction data) * _m_ is the message (transaction data)
@ -844,18 +841,18 @@ where:
- _m_ is the message (the transaction data that was signed) - _m_ is the message (the transaction data that was signed)
- _G_ is the elliptic curve generator point - _G_ is the elliptic curve generator point
If the x coordinate of the calculated point _K_ is equal to _R_, then If the _x_ coordinate of the calculated point _K_ is equal to _R_, then
the verifier can conclude that the signature is valid. the verifier can conclude that the signature is valid.
[TIP] [TIP]
==== ====
ECDSA is necessarily a fairly complicated piece of math; a full ECDSA is necessarily a fairly complicated piece of math; a full
explanation is beyond the scope of this book. A number of great guides explanation is beyond the scope of this book. A number of great guides
online take you through it step by step: search for "ECDSA explained". online take you through it step by step: search for "ECDSA explained."
==== ====
[[serialization_of_signatures_der]] [[serialization_of_signatures_der]]
==== Serialization of ECDSA signatures (DER) ==== Serialization of ECDSA Signatures (DER)
Let's look at Let's look at
the following DER-encoded signature: the following DER-encoded signature:
@ -864,19 +861,19 @@ the following DER-encoded signature:
3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e381301 3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e381301
---- ----
That signature is a serialized byte-stream of the +R+ and +S+ values That signature is a serialized byte stream of the +R+ and +S+ values
produced by to prove control of the private key authorized produced by the signer to prove control of the private key authorized
to spend an output. The serialization format consists of nine elements to spend an output. The serialization format consists of nine elements
as follows: as follows:
* +0x30+&#x2014;indicating the start of a DER sequence * +0x30+, indicating the start of a DER sequence
* +0x45+&#x2014;the length of the sequence (69 bytes) * +0x45+, the length of the sequence (69 bytes)
* +0x02+&#x2014;an integer value follows * +0x02+, an integer value follows
* +0x21+&#x2014;the length of the integer (33 bytes) * +0x21+, the length of the integer (33 bytes)
* +R+&#x2014;++00884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb++ * +R+, ++00884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb++
* +0x02+&#x2014;another integer follows * +0x02+, another integer follows
* +0x20+&#x2014;the length of the integer (32 bytes) * +0x20+, the length of the integer (32 bytes)
* +S+&#x2014;++4b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813++ * +S+, ++4b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813++
* A suffix (+0x01+) indicating the type of hash used (+SIGHASH_ALL+) * A suffix (+0x01+) indicating the type of hash used (+SIGHASH_ALL+)
[[nonce_warning]] [[nonce_warning]]
@ -932,7 +929,7 @@ ensure you generate a different _k_ for each transaction.
=== Segregated Witness's New Signing Algorithm === Segregated Witness's New Signing Algorithm
Signatures in bitcoin transactions are applied on a _commitment hash_, Signatures in Bitcoin transactions are applied on a _commitment hash_,
which is calculated from the transaction data, locking specific parts of which is calculated from the transaction data, locking specific parts of
the data indicating the signer's commitment to those values. For the data indicating the signer's commitment to those values. For
example, in a simple +SIGHASH_ALL+ type signature, the commitment hash example, in a simple +SIGHASH_ALL+ type signature, the commitment hash

82
ch09_fees.adoc
View File

@ -38,7 +38,7 @@ transaction pinning, and are largely uninteresting to the following
discussion about fees--so we're going to ignore them in this chapter. discussion about fees--so we're going to ignore them in this chapter.
==== ====
The criteria that almost all miners use to select which transactions to The criterion that almost all miners use to select which transactions to
include in their blocks is to maximize their revenue. Bitcoin was include in their blocks is to maximize their revenue. Bitcoin was
specifically designed to accommodate this by providing a mechanism that specifically designed to accommodate this by providing a mechanism that
allows a transaction to give money to the miner who includes that allows a transaction to give money to the miner who includes that
@ -54,7 +54,7 @@ In this chapter, we'll explore various aspects of those
bids--transaction fees--and how they influence the creation and bids--transaction fees--and how they influence the creation and
management of Bitcoin transactions. management of Bitcoin transactions.
=== Who pays the transaction fee? === Who Pays the Transaction Fee?
Most payment systems involve some sort of fee for transacting, but Most payment systems involve some sort of fee for transacting, but
often this fee is hidden from typical buyers. For example, a merchant often this fee is hidden from typical buyers. For example, a merchant
@ -85,16 +85,16 @@ normally allowing only spenders to choose fees can sometimes be a
problem; we'll look at a solution to that problem in <<cpfp>>. However, problem; we'll look at a solution to that problem in <<cpfp>>. However,
in many common payment flows, the parties with the highest desire to see a in many common payment flows, the parties with the highest desire to see a
transaction confirm quickly--that is, the parties who'd be the most transaction confirm quickly--that is, the parties who'd be the most
willing to pay higher fees--are the the spenders. willing to pay higher fees--are the spenders.
For those reasons, both technical and practical, it is customary in For those reasons, both technical and practical, it is customary in
Bitcoin for spenders to pay transaction fees. There are exceptions, Bitcoin for spenders to pay transaction fees. There are exceptions,
such as for merchants which accept unconfirmed transactions and in such as for merchants that accept unconfirmed transactions and in
protocols that don't immediately broadcast transactions after they are protocols that don't immediately broadcast transactions after they are
signed (preventing the spender from being able to choose an appropriate signed (preventing the spender from being able to choose an appropriate
fee for the current market). We'll explore those exceptions later. fee for the current market). We'll explore those exceptions later.
=== Fees and fee rates === Fees and Fee Rates
Each transaction only pays a single fee--it doesn't matter how large the Each transaction only pays a single fee--it doesn't matter how large the
transaction is. However, the larger transactions become, the fewer of transaction is. However, the larger transactions become, the fewer of
@ -140,7 +140,7 @@ by accident. There are legitimate reasons for users to overpay fees on
rare occasions. rare occasions.
==== ====
=== Estimating appropriate fee rates === Estimating Appropriate Fee Rates
We've established that you can pay a lower fee rate if you're willing to We've established that you can pay a lower fee rate if you're willing to
wait longer for your transaction to be confirmed, with the exception wait longer for your transaction to be confirmed, with the exception
@ -163,7 +163,7 @@ similar number of blocks to confirm. Bitcoin Core includes a fee rate
estimator that uses these principles, which can be called using the estimator that uses these principles, which can be called using the
`estimatesmartfee` RPC with a parameter specifying how many blocks `estimatesmartfee` RPC with a parameter specifying how many blocks
you're willing to wait before the transaction is highly likely to you're willing to wait before the transaction is highly likely to
confirm (for example, 144 blocks is about 1 day). confirm (for example, 144 blocks is about 1 day):
---- ----
$ bitcoin-cli -named estimatesmartfee conf_target=144 $ bitcoin-cli -named estimatesmartfee conf_target=144
@ -174,12 +174,12 @@ $ bitcoin-cli -named estimatesmartfee conf_target=144
---- ----
Many web-based services also provide fee estimation as an API. For a Many web-based services also provide fee estimation as an API. For a
current list, see https://www.lopp.net/bitcoin-information/fee-estimates.html current list, see https://www.lopp.net/bitcoin-information/fee-estimates.html.
As mentioned, fee rate estimation can never be perfect. One common As mentioned, fee rate estimation can never be perfect. One common
problem is that the fundamental demand might change, adjusting the problem is that the fundamental demand might change, adjusting the
equilibrium and either increasing prices (fees) to new heights or equilibrium and either increasing prices (fees) to new heights or
decreasing them towards the minimum. decreasing them toward the minimum.
If fee rates go down, then a transaction If fee rates go down, then a transaction
that previously paid a normal fee rate might now be paying a high fee that previously paid a normal fee rate might now be paying a high fee
rate and it will be confirmed earlier than expected. There's no way to rate and it will be confirmed earlier than expected. There's no way to
@ -187,18 +187,18 @@ lower the fee rate on a transaction you've already sent, so you're stuck
paying a higher fee rate. But, when fee rates go up, there's a need for paying a higher fee rate. But, when fee rates go up, there's a need for
methods to be able to increase the fee rates on those transactions, methods to be able to increase the fee rates on those transactions,
which is called _fee bumping_. There are two commonly used types of fee which is called _fee bumping_. There are two commonly used types of fee
bumping in Bitcoin, Replace-By-Fee (RBF) and Child Pays For Parent bumping in Bitcoin, replace by fee (RBF) and child pays for parent
(CPFP). (CPFP).
[[rbf]] [[rbf]]
=== Replace-By-Fee (RBF) Fee Bumping === Replace By Fee (RBF) Fee Bumping
To increase the fee of a transaction using RBF fee bumping, you create To increase the fee of a transaction using RBF fee bumping, you create
a conflicting version of the transaction which pays a higher fee. Two a conflicting version of the transaction that pays a higher fee. Two
or more transactions are considered to be _conflicting transactions_ if or more transactions are considered to be _conflicting transactions_ if
only one of them can be included in a valid blockchain, forcing a miner only one of them can be included in a valid blockchain, forcing a miner
to chose only one of them. Conflicts occur when two or more transactions to choose only one of them. Conflicts occur when two or more transactions
each try to spend one of the same UTXOs, i.e. they each include an input each try to spend one of the same UTXOs, i.e., they each include an input
that has the same outpoint (reference to the output of a previous that has the same outpoint (reference to the output of a previous
transaction). transaction).
@ -226,7 +226,7 @@ Full RBF::
version. As of this writing, this can be optionally enabled in version. As of this writing, this can be optionally enabled in
Bitcoin Core (but it is disabled by default). Bitcoin Core (but it is disabled by default).
.Why are there two variants of RBF? .Why Are There Two Variants of RBF?
**** ****
The reason for the two different versions of RBF is that full RBF has The reason for the two different versions of RBF is that full RBF has
been controversial. Early versions of Bitcoin allowed transaction been controversial. Early versions of Bitcoin allowed transaction
@ -260,7 +260,7 @@ based on the first version, but the second version gets confirmed, then
the merchant will not receive payment for its costs. the merchant will not receive payment for its costs.
Some merchants, and people supporting them, requested that transaction Some merchants, and people supporting them, requested that transaction
replacement not be re-enabled in Bitcoin Core. Other people pointed out replacement not be reenabled in Bitcoin Core. Other people pointed out
that transaction replacement provides benefits, including the ability to that transaction replacement provides benefits, including the ability to
fee bump transactions that initially paid too low of a fee rate. fee bump transactions that initially paid too low of a fee rate.
@ -289,7 +289,7 @@ whether or not that has happened yet.
As a user, if you plan to use RBF fee bumping, you will first need to As a user, if you plan to use RBF fee bumping, you will first need to
choose a wallet that supports it, such as one of the wallets listed as choose a wallet that supports it, such as one of the wallets listed as
having "Sending support" on having "Sending support" on
https://bitcoinops.org/en/compatibility/#replace-by-fee-rbf https://bitcoinops.org/en/compatibility/#replace-by-fee-rbf.
As a developer, if you plan to implement RBF fee bumping, you will first As a developer, if you plan to implement RBF fee bumping, you will first
need to decided whether to perform opt-in RBF or full RBF. At the time need to decided whether to perform opt-in RBF or full RBF. At the time
@ -306,8 +306,8 @@ same for both approaches.
When you need to fee bump a transaction, you will simply create a new When you need to fee bump a transaction, you will simply create a new
transaction that spends at least one of the same UTXOs as the original transaction that spends at least one of the same UTXOs as the original
transaction you want to replace. You will likely want to keep the transaction you want to replace. You will likely want to keep the
same outputs in the transaction which pay the receiver (or receivers). same outputs in the transaction that pay the receiver (or receivers).
You may pay the increased fee by reducing the the value of your change You may pay the increased fee by reducing the value of your change
output or by adding additional inputs to the transaction. Developers output or by adding additional inputs to the transaction. Developers
should provide users with a fee-bumping interface that does all of this should provide users with a fee-bumping interface that does all of this
work for them and simply asks them (or suggests to them) how much the work for them and simply asks them (or suggests to them) how much the
@ -324,11 +324,11 @@ overpay the receivers. For example:
- Transaction version 0 includes input _A_. - Transaction version 0 includes input _A_.
- Transaction version 1 includes inputs _A_ and _B_ (e.g., you had to add - Transaction version 1 includes inputs _A_ and _B_ (e.g., you had to add
input _B_ to pay the extra fees) input _B_ to pay the extra fees)
- Transaction version 2 includes inputs B and C (e.g., you had to add input - Transaction version 2 includes inputs _B_ and _C_ (e.g., you had to add input
_C_ to pay the extra fees but _C_ was large enough that you no longer _C_ to pay the extra fees but _C_ was large enough that you no longer
need input _A_). need input _A_).
In the above scenario, any miner who saved version 0 of the transaction In this scenario, any miner who saved version 0 of the transaction
will be able to confirm both it and version 2 of the transaction. If will be able to confirm both it and version 2 of the transaction. If
both versions pay the same receivers, they'll be paid twice (and the both versions pay the same receivers, they'll be paid twice (and the
miner will receive transaction fees from two separate transactions). miner will receive transaction fees from two separate transactions).
@ -347,27 +347,26 @@ have created if they had paid the increased fee rate in the first place.
The fundamental disadvantage of RBF fee bumping is that it can normally The fundamental disadvantage of RBF fee bumping is that it can normally
only be performed by the creator of the transaction--the person or only be performed by the creator of the transaction--the person or
people who were required to provide signatures or other authentication people who were required to provide signatures or other authentication
data for the transaction. An exception to this is transactions which data for the transaction. An exception to this is transactions that
were designed to allow additional inputs to be added by using sighash were designed to allow additional inputs to be added by using sighash
flags, see <<sighash_types>>, but that presents its own challenges. In flags (see <<sighash_types>>), but that presents its own challenges. In
general, if you're the receiver of an unconfirmed transaction and you general, if you're the receiver of an unconfirmed transaction and you
want to make it confirm faster (or at all), you can't use an RBF fee want to make it confirm faster (or at all), you can't use an RBF fee
bump; you need some other method. bump; you need some other method.
There are additional problems with RBF that we'll explore in the There are additional problems with RBF that we'll explore in <<transaction_pinning>>.
subsequent <<transaction_pinning>> section.
[[cpfp]] [[cpfp]]
=== Child Pays For Parent (CPFP) Fee Bumping === Child Pays for Parent (CPFP) Fee Bumping
Anyone who receives the output of an unconfirmed transaction can Anyone who receives the output of an unconfirmed transaction can
incentivize miners to confirm that transaction by spending that output. incentivize miners to confirm that transaction by spending that output.
The transaction you want to get confirmed is called the _parent The transaction you want to get confirmed is called the _parent
transaction_. A transaction which spends an output of the parent transaction_. A transaction that spends an output of the parent
transaction is called a _child transaction_. transaction is called a _child transaction_.
As we learned in <<outpoints>>, every input in a confirmed transaction As we learned in <<outpoints>>, every input in a confirmed transaction
must reference the unspent output of a transaction which appears earlier must reference the unspent output of a transaction that appears earlier
in the blockchain (whether earlier in the same block or in a previous in the blockchain (whether earlier in the same block or in a previous
block). That means a miner who wants to confirm a child transaction block). That means a miner who wants to confirm a child transaction
must also ensure that its parent transaction is confirmed. If the must also ensure that its parent transaction is confirmed. If the
@ -384,7 +383,7 @@ about by fee rate and include the highest-revenue ones in the block
they're attempting to mine, up to the maximum size (weight) allowed to they're attempting to mine, up to the maximum size (weight) allowed to
be included in a block. To find even more packages that might be be included in a block. To find even more packages that might be
profitable to mine, the miner can evaluate packages across multiple profitable to mine, the miner can evaluate packages across multiple
generations (e.g. an unconfirmed parent transaction being combined with generations (e.g., an unconfirmed parent transaction being combined with
both its child and grandchild). This is called _ancestor fee rate both its child and grandchild). This is called _ancestor fee rate
mining_. mining_.
@ -404,14 +403,14 @@ The primary disadvantage of CPFP compared to RBF is that CPFP typically
uses more block space. In RBF, a fee bump transaction is often the same uses more block space. In RBF, a fee bump transaction is often the same
size as the transaction it replaces. In CPFP, a fee bump adds a whole size as the transaction it replaces. In CPFP, a fee bump adds a whole
separate transaction. Using extra block space requires paying extra separate transaction. Using extra block space requires paying extra
fees beyond the the cost of the fee bump. fees beyond the cost of the fee bump.
There are several challenges with CPFP, some of which we'll explore in There are several challenges with CPFP, some of which we'll explore in
<<transaction_pinning>>. One other problem which we <<transaction_pinning>>. One other problem that we
specifically need to mention is the minimum relay fee rate problem, specifically need to mention is the minimum relay fee rate problem,
which is addressed by package relay. which is addressed by package relay.
==== Package Relay === Package Relay
Early versions of Bitcoin Core didn't place any limits on the number of Early versions of Bitcoin Core didn't place any limits on the number of
unconfirmed transactions they stored for later relay and mining in their unconfirmed transactions they stored for later relay and mining in their
@ -454,25 +453,25 @@ problem.
Although both RBF and CPFP fee bumping work in the basic cases we Although both RBF and CPFP fee bumping work in the basic cases we
described, there are rules related to both described, there are rules related to both
methods that are designed to prevent denial of service attacks on miners methods that are designed to prevent denial-of-service attacks on miners
and relaying full nodes. An unfortunate side effect of those rules and relaying full nodes. An unfortunate side effect of those rules
is that they can sometimes prevent someone from being able to use fee is that they can sometimes prevent someone from being able to use fee
bumping. Making it impossible or difficult to fee bump a transaction is bumping. Making it impossible or difficult to fee bump a transaction is
called _transaction pinning_. called _transaction pinning_.
One of the major denial of service concerns revolve's around the effect of One of the major denial of service concerns revolves around the effect of
transaction relationships. Whenever the output of a transaction is transaction relationships. Whenever the output of a transaction is
spent, that transaction's identifier (txid) is referenced by the child spent, that transaction's identifier (txid) is referenced by the child
transaction. However, when a transaction is replaced, the replacement transaction. However, when a transaction is replaced, the replacement
has a different txid. If that replacement transaction gets confirmed, has a different txid. If that replacement transaction gets confirmed,
none of its descendants can be included in the same blockchain. It's none of its descendants can be included in the same blockchain. It's
possible to recreate and re-sign the descendant transactions, but that's possible to re-create and re-sign the descendant transactions, but that's
not guaranteed to happen. This has related but divergent implications not guaranteed to happen. This has related but divergent implications
for RBF and CPFP: for RBF and CPFP:
- In the context of RBF, when Bitcoin Core accepts a replacement - In the context of RBF, when Bitcoin Core accepts a replacement
transaction, it keeps things simple by forgetting about the original transaction, it keeps things simple by forgetting about the original
transaction and all descendant transactions which depended on that transaction and all descendant transactions that depended on that
original. To ensure that it's more profitable for miners to accept original. To ensure that it's more profitable for miners to accept
replacements, Bitcoin Core only accepts a replacement transaction if it replacements, Bitcoin Core only accepts a replacement transaction if it
pays more fees than all the transactions that will be forgotten. pays more fees than all the transactions that will be forgotten.
@ -496,7 +495,7 @@ transaction.
Similarly, if a transaction with 25 descendants is removed from a Similarly, if a transaction with 25 descendants is removed from a
node's mempool (such as for being included in a block), and each of node's mempool (such as for being included in a block), and each of
those descendants has 25 other ancestors, another 625 packages need to those descendants has 25 other ancestors, another 625 packages need to
be updated. Each time we double our parameter (e.g. from 25 to 50), be updated. Each time we double our parameter (e.g., from 25 to 50),
we quadruple the amount of work our node needs to perform. we quadruple the amount of work our node needs to perform.
+ +
Additionally, a transaction and all of its descendants is not Additionally, a transaction and all of its descendants is not
@ -526,14 +525,13 @@ Protocol developers have been working on mitigating problems with
transaction pinning for several years. One partial solution is transaction pinning for several years. One partial solution is
described in <<cpfp_carve_out>>. Several other solutions have been described in <<cpfp_carve_out>>. Several other solutions have been
proposed, and at least one solution is being actively developed as of proposed, and at least one solution is being actively developed as of
this writing (ephemeral anchors, see this writing—https://bitcoinops.org/en/topics/ephemeral-anchors/[ephemeral anchors].
https://bitcoinops.org/en/topics/ephemeral-anchors/).
[[cpfp_carve_out]] [[cpfp_carve_out]]
=== CPFP Carve Out and Anchor Outputs === CPFP Carve Out and Anchor Outputs
In 2018, developers working on Lightning Network (LN) had a problem. In 2018, developers working on Lightning Network (LN) had a problem.
Their protocol uses transactions which require signatures from two Their protocol uses transactions that require signatures from two
different parties. Neither party wants to trust the other, so they sign different parties. Neither party wants to trust the other, so they sign
transactions at a point in the protocol when trust isn't needed, transactions at a point in the protocol when trust isn't needed,
allowing either of them to broadcast one of those transactions at a allowing either of them to broadcast one of those transactions at a
@ -620,7 +618,7 @@ you intended.
[[fee_sniping]] [[fee_sniping]]
=== Timelock Defense Against Fee Sniping === Timelock Defense Against Fee Sniping
Fee-sniping is a theoretical Fee sniping is a theoretical
attack scenario, where miners attempting to rewrite past blocks "snipe" attack scenario, where miners attempting to rewrite past blocks "snipe"
higher-fee transactions from future blocks to maximize their higher-fee transactions from future blocks to maximize their
profitability. profitability.
@ -666,4 +664,4 @@ As Bitcoin continues to mature, and as the subsidy continues to decline,
fees become more and more important to Bitcoin users, both in their fees become more and more important to Bitcoin users, both in their
day-to-day use for getting transactions confirmed quickly and in day-to-day use for getting transactions confirmed quickly and in
providing an incentive for miners to continue securing Bitcoin providing an incentive for miners to continue securing Bitcoin
transactions with new proof-of-work. transactions with new proof of work.

120
ch10_network.adoc
View File

@ -3,7 +3,7 @@
Bitcoin is structured as a peer-to-peer network architecture on Bitcoin is structured as a peer-to-peer network architecture on
top of the internet. The term peer-to-peer, or P2P, means that the top of the internet. The term peer-to-peer, or P2P, means that the
full nodes which participate in the network are peers to each other, that full nodes that participate in the network are peers to each other, that
they can all perform the same functions, and that there are no "special" nodes. they can all perform the same functions, and that there are no "special" nodes.
The network nodes The network nodes
interconnect in a mesh network with a "flat" topology. There is no interconnect in a mesh network with a "flat" topology. There is no
@ -52,7 +52,7 @@ only a subset of the blockchain and partly verify transactions using a method
called _simplified payment verification_, or SPV. These clients are known as lightweight clients. called _simplified payment verification_, or SPV. These clients are known as lightweight clients.
Miners compete to create new blocks by Miners compete to create new blocks by
running specialized hardware to solve the Proof-of-Work algorithm. Some running specialized hardware to solve the proof-of-work algorithm. Some
miners operate full nodes, validating every block on the miners operate full nodes, validating every block on the
blockchain, while others are clients participating in pool blockchain, while others are clients participating in pool
mining and depending on a pool server to provide them with work. mining and depending on a pool server to provide them with work.
@ -119,7 +119,7 @@ some of those transactions are confirmed in a new block, the node
doesn't need to receive a second copy of those transactions. doesn't need to receive a second copy of those transactions.
Instead of receiving redundant unconfirmed transactions, compact blocks Instead of receiving redundant unconfirmed transactions, compact blocks
allows a peer to instead send a short 6-byte identifier for each transaction. allow a peer to instead send a short 6-byte identifier for each transaction.
When your node receives a compact block with one or more identifiers, it When your node receives a compact block with one or more identifiers, it
checks its mempool for those transactions and uses them if they are checks its mempool for those transactions and uses them if they are
found. For any transaction that isn't found in your local node's found. For any transaction that isn't found in your local node's
@ -133,7 +133,7 @@ always sends a block's coinbase transaction.
If the remote peer guesses correctly about what transactions your node If the remote peer guesses correctly about what transactions your node
has in its mempool, and which it does not, it will send a block nearly has in its mempool, and which it does not, it will send a block nearly
as efficiently as is theoretically possible (for a typical block, it'll as efficiently as is theoretically possible (for a typical block, it'll
be between 97% to 99% efficient). be between 97% and 99% efficient).
[TIP] [TIP]
==== ====
@ -175,7 +175,7 @@ quickly announcing blocks).
// released into the public domain by Nicolas Dorier // released into the public domain by Nicolas Dorier
[[bip152_illustration]] [[bip152_illustration]]
.BIP152 modes compared (from BIP152). They grey bar indicates the time it takes the node to validate the block .BIP152 modes compared (from BIP152). The gray bar indicates the time it takes the node to validate the block.
image::images/mbc3_1002.png["BIP152"] image::images/mbc3_1002.png["BIP152"]
The names of the two methods (which are taken from BIP152) can be a bit The names of the two methods (which are taken from BIP152) can be a bit
@ -186,15 +186,15 @@ relay before compact blocks were implemented.
=== Private Block Relay Networks === Private Block Relay Networks
Although compact blocks go a long way towards minimizing the time it Although compact blocks go a long way toward minimizing the time it
takes for blocks to propagate across the network, takes for blocks to propagate across the network,
it's possible to minimize latency further. Unlike it's possible to minimize latency further. Unlike
compact blocks, though, the other solutions involve tradeoffs that compact blocks, though, the other solutions involve trade-offs that
make them unavailable or unsuitable for the public P2P relay network. make them unavailable or unsuitable for the public P2P relay network.
For that reason, there has been experimentation with private relay For that reason, there has been experimentation with private relay
networks for blocks. networks for blocks.
One simple technique is to pre-select a route between endpoints. For One simple technique is to preselect a route between endpoints. For
example, a relay network with servers running in datacenters near major example, a relay network with servers running in datacenters near major
trans-oceanic fiber optic lines might be able to forward new blocks trans-oceanic fiber optic lines might be able to forward new blocks
faster than waiting for the block to arrive at the node run by some home faster than waiting for the block to arrive at the node run by some home
@ -214,18 +214,18 @@ because we mostly use protocols that automatically re-request the
missing data. However, requesting missing data triples the time to missing data. However, requesting missing data triples the time to
receive it. For example: receive it. For example:
1. Alice sends some data to Bob 1. Alice sends some data to Bob.
2. Bob doesn't receive the data (or it is damaged). Bob re-requests 2. Bob doesn't receive the data (or it is damaged). Bob re-requests
the data from Alice the data from Alice.
3. Alice sends the data again 3. Alice sends the data again.
A third technique is to assume all nodes receiving the data have A third technique is to assume all nodes receiving the data have
almost all of the same transactions in their mempool, so they can all almost all of the same transactions in their mempool, so they can all
accept the same compact block. That not only saves us time computing accept the same compact block. That not only saves us time computing
a compact block at each hop but it means that all each hop can simply a compact block at each hop, but it means that all each hop can simply
relay the FEC packets to the next hop even before validating them. relay the FEC packets to the next hop even before validating them.
The tradeoff for each of the above methods is that they work well with The trade-off for each of the preceding methods is that they work well with
centralization but not in a decentralized network where individual nodes centralization but not in a decentralized network where individual nodes
can't trust other nodes. Servers in datacenters cost money and can can't trust other nodes. Servers in datacenters cost money and can
often be accessed by operators of the datacenter, making them less often be accessed by operators of the datacenter, making them less
@ -270,7 +270,7 @@ information, including:
+nTime+:: The current time +nTime+:: The current time
+addrYou+:: The IP address of the remote node as seen from this node +addrYou+:: The IP address of the remote node as seen from this node
+addrMe+:: The IP address of the local node, as discovered by the local node +addrMe+:: The IP address of the local node, as discovered by the local node
+subver+:: A sub-version showing the type of software running on this node (e.g., pass:[<span class="keep-together"><code>/Satoshi:0.9.2.1/</code></span>]) +subver+:: A subversion showing the type of software running on this node (e.g., pass:[<span class="keep-together"><code>/Satoshi:0.9.2.1/</code></span>])
+BestHeight+:: The block height of this node's blockchain +BestHeight+:: The block height of this node's blockchain
+fRelay+:: A field added by BIP37 for requesting not to receive unconfirmed transactions +fRelay+:: A field added by BIP37 for requesting not to receive unconfirmed transactions
@ -435,8 +435,8 @@ decisions. However, the most common
implementation is Bitcoin Core. implementation is Bitcoin Core.
More than 95% of full nodes on the Bitcoin network run More than 95% of full nodes on the Bitcoin network run
various versions of Bitcoin Core. It is identified as "Satoshi" in the various versions of Bitcoin Core. It is identified as "Satoshi" in the
sub-version string sent in the +version+ message and shown by the subversion string sent in the +version+ message and shown by the
command +getpeerinfo+ as we saw earlier; for example, +/Satoshi:24.0.1/+. command +getpeerinfo+ as we saw earlier; for example, [.keep-together]#+/Satoshi:24.0.1/+#.
=== Exchanging "Inventory" === Exchanging "Inventory"
@ -458,7 +458,7 @@ a +getheaders+ message that contains the hash of the top
block on their local blockchain. One of the peers will be able to block on their local blockchain. One of the peers will be able to
identify the received hash as belonging to a block that is not at the identify the received hash as belonging to a block that is not at the
top, but rather belongs to an older block, thus deducing that its own top, but rather belongs to an older block, thus deducing that its own
local blockchain is longer than its peer's. local blockchain is longer than the remote node's blockchain.
The peer that has the longer blockchain has more blocks than the other The peer that has the longer blockchain has more blocks than the other
node and can identify which headers the other node needs in order to node and can identify which headers the other node needs in order to
@ -474,7 +474,7 @@ connections to peers that are significantly slower than the average in
order to find newer (and possibly faster) peers. order to find newer (and possibly faster) peers.
Let's assume, for example, that a node only has the genesis block. It Let's assume, for example, that a node only has the genesis block. It
will then receive an +headers+ message from its peers containing the headers will then receive a +headers+ message from its peers containing the headers
of the next 2,000 blocks in the chain. It will start requesting blocks of the next 2,000 blocks in the chain. It will start requesting blocks
from all of its connected peers, keeping a queue of up to 1,024 blocks. from all of its connected peers, keeping a queue of up to 1,024 blocks.
Blocks need to be validated in order, so if the oldest block in the Blocks need to be validated in order, so if the oldest block in the
@ -542,7 +542,7 @@ on top of it is proof, by proxy, that the transaction actually exists.
A lightweight client cannot normally be persuaded that a transaction exists in a block A lightweight client cannot normally be persuaded that a transaction exists in a block
when the transaction does not in fact exist. The lightweight client establishes when the transaction does not in fact exist. The lightweight client establishes
the existence of a transaction in a block by requesting a merkle path the existence of a transaction in a block by requesting a merkle path
proof and by validating the Proof-of-Work in the chain of blocks. proof and by validating the proof of work in the chain of blocks.
However, a transaction's existence can be "hidden" from a lightweight client. A However, a transaction's existence can be "hidden" from a lightweight client. A
lightweight client can definitely verify that a transaction exists but cannot lightweight client can definitely verify that a transaction exists but cannot
verify that a transaction, such as a double-spend of the same UTXO, verify that a transaction, such as a double-spend of the same UTXO,
@ -565,7 +565,7 @@ node.
==== ====
A full node verifies a transaction by checking the entire chain of A full node verifies a transaction by checking the entire chain of
thousands of blocks below it in order to guarantee that the UTXO exists thousands of blocks below it in order to guarantee that the UTXO exists
and is not spent, whereas an lightweight client only proves that a transaction and is not spent, whereas a lightweight client only proves that a transaction
exists and checks that the block containing that transaction is exists and checks that the block containing that transaction is
buried by a handful of blocks above it. buried by a handful of blocks above it.
==== ====
@ -607,19 +607,19 @@ transactions matching a specific pattern, without revealing exactly
which addresses, keys, or transactions they are searching for. which addresses, keys, or transactions they are searching for.
In our previous analogy, a tourist without a map is asking for In our previous analogy, a tourist without a map is asking for
directions to a specific address, "23 Church St." If she asks strangers directions to a specific address, "23 Church St." If they ask a stranger
for directions to this street, she inadvertently reveals her for directions to this street, they inadvertently reveal their
destination. A bloom filter is like asking, "Are there any streets in destination. A bloom filter is like asking, "Are there any streets in
this neighborhood whose name ends in R-C-H?" A question like that this neighborhood whose name ends in R-C-H?" A question like that
reveals slightly less about the desired destination than asking for "23 reveals slightly less about the desired destination than asking for "23
Church St." Using this technique, a tourist could specify the desired Church St." Using this technique, a tourist could specify the desired
address in more detail such as "ending in U-R-C-H" or less detail as address in more detail such as "ending in U-R-C-H" or less detail such as
"ending in H." By varying the precision of the search, the tourist "ending in H." By varying the precision of the search, the tourist
reveals more or less information, at the expense of getting more or less reveals more or less information, at the expense of getting more or less
specific results. If she asks a less specific pattern, she gets a lot specific results. If they ask for a less specific pattern, they get a lot
more possible addresses and better privacy, but many of the results are more possible addresses and better privacy, but many of the results are
irrelevant. If she asks for a very specific pattern, she gets fewer irrelevant. If they ask for a very specific pattern, they get fewer
results but loses privacy. results but lose privacy.
Bloom filters serve this function by allowing a lightweight client to specify a Bloom filters serve this function by allowing a lightweight client to specify a
search pattern for transactions that can be tuned toward precision or search pattern for transactions that can be tuned toward precision or
@ -782,14 +782,14 @@ that they didn't offer very much privacy. A full node receiving a bloom
filter from a peer could apply that filter to the entire blockchain to filter from a peer could apply that filter to the entire blockchain to
find all of the client's transactions (plus false positives). It could find all of the client's transactions (plus false positives). It could
then look for patterns and relationships between the transactions. then look for patterns and relationships between the transactions.
Randomly-selected false positive transactions would be unlikely to have Randomly selected false positive transactions would be unlikely to have
a parent-child relationship from output to input, but transactions from a parent-child relationship from output to input, but transactions from
the user's wallet would be very likely to have that relationship. If the user's wallet would be very likely to have that relationship. If
all of the related transactions have certain characteristics, such as all of the related transactions have certain characteristics, such as
at least one P2PKH output, then transactions without that characteristic at least one P2PKH output, then transactions without that characteristic
can be assumed not to belong to the wallet. can be assumed not to belong to the wallet.
It was also discovered that specially-constructed filters could force It was also discovered that specially constructed filters could force
the full nodes that processed them to perform a large amount of work, the full nodes that processed them to perform a large amount of work,
which could lead to denial-of-service attacks. which could lead to denial-of-service attacks.
@ -814,7 +814,7 @@ the entire block.
[NOTE] [NOTE]
==== ====
Despite the similarities in names, BIP152 _compact blocks_ and Despite the similarities in names, BIP152 _compact blocks_ and
BIP157/158 _compact block filters_ are unrelated BIP157/158 _compact block filters_ are unrelated.
==== ====
This allows nodes to create a single filter for every block, which they This allows nodes to create a single filter for every block, which they
@ -863,8 +863,8 @@ numerical order:
Then, Alice sends the first number. For the remaining numbers, she Then, Alice sends the first number. For the remaining numbers, she
sends the difference between that number and the preceding number. For sends the difference between that number and the preceding number. For
example, for the second number, she sends 97 (476 - 379); for the third example, for the second number, she sends 97 (476 – 379); for the third
number, she sends 177 (653 - 476); and so on: number, she sends 177 (653 – 476); and so on:
---- ----
379 379
@ -885,8 +885,8 @@ more numbers we select, the smaller the average (mean) size of the
differences. That means the amount of data we need to transfer doesn't differences. That means the amount of data we need to transfer doesn't
increase as fast as the length of our list increases (up to a point). increase as fast as the length of our list increases (up to a point).
Even more usefully, the length of the randomly-selected numbers in a Even more usefully, the length of the randomly selected numbers in a
list of differences is naturally biased towards smaller lengths. list of differences is naturally biased toward smaller lengths.
Consider selecting two random numbers from 1 to 6; this is the same Consider selecting two random numbers from 1 to 6; this is the same
as rolling two dice. There are 36 distinct combinations of two dice: as rolling two dice. There are 36 distinct combinations of two dice:
@ -939,17 +939,17 @@ Golomb coding provides that facility. Rice coding is a subset of Golomb
coding that's more convenient to use in some situations, including the coding that's more convenient to use in some situations, including the
application of Bitcoin block filters. application of Bitcoin block filters.
==== What data to include in a block filter ==== What Data to Include in a Block Filter
Our primary goal is to allow wallets to learn whether a block contains a Our primary goal is to allow wallets to learn whether a block contains a
transaction affecting that wallet. For a wallet to be effective, it transaction affecting that wallet. For a wallet to be effective, it
needs to learn two types of information: needs to learn two types of information:
1. When it has received money. Specifically, when a transaction . When it has received money. Specifically, when a transaction
output contains a script that the wallet controls (such as by output contains a script that the wallet controls (such as by
controlling the authorized private key). controlling the authorized private key).
2. When it has spent money. Specifically, when a transaction input . When it has spent money. Specifically, when a transaction input
references a previous transaction output that the wallet controlled. references a previous transaction output that the wallet controlled.
A secondary goal during the design of compact block filters was to allow A secondary goal during the design of compact block filters was to allow
@ -963,9 +963,9 @@ accurate filter.
For both the primary and secondary goals to be met, a filter would need For both the primary and secondary goals to be met, a filter would need
to reference two types of information: to reference two types of information:
1. The script for every output in every transaction in a block. * The script for every output in every transaction in a block.
2. The outpoint for every input in every transaction in a block. * The outpoint for every input in every transaction in a block.
An early design for compact block filters included both of those pieces An early design for compact block filters included both of those pieces
of information, but it was realized that there was a more efficient way of information, but it was realized that there was a more efficient way
@ -973,15 +973,15 @@ to accomplish the primary goal if we sacrificed the secondary goal. In
the new design, a block filter would still references two types of the new design, a block filter would still references two types of
information, but they'd be more closely related: information, but they'd be more closely related:
1. As before, the script for every output in every transaction in a * As before, the script for every output in every transaction in a
block. block.
2. In a change, it would also reference the script of the output * In a change, it would also reference the script of the output
referenced by the outpoint for every input in every transaction in a referenced by the outpoint for every input in every transaction in a
block. In other words, the output script being spent. block. In other words, the output script being spent.
This had several advantages. First, it meant that wallets didn't need This had several advantages. First, it meant that wallets didn't need
to track outpoints; they could instead just scan for the the to track outpoints; they could instead just scan for the
output scripts to which they expected to receive money. Second, any time a output scripts to which they expected to receive money. Second, any time a
later transaction in a block spends the output of an earlier later transaction in a block spends the output of an earlier
transaction in the same block, they'll both reference the same transaction in the same block, they'll both reference the same
@ -999,12 +999,12 @@ to verify a block filter was built correctly. However, there is an
alternative that can help a client detect if a peer is lying to it: alternative that can help a client detect if a peer is lying to it:
obtaining the same filter from multiple peers. obtaining the same filter from multiple peers.
==== Downloading block filters from multiple peers ==== Downloading Block Filters from Multiple Peers
A peer can provide a wallet with an inaccurate filter. There's two ways A peer can provide a wallet with an inaccurate filter. There are two ways
to create an inaccurate filter. The peer can create a filter that to create an inaccurate filter. The peer can create a filter that
references transactions that don't actually appear in the associated references transactions that don't actually appear in the associated
block (a false positive). Alternatively, the peer can crate a filter block (a false positive). Alternatively, the peer can create a filter
that doesn't reference transactions that do actually appear in the that doesn't reference transactions that do actually appear in the
associated block (a false negative). associated block (a false negative).
@ -1021,7 +1021,7 @@ block, the client can download all of them. It can then also download
the associated block. If the block contains any transaction related to the associated block. If the block contains any transaction related to
the wallet that is not part of one of the filters, then the wallet can the wallet that is not part of one of the filters, then the wallet can
be sure that whichever peer created that filter was be sure that whichever peer created that filter was
inaccurate--Golomb-Rice Coded Sets (GCSes) will always include a inaccurate--Golomb-Rice Coded Sets will always include a
potential match. potential match.
Alternatively, if the block doesn't contain a transaction that the Alternatively, if the block doesn't contain a transaction that the
@ -1035,7 +1035,7 @@ designed to use, the wallet can stop using that peer. In most cases,
the only consequence of the inaccurate filter is that the wallet uses the only consequence of the inaccurate filter is that the wallet uses
more bandwidth than expected. more bandwidth than expected.
==== Reducing bandwidth with lossy encoding ==== Reducing Bandwidth with Lossy Encoding
The data about the transactions in a block that we want to communicate The data about the transactions in a block that we want to communicate
is an output script. Output scripts vary in length and follow patterns, is an output script. Output scripts vary in length and follow patterns,
@ -1051,7 +1051,7 @@ faster and more configurable non-cryptographic hash functions, such as
the SipHash function we'll use for compact block filters. the SipHash function we'll use for compact block filters.
The details of the algorithm used are described in BIP158, but the gist The details of the algorithm used are described in BIP158, but the gist
is that each output script is reduced to a 64 bit commitment using is that each output script is reduced to a 64-bit commitment using
SipHash and some arithmetic operations. You can think of this as SipHash and some arithmetic operations. You can think of this as
taking a set of large numbers and truncating them to shorter numbers, a taking a set of large numbers and truncating them to shorter numbers, a
process that loses data (so it's called _lossy encoding_). By losing process that loses data (so it's called _lossy encoding_). By losing
@ -1059,9 +1059,9 @@ some information, we don't need to store as much information later,
which saves space. In this case, we go from a typical output script which saves space. In this case, we go from a typical output script
that's 160 bits or longer down to just 64 bits. that's 160 bits or longer down to just 64 bits.
==== Using compact block filters ==== Using Compact Block Filters
The 64 bit value for every commitment to an output script in a block are The 64-bit values for every commitment to an output script in a block are
sorted, duplicate entries are removed, and the GCS is constructed by sorted, duplicate entries are removed, and the GCS is constructed by
finding the differences (deltas) between each entry. That compact block finding the differences (deltas) between each entry. That compact block
filter is then distributed by peers to their clients (such as wallets). filter is then distributed by peers to their clients (such as wallets).
@ -1074,23 +1074,23 @@ the filter.
Recall our example of the lossiness of compact block filters being Recall our example of the lossiness of compact block filters being
similar to truncating a number. Imagine a client is looking for a block similar to truncating a number. Imagine a client is looking for a block
that contains the number 123456 and that an an accurate (but lossy) that contains the number 123456 and that an accurate (but lossy)
compact block filter contains the number 1234. When a client sees that compact block filter contains the number 1234. When a client sees that
1234, it will download the associated block. 1234, it will download the associated block.
There's a 100% guarantee that an accurate filter containing 1234 will There's a 100% guarantee that an accurate filter containing 1234 will
allow a client to learn about a block containing 123456, called a _true allow a client to learn about a block containing 123456, called a _true
positive_. However, there's also chance that the block might contain positive_. However, there's also chance that the block might contain
123400, 123401, or almost a hundred other entries that are not when the 123400, 123401, or almost a hundred other entries that are not what the
client is looking for (in this example), called a _false positive_. client is looking for (in this example), called a _false positive_.
A 100% true positive match rate is great. It means that a wallet can A 100% true positive match rate is great. It means that a wallet can
depend on compact block filters to find every transaction affecting that depend on compact block filters to find every transaction affecting that
wallet. A non-zero false positive rate means that the wallet will end wallet. A nonzero false positive rate means that the wallet will end
up downloading some blocks that don't contain transactions interesting up downloading some blocks that don't contain transactions interesting
to the wallet. The main consequence of this is that the client will use to the wallet. The main consequence of this is that the client will use
extra bandwidth, which is not a huge problem. The actual extra bandwidth, which is not a huge problem. The actual
false-positive rate for BIP158 compact block filters is very low, so false positive rate for BIP158 compact block filters is very low, so
it's not a major problem. A false positive rate can also help improve a it's not a major problem. A false positive rate can also help improve a
client's privacy, as it does with bloom filters, although anyone wanting client's privacy, as it does with bloom filters, although anyone wanting
the best possible privacy should still use their own full node. the best possible privacy should still use their own full node.
@ -1129,7 +1129,7 @@ at the time of writing.
As a way to increase the privacy and security of the Bitcoin P2P As a way to increase the privacy and security of the Bitcoin P2P
network, there is a solution that provides encryption of the network, there is a solution that provides encryption of the
communications: _Tor Transport_. communications: _Tor transport_.
==== Tor Transport ==== Tor Transport
@ -1155,7 +1155,7 @@ debugging for the Tor service like this:
$ bitcoind --daemon --debug=tor $ bitcoind --daemon --debug=tor
---- ----
You should see "tor: ADD_ONION successful" in the logs, indicating that You should see ++tor: ADD_ONION successful++ in the logs, indicating that
Bitcoin Core has added a hidden service to the Tor network. Bitcoin Core has added a hidden service to the Tor network.
You can find more instructions on running Bitcoin Core as a Tor hidden You can find more instructions on running Bitcoin Core as a Tor hidden
@ -1163,7 +1163,7 @@ service in the Bitcoin Core documentation (_docs/tor.md_) and various
online tutorials. online tutorials.
[[mempool]] [[mempool]]
=== Mempools and orphan pools === Mempools and Orphan Pools
Almost every node on the Bitcoin Almost every node on the Bitcoin
network maintains a temporary list of unconfirmed transactions called network maintains a temporary list of unconfirmed transactions called
@ -1190,10 +1190,10 @@ of the newly added transaction, which is no longer an orphan, the
process is repeated recursively looking for any further descendants, process is repeated recursively looking for any further descendants,
until no more descendants are found. Through this process, the arrival until no more descendants are found. Through this process, the arrival
of a parent transaction triggers a cascade reconstruction of an entire of a parent transaction triggers a cascade reconstruction of an entire
chain of interdependent transactions by re-uniting the orphans with chain of interdependent transactions by reuniting the orphans with
their parents all the way down the chain. their parents all the way down the chain.
Some implementations of the Bitcoin also maintain an UTXO Some implementations of Bitcoin also maintain a UTXO
database, which is the set of all unspent outputs on the database, which is the set of all unspent outputs on the
blockchain. This represents a different set of data from the mempool. Unlike the blockchain. This represents a different set of data from the mempool. Unlike the
mempool and orphan pools, the UTXO database mempool and orphan pools, the UTXO database
@ -1204,11 +1204,11 @@ table on persistent storage.
Whereas the mempool and orphan pools represent a single node's local Whereas the mempool and orphan pools represent a single node's local
perspective and might vary significantly from node to node depending perspective and might vary significantly from node to node depending
upon when the node was started or restarted, the UTXO database represents on when the node was started or restarted, the UTXO database represents
the emergent consensus of the network and therefore will not usually the emergent consensus of the network and therefore will not usually
vary between nodes. vary between nodes.
Now that we have a understanding of many of the data types and Now that we have an understanding of many of the data types and
structures used by nodes and clients to send data across the Bitcoin structures used by nodes and clients to send data across the Bitcoin
network, it's time to look at the software that's responsible for network, it's time to look at the software that's responsible for
keeping the network secure and operational. keeping the network secure and operational.

46
ch11_blockchain.adoc
View File

@ -84,7 +84,7 @@ header. <<block_structure1>> describes how Bitcoin Core stores the structure of
|Size| Field | Description |Size| Field | Description
| 4 bytes | Block Size | The size of the block, in bytes, following this field | 4 bytes | Block Size | The size of the block, in bytes, following this field
| 80 bytes | Block Header | Several fields form the block header | 80 bytes | Block Header | Several fields form the block header
| 1-3 bytes (compactSize) | Transaction Counter | How many transactions follow | 1–3 bytes (compactSize) | Transaction Counter | How many transactions follow
| Variable | Transactions | The transactions recorded in this block | Variable | Transactions | The transactions recorded in this block
|======= |=======
@ -103,8 +103,8 @@ block metadata as shown in <<block_header_structure_ch09>>.
| 32 bytes | Previous Block Hash | A hash of the previous (parent) block in the chain | 32 bytes | Previous Block Hash | A hash of the previous (parent) block in the chain
| 32 bytes | Merkle Root | The root hash of the merkle tree of this block's transactions | 32 bytes | Merkle Root | The root hash of the merkle tree of this block's transactions
| 4 bytes | Timestamp | The approximate creation time of this block (Unix epoch time) | 4 bytes | Timestamp | The approximate creation time of this block (Unix epoch time)
| 4 bytes | Target | A compact encoding of the Proof-of-Work target for this block | 4 bytes | Target | A compact encoding of the proof-of-work target for this block
| 4 bytes | Nonce | Arbitrary data used for the Proof-of-Work algorithm | 4 bytes | Nonce | Arbitrary data used for the proof-of-work algorithm
|======= |=======
The nonce, target, and timestamp are used in the mining The nonce, target, and timestamp are used in the mining
@ -185,7 +185,7 @@ even the single transaction within. Thus, every node has the starting
point for the blockchain, a secure "root" from which to build a trusted point for the blockchain, a secure "root" from which to build a trusted
blockchain. blockchain.
See the statically encoded genesis block inside the Bitcoin Core client, See the statically encoded genesis block inside the Bitcoin Core client
in https://github.com/bitcoin/bitcoin/blob/3955c3940eff83518c186facfec6f50545b5aab5/src/chainparams.cpp#L123[_chainparams.cpp_]. in https://github.com/bitcoin/bitcoin/blob/3955c3940eff83518c186facfec6f50545b5aab5/src/chainparams.cpp#L123[_chainparams.cpp_].
The following identifier hash belongs to the genesis block: The following identifier hash belongs to the genesis block:
@ -200,7 +200,7 @@ of this block, with a URL containing that hash:
https://blockstream.info/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f https://blockstream.info/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
Using the Bitcoin Core reference client on the command line: Alternatively, you can get the block using Bitcoin Core on the command line:
---- ----
$ bitcoin-cli getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f $ bitcoin-cli getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
@ -311,7 +311,7 @@ these trees are usually displayed upside down with the "root" at the top
and the "leaves" at the bottom of a diagram, as you will see in the and the "leaves" at the bottom of a diagram, as you will see in the
examples that follow. examples that follow.
Merkle trees are used in bitcoin to summarize all the transactions in a Merkle trees are used in Bitcoin to summarize all the transactions in a
block, producing an overall commitment to the entire set of block, producing an overall commitment to the entire set of
transactions and permitting a very efficient process to verify whether a transactions and permitting a very efficient process to verify whether a
transaction is included in a block. A merkle tree is constructed by transaction is included in a block. A merkle tree is constructed by
@ -361,7 +361,7 @@ of the nodes.
image::images/mbc3_1102.png["merkle_tree"] image::images/mbc3_1102.png["merkle_tree"]
Because the merkle tree is a binary tree, it needs Because the merkle tree is a binary tree, it needs
an even number of leaf nodes. If there is an odd number of transactions an even number of leaf nodes. If there are an odd number of transactions
to summarize, the last transaction hash will be duplicated to create an to summarize, the last transaction hash will be duplicated to create an
even number of leaf nodes, also known as a _balanced tree_. This is even number of leaf nodes, also known as a _balanced tree_. This is
shown in <<merkle_tree_odd>>, where transaction C is duplicated. shown in <<merkle_tree_odd>>, where transaction C is duplicated.
@ -372,10 +372,10 @@ the last hash is duplicated.
.Duplicating one data element achieves an even number of data elements .Duplicating one data element achieves an even number of data elements
image::images/mbc3_1103.png["merkle_tree_odd"] image::images/mbc3_1103.png["merkle_tree_odd"]
.A design flaw in Bitcoin's merkle tree .A Design Flaw in Bitcoin's Merkle Tree
**** ****
An extended comment in Bitcoin Core's source code describes a An extended comment in Bitcoin Core's source code, reproduced here with slight revisions, describes a
significant problems in the design of Bitcoin's duplication of odd significant problem in the design of Bitcoin's duplication of odd
elements in its merkle tree: elements in its merkle tree:
[quote,Bitcoin Core src/consensus/merkle.cpp] [quote,Bitcoin Core src/consensus/merkle.cpp]
@ -388,14 +388,14 @@ duplicate txids, resulting in a vulnerability (CVE-2012-2459).
The reason is that if the number of hashes in the list at a given level The reason is that if the number of hashes in the list at a given level
is odd, the last one is duplicated before computing the next level (which is odd, the last one is duplicated before computing the next level (which
is unusual in Merkle trees). This results in certain sequences of is unusual in Merkle trees). This results in certain sequences of
transactions leading to the same merkle root. For example, these two transactions leading to the same merkle root. For example, the two
trees: trees in <<cve_tree>>:
[[cve_tree]] [[cve_tree]]
.Two Bitcoin-style merkle tree with the same root but a different number of leaves .Two Bitcoin-style merkle tree with the same root but a different number of leaves
image::images/mbc3_1104.png["Two Bitcoin-style merkle tree with the same root but a different number of leaves"] image::images/mbc3_1104.png["Two Bitcoin-style merkle trees with the same root but a different number of leaves"]
For transaction lists [1,2,3,4,5,6] and [1,2,3,4,5,6,5,6] (where 5 and The transaction lists [1,2,3,4,5,6] and [1,2,3,4,5,6,5,6] (where 5 and
6 are repeated) result in the same root hash A (because the hash of both 6 are repeated) result in the same root hash A (because the hash of both
of (F) and (F,F) is C). of (F) and (F,F) is C).
@ -433,7 +433,7 @@ especially important as the number of transactions increases, because
the base-2 logarithm of the number of transactions increases much more the base-2 logarithm of the number of transactions increases much more
slowly. This allows Bitcoin nodes to efficiently produce paths of 10 or slowly. This allows Bitcoin nodes to efficiently produce paths of 10 or
12 hashes (320–384 bytes), which can provide proof of a single 12 hashes (320–384 bytes), which can provide proof of a single
transaction out of more than a thousand transactions in a multi-megabyte transaction out of more than a thousand transactions in a multimegabyte
block. block.
[[merkle_tree_large]] [[merkle_tree_large]]
@ -457,7 +457,7 @@ image::images/mbc3_1106.png["merkle_tree_path"]
The efficiency of merkle trees becomes obvious as the scale increases. The efficiency of merkle trees becomes obvious as the scale increases.
The largest possible block can hold almost 16,000 transactions in 4,000,000 The largest possible block can hold almost 16,000 transactions in 4,000,000
bytes, but proving any particular one of those sixteen thousand transactions bytes, but proving any particular one of those 16,000 transactions
is a part of that block only requires a copy of the transaction, a copy is a part of that block only requires a copy of the transaction, a copy
of the 80-byte block header, and 448 bytes for the merkle proof. That of the 80-byte block header, and 448 bytes for the merkle proof. That
makes the largest possible proof almost 10,000 times smaller than the makes the largest possible proof almost 10,000 times smaller than the
@ -576,14 +576,14 @@ as +btcd+ (written in Go) and +bcoin+ (written in JavaScript), to
experiment and learn in other programming languages and frameworks. experiment and learn in other programming languages and frameworks.
Testnet3 supports all the features of mainnet, including Testnet3 supports all the features of mainnet, including
Segregated Witness v0 and v1 (see <<segwit>> and <<taproot>>). Therefore, testnet3 can also be segregated witness v0 and v1 (see <<segwit>> and <<taproot>>). Therefore, testnet3 can also be
used to test Segregated Witness features. used to test segregated witness features.
===== Problems With Testnet ===== Problems with testnet
Testnet doesn't just use the same data structures as Bitcoin, it also Testnet doesn't just use the same data structures as Bitcoin, it also
uses almost exactly the same Proof-of-Work (PoW) security mechanism as uses almost exactly the same proof-of-work (PoW) security mechanism as
Bitcoin. The notable differences for testnet are that it's minimum Bitcoin. The notable differences for testnet are that its minimum
difficulty is half that of Bitcoin and that it's allowed to include a difficulty is half that of Bitcoin and that it's allowed to include a
block at the minimum difficulty if that block's timestamp is more than block at the minimum difficulty if that block's timestamp is more than
20 minutes after the previous block. 20 minutes after the previous block.
@ -630,7 +630,7 @@ BIP325 signets are designed to make it very easy to create your own. If
you disagree with how someone else is running their signet, you can you disagree with how someone else is running their signet, you can
start your own signet and connect your software to it. start your own signet and connect your software to it.
===== The Default Signet and Custom Signets ===== The default signet and custom signets
Bitcoin Core supports a default signet, which we believe to be the most Bitcoin Core supports a default signet, which we believe to be the most
widely used signet at the time of writing. It is currently operated by widely used signet at the time of writing. It is currently operated by
@ -757,7 +757,7 @@ $ bitcoin-cli -regtest getbalance
=== Using Test Blockchains for Development === Using Test Blockchains for Development
Bitcoin's various Bitcoin's various
blockchains (+regtest+, +signet+, +testnet3+, +mainnet+) offer a range blockchains (regtest, signet, testnet3, mainnet) offer a range
of testing environments for bitcoin development. Use the test of testing environments for bitcoin development. Use the test
blockchains whether you are developing for Bitcoin Core, or another blockchains whether you are developing for Bitcoin Core, or another
full-node consensus client; an application such as a wallet, exchange, full-node consensus client; an application such as a wallet, exchange,