arno/bitcoinbook
1
0
Fork 0
mirror of https://github.com/bitcoinbook/bitcoinbook synced 2024-11-21 23:58:09 +00:00
Code Issues Releases Wiki Activity

part 3 CE

Browse Source
This commit is contained in:
Clare Laylock 2023-10-03 09:25:59 -04:00
parent 11577d434c
commit 311eac3b34
4 changed files with 230 additions and 235 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

217
ch08_signatures.adoc
View File

@ -27,7 +27,7 @@ intact.
Each transaction input and any signatures it may contain is _completely_
independent of any other input or signature. Multiple parties can
collaborate to construct transactions and sign only one input each.
Several protocols uses this fact to create multi-party transactions for
Several protocols use this fact to create multiparty transactions for
privacy.
====
@ -44,7 +44,7 @@ signing key). The second part is an algorithm
that allows anyone to verify the signature, given also the message and the corresponding
public key.
==== Creating a digital signature
==== Creating a Digital Signature
In Bitcoin's use of digital signature algorithms, the "message" being
signed is the transaction, or more accurately a hash of a specific
@ -62,7 +62,7 @@ where:
* _F_~_sig_~ is the signing algorithm
* _Sig_ is the resulting signature
More details on the mathematics of schnorr and ECDSA signatures can be found in <<schnorr_signatures>>
You can find more details on the mathematics of schnorr and ECDSA signatures in <<schnorr_signatures>>
and <<ecdsa_signatures>>.
In both schnorr and ECDSA signatures, the function _F_~_sig_~ produces a signature +Sig+ that is composed of
@ -70,7 +70,7 @@ two values. There are differences between the two values in the
different algorithms, which we'll explore later.
After the two values
are calculated, they are serialized into a byte-stream. For ECDSA
are calculated, they are serialized into a byte stream. For ECDSA
signatures, the encoding uses an international standard encoding scheme
called the
_Distinguished Encoding Rules_, or _DER_. For schnorr signatures, a
@ -78,10 +78,7 @@ simpler serialization format is used.
==== Verifying the Signature
The signature verification
algorithm takes the message (a hash of parts of the transaction and
related data), the signer's public key and the signature, and returns
TRUE if the signature is valid for this message and public key.
The signature verification algorithm takes the message (a hash of parts of the transaction and related data), the signer's public key and the signature, and returns ++TRUE++ if the signature is valid for this message and public key.
To verify the signature, one must have the signature, the serialized
transaction, some data about the output being spent, and the public key
@ -127,16 +124,16 @@ There are three +SIGHASH+ flags: +ALL+, +NONE+, and +SINGLE+, as shown
in <<sighash_types_and_their>>.
[[sighash_types_and_their]]
.SIGHASH types and their meanings
.++SIGHASH++ types and their meanings
[options="header"]
|=======================
|+SIGHASH+ flag| Value | Description
| +ALL+ | 0x01 | Signature applies to all inputs and outputs
| +NONE+ | 0x02 | Signature applies to all inputs, none of the outputs
| +SINGLE+ | 0x03 | Signature applies to all inputs but only the one output with the same index number as the signed input
|++SIGHASH++ flag| Value | Description
| ++ALL++ | ++0x01++ | Signature applies to all inputs and outputs
| ++NONE++ | ++0x02++ | Signature applies to all inputs, none of the outputs
| ++SINGLE++ | ++0x03++ | Signature applies to all inputs but only the one output with the same index number as the signed input
|=======================
In addition, there is a modifier flag +SIGHASH_ANYONECANPAY+, which can
In addition, there is a modifier flag, +SIGHASH_ANYONECANPAY+, which can
be combined with each of the preceding flags. When +ANYONECANPAY+ is
set, only one input is signed, leaving the rest (and their sequence
numbers) open for modification. The +ANYONECANPAY+ has the value +0x80+
@ -144,13 +141,13 @@ and is applied by bitwise OR, resulting in the combined flags as shown
in <<sighash_types_with_modifiers>>.
[[sighash_types_with_modifiers]]
.SIGHASH types with modifiers and their meanings
.++SIGHASH++ types with modifiers and their meanings
[options="header"]
|=======================
|SIGHASH flag| Value | Description
| ALL\|ANYONECANPAY | 0x81 | Signature applies to one input and all outputs
| NONE\|ANYONECANPAY | 0x82 | Signature applies to one input, none of the outputs
| SINGLE\|ANYONECANPAY | 0x83 | Signature applies to one input and the output with the same index number
|++SIGHASH++ flag| Value | Description
| ++ALL\++|++ANYONECANPAY++ | ++0x81++ | Signature applies to one input and all outputs
| ++NONE\++|++ANYONECANPAY++ | ++0x82++ | Signature applies to one input, none of the outputs
| ++SINGLE\++|++ANYONECANPAY++ | ++0x83++ | Signature applies to one input and the output with the same index number
|=======================
The way +SIGHASH+ flags are applied during signing and verification is
@ -195,12 +192,12 @@ By itself, this allows any miner to change
the output destination and claim the funds for themselves, but if other
required signatures in the transaction use +SIGHASH_ALL+ or another type
that commits to the output, it allows those spenders to change the
destination without allowing any third-parties (like miners) to modify
destination without allowing any third parties (like miners) to modify
the outputs.
+NONE|ANYONECANPAY+ :: This construction can be used to build a "dust
collector." Users who have tiny UTXOs in their wallets can't spend these
without the cost in fees exceeding the value of the UTXO, see
without the cost in fees exceeding the value of the UTXO; see
<<uneconomical_outputs>>. With this type
of signature, the uneconomical UTXOs can be donated for anyone to aggregate and
spend whenever they want.
@ -212,23 +209,22 @@ new sighash flags. A signature using +SIGHASH_ANYPREVOUT+ would not
commit to an input's outpoint field, allowing it to be used to spend any
previous output for a particular witness program. For example, if Alice
receives two outputs for the same amount to the same witness program
(e.g. requiring a single signature from her wallet), a
(e.g., requiring a single signature from her wallet), a
+SIGHASH_ANYPREVOUT+ signature for spending either one of those outputs
could be copied and used to spend the other output to the same
destination.
A signature using +SIGHASH_ANYPREVOUTANYSCRIPT+ would not
commit to the outpoint, the amount, the witness program, or the
specific leaf in the taproot merkle tree (script tree), allowing it to spend any previous output
which the signature could satisfy. For example, if Alice received two
outputs for different amounts and different witness programs (e.g. one
specific leaf in the taproot merkle tree (script tree), allowing it to spend any previous output that the signature could satisfy. For example, if Alice received two
outputs for different amounts and different witness programs (e.g., one
requiring a single signature and another requiring her signature plus some
other data), a +SIGHASH_ANYPREVOUTANYSCRIPT+ signature for spending
either one of those outputs could be copied and used to spend the other
output to the same destination (assuming the extra data for the second
output was known).
The main expected use for the two SIGHASH_ANYPREVOUT opcodes is improved
The main expected use for the two ++SIGHASH_ANYPREVOUT++ opcodes is improved
payment channels, such as those used in the Lightning Network, although
several other uses have been described.
@ -236,27 +232,27 @@ several other uses have been described.
====
You will not often see +SIGHASH+ flags presented as an option in a user's
wallet application. Simple wallet applications
sign with +SIGHASH_ALL+ flags. More sophisticated applications, such as
sign with [.keep-together]#+SIGHASH_ALL+# flags. More sophisticated applications, such as
Lightning Network nodes, may use alternative +SIGHASH+ flags, but they
use protocols that have been extensively reviewed to understand the
influence of the alternative flags.
====
[[schnorr_signatures]]
=== Schnorr signatures
=== Schnorr Signatures
In 1989, Claus Schnorr published a paper describing the signature
algorithm that's become eponymous with him. The algorithm isn't
specific to the Elliptic Curve Cryptography (ECC) that Bitcoin and many
specific to the elliptic curve cryptography (ECC) that Bitcoin and many
other applications use, although it is perhaps most strongly associated
with ECC today. Schnorr signatures have a number of nice properties:
Provable security::
A mathematical proof of the security of schnorr signatures depends on
only the difficulty of solving the Discrete Logarithm Problem (DLP),
particularly for Elliptic Curves (EC) for Bitcoin, and the ability of
particularly for elliptic curves (EC) for Bitcoin, and the ability of
a hash function (like the SHA256 function used in Bitcoin) to produce
unpredictable values, called the Random Oracle Model (ROM). Other
unpredictable values, called the random oracle model (ROM). Other
signature algorithms have additional dependencies or require much
larger public keys or signatures for equivalent security to
ECC-Schnorr (when the threat is defined as classical computers; other
@ -269,12 +265,12 @@ Linearity::
properties. The first property is that summing together two or more
variables and then running a function on that sum will produce the
same value as running the function on each of the variables
independently and then summing together the results, e.g.
independently and then summing together the results, e.g.,
+f(x + y + z) == f(x) + f(y) + f(z)+; this property is called
_additivity_. The second property is that multiplying a variable and
then running a function on that product will produce the same value as
running the function on the variable and then multiplying it by the
same amount, e.g. +f(a * x) == a * f(x)+; this property is called
same amount, e.g., +f(a * x) == a * f(x)+; this property is called
_homogeneity of degree 1_.
+
In cryptographic operations, some functions may be private (such
@ -285,7 +281,7 @@ In cryptographic operations, some functions may be private (such
specific benefits of linearity in schnorr signatures in
<<schnorr_multisignatures>> and <<schnorr_threshold_signatures>>.
Batch Verification::
Batch verification::
When used in a certain way (which Bitcoin does), one consequence of
schnorr's linearity is that it's relatively straightforward to verify
more than one schnorr signature at the same time in less time than it
@ -297,17 +293,16 @@ Batch Verification::
Later in this chapter, we'll describe the schnorr signature algorithm
exactly as it's used in Bitcoin, but we're going to start with a
simplified version of it and work our way towards the actual protocol in
simplified version of it and work our way toward the actual protocol in
stages.
Alice starts by choosing a large random number (+x+), which we call her
_private key_. She also knows a public point on Bitcoin's Elliptic
Curve (EC) called the Generator (+G+) (see <<public_key_derivation>>). Alice uses EC
_private key_. She also knows a public point on Bitcoin's elliptic
curve called the Generator (+G+) (see <<public_key_derivation>>). Alice uses EC
multiplication to multiply +G+ by her private key +x+, in which case +x+
is called a _scalar_ because it scales up +G+. The result is +xG+,
which we call Alice's _public key_. Alice gives her public key to Bob.
Even though Bob also knows +G+, the Discrete Logarithm Problem (DLP)
prevents Bob from being able to divide +xG+ by +G+ to derive Alice's
Even though Bob also knows +G+, the Discrete Logarithm Problem prevents Bob from being able to divide +xG+ by +G+ to derive Alice's
private key.
At some later time, Bob wants Alice to identify herself by proving
@ -330,7 +325,7 @@ process:
be used to multiply an EC point.
3. Alice now has the numbers (scalars) +x+, +k+, and +e+. She combines
them together to produce a final scalar +s+ using the formula:
them together to produce a final scalar +s+ using the formula
+s = k + ex+. She gives +s+ to Bob.
4. Bob now knows the scalars +s+ and +e+, but not +x+ or +k+. However,
@ -339,11 +334,10 @@ process:
the operation Alice performed: +sG == kG + exG+. If that is equal,
then Bob can be sure that Alice knew +x+ when she generated +s+.
.Schnorr identity protocol with integers instead of points
.Schnorr Identity Protocol with Integers Instead of Points
****
It might be easier to understand the interactive schnorr identity
protocol if we create an insecure oversimplification by substituting each of the values above
(including +G+) with simple integers instead of points on a elliptic curve.
protocol if we create an insecure oversimplification by substituting each of the preceding values (including +G+) with simple integers instead of points on an elliptic curve.
For example, we'll use the prime numbers starting with 3:
Setup: Alice chooses +x=3+ as her private key. She multiplies it by the
@ -364,7 +358,7 @@ generator +G=5+ to get her public key +xG=15+. She gives Bob +15+.
Of course, this is an oversimplified example. When working with simple
integers, we can divide products by the generator +G+ to get the
underlying scalar, which isn't secure. This is why a critical property
of the Elliptic Curve Cryptography (ECC) used in Bitcoin is that
of the elliptic curve cryptography used in Bitcoin is that
multiplication is easy but division by a point on the curve is impractical. Also, with numbers
this small, finding underlying values (or valid substitutes) through
brute force is easy; the numbers used in Bitcoin are much larger.
@ -373,7 +367,8 @@ brute force is easy; the numbers used in Bitcoin are much larger.
Let's discuss some of the features of the interactive schnorr
identity protocol that make it secure:
- The nonce (+k+). In step 1, Alice chooses a number that Bob doesn't
The nonce (+k+)::
In step 1, Alice chooses a number that Bob doesn't
know and can't guess and gives him the scaled form of that number,
+kG+. At that point, Bob also already has her public key (+xG+),
which is the scaled form of +x+, her private key. That means when Bob is working on
@ -387,7 +382,8 @@ identity protocol that make it secure:
be able to leverage that into figuring out Alice's private key. See
<<nonce_warning>> for more details.
- The challenge scalar (+e+). Bob waits to receive Alice's public nonce
The challenge scalar (+e+)::
Bob waits to receive Alice's public nonce
and then proceeds in step 2 to give her a number (the challenge
scalar) that Alice didn't previously know and couldn't have guessed.
It's critical that Bob only give her the challenge scalar after she
@ -395,8 +391,8 @@ identity protocol that make it secure:
who didn't know +x+ wanted to impersonate Alice, and Bob accidentally
gave them the challenge scalar +e+ before they told him the public
nonce +kG+. This allows the impersonator to change parameters on both sides of
the equation that Bob will use for verification, +sG == kG + exG+,
specifically they can change both +sG+ and +kG+. Think about a
the equation that Bob will use for verification, +sG == kG + exG+;
specifically, they can change both +sG+ and +kG+. Think about a
simplified form of that expression: x = y + a. If you can change both
+x+ and +y+, you can cancel out +a+ using +x' = (x - a) + a+. Any
value you choose for +x+ will now satisfy the equation. For the
@ -426,7 +422,7 @@ node that wants to authenticate her.
A simple technique, known as the Fiat-Shamir transform after its
discoverers, can turn the schnorr interactive identity protocol
into a non-interactive digital signature scheme. Recall the importance
into a noninteractive digital signature scheme. Recall the importance
of steps 1 and 2--including that they be performed in order. Alice must
commit to an unpredictable nonce; Bob must give Alice an unpredictable
challenge scalar only after he has received her commitment. Recall also
@ -460,9 +456,9 @@ We've now defined a version of the schnorr signature protocol, but
there's one more thing we need to do to address a Bitcoin-specific
concern. In BIP32 key derivation, as described in
<<public_child_key_derivation>>, the algorithm for unhardened derivation
takes a public key and adds to it a non-secret value to produce a
takes a public key and adds to it a nonsecret value to produce a
derived public key. That means it's also possible to add that
non-secret value to a valid signature for one key to produce a signature
nonsecret value to a valid signature for one key to produce a signature
for a related key. That related signature is valid but it wasn't
authorized by the person possessing the private key, which is a major
security failure. To protect BIP32 unhardened derivation and
@ -486,13 +482,13 @@ generate a private key from a large random seed value). She uses the
parameters defined in secp256k1 (see <<elliptic_curve>>) to multiply the
generator +G+ by her scalar +x+, producing +xG+ (her public key). She
gives her public key to everyone who will later authenticate her Bitcoin
transactions (e.g. by having +xG+ included in a transaction output). When
transactions (e.g., by having +xG+ included in a transaction output). When
she's ready to spend, she begins generating her signature:
1. Alice chooses a large random private nonce +k+ and derives the public
nonce +kG+.
2. She chooses her message +m+ (e.g. transaction data) and generates the
2. She chooses her message +m+ (e.g., transaction data) and generates the
challenge scalar +e = hash(kG || xG || m)+.
3. She produces the scalar +s = k + ex+. The two values +kG+ and +s+
@ -502,19 +498,19 @@ she's ready to spend, she begins generating her signature:
the witness structure of her spending transaction and then relaying that
transaction to full nodes.
4. The verifiers (e.g. full nodes) use +s+ to derive +sG+ and then
4. The verifiers (e.g., full nodes) use +s+ to derive +sG+ and then
verify that +sG == kG + hash(kG || xG || m)*xG+. If the equation is
valid, Alice proved that she knows her private key +x+ (without
revealing it) and committed to the message +m+ (containing the
transaction data).
==== Serialization of schnorr signatures
==== Serialization of Schnorr Signatures
A schnorr signature consists of two values, +kG+ and +s+. The value
+kG+ is a point on Bitcoin's elliptic curve (called secp256k1) and so
would normally be represented by two 32-byte coordinates, e.g. +(x,y)+.
However, only the x coordinate is needed, so only that value is
included. When you see +kG+ below, note that it's only that point's x
would normally be represented by two 32-byte coordinates, e.g., +(x,y)+.
However, only the _x_ coordinate is needed, so only that value is
included. When you see +kG+ in schnorr signatures for Bitcoin, note that it's only that point's _x_
coordinate.
The value +s+ is a scalar (a number meant to multiply other numbers). For
@ -522,8 +518,8 @@ Bitcoin's secp256k1 curve, it can never be more than 32 bytes long.
Although both +kG+ and +s+ can sometimes be values that can be
represented with fewer than 32 bytes, it's improbable that they'd be
much smaller than 32 bytes, and so they're serialized as two 32 byte
values (i.e., values smaller than 32 bytes have leading zeroes).
much smaller than 32 bytes, and so they're serialized as two 32-byte
values (i.e., values smaller than 32 bytes have leading zeros).
They're serialized in the order of +kG+ and then +s+, producing exactly
64 bytes.
@ -541,7 +537,7 @@ the serialization used for ECDSA signatures described in
<<serialization_of_signatures_der>>.
[[schnorr_multisignatures]]
==== Schnorr-based scriptless multisignatures
==== Schnorr-based Scriptless Multisignatures
In the single-signature schnorr protocol described in <<schnorr_signatures>>, Alice
uses a signature (+kG+, +s+) to publicly prove her knowledge of her
@ -560,9 +556,9 @@ that are believed to be secure.
====
Alice and Bob need to derive the public key for +x+, which is +xG+.
Since it's possible to use Elliptic Curve (EC) operations to add two EC
Since it's possible to use elliptic curve operations to add two EC
points together, they start by Alice deriving +yG+ and Bob deriving
+zG+. Then then add them together to create +xG = yG + zG+. The point
+zG+. They then add them together to create +xG = yG + zG+. The point
+xG+ is their _aggregated public key_. To create a signature, they begin the
simple multisignature protocol:
@ -571,8 +567,8 @@ simple multisignature protocol:
public nonce +aG+ and +bG+. Together, they produce an aggregated
public nonce +kG = aG + bG+.
2. They agree on the message to sign, +m+ (e.g. a transaction), and
each generate a copy of the challenge scalar: +e = hash(kG || xG || m)+.
2. They agree on the message to sign, +m+ (e.g., a transaction), and
each generates a copy of the challenge scalar: +e = hash(kG || xG || m)+.
3. Alice produces the scalar +q = a + ey+. Bob produces the scalar
+r = b + ez+. They add the scalars together to produce
@ -583,17 +579,17 @@ simple multisignature protocol:
Alice and Bob have proven that they know the sum of their private keys without
either one of them revealing their private key to the other or anyone
else. The protocol can be extended to any number of participants, e.g.
else. The protocol can be extended to any number of participants; e.g.,
a million people could prove they knew the sum of their million
different keys.
The protocol above has several security problems. Most notable is that one
The preceding protocol has several security problems. Most notable is that one
party might learn the public keys of the other parties before committing
to their own public key. For example, Alice generates her public key
+yG+ honestly and shares it with Bob. Bob generates his public key
using +zG - yG+. When their two keys are combined (+yG + zG - yG+), the
positive and negative +yG+ terms cancel out so the public key only represents
the private key for +z+, i.e. Bob's private key. Now Bob can create a
the private key for +z+, i.e., Bob's private key. Now Bob can create a
valid signature without any assistance from Alice. This is called a
_key cancellation attack_.
@ -623,14 +619,14 @@ signature parameters. The different nonces must not be related in any
way. For a multisignature, every participant must follow these rules or
it could compromise the security of other participants. In addition,
cancellation and other attacks need to be prevented. Different
protocols that accomplish these aims make different tradeoffs, so
protocols that accomplish these aims make different trade-offs, so
there's no single multisignature protocol to recommend in all cases.
Instead, we'll note three from the MuSig family of protocols:
MuSig::
Also called _MuSig1_, this protocol requires three rounds of
communication during the signing process, making it similar to the
process we described above. MuSig1's greatest advantage is its
process we just described. MuSig1's greatest advantage is its
simplicity.
MuSig2::
@ -645,13 +641,13 @@ MuSig-DN::
DN stands for Deterministic Nonce, which eliminates as a concern a
problem known as the _repeated session attack_. It can't be combined
with key exchange and it's significantly more complex to implement
than MuSig or Musig2.
than MuSig or MuSig2.
For most applications, MuSig2 is the best multisignature protocol
available at the time of writing.
[[schnorr_threshold_signatures]]
==== Schnorr-based scriptless threshold signatures
==== Schnorr-based Scriptless Threshold Signatures
Scriptless multisignature protocols only work for k-of-k signing.
Everyone with a partial public key that becomes part of the aggregated
@ -664,10 +660,10 @@ _threshold signature_.
We saw script-based threshold signatures in
<<multisig>>. But just as
scriptless multisignatures save space and increase privacy compared to
scripted multisigantures, _scriptless threshold signatures_ save space and
scripted multisignatures, _scriptless threshold signatures_ save space and
increase privacy compared to _scripted threshold signatures_. To anyone
not involved in the signing, a _scriptless threshold signatures_ looks
like any other signature which could've been created by a single-sig
not involved in the signing, a _scriptless threshold signature_ looks
like any other signature that could've been created by a single-sig
user or through a scriptless multisignature protocol.
Various methods are known for generating scriptless threshold
@ -685,7 +681,7 @@ on some additional information to each share, called a correction code,
that allows any two of them to recover the number. This scheme is not
secure because each share gives its holder partial knowledge of Alice's
secret, making it easier for the participant to guess Alice's secret
than a non-participant who didn't have a share.
than a nonparticipant who didn't have a share.
A secure secret sharing scheme prevents participants from learning
anything about the secret unless they combine the minimum threshold
@ -697,7 +693,7 @@ after its discoverer, one of the same discoverers of the Fiat-Shamir
transform we saw in <<schnorr_signatures>>.
In some cryptographic protocols, such as the scriptless threshold signature
schemes we're working towards, it's critical for Bob, Carol, and Dan to
schemes we're working toward, it's critical for Bob, Carol, and Dan to
know that Alice followed her side of the protocol correctly. They need to
know that the shares she creates all derive from the same secret, that
she used the threshold value she claims, and that she gave each one of
@ -705,7 +701,7 @@ them a different share. A protocol that can accomplish all of that,
and still be a secure secret sharing scheme, is a _verifiable secret
sharing scheme_.
To see how multisignatures and verifiable secret sharing works for
To see how multisignatures and verifiable secret sharing work for
Alice, Bob, and Carol, imagine they each wish to receive funds that can
be spent by any two of them. They collaborate as described in
<<schnorr_multisignatures>> to produce a regular multisignature public
@ -726,8 +722,8 @@ partial private key. Alice and Bob also have their private keys,
allowing them to create a scriptless multisignature with all three
necessary keys.
In other words, the scriptless threshold signature scheme described
above is the same as a scriptless multisignature scheme except that
In other words, the scriptless threshold signature scheme just described
is the same as a scriptless multisignature scheme except that
a threshold number of participants have the ability to reconstruct the
partial private keys of any other participants who are unable or
unwilling to sign.
@ -735,7 +731,8 @@ unwilling to sign.
This does point to a few things to be aware about when considering a
scriptless threshold signature protocol:
1. No accountability: because Alice and Bob reconstruct Carol's partial
No accountability::
Because Alice and Bob reconstruct Carol's partial
private key, there can be no fundamental difference between a scriptless
multisignature produced by a process that involved Carol and one that
didn't. Even if Alice, Bob, or Carol claim that they didn't sign,
@ -743,15 +740,16 @@ there's no guaranteed way for them to prove that they didn't
help produce the signature. If it's important to know which members of
the group signed, you will need to use a script.
2. Manipulation attacks: imagine that Bob tells Alice that Carol is
Manipulation attacks::
Imagine that Bob tells Alice that Carol is
unavailable, so they work together to reconstruct Carol's partial
private key. Then Bob tells Carol that Alice is unavailable, so they
work together to reconstruct Alice's partial private key. Now Bob has
his own partial private key plus the keys of Alice and Carol, allowing
him to spend the funds himself without their involvement. This attack can
be addressed if all of the participants agree to only communicate using a
scheme that allows any one of them to see all of the other's messages,
e.g. if Bob tells Alice that Carol is unavailable, Carol is able to see
scheme that allows any one of them to see all of the other's messages;
e.g., if Bob tells Alice that Carol is unavailable, Carol is able to see
that message before she begins working with Bob. Other solutions,
possibly more robust solutions, to this problem were being researched at
the time of writing.
@ -762,7 +760,7 @@ by multiple Bitcoin contributors and we expect peer-reviewed solutions
will become available after the publication of this book.
[[ecdsa_signatures]]
=== ECDSA signatures
=== ECDSA Signatures
Unfortunately for the future development of Bitcoin and many other
applications, Claus Schnorr patented the algorithm he discovered and
@ -770,7 +768,7 @@ prevented its use in open standards and open source software for almost
two decades. Cryptographers in the early 1990s who were blocked from
using the schnorr signature scheme developed an alternative construction
called the _Digital Signature Algorithm_ (DSA), with a version adapted
to Elliptic Curves called ECDSA.
to elliptic curves called ECDSA.
The ECDSA scheme and standardized parameters for suggested curves it could be used
with were widely implemented in cryptographic libraries by the time
@ -790,20 +788,20 @@ More complex::
Less provable security::
The interactive schnorr signature identification protocol depends only
on the strength of the Elliptic Curve Discrete Logarithm Problem
on the strength of the elliptic curve Discrete Logarithm Problem
(ECDLP). The non-interactive authentication protocol used in Bitcoin
also relies on the Random Oracle Model (ROM). However, ECDSA's extra
also relies on the random oracle model (ROM). However, ECDSA's extra
complexity has prevented a complete proof of its security being
published (to the best of our knowledge). We are not experts in
proving cryptographic algorithms, but it seems unlikely after 30 years
that ECDSA will be proven to only require the same two assumptions as
schnorr.
Non-linear::
Nonlinear::
ECDSA signatures cannot be easily combined to create scriptless
multisignatures or used in related advanced applications such as
multi-party signature adaptors. There are workarounds for this
problem, but they involve additional extra complexity which
multiparty signature adaptors. There are workarounds for this
problem, but they involve additional extra complexity that
significantly slows down operations and which, in some cases, has
resulted in software accidentally leaking private keys.
@ -814,19 +812,18 @@ values are _R_ and _s_.
The signature
algorithm first generates a private nonce (_k_) and derives from it a public
nonce (_K_). The _R_ value of the digital signature is then the x
nonce (_K_). The _R_ value of the digital signature is then the _x_
coordinate of the nonce _K_.
From there, the algorithm calculates the _s_ value of the signature,
such that. Like we did with schnorr signatures, operations involving
integers are modulus p.
From there, the algorithm calculates the _s_ value of the signature. Like we did with schnorr signatures, operations involving
integers are modulus p:
_s_ = __k__^-1^ (__Hash__(__m__) + __x__ * __R__)
where:
* _k_ is the private nonce
* _R_ is the x coordinate of the public nonce
* _R_ is the _x_ coordinate of the public nonce
* _x_ is the Alice's private key
* _m_ is the message (transaction data)
@ -844,18 +841,18 @@ where:
- _m_ is the message (the transaction data that was signed)
- _G_ is the elliptic curve generator point
If the x coordinate of the calculated point _K_ is equal to _R_, then
If the _x_ coordinate of the calculated point _K_ is equal to _R_, then
the verifier can conclude that the signature is valid.
[TIP]
====
ECDSA is necessarily a fairly complicated piece of math; a full
explanation is beyond the scope of this book. A number of great guides
online take you through it step by step: search for "ECDSA explained".
online take you through it step by step: search for "ECDSA explained."
====
[[serialization_of_signatures_der]]
==== Serialization of ECDSA signatures (DER)
==== Serialization of ECDSA Signatures (DER)
Let's look at
the following DER-encoded signature:
@ -864,19 +861,19 @@ the following DER-encoded signature:
3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e381301
----
That signature is a serialized byte-stream of the +R+ and +S+ values
produced by to prove control of the private key authorized
That signature is a serialized byte stream of the +R+ and +S+ values
produced by the signer to prove control of the private key authorized
to spend an output. The serialization format consists of nine elements
as follows:
* +0x30+&#x2014;indicating the start of a DER sequence
* +0x45+&#x2014;the length of the sequence (69 bytes)
* +0x02+&#x2014;an integer value follows
* +0x21+&#x2014;the length of the integer (33 bytes)
* +R+&#x2014;++00884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb++
* +0x02+&#x2014;another integer follows
* +0x20+&#x2014;the length of the integer (32 bytes)
* +S+&#x2014;++4b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813++
* +0x30+, indicating the start of a DER sequence
* +0x45+, the length of the sequence (69 bytes)
* +0x02+, an integer value follows
* +0x21+, the length of the integer (33 bytes)
* +R+, ++00884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb++
* +0x02+, another integer follows
* +0x20+, the length of the integer (32 bytes)
* +S+, ++4b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813++
* A suffix (+0x01+) indicating the type of hash used (+SIGHASH_ALL+)
[[nonce_warning]]
@ -932,7 +929,7 @@ ensure you generate a different _k_ for each transaction.
=== Segregated Witness's New Signing Algorithm
Signatures in bitcoin transactions are applied on a _commitment hash_,
Signatures in Bitcoin transactions are applied on a _commitment hash_,
which is calculated from the transaction data, locking specific parts of
the data indicating the signer's commitment to those values. For
example, in a simple +SIGHASH_ALL+ type signature, the commitment hash

82
ch09_fees.adoc
View File

@ -38,7 +38,7 @@ transaction pinning, and are largely uninteresting to the following
discussion about fees--so we're going to ignore them in this chapter.
====
The criteria that almost all miners use to select which transactions to
The criterion that almost all miners use to select which transactions to
include in their blocks is to maximize their revenue. Bitcoin was
specifically designed to accommodate this by providing a mechanism that
allows a transaction to give money to the miner who includes that
@ -54,7 +54,7 @@ In this chapter, we'll explore various aspects of those
bids--transaction fees--and how they influence the creation and
management of Bitcoin transactions.
=== Who pays the transaction fee?
=== Who Pays the Transaction Fee?
Most payment systems involve some sort of fee for transacting, but
often this fee is hidden from typical buyers. For example, a merchant
@ -85,16 +85,16 @@ normally allowing only spenders to choose fees can sometimes be a
problem; we'll look at a solution to that problem in <<cpfp>>. However,
in many common payment flows, the parties with the highest desire to see a
transaction confirm quickly--that is, the parties who'd be the most
willing to pay higher fees--are the the spenders.
willing to pay higher fees--are the spenders.
For those reasons, both technical and practical, it is customary in
Bitcoin for spenders to pay transaction fees. There are exceptions,
such as for merchants which accept unconfirmed transactions and in
such as for merchants that accept unconfirmed transactions and in
protocols that don't immediately broadcast transactions after they are
signed (preventing the spender from being able to choose an appropriate
fee for the current market). We'll explore those exceptions later.
=== Fees and fee rates
=== Fees and Fee Rates
Each transaction only pays a single fee--it doesn't matter how large the
transaction is. However, the larger transactions become, the fewer of
@ -140,7 +140,7 @@ by accident. There are legitimate reasons for users to overpay fees on
rare occasions.
====
=== Estimating appropriate fee rates
=== Estimating Appropriate Fee Rates
We've established that you can pay a lower fee rate if you're willing to
wait longer for your transaction to be confirmed, with the exception
@ -163,7 +163,7 @@ similar number of blocks to confirm. Bitcoin Core includes a fee rate
estimator that uses these principles, which can be called using the
`estimatesmartfee` RPC with a parameter specifying how many blocks
you're willing to wait before the transaction is highly likely to
confirm (for example, 144 blocks is about 1 day).
confirm (for example, 144 blocks is about 1 day):
----
$ bitcoin-cli -named estimatesmartfee conf_target=144
@ -174,12 +174,12 @@ $ bitcoin-cli -named estimatesmartfee conf_target=144
----
Many web-based services also provide fee estimation as an API. For a
current list, see https://www.lopp.net/bitcoin-information/fee-estimates.html
current list, see https://www.lopp.net/bitcoin-information/fee-estimates.html.
As mentioned, fee rate estimation can never be perfect. One common
problem is that the fundamental demand might change, adjusting the
equilibrium and either increasing prices (fees) to new heights or
decreasing them towards the minimum.
decreasing them toward the minimum.
If fee rates go down, then a transaction
that previously paid a normal fee rate might now be paying a high fee
rate and it will be confirmed earlier than expected. There's no way to
@ -187,18 +187,18 @@ lower the fee rate on a transaction you've already sent, so you're stuck
paying a higher fee rate. But, when fee rates go up, there's a need for
methods to be able to increase the fee rates on those transactions,
which is called _fee bumping_. There are two commonly used types of fee
bumping in Bitcoin, Replace-By-Fee (RBF) and Child Pays For Parent
bumping in Bitcoin, replace by fee (RBF) and child pays for parent
(CPFP).
[[rbf]]
=== Replace-By-Fee (RBF) Fee Bumping
=== Replace By Fee (RBF) Fee Bumping
To increase the fee of a transaction using RBF fee bumping, you create
a conflicting version of the transaction which pays a higher fee. Two
a conflicting version of the transaction that pays a higher fee. Two
or more transactions are considered to be _conflicting transactions_ if
only one of them can be included in a valid blockchain, forcing a miner
to chose only one of them. Conflicts occur when two or more transactions
each try to spend one of the same UTXOs, i.e. they each include an input
to choose only one of them. Conflicts occur when two or more transactions
each try to spend one of the same UTXOs, i.e., they each include an input
that has the same outpoint (reference to the output of a previous
transaction).
@ -226,7 +226,7 @@ Full RBF::
version. As of this writing, this can be optionally enabled in
Bitcoin Core (but it is disabled by default).
.Why are there two variants of RBF?
.Why Are There Two Variants of RBF?
****
The reason for the two different versions of RBF is that full RBF has
been controversial. Early versions of Bitcoin allowed transaction
@ -260,7 +260,7 @@ based on the first version, but the second version gets confirmed, then
the merchant will not receive payment for its costs.
Some merchants, and people supporting them, requested that transaction
replacement not be re-enabled in Bitcoin Core. Other people pointed out
replacement not be reenabled in Bitcoin Core. Other people pointed out
that transaction replacement provides benefits, including the ability to
fee bump transactions that initially paid too low of a fee rate.
@ -289,7 +289,7 @@ whether or not that has happened yet.
As a user, if you plan to use RBF fee bumping, you will first need to
choose a wallet that supports it, such as one of the wallets listed as
having "Sending support" on
https://bitcoinops.org/en/compatibility/#replace-by-fee-rbf
https://bitcoinops.org/en/compatibility/#replace-by-fee-rbf.
As a developer, if you plan to implement RBF fee bumping, you will first
need to decided whether to perform opt-in RBF or full RBF. At the time
@ -306,8 +306,8 @@ same for both approaches.
When you need to fee bump a transaction, you will simply create a new
transaction that spends at least one of the same UTXOs as the original
transaction you want to replace. You will likely want to keep the
same outputs in the transaction which pay the receiver (or receivers).
You may pay the increased fee by reducing the the value of your change
same outputs in the transaction that pay the receiver (or receivers).
You may pay the increased fee by reducing the value of your change
output or by adding additional inputs to the transaction. Developers
should provide users with a fee-bumping interface that does all of this
work for them and simply asks them (or suggests to them) how much the
@ -324,11 +324,11 @@ overpay the receivers. For example:
- Transaction version 0 includes input _A_.
- Transaction version 1 includes inputs _A_ and _B_ (e.g., you had to add
input _B_ to pay the extra fees)
- Transaction version 2 includes inputs B and C (e.g., you had to add input
- Transaction version 2 includes inputs _B_ and _C_ (e.g., you had to add input
_C_ to pay the extra fees but _C_ was large enough that you no longer
need input _A_).
In the above scenario, any miner who saved version 0 of the transaction
In this scenario, any miner who saved version 0 of the transaction
will be able to confirm both it and version 2 of the transaction. If
both versions pay the same receivers, they'll be paid twice (and the
miner will receive transaction fees from two separate transactions).
@ -347,27 +347,26 @@ have created if they had paid the increased fee rate in the first place.
The fundamental disadvantage of RBF fee bumping is that it can normally
only be performed by the creator of the transaction--the person or
people who were required to provide signatures or other authentication
data for the transaction. An exception to this is transactions which
data for the transaction. An exception to this is transactions that
were designed to allow additional inputs to be added by using sighash
flags, see <<sighash_types>>, but that presents its own challenges. In
flags (see <<sighash_types>>), but that presents its own challenges. In
general, if you're the receiver of an unconfirmed transaction and you
want to make it confirm faster (or at all), you can't use an RBF fee
bump; you need some other method.
There are additional problems with RBF that we'll explore in the
subsequent <<transaction_pinning>> section.
There are additional problems with RBF that we'll explore in <<transaction_pinning>>.
[[cpfp]]
=== Child Pays For Parent (CPFP) Fee Bumping
=== Child Pays for Parent (CPFP) Fee Bumping
Anyone who receives the output of an unconfirmed transaction can
incentivize miners to confirm that transaction by spending that output.
The transaction you want to get confirmed is called the _parent
transaction_. A transaction which spends an output of the parent
transaction_. A transaction that spends an output of the parent
transaction is called a _child transaction_.
As we learned in <<outpoints>>, every input in a confirmed transaction
must reference the unspent output of a transaction which appears earlier
must reference the unspent output of a transaction that appears earlier
in the blockchain (whether earlier in the same block or in a previous
block). That means a miner who wants to confirm a child transaction
must also ensure that its parent transaction is confirmed. If the
@ -384,7 +383,7 @@ about by fee rate and include the highest-revenue ones in the block
they're attempting to mine, up to the maximum size (weight) allowed to
be included in a block. To find even more packages that might be
profitable to mine, the miner can evaluate packages across multiple
generations (e.g. an unconfirmed parent transaction being combined with
generations (e.g., an unconfirmed parent transaction being combined with
both its child and grandchild). This is called _ancestor fee rate
mining_.
@ -404,14 +403,14 @@ The primary disadvantage of CPFP compared to RBF is that CPFP typically
uses more block space. In RBF, a fee bump transaction is often the same
size as the transaction it replaces. In CPFP, a fee bump adds a whole
separate transaction. Using extra block space requires paying extra
fees beyond the the cost of the fee bump.
fees beyond the cost of the fee bump.
There are several challenges with CPFP, some of which we'll explore in
<<transaction_pinning>>. One other problem which we
<<transaction_pinning>>. One other problem that we
specifically need to mention is the minimum relay fee rate problem,
which is addressed by package relay.
==== Package Relay
=== Package Relay
Early versions of Bitcoin Core didn't place any limits on the number of
unconfirmed transactions they stored for later relay and mining in their
@ -454,25 +453,25 @@ problem.
Although both RBF and CPFP fee bumping work in the basic cases we
described, there are rules related to both
methods that are designed to prevent denial of service attacks on miners
methods that are designed to prevent denial-of-service attacks on miners
and relaying full nodes. An unfortunate side effect of those rules
is that they can sometimes prevent someone from being able to use fee
bumping. Making it impossible or difficult to fee bump a transaction is
called _transaction pinning_.
One of the major denial of service concerns revolve's around the effect of
One of the major denial of service concerns revolves around the effect of
transaction relationships. Whenever the output of a transaction is
spent, that transaction's identifier (txid) is referenced by the child
transaction. However, when a transaction is replaced, the replacement
has a different txid. If that replacement transaction gets confirmed,
none of its descendants can be included in the same blockchain. It's
possible to recreate and re-sign the descendant transactions, but that's
possible to re-create and re-sign the descendant transactions, but that's
not guaranteed to happen. This has related but divergent implications
for RBF and CPFP:
- In the context of RBF, when Bitcoin Core accepts a replacement
transaction, it keeps things simple by forgetting about the original
transaction and all descendant transactions which depended on that
transaction and all descendant transactions that depended on that
original. To ensure that it's more profitable for miners to accept
replacements, Bitcoin Core only accepts a replacement transaction if it
pays more fees than all the transactions that will be forgotten.
@ -496,7 +495,7 @@ transaction.
Similarly, if a transaction with 25 descendants is removed from a
node's mempool (such as for being included in a block), and each of
those descendants has 25 other ancestors, another 625 packages need to
be updated. Each time we double our parameter (e.g. from 25 to 50),
be updated. Each time we double our parameter (e.g., from 25 to 50),
we quadruple the amount of work our node needs to perform.
+
Additionally, a transaction and all of its descendants is not
@ -526,14 +525,13 @@ Protocol developers have been working on mitigating problems with
transaction pinning for several years. One partial solution is
described in <<cpfp_carve_out>>. Several other solutions have been
proposed, and at least one solution is being actively developed as of
this writing (ephemeral anchors, see
https://bitcoinops.org/en/topics/ephemeral-anchors/).
this writing—https://bitcoinops.org/en/topics/ephemeral-anchors/[ephemeral anchors].
[[cpfp_carve_out]]
=== CPFP Carve Out and Anchor Outputs
In 2018, developers working on Lightning Network (LN) had a problem.
Their protocol uses transactions which require signatures from two
Their protocol uses transactions that require signatures from two
different parties. Neither party wants to trust the other, so they sign
transactions at a point in the protocol when trust isn't needed,
allowing either of them to broadcast one of those transactions at a
@ -620,7 +618,7 @@ you intended.
[[fee_sniping]]
=== Timelock Defense Against Fee Sniping
Fee-sniping is a theoretical
Fee sniping is a theoretical
attack scenario, where miners attempting to rewrite past blocks "snipe"
higher-fee transactions from future blocks to maximize their
profitability.
@ -666,4 +664,4 @@ As Bitcoin continues to mature, and as the subsidy continues to decline,
fees become more and more important to Bitcoin users, both in their
day-to-day use for getting transactions confirmed quickly and in
providing an incentive for miners to continue securing Bitcoin
transactions with new proof-of-work.
transactions with new proof of work.

120
ch10_network.adoc
View File

@ -3,7 +3,7 @@
Bitcoin is structured as a peer-to-peer network architecture on
top of the internet. The term peer-to-peer, or P2P, means that the
full nodes which participate in the network are peers to each other, that
full nodes that participate in the network are peers to each other, that
they can all perform the same functions, and that there are no "special" nodes.
The network nodes
interconnect in a mesh network with a "flat" topology. There is no
@ -52,7 +52,7 @@ only a subset of the blockchain and partly verify transactions using a method
called _simplified payment verification_, or SPV. These clients are known as lightweight clients.
Miners compete to create new blocks by
running specialized hardware to solve the Proof-of-Work algorithm. Some
running specialized hardware to solve the proof-of-work algorithm. Some
miners operate full nodes, validating every block on the
blockchain, while others are clients participating in pool
mining and depending on a pool server to provide them with work.
@ -119,7 +119,7 @@ some of those transactions are confirmed in a new block, the node
doesn't need to receive a second copy of those transactions.
Instead of receiving redundant unconfirmed transactions, compact blocks
allows a peer to instead send a short 6-byte identifier for each transaction.
allow a peer to instead send a short 6-byte identifier for each transaction.
When your node receives a compact block with one or more identifiers, it
checks its mempool for those transactions and uses them if they are
found. For any transaction that isn't found in your local node's
@ -133,7 +133,7 @@ always sends a block's coinbase transaction.
If the remote peer guesses correctly about what transactions your node
has in its mempool, and which it does not, it will send a block nearly
as efficiently as is theoretically possible (for a typical block, it'll
be between 97% to 99% efficient).
be between 97% and 99% efficient).
[TIP]
====
@ -175,7 +175,7 @@ quickly announcing blocks).
// released into the public domain by Nicolas Dorier
[[bip152_illustration]]
.BIP152 modes compared (from BIP152). They grey bar indicates the time it takes the node to validate the block
.BIP152 modes compared (from BIP152). The gray bar indicates the time it takes the node to validate the block.
image::images/mbc3_1002.png["BIP152"]
The names of the two methods (which are taken from BIP152) can be a bit
@ -186,15 +186,15 @@ relay before compact blocks were implemented.
=== Private Block Relay Networks
Although compact blocks go a long way towards minimizing the time it
Although compact blocks go a long way toward minimizing the time it
takes for blocks to propagate across the network,
it's possible to minimize latency further. Unlike
compact blocks, though, the other solutions involve tradeoffs that
compact blocks, though, the other solutions involve trade-offs that
make them unavailable or unsuitable for the public P2P relay network.
For that reason, there has been experimentation with private relay
networks for blocks.
One simple technique is to pre-select a route between endpoints. For
One simple technique is to preselect a route between endpoints. For
example, a relay network with servers running in datacenters near major
trans-oceanic fiber optic lines might be able to forward new blocks
faster than waiting for the block to arrive at the node run by some home
@ -214,18 +214,18 @@ because we mostly use protocols that automatically re-request the
missing data. However, requesting missing data triples the time to
receive it. For example:
1. Alice sends some data to Bob
1. Alice sends some data to Bob.
2. Bob doesn't receive the data (or it is damaged). Bob re-requests
the data from Alice
3. Alice sends the data again
the data from Alice.
3. Alice sends the data again.
A third technique is to assume all nodes receiving the data have
almost all of the same transactions in their mempool, so they can all
accept the same compact block. That not only saves us time computing
a compact block at each hop but it means that all each hop can simply
a compact block at each hop, but it means that all each hop can simply
relay the FEC packets to the next hop even before validating them.
The tradeoff for each of the above methods is that they work well with
The trade-off for each of the preceding methods is that they work well with
centralization but not in a decentralized network where individual nodes
can't trust other nodes. Servers in datacenters cost money and can
often be accessed by operators of the datacenter, making them less
@ -270,7 +270,7 @@ information, including:
+nTime+:: The current time
+addrYou+:: The IP address of the remote node as seen from this node
+addrMe+:: The IP address of the local node, as discovered by the local node
+subver+:: A sub-version showing the type of software running on this node (e.g., pass:[<span class="keep-together"><code>/Satoshi:0.9.2.1/</code></span>])
+subver+:: A subversion showing the type of software running on this node (e.g., pass:[<span class="keep-together"><code>/Satoshi:0.9.2.1/</code></span>])
+BestHeight+:: The block height of this node's blockchain
+fRelay+:: A field added by BIP37 for requesting not to receive unconfirmed transactions
@ -435,8 +435,8 @@ decisions. However, the most common
implementation is Bitcoin Core.
More than 95% of full nodes on the Bitcoin network run
various versions of Bitcoin Core. It is identified as "Satoshi" in the
sub-version string sent in the +version+ message and shown by the
command +getpeerinfo+ as we saw earlier; for example, +/Satoshi:24.0.1/+.
subversion string sent in the +version+ message and shown by the
command +getpeerinfo+ as we saw earlier; for example, [.keep-together]#+/Satoshi:24.0.1/+#.
=== Exchanging "Inventory"
@ -458,7 +458,7 @@ a +getheaders+ message that contains the hash of the top
block on their local blockchain. One of the peers will be able to
identify the received hash as belonging to a block that is not at the
top, but rather belongs to an older block, thus deducing that its own
local blockchain is longer than its peer's.
local blockchain is longer than the remote node's blockchain.
The peer that has the longer blockchain has more blocks than the other
node and can identify which headers the other node needs in order to
@ -474,7 +474,7 @@ connections to peers that are significantly slower than the average in
order to find newer (and possibly faster) peers.
Let's assume, for example, that a node only has the genesis block. It
will then receive an +headers+ message from its peers containing the headers
will then receive a +headers+ message from its peers containing the headers
of the next 2,000 blocks in the chain. It will start requesting blocks
from all of its connected peers, keeping a queue of up to 1,024 blocks.
Blocks need to be validated in order, so if the oldest block in the
@ -542,7 +542,7 @@ on top of it is proof, by proxy, that the transaction actually exists.
A lightweight client cannot normally be persuaded that a transaction exists in a block
when the transaction does not in fact exist. The lightweight client establishes
the existence of a transaction in a block by requesting a merkle path
proof and by validating the Proof-of-Work in the chain of blocks.
proof and by validating the proof of work in the chain of blocks.
However, a transaction's existence can be "hidden" from a lightweight client. A
lightweight client can definitely verify that a transaction exists but cannot
verify that a transaction, such as a double-spend of the same UTXO,
@ -565,7 +565,7 @@ node.
====
A full node verifies a transaction by checking the entire chain of
thousands of blocks below it in order to guarantee that the UTXO exists
and is not spent, whereas an lightweight client only proves that a transaction
and is not spent, whereas a lightweight client only proves that a transaction
exists and checks that the block containing that transaction is
buried by a handful of blocks above it.
====
@ -607,19 +607,19 @@ transactions matching a specific pattern, without revealing exactly
which addresses, keys, or transactions they are searching for.
In our previous analogy, a tourist without a map is asking for
directions to a specific address, "23 Church St." If she asks strangers
for directions to this street, she inadvertently reveals her
directions to a specific address, "23 Church St." If they ask a stranger
for directions to this street, they inadvertently reveal their
destination. A bloom filter is like asking, "Are there any streets in
this neighborhood whose name ends in R-C-H?" A question like that
reveals slightly less about the desired destination than asking for "23
Church St." Using this technique, a tourist could specify the desired
address in more detail such as "ending in U-R-C-H" or less detail as
address in more detail such as "ending in U-R-C-H" or less detail such as
"ending in H." By varying the precision of the search, the tourist
reveals more or less information, at the expense of getting more or less
specific results. If she asks a less specific pattern, she gets a lot
specific results. If they ask for a less specific pattern, they get a lot
more possible addresses and better privacy, but many of the results are
irrelevant. If she asks for a very specific pattern, she gets fewer
results but loses privacy.
irrelevant. If they ask for a very specific pattern, they get fewer
results but lose privacy.
Bloom filters serve this function by allowing a lightweight client to specify a
search pattern for transactions that can be tuned toward precision or
@ -782,14 +782,14 @@ that they didn't offer very much privacy. A full node receiving a bloom
filter from a peer could apply that filter to the entire blockchain to
find all of the client's transactions (plus false positives). It could
then look for patterns and relationships between the transactions.
Randomly-selected false positive transactions would be unlikely to have
Randomly selected false positive transactions would be unlikely to have
a parent-child relationship from output to input, but transactions from
the user's wallet would be very likely to have that relationship. If
all of the related transactions have certain characteristics, such as
at least one P2PKH output, then transactions without that characteristic
can be assumed not to belong to the wallet.
It was also discovered that specially-constructed filters could force
It was also discovered that specially constructed filters could force
the full nodes that processed them to perform a large amount of work,
which could lead to denial-of-service attacks.
@ -814,7 +814,7 @@ the entire block.
[NOTE]
====
Despite the similarities in names, BIP152 _compact blocks_ and
BIP157/158 _compact block filters_ are unrelated
BIP157/158 _compact block filters_ are unrelated.
====
This allows nodes to create a single filter for every block, which they
@ -863,8 +863,8 @@ numerical order:
Then, Alice sends the first number. For the remaining numbers, she
sends the difference between that number and the preceding number. For
example, for the second number, she sends 97 (476 - 379); for the third
number, she sends 177 (653 - 476); and so on:
example, for the second number, she sends 97 (476 – 379); for the third
number, she sends 177 (653 – 476); and so on:
----
379
@ -885,8 +885,8 @@ more numbers we select, the smaller the average (mean) size of the
differences. That means the amount of data we need to transfer doesn't
increase as fast as the length of our list increases (up to a point).
Even more usefully, the length of the randomly-selected numbers in a
list of differences is naturally biased towards smaller lengths.
Even more usefully, the length of the randomly selected numbers in a
list of differences is naturally biased toward smaller lengths.
Consider selecting two random numbers from 1 to 6; this is the same
as rolling two dice. There are 36 distinct combinations of two dice:
@ -939,17 +939,17 @@ Golomb coding provides that facility. Rice coding is a subset of Golomb
coding that's more convenient to use in some situations, including the
application of Bitcoin block filters.
==== What data to include in a block filter
==== What Data to Include in a Block Filter
Our primary goal is to allow wallets to learn whether a block contains a
transaction affecting that wallet. For a wallet to be effective, it
needs to learn two types of information:
1. When it has received money. Specifically, when a transaction
. When it has received money. Specifically, when a transaction
output contains a script that the wallet controls (such as by
controlling the authorized private key).
2. When it has spent money. Specifically, when a transaction input
. When it has spent money. Specifically, when a transaction input
references a previous transaction output that the wallet controlled.
A secondary goal during the design of compact block filters was to allow
@ -963,9 +963,9 @@ accurate filter.
For both the primary and secondary goals to be met, a filter would need
to reference two types of information:
1. The script for every output in every transaction in a block.
* The script for every output in every transaction in a block.
2. The outpoint for every input in every transaction in a block.
* The outpoint for every input in every transaction in a block.
An early design for compact block filters included both of those pieces
of information, but it was realized that there was a more efficient way
@ -973,15 +973,15 @@ to accomplish the primary goal if we sacrificed the secondary goal. In
the new design, a block filter would still references two types of
information, but they'd be more closely related:
1. As before, the script for every output in every transaction in a
* As before, the script for every output in every transaction in a
block.
2. In a change, it would also reference the script of the output
* In a change, it would also reference the script of the output
referenced by the outpoint for every input in every transaction in a
block. In other words, the output script being spent.
This had several advantages. First, it meant that wallets didn't need
to track outpoints; they could instead just scan for the the
to track outpoints; they could instead just scan for the
output scripts to which they expected to receive money. Second, any time a
later transaction in a block spends the output of an earlier
transaction in the same block, they'll both reference the same
@ -999,12 +999,12 @@ to verify a block filter was built correctly. However, there is an
alternative that can help a client detect if a peer is lying to it:
obtaining the same filter from multiple peers.
==== Downloading block filters from multiple peers
==== Downloading Block Filters from Multiple Peers
A peer can provide a wallet with an inaccurate filter. There's two ways
A peer can provide a wallet with an inaccurate filter. There are two ways
to create an inaccurate filter. The peer can create a filter that
references transactions that don't actually appear in the associated
block (a false positive). Alternatively, the peer can crate a filter
block (a false positive). Alternatively, the peer can create a filter
that doesn't reference transactions that do actually appear in the
associated block (a false negative).
@ -1021,7 +1021,7 @@ block, the client can download all of them. It can then also download
the associated block. If the block contains any transaction related to
the wallet that is not part of one of the filters, then the wallet can
be sure that whichever peer created that filter was
inaccurate--Golomb-Rice Coded Sets (GCSes) will always include a
inaccurate--Golomb-Rice Coded Sets will always include a
potential match.
Alternatively, if the block doesn't contain a transaction that the
@ -1035,7 +1035,7 @@ designed to use, the wallet can stop using that peer. In most cases,
the only consequence of the inaccurate filter is that the wallet uses
more bandwidth than expected.
==== Reducing bandwidth with lossy encoding
==== Reducing Bandwidth with Lossy Encoding
The data about the transactions in a block that we want to communicate
is an output script. Output scripts vary in length and follow patterns,
@ -1051,7 +1051,7 @@ faster and more configurable non-cryptographic hash functions, such as
the SipHash function we'll use for compact block filters.
The details of the algorithm used are described in BIP158, but the gist
is that each output script is reduced to a 64 bit commitment using
is that each output script is reduced to a 64-bit commitment using
SipHash and some arithmetic operations. You can think of this as
taking a set of large numbers and truncating them to shorter numbers, a
process that loses data (so it's called _lossy encoding_). By losing
@ -1059,9 +1059,9 @@ some information, we don't need to store as much information later,
which saves space. In this case, we go from a typical output script
that's 160 bits or longer down to just 64 bits.
==== Using compact block filters
==== Using Compact Block Filters
The 64 bit value for every commitment to an output script in a block are
The 64-bit values for every commitment to an output script in a block are
sorted, duplicate entries are removed, and the GCS is constructed by
finding the differences (deltas) between each entry. That compact block
filter is then distributed by peers to their clients (such as wallets).
@ -1074,23 +1074,23 @@ the filter.
Recall our example of the lossiness of compact block filters being
similar to truncating a number. Imagine a client is looking for a block
that contains the number 123456 and that an an accurate (but lossy)
that contains the number 123456 and that an accurate (but lossy)
compact block filter contains the number 1234. When a client sees that
1234, it will download the associated block.
There's a 100% guarantee that an accurate filter containing 1234 will
allow a client to learn about a block containing 123456, called a _true
positive_. However, there's also chance that the block might contain
123400, 123401, or almost a hundred other entries that are not when the
123400, 123401, or almost a hundred other entries that are not what the
client is looking for (in this example), called a _false positive_.
A 100% true positive match rate is great. It means that a wallet can
depend on compact block filters to find every transaction affecting that
wallet. A non-zero false positive rate means that the wallet will end
wallet. A nonzero false positive rate means that the wallet will end
up downloading some blocks that don't contain transactions interesting
to the wallet. The main consequence of this is that the client will use
extra bandwidth, which is not a huge problem. The actual
false-positive rate for BIP158 compact block filters is very low, so
false positive rate for BIP158 compact block filters is very low, so
it's not a major problem. A false positive rate can also help improve a
client's privacy, as it does with bloom filters, although anyone wanting
the best possible privacy should still use their own full node.
@ -1129,7 +1129,7 @@ at the time of writing.
As a way to increase the privacy and security of the Bitcoin P2P
network, there is a solution that provides encryption of the
communications: _Tor Transport_.
communications: _Tor transport_.
==== Tor Transport
@ -1155,7 +1155,7 @@ debugging for the Tor service like this:
$ bitcoind --daemon --debug=tor
----
You should see "tor: ADD_ONION successful" in the logs, indicating that
You should see ++tor: ADD_ONION successful++ in the logs, indicating that
Bitcoin Core has added a hidden service to the Tor network.
You can find more instructions on running Bitcoin Core as a Tor hidden
@ -1163,7 +1163,7 @@ service in the Bitcoin Core documentation (_docs/tor.md_) and various
online tutorials.
[[mempool]]
=== Mempools and orphan pools
=== Mempools and Orphan Pools
Almost every node on the Bitcoin
network maintains a temporary list of unconfirmed transactions called
@ -1190,10 +1190,10 @@ of the newly added transaction, which is no longer an orphan, the
process is repeated recursively looking for any further descendants,
until no more descendants are found. Through this process, the arrival
of a parent transaction triggers a cascade reconstruction of an entire
chain of interdependent transactions by re-uniting the orphans with
chain of interdependent transactions by reuniting the orphans with
their parents all the way down the chain.
Some implementations of the Bitcoin also maintain an UTXO
Some implementations of Bitcoin also maintain a UTXO
database, which is the set of all unspent outputs on the
blockchain. This represents a different set of data from the mempool. Unlike the
mempool and orphan pools, the UTXO database
@ -1204,11 +1204,11 @@ table on persistent storage.
Whereas the mempool and orphan pools represent a single node's local
perspective and might vary significantly from node to node depending
upon when the node was started or restarted, the UTXO database represents
on when the node was started or restarted, the UTXO database represents
the emergent consensus of the network and therefore will not usually
vary between nodes.
Now that we have a understanding of many of the data types and
Now that we have an understanding of many of the data types and
structures used by nodes and clients to send data across the Bitcoin
network, it's time to look at the software that's responsible for
keeping the network secure and operational.

46
ch11_blockchain.adoc
View File

@ -84,7 +84,7 @@ header. <<block_structure1>> describes how Bitcoin Core stores the structure of
|Size| Field | Description
| 4 bytes | Block Size | The size of the block, in bytes, following this field
| 80 bytes | Block Header | Several fields form the block header
| 1-3 bytes (compactSize) | Transaction Counter | How many transactions follow
| 1–3 bytes (compactSize) | Transaction Counter | How many transactions follow
| Variable | Transactions | The transactions recorded in this block
|=======
@ -103,8 +103,8 @@ block metadata as shown in <<block_header_structure_ch09>>.
| 32 bytes | Previous Block Hash | A hash of the previous (parent) block in the chain
| 32 bytes | Merkle Root | The root hash of the merkle tree of this block's transactions
| 4 bytes | Timestamp | The approximate creation time of this block (Unix epoch time)
| 4 bytes | Target | A compact encoding of the Proof-of-Work target for this block
| 4 bytes | Nonce | Arbitrary data used for the Proof-of-Work algorithm
| 4 bytes | Target | A compact encoding of the proof-of-work target for this block
| 4 bytes | Nonce | Arbitrary data used for the proof-of-work algorithm
|=======
The nonce, target, and timestamp are used in the mining
@ -185,7 +185,7 @@ even the single transaction within. Thus, every node has the starting
point for the blockchain, a secure "root" from which to build a trusted
blockchain.
See the statically encoded genesis block inside the Bitcoin Core client,
See the statically encoded genesis block inside the Bitcoin Core client
in https://github.com/bitcoin/bitcoin/blob/3955c3940eff83518c186facfec6f50545b5aab5/src/chainparams.cpp#L123[_chainparams.cpp_].
The following identifier hash belongs to the genesis block:
@ -200,7 +200,7 @@ of this block, with a URL containing that hash:
https://blockstream.info/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
Using the Bitcoin Core reference client on the command line:
Alternatively, you can get the block using Bitcoin Core on the command line:
----
$ bitcoin-cli getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
@ -311,7 +311,7 @@ these trees are usually displayed upside down with the "root" at the top
and the "leaves" at the bottom of a diagram, as you will see in the
examples that follow.
Merkle trees are used in bitcoin to summarize all the transactions in a
Merkle trees are used in Bitcoin to summarize all the transactions in a
block, producing an overall commitment to the entire set of
transactions and permitting a very efficient process to verify whether a
transaction is included in a block. A merkle tree is constructed by
@ -361,7 +361,7 @@ of the nodes.
image::images/mbc3_1102.png["merkle_tree"]
Because the merkle tree is a binary tree, it needs
an even number of leaf nodes. If there is an odd number of transactions
an even number of leaf nodes. If there are an odd number of transactions
to summarize, the last transaction hash will be duplicated to create an
even number of leaf nodes, also known as a _balanced tree_. This is
shown in <<merkle_tree_odd>>, where transaction C is duplicated.
@ -372,10 +372,10 @@ the last hash is duplicated.
.Duplicating one data element achieves an even number of data elements
image::images/mbc3_1103.png["merkle_tree_odd"]
.A design flaw in Bitcoin's merkle tree
.A Design Flaw in Bitcoin's Merkle Tree
****
An extended comment in Bitcoin Core's source code describes a
significant problems in the design of Bitcoin's duplication of odd
An extended comment in Bitcoin Core's source code, reproduced here with slight revisions, describes a
significant problem in the design of Bitcoin's duplication of odd
elements in its merkle tree:
[quote,Bitcoin Core src/consensus/merkle.cpp]
@ -388,14 +388,14 @@ duplicate txids, resulting in a vulnerability (CVE-2012-2459).
The reason is that if the number of hashes in the list at a given level
is odd, the last one is duplicated before computing the next level (which
is unusual in Merkle trees). This results in certain sequences of
transactions leading to the same merkle root. For example, these two
trees:
transactions leading to the same merkle root. For example, the two
trees in <<cve_tree>>:
[[cve_tree]]
.Two Bitcoin-style merkle tree with the same root but a different number of leaves
image::images/mbc3_1104.png["Two Bitcoin-style merkle tree with the same root but a different number of leaves"]
image::images/mbc3_1104.png["Two Bitcoin-style merkle trees with the same root but a different number of leaves"]
For transaction lists [1,2,3,4,5,6] and [1,2,3,4,5,6,5,6] (where 5 and
The transaction lists [1,2,3,4,5,6] and [1,2,3,4,5,6,5,6] (where 5 and
6 are repeated) result in the same root hash A (because the hash of both
of (F) and (F,F) is C).
@ -433,7 +433,7 @@ especially important as the number of transactions increases, because
the base-2 logarithm of the number of transactions increases much more
slowly. This allows Bitcoin nodes to efficiently produce paths of 10 or
12 hashes (320–384 bytes), which can provide proof of a single
transaction out of more than a thousand transactions in a multi-megabyte
transaction out of more than a thousand transactions in a multimegabyte
block.
[[merkle_tree_large]]
@ -457,7 +457,7 @@ image::images/mbc3_1106.png["merkle_tree_path"]
The efficiency of merkle trees becomes obvious as the scale increases.
The largest possible block can hold almost 16,000 transactions in 4,000,000
bytes, but proving any particular one of those sixteen thousand transactions
bytes, but proving any particular one of those 16,000 transactions
is a part of that block only requires a copy of the transaction, a copy
of the 80-byte block header, and 448 bytes for the merkle proof. That
makes the largest possible proof almost 10,000 times smaller than the
@ -576,14 +576,14 @@ as +btcd+ (written in Go) and +bcoin+ (written in JavaScript), to
experiment and learn in other programming languages and frameworks.
Testnet3 supports all the features of mainnet, including
Segregated Witness v0 and v1 (see <<segwit>> and <<taproot>>). Therefore, testnet3 can also be
used to test Segregated Witness features.
segregated witness v0 and v1 (see <<segwit>> and <<taproot>>). Therefore, testnet3 can also be
used to test segregated witness features.
===== Problems With Testnet
===== Problems with testnet
Testnet doesn't just use the same data structures as Bitcoin, it also
uses almost exactly the same Proof-of-Work (PoW) security mechanism as
Bitcoin. The notable differences for testnet are that it's minimum
uses almost exactly the same proof-of-work (PoW) security mechanism as
Bitcoin. The notable differences for testnet are that its minimum
difficulty is half that of Bitcoin and that it's allowed to include a
block at the minimum difficulty if that block's timestamp is more than
20 minutes after the previous block.
@ -630,7 +630,7 @@ BIP325 signets are designed to make it very easy to create your own. If
you disagree with how someone else is running their signet, you can
start your own signet and connect your software to it.
===== The Default Signet and Custom Signets
===== The default signet and custom signets
Bitcoin Core supports a default signet, which we believe to be the most
widely used signet at the time of writing. It is currently operated by
@ -757,7 +757,7 @@ $ bitcoin-cli -regtest getbalance
=== Using Test Blockchains for Development
Bitcoin's various
blockchains (+regtest+, +signet+, +testnet3+, +mainnet+) offer a range
blockchains (regtest, signet, testnet3, mainnet) offer a range
of testing environments for bitcoin development. Use the test
blockchains whether you are developing for Bitcoin Core, or another
full-node consensus client; an application such as a wallet, exchange,