arno/bitcoinbook
1
0
Fork 0
mirror of https://github.com/bitcoinbook/bitcoinbook synced 2024-12-23 15:18:11 +00:00
Code Issues Releases Wiki Activity

CH09: add info about fees

Browse Source 
- New introduction to fees
- More detail about how the fee market works
- Adds RBF and CPFP fee bumping
- Adds transaction pinning
- Adds package relay
- Adds CPFP carve out
- Small edits to 'Adding fees'
- Tiny edits to fee sniping
This commit is contained in:
David A. Harding 2023-05-07 09:14:14 -10:00
parent 02d7401f9d
commit 3ff08d8fff
3 changed files with 666 additions and 174 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
ch02.asciidoc
View File

@ -599,6 +599,7 @@ mine for bitcoin 24 hours a day. The company pays its electricity costs
by selling the bitcoin it is able to generate from mining, creating some
income from the profits.
[[confirmation_score]]
=== Mining Transactions in Blocks
((("blocks", "mining transactions in")))New transactions are constantly

836
chapters/fees.adoc
View File

@ -1,134 +1,669 @@
//FIXME
// - Economic incentives for segwit
// - RBF (opt-in, full)
// - CPFP
// - CPFP carve out
// - Package relay
[[tx_fees]]
==== Transaction Fees
== Transaction Fees
((("transactions", "outputs and inputs", "transaction fees")))((("fees",
"transaction fees")))((("mining and consensus", "rewards and
fees")))Most transactions include transaction fees, which compensate the
bitcoin miners for securing the network. Fees also serve as a security
mechanism themselves, by making it economically infeasible for attackers
to flood the network with transactions. Mining and the fees and rewards
collected by miners are discussed in more detail in <<mining>>.
The digital signature we saw Alice create in <<c_signatures>> only
proves that she knows her private key and that she committed to a
transaction that pays Bob. She can create another signature that
instead commits to a transaction paying Carol--a transaction that spends
the same output (bitcoins) that she used to pay Bob. Those two
transactions are now _conflicting transactions_ because only one
transaction spending a particular output can be included in the valid
blockchain with the most proof of work--the blockchain that full nodes
use to determine which keys control which bitcoins.
This section examines how transaction fees are included in a typical
transaction. Most wallets calculate and include transaction fees
automatically. However, if you are constructing transactions
programmatically, or using a command-line interface, you must manually
account for and include these fees.
To protect himself against conflicting transactions, it would be wise
for Bob to wait until the transaction from Alice is included in the
blockchain before he considers the money he received as his to spend.
He may want to wait even longer than that, as we discussed in
<<confirmation_score>>. For Alice's transaction to be included in the
blockchain, it must be included in a _block_ of transactions. There are
a limited number of blocks produced in a given amount of time and each
block only has a limited amount of space. Only the miner who creates
that block gets to choose which transactions to include. Miners may
select transactions by any criteria they want, including refusing to
include any transactions at all.
Transaction fees serve as an incentive to include (mine) a transaction
into the next block and also as a disincentive against abuse of the
system by imposing a small cost on every transaction. Transaction fees
are collected by the miner who mines the block that records the
transaction on the blockchain.
Transaction fees are calculated based on the size of the transaction in
kilobytes, not the value of the transaction in bitcoin. Overall,
transaction fees are set based on market forces within the Bitcoin
network. Miners prioritize transactions based on many different
criteria, including fees, and might even process transactions for free
under certain circumstances. Transaction fees affect the processing
priority, meaning that a transaction with sufficient fees is likely to
be included in the next block mined, whereas a transaction with
insufficient or no fees might be delayed, processed on a best-effort
basis after a few blocks, or not processed at all. Transaction fees are
not mandatory, and transactions without fees might be processed
eventually; however, including transaction fees encourages priority
processing.
Over time, the way transaction fees are calculated and the effect they
have on transaction prioritization has evolved. At first, transaction
fees were fixed and constant across the network. Gradually, the fee
structure relaxed and may be influenced by market forces, based on
network capacity and transaction volume. Since at least the beginning of
2016, capacity limits in bitcoin have created competition between
transactions, resulting in higher fees and effectively making free
transactions a thing of the past. Zero fee or very low fee transactions
rarely get mined and sometimes will not even be propagated across the
network.
((("fees", "fee relay policies")))((("minrelaytxfee option")))In Bitcoin
Core, fee relay policies are set by the +minrelaytxfee+ option. The
current default +minrelaytxfee+ is 0.00001 bitcoin or a hundredth of a
millibitcoin per kilobyte. Therefore, by default, transactions with a
fee less than 0.00001 bitcoin are treated as free and are only relayed
if there is space in the mempool; otherwise, they are dropped. Bitcoin
nodes can override the default fee relay policy by adjusting the value
of +minrelaytxfee+.
((("dynamic fees")))((("fees", "dynamic fees")))Any bitcoin service that
creates transactions, including wallets, exchanges, retail applications,
etc., _must_ implement dynamic fees. Dynamic fees can be implemented
through a third-party fee estimation service or with a built-in fee
estimation algorithm. If you're unsure, begin with a third-party service
and as you gain experience design and implement your own algorithm if
you wish to remove the third-party dependency.
Fee estimation algorithms calculate the appropriate fee, based on
capacity and the fees offered by "competing" transactions. These
algorithms range from simplistic (average or median fee in the last
block) to sophisticated (statistical analysis). They estimate the
necessary fee (in satoshis per byte) that will give a transaction a high
probability of being selected and included within a certain number of
blocks. Most services offer users the option of choosing high, medium,
or low priority fees. High priority means users pay higher fees but the
transaction is likely to be included in the next block. Medium and low
priority means users pay lower transaction fees but the transactions may
take much longer to confirm.
((("bitcoinfees (third-party service)")))Many wallet applications use
third-party services for fee calculations. One popular service is
http://bitcoinfees.21.co/[_http://bitcoinfees.21.co_], which provides an
API and a visual chart showing the fee in satoshi/byte for different
priorities.
[TIP]
[NOTE]
====
((("static fees")))((("fees", "static fees")))Static fees are no longer
viable on the Bitcoin network. Wallets that set static fees will produce
a poor user experience as transactions will often get "stuck" and remain
unconfirmed. Users who don't understand bitcoin transactions and fees
are dismayed by "stuck" transactions because they think they've lost
their money.
When we say "transactions" in this chapter, we refer to every
transaction in a block except for the first transaction. The first
transaction in a block is a _coinbase transaction_, described in
<<coinbase_transactions>>, which allows the miner of the block to
collect their reward for producing the block. Unlike other
transactions, a coinbase transaction doesn't spend the output of a
previous transaction and is also an exception to several other rules
that apply to other transactions. Coinbase transactions don't pay
transaction fees, don't need to be fee bumped, aren't subject to
transaction pinning, and are largely uninteresting to the following
discussion about fees--so we're going to ignore them in this chapter.
====
The chart in <<bitcoinfees21co>> shows the real-time estimate of fees in
10 satoshi/byte increments and the expected confirmation time (in
minutes and number of blocks) for transactions with fees in each range.
For each fee range (e.g., 61&#x2013;70 satoshi/byte), two horizontal
bars show the number of unconfirmed transactions (1405) and total number
of transactions in the past 24 hours (102,975), with fees in that range.
Based on the graph, the recommended high-priority fee at this time was
80 satoshi/byte, a fee likely to result in the transaction being mined
in the very next block (zero block delay). For perspective, the median
transaction size is 226 bytes, so the recommended fee for a transaction
size would be 18,080 satoshis (0.00018080 BTC).
The criteria that almost all miners use to select which transactions to
include in their blocks is to maximize their revenue. Bitcoin was
specifically designed to accommodate this by providing a mechanism that
allows a transaction to give money to the miner who includes that
transaction in a block. We call that mechanism _transaction fees_,
although it's not a fee in the usual sense of that word. It's not an
amount set by the protocol or by any particular miner---it's much more
like a bid in an auction. The good being purchased is the portion of
limited space in a block that a transaction will consume. Miners choose
the set of transactions whose bids will allow them to earn the greatest
revenue.
The fee estimation data can be retrieved via a simple HTTP REST API, at
https://bitcoinfees.21.co/api/v1/fees/recommended[https://bitcoinfees.21.co/api/v1/fees/recommended].
For example, on the command line using the +curl+ command:
In this chapter, we'll explore various aspects of those
bids--transaction fees--and how they influence the creation and
management of Bitcoin transactions.
=== Bids on space and time
There's a chart taught in most economics courses that shows simplified
supply and demand curves, shown in <<img_supply_demand>>. In these
simple scenarios, increasing aggregate demand or decreasing aggregate
supply results in higher prices; alternatively, decreasing demand or
increasing supply results in lower prices.
[[img_supply_demand]]
.Traditional supply and demand curves
image::../images/supply-demand.png[Traditional supply and demand curves]
It's possible for the supply of block space in Bitcoin to significantly
change, but normally it's very consistent when averaged over the course
of several hours or longer. That means, for the purposes of considering
transaction fees, price is almost entirely a function of aggregate demand,
as shown in <<img_block_supply_demand>>.
[[img_block_supply_demand]]
.Bitcoin block space supply and demand
image::../images/supply-demand-constant.png[Bitcoin block space supply and demand]
When demand is less than supply, the transaction fees should be nearly
zero. Most full nodes which help relay transactions from users to
miners require that every transaction they relay pay a minimum
transaction fee. This prevents their relay bandwidth being wasted.
That minimum fee is the effective floor on transaction fees. Note, the
minimum transaction relay fee must be a fairly small amount or it would
create an incentive for users to begin sending transactions to miners
directly, which would be bad for mining decentralization.
When demand exceeds supply, then the only thing that prevents fee costs
from rising to infinity is that higher prices curtail demand. Each time
the price increases, the number of people who want to pay that price to
include a transaction in a block decreases. Conversely, when prices
decrease, the the number of people willing to pay that price to confirm
a transaction increases. Engineers call this a _negative feedback
loop_, as shown in <<img_negative_feedback>>.
[[img_negative_feedback]]
.Negative feedback loop
image::../images/fee-negative-feedback.png[Negative feedback loop]
In particular, people who want to send a transaction during a period of
high fees will try to find an alternative. There are many alternatives
for most reasons people need to send a transaction, such as moving to an
offchain payment system like Lightning Network, using an alternative
payment network, or taking care to construct the smallest transaction
possible. But it's also possible to take advantage of the negative
feedback loop to pay less fees. All you need to do is create a
transaction now which pays a fee that's equal to or higher than what
miners will accept in the future--even if that price is below what they
accept now. As the market of other people adapts to the higher price by
lowering demand, and as miners confirm all transactions paying a higher
price, the price will drop until miners are willing to accept the
lower fee you offered earlier.
We can imagine four separate transactions that are equivalent except
that each pays a different fee. The transaction paying the highest fee
gets confirmed first. As the high prices result in lower demand, which
subsequently leads to lower prices, the second highest fee transaction
gets confirmed, as does the third. Eventually the lowest fee
transaction gets confirmed. However, the lower the price becomes, the
more demand there is to confirm transactions at that price, so a
transaction paying too low of a fee might never be confirmed. We can
see this scenario illustrated in <<img_wait_times>>.
[[img_wait_times]]
.Wait times for different fee rates
image::../images/fee-patience.png[Wait times for different feerates]
In other words, you don't need to pay peak prices, or switch to another
payment method, if you're willing to wait longer for your transaction to
confirm.
Miners keep a pool of transactions that they will attempt to mine,
either presently or in the near future. Full nodes which relay
unconfirmed transactions will also keep their own pool. In some
Bitcoin implementations these pools are kept in memory and are called
memory pools or _mempools_. The mempools across most nodes tend to be
very similar, so it's common to talk about _the mempool_ as if it was a
single pool synchronized across multiple peers. Although that's
convenient most of the time, anyone evaluating potential problems with
transaction relay (both accidental problems and deliberate attacks)
needs to remember that every relay node keeps its own mempool.
=== Who pays the transaction fee?
Most payment systems involve some sort of fee for transacting, but
often this fee is hidden from typical buyers. For example, a merchant
may advertise the same item for the same price whether you pay with cash
or a credit card even though their payment processor may charge them
a higher fee for credit transactions than their bank charges them for
cash deposits.
In Bitcoin, every spend of bitcoins must be authenticated (typically
with a signature), so it's not possible for a transaction to pay a fee
without the permission of the spender. It is possible for the receiver
of a transaction to pay a fee in a different transaction--and we'll see
that in use later--but if we want a single transaction to pay its own
fee, that fee needs to be something agreed upon by the spender. It
can't be hidden.
Bitcoin transactions are designed so that it doesn't take any extra
space in a transaction for a spender to commit to the fee it pays. That
means that, even though it's possible to pay the fee in a different
transaction, it's most efficient (and thus cheapest) to pay the fee in a
single transaction.
Those technicalities leads us to design Bitcoin so that spenders are
normally responsible for fees.
That's not always what's most convenient for the situation. In Bitcoin,
the fee is a bid and the amount paid contributes to determining how long
it will take the transaction to confirm. Both spenders and receivers of
a payment typically have an interest in having it confirming quickly, so
normally allowing only spenders to choose fees can sometimes be a
problem; we'll look at a solution to that problem in <<cpfp>>. However,
in many common payment flows, the parties with the highest desire to see a
transaction confirm quickly--that is, the parties who'd be the most
willing to pay higher fees--are the the spenders.
For those reasons, both technical and practical, it is customary in
Bitcoin for spenders to pay transaction fees. There are exceptions,
such as for merchants which accept unconfirmed transactions and in
protocols that don't immediately broadcast transactions after they are
signed (preventing the spender from being able to choose an appropriate
fee for the current market).
=== Fees and fee rates
Each transaction only pays a single fee--it doesn't matter how large the
transaction is. However, the larger transactions become, the fewer of
them a miner will be able to fit in a block. For that reason, miners
evaluate transactions the same way you might comparison shop between
several equivalent items at the market: they divide quantity by price.
Whereas you might divide the weight of several different bags of rice by
their cost to find the lowest price per weight (best deal), miners
divide the size of a transaction (also called its weight) by the fee to
find the highest fee per weight (most revenue). In Bitcoin, we use the
term _fee rate_ for a transaction's size divided by weight. Due to
changes in Bitcoin over the years, fee rate can be expressed in
different units:
- Bytes/BTC (a legacy unit rarely used anymore)
- Kilobytes/BTC (a legacy unit rarely used anymore)
- vBytes/BTC (rarely used)
- Kilo-vByte/BTC (used mainly in Bitcoin Core)
- vByte/satoshi (most commonly used today)
- weight/satoshi (also commonly used today)
We recommend either the vByte/sat or weight/sat units for displaying
feerates.
[WARNING]
====
Be careful accepting input for feerates. If a user copy and pastes a
feerate printed in one enumerator into a field using a different
enumerator, they could overpay fees by 1,000 times. If they instead
switch the denominator, they could theoretically overpay by 100,000,000
times. Wallets should make it hard for the user to pay an excessive
feerate and may want to prompt the user to confirm any feerate that was
not generated by the wallet itself using a trusted data source.
An excessive fee, also called an _absurd fee_, is any feerate that's
significantly higher than the amount that feerate estimators currently
expect is necessary to get a transaction confirmed in the next block.
Note that wallets should not entirely prevent users from choosing an
excessive feerate--they should only make using such a feerate hard to do
by accident. There are legitimate reasons for users to overpay fees on
rare occasions.
====
=== Estimating appropriate feerates
We've established that you can pay a lower fee rate if you're willing to
wait longer for your transaction to be confirmed, with the exception
that paying too low of a fee rate could result in your transaction never
confirming. Because fee rates are bids in an open auction for block
space, it's not possible to perfectly predict what fee rate you need to
pay to get your transaction confirmed by a certain time. However, we
can generate a rough estimate based on what fee rates other transactions
have paid in the recent past.
A full node can record three pieces of information about each
transactions it sees: the time (block height) when it first received
that transaction, the block height when that transaction was confirmed,
and the fee rate paid by that transaction. By grouping together
transactions that arrived at similar heights, were confirmed at similar
heights, and which paid similar fees, we can calculate how many blocks it
took to confirm transactions paying a certain fee rate. We can then
assume that a transaction paying a similar fee rate now will take a
similar number of blocks to confirm. Bitcoin Core includes a fee rate
estimator that uses these principles, which can be called using the
`estimatesmartfee` RPC with a parameter specifying how many blocks
you're willing to wait before the transaction is highly likely to
confirm (for example, 144 blocks is about 1 day).
.Using the fee estimation API
----
$ curl https://bitcoinfees.21.co/api/v1/fees/recommended
{"fastestFee":80,"halfHourFee":80,"hourFee":60}
$ bitcoin-cli -named estimatesmartfee conf_target=144
{
"feerate": 0.00006570,
"blocks": 144
}
----
The API returns a JSON object with the current fee estimate for fastest
confirmation (+fastestFee+), confirmation within three blocks
(+halfHourFee+) and six blocks (+hourFee+), in satoshi per byte.
Many web-based services also provide fee estimation as an API. For a
current list, see https://www.lopp.net/bitcoin-information/fee-estimates.html
[[bitcoinfees21co]]
.Fee estimation service bitcoinfees.21.co
image::images/mbc2_0602.png[Fee Estimation Service bitcoinfees.21.co]
As mentioned, fee rate estimation can never be perfect. One common
problem is that the fundamental demand might change, adjusting the
equilibrium and either increasing prices (fees) to new heights or
decreasing them towards the minimum, as show in
<<img_equilibrium_change>>. If fee rates go down, then a transaction
that previously paid a normal fee rate might now be paying a high fee
rate and it will be confirmed earlier than expected. There's no way to
lower the fee rate on a transaction you've already sent, so you're stuck
paying a higher fee rate. But, when fee rates go up, there's a need for
methods to be able to increase the fee rates on those transactions,
which is called _fee bumping_. There are two commonly used types of fee
bumping in Bitcoin, Replace-By-Fee (RBF) and Child Pays For Parent
(CPFP).
==== Adding Fees to Transactions
[[img_equilibrium_change]]
.Equilibrium change
image::../images/equilibrium-change.png[Equilibrium change]
[[rbf]]
=== Replace-By-Fee (RBF) Fee Bumping
To increase the fee of a transaction using RBF fee bumping, you create
a conflicting version of the transaction which pays a higher fee. Two
or more transactions are considered to be _conflicting transactions_ if
only one of them can be included in a valid block chain, forcing a miner
to chose only one of them. Conflicts occur when two or more transaction
both try to spend one of the same UTXOs, i.e. they each include an input
that has the same outpoint (reference to the output of a previous
transaction).
To prevent someone from consuming large amounts of bandwidth by creating
an unlimited number of conflicting transactions and sending them through
the network of relaying full nodes, Bitcoin Core and other full nodes
that support transaction replacement require each replacement
transaction to pay a higher fee rate than the transaction being
replaced. Bitcoin Core also currently requires the replacement
transaction to pay a higher total fee than the original transaction, but
this requirement has undesired side effects and developers have been
looking for ways to remove it at the time of writing.
Bitcoin Core currently supports two variations of RBF:
Opt-in RBF::
An unconfirmed transaction can signal to miners and full nodes that
the creator of the transaction wants to allow it to be replaced by a
higher fee rate version. This signal and the rules for using it
are specified in BIP125. As of this writing, this has been enabled by
default in Bitcoin Core for several years.
Full RBF::
Any unconfirmed transaction can be replaced by a higher fee rate
version. As of this writing, this can be optionally enabled in
Bitcoin Core (but it is disabled by default).
.Why are there two variants of RBF?
****
The reason for the two different versions of RBF is that full RBF has
been controversial. Early versions of Bitcoin allowed transaction
replacement but this behavior was disabled for several releases. During
that time, a miner or full node using the software now called Bitcoin
Core would not replace the first version of an unconfirmed transaction
they received with any different version. Some merchants came to expect
this behavior: they assumed that any valid unconfirmed transaction which
paid an appropriate fee rate would eventually become a confirmed
transaction, so they provided their goods or services shortly after
receiving such an unconfirmed transaction.
However, there's no way for the Bitcoin protocol to guarantee that any
unconfirmed transaction will eventually be confirmed. As mentioned
earlier in this chapter, every miner gets to choose for themselves which
transactions they will try to confirm--including which versions of those
transactions. Bitcoin Core is open source software, so anyone with a
copy of its source code can add (or remove) transaction replacement.
Even if Bitcoin Core wasn't open source, Bitcoin is an open protocol
that can be reimplemented from scratch by a sufficiently competent
programmer, allowing the reimplementor to include or not include
transaction replacement.
Transaction replacement breaks the assumption of some merchants that
every reasonable unconfirmed transaction will eventually be confirmed.
An alternative version of a transaction can pay the same outputs as the
original, but it isn't required to pay any of those outputs. If the
first version of an unconfirmed transaction pays a merchant, the second
version might not pay them. If the merchant provided goods or services
based on the first version, but the second version gets confirmed, then
the merchant will not receive payment for its costs.
Some merchants, and people supporting them, requested that transaction
replacement not be re-enabled in Bitcoin Core. Other people pointed out
that transaction replacement provides benefits, including the ability to
fee bump transactions that initially paid too low of a fee rate.
Eventually, developers working on Bitcoin Core implemented a compromise:
instead of allowing every unconfirmed transaction to be replaced (full
RBF), they only programmed Bitcoin Core to allow replacement of
transactions that signaled they wanted to allow replacement (opt-in RBF).
Merchants can check the transactions they receive for the opt-in
signal and treat those transactions differently than those without the
signal.
This doesn't change the fundamental concern: anyone can still alter
their copy of Bitcoin Core, or create a reimplementation, to allow full
RBF--and some developers even did this, but seemingly few people used
their software.
After several years, developers working on Bitcoin Core changed the
compromise slightly. In addition to keeping opt-in RBF by default, they
added an option that allows users to enable full RBF. If enough mining
hash rate and relaying full nodes enable this option, it will be
possible for any unconfirmed transaction to eventually be replaced by a
version paying a higher fee rate. As of this writing, it's not clear
whether or not that has happened yet.
****
As a user, if you plan to use RBF fee bumping, you will first need to
choose a wallet that supports it, such as one of the wallets listed as
having "Sending support" on
https://bitcoinops.org/en/compatibility/#replace-by-fee-rbf
As a developer, if you plan to implement RBF fee bumping, you will first
need to implement the signaling specified in BIP125, which is a simple
modification to any one of the nSequence fields in a transaction (see
<<nsequence>>).
When you need to fee bump a transaction, you will simply create a new
transaction that spends at least one of the same UTXOs as the original
transaction you want to replace. You will likely want to keep the
same outputs in the transaction which pay the receiver (or receivers).
You may pay the increased fee by reducing the the value of your change
output or by adding additional inputs to the transaction. Developers
should provide users with a fee-bumping interface that does all of this
work for them and simply asks them (or suggests to them) how much the
fee rate should be increased.
[WARNING]
====
Be very careful when creating more than one replacement of the same
transaction. You must ensure than all versions of the transactions
conflict with each other. If they aren't all conflicts, it may be
possible for multiple separate transactions to confirm, leading you to
overpay the receivers. For example:
- Transaction version 0 includes input _A_.
- Transaction version 1 includes inputs _A_ and _B_ (e.g., you had to add
input _B_ to pay the extra fees)
- Transaction version 2 includes inputs B and C (e.g., you had to add input
_C_ to pay the extra fees but _C_ was large enough that you no longer
need input _A_).
In the above scenario, any miner who saved version 0 of the transaction
will be able to confirm both it and version 2 of the transaction. If
both versions pay the same receivers, they'll be paid twice (and the
miner will receive transaction fees from two separate transactions).
A simple method to avoid this problem is to ensure the replacement
transaction always includes all of the same inputs as the previous
version of the transaction.
====
The advantage of RBF fee bumping over other types of fee bumping is that
it can be very efficient at using block space. Often, a replacement
transaction is the same size as the transaction it replaces. Even when
it's larger, it's often the same size as the transaction the user would
have created if they had paid the increased fee rate in the first place.
The fundamental disadvantage of RBF fee bumping is that it can normally
only be performed by the creator of the transaction--the person or
people who were required to provide signatures or other authentication
data for the transaction. An exception to this is transactions which
were designed to allow additional inputs to be added by using sighash
flags, see <<sighash_types>>, but that presents its own challenges. In
general, if you're the receiver of an unconfirmed transaction and you
want to make it confirm faster (or at all), you can't use an RBF fee
bump; you need some other method.
There are additional problems with RBF that we'll explore in the
subsequent <<transaction_pinning>> section.
[[cpfp]]
=== Child Pays For Parent (CPFP) Fee Bumping
Anyone who receives the output of an unconfirmed transaction can
incentivize miners to confirm that transaction by spending that output.
The transaction you want to get confirmed is called the _parent
transaction_. A transaction which spends an output of the parent
transaction is called a _child transaction_.
As we learned in <<outpoints>>, every input in a confirmed transaction
must reference the unspent output of a transaction which appears earlier
in the block chain (whether earlier in the same block or in a previous
block). That means a miner who wants to confirm a child transaction
must also ensure that its parent transaction is confirmed. If the
parent transaction hasn't been confirmed yet but the child transaction
pays a high enough fee, the miner can consider whether it would be
profitable to confirm both of them in the same block.
To evaluate the profitability of mining both a parent and child
transaction, the miner looks at them as a _package of transactions_ with
an aggregate size and aggregate fees, from which the fees can be divided
by the size to calculate a _package fee rate_. The miner can then sort
all of the individual transactions and transaction packages they know
about by fee rate and include the highest-revenue ones in the block
they're attempting to mine, up to the maximum size (weight) allowed to
be included in a block. To find even more packages that might be
profitable to mine, the miner can evaluate packages across multiple
generations (e.g. an unconfirmed parent transaction being combined with
both its child and grandchild). This is called _ancestor fee rate
mining_.
Bitcoin Core has implemented ancestor fee rate mining for many years,
and it's believed that almost all miners use it at the time of writing.
That means it's practical for wallets to use this feature to fee bump an
incoming transaction by using a child transaction to pay for its parent
(CPFP).
CPFP has several advantages over RBF. Anyone who receives an output
from a transaction can use CPFP--that includes both the receivers of
payments and the spender (if the spender included a change output). It
also doesn't require replacing the original transaction, which makes it
less disruptive to some merchants than RBF.
The primary disadvantage of CPFP compared to RBF is that CPFP typically
uses more block space. In RBF, a fee bump transaction is often the same
size as the transaction it replaces. In CPFP, a fee bump is a whole
separate transaction. Using extra block space requires paying extra
fees beyond the the cost of the fee bump.
There are several challenges with CPFP, some of which we'll explore in
the <<transaction_pinning>> section. One other problem which we
specifically need to mention is the minimum relay fee rate problem,
which is addressed by package relay.
==== Package Relay
Early versions of Bitcoin Core didn't place any limits on the size of
each node's mempool. Of course, computers have physical limits, whether
it's the memory (RAM) or disk space--it's not possible for a full node
to store an unlimited number of unconfirmed transactions. Later
versions of Bitcoin Core limited the size of the mempool to hold about
one day worth of transactions, storing only the transactions or packages
with the highest fee rate.
That works extremely well for most things, but it creates a dependency
problem. In order to calculate the fee rate for a transaction package,
we need both the parent and descendant transactions--but if the parent
transaction doesn't pay a high enough fee rate, it won't be kept in a
node's mempool. If a node receives a child transaction without having
access to its parent, it can't do anything with that transaction.
The solution to this problem is the ability to relay transactions as a
package, called _package relay_, allowing the receiving node to evaluate
the fee rate of the entire package before operating on any individual
transaction. As of this writing, developers working on Bitcoin Core
have made significant progress on implementing package relay and a
limited early version of it may be available by the time this book is
published.
Package relay is especially important for protocols based on
time-sensitive presigned transactions, such as Lightning Network. In
non-cooperative cases, some presigned transactions can't be fee bumped
using RBF, forcing them to depend on CPFP. In those protocols, some
transactions may also be created long before they need to be broadcast,
making it effectively impossible to estimate an appropriate fee rate.
If a presigned transaction pays a fee rate below the amount necessary to
get into a node's mempool, there's no way to fee bump it with a child.
If that prevents the transaction from confirming in time, an honest user
might lose money. Package relay is the solution for this critical
problem.
[[transaction_pinning]]
=== Transaction Pinning
Although both RBF and CPFP fee bumping work in the basic cases we
described in the sections about them, there are rules related to both
methods that are designed to prevent denial of service attacks on miners
and relaying full nodes. An unfortunate side effect of those rules
is that they can sometimes prevent someone from being able to use fee
bumping. Making it impossible or difficult to fee bump a transaction is
called _transaction pinning_.
One of the major denial of service concerns revolve around the effect of
transaction relationships. Whenever the output of a transaction is
spent, that transaction's identifier (txid) is referenced by the child
transaction. However, when a transaction is replaced, the replacement
has a different txid. If that replacement transaction gets confirmed,
none of its descendants can be included in the same block chain. It's
possible to recreate and re-sign the descendant transactions, but that's
not guaranteed to happen. This has related but divergent implications
for RBF and CPFP:
- In the context of RBF, when Bitcoin Core accepts a replacement
transaction, it keeps things simple by forgetting about the original
transaction and all descendant transactions which depended on that
original. To ensure that it's more profitable for miners to accept
replacements, Bitcoin Core only accepts a replacement transaction if it
pays more fees than all the transactions that will be forgotten.
+
The downside of this approach is that Alice can create a small
transaction that pays Bob. Bob can then use his output to create a
large child transaction. If Alice then wants to replace her original
transaction, she needs to pay a fee that's larger than what both she and
Bob originally paid. For example, if Alice's original transaction was
about 100 vbytes and Bob's transaction was about 100,000 vbytes, and
they both used the same feerate, Alice now needs to pay more than 1,000
times as much as she originally paid in order to RBF fee bump her
transaction.
- In the context of CPFP, any time the node considers including a
package in a block, it must remove the transactions in that package
from any other package it wants to consider for the same block. For
example, if a child transaction pays for 25 ancestors, and each of
those ancestors has 25 other children, then including the package in
the block requires updating approximately 625 packages (25^2^).
Similarly, if a transaction with 25 descendants is removed from a
node's mempool (such as for being included in a block), and each of
those descendants has 25 other ancestors, another 625 packages need to
be updated. Each time we double our parameter (e.g. from 25 to 50),
we quadruple the amount of work our node needs to perform.
+
Additionally, a transaction and all of its descendants is not
useful to keep in a mempool long term if an alternative version of
that transaction is mined--none of those transactions can now be
confirmed unless there's a rare blockchain reorganization. Bitcoin
Core will remove from its mempool every transaction that can no longer
be confirmed on the current block chain. At it's worst, that can
waste an enormous amount of your node's bandwidth and possibly be used
to prevent transactions from propagating correctly.
+
To prevent these problems, and other related
problems, Bitcoin Core limits a parent transaction to having a maximum
of 25 ancestors or descendants in its mempool pool and limits the
total size of all those transactions to 100,000 vbytes. The downside
of this approach is that users are prevented from creating CPFP fee
bumps if a transaction already has too many descendants (or if it and
its descendants are too large).
Transaction pinning can happen by accident, but it also represents a
serious vulnerability for multiparty time-sensitive protocols such as
Lightning Network. If your counterparty can prevent one of your
transactions from confirming by a deadline, they may be able to steal
money from you.
Protocol developers have been working on mitigating problems with
transaction pinning for several years. One partial solution is
described in <<cpfp_carve_out>>. Several other solutions have been
proposed, and at least one solution is being actively developed as of
this writing (ephemeral anchors, see
https://bitcoinops.org/en/topics/ephemeral-anchors/).
[[cpfp_carve_out]]
=== CPFP Carve Out and Anchor Outputs
In 2018, developers working on Lightning Network (LN) had a problem.
Their protocol uses transactions which require signatures from two
different parties. Neither party wants to trust the other, so they sign
transactions at a point in the protocol when trust isn't needed,
allowing either of them to broadcast one of those transactions at a
later time when the other party may not want to (or be able to) fulfill
its obligations. The problem with this approach is that the
transactions might need to be broadcast far in the future, beyond any
reasonable ability to estimate an appropriate fee rate for the
transactions.
In theory, the developers could have designed their transactions to
allow fee bumping with either RBF (using special sighash flags) or CPFP,
but both of those protocols are vulnerable to transaction pinning.
Given that the involved transactions are time sensitive, allowing a
counterparty to use transaction pinning to delay confirmation of a
transaction can easily lead to an repeatable exploit that malicious
parties could use to steal money from honest parties.
LN developer Matt Corallo proposed a solution: give the rules for CPFP
fee bumping a special exception, called _CPFP carve out_. The normal
rules for CPFP forbid the inclusion of an additional descendant if it
would cause a parent transaction to have 26 or more descendants or if it
would cause a parent and all of its descendants to exceed 100,000 vbytes
in size. Under the rules of CPFP carve out, a single additional
transaction up to 1,000 vbytes in size can be added to a package even if
it would exceed the other limits as long as it is a direct child of an
unconfirmed transaction with no unconfirmed ancestors.
For example, Bob and Mallory both co-sign a transaction with two
outputs, one to each of them. Mallory broadcasts that transaction and
uses her output to attach either 25 child transactions or any smaller
number of child transactions equaling 100,000 vbytes in size. Without
carve-out, Bob would be unable to attach another child transaction to
his output for CPFP fee bumping. With carve-out, he can do that as long
as his child transaction is less than 1,000 vbytes in size (which should
be more than enough space).
It's not allowed to use CPFP carve-out more than once, so it only works
for two-party protocols. There have been proposals to extend it to
protocols involving more participants, but there hasn't been much demand
for that and developers are focused on building more generic solutions
to transaction pinning attacks.
As of this writing, most popular LN implementations use a transaction
template called _anchor outputs_, which is designed to be used with CPFP
carve out.
=== Adding Fees to Transactions
The data structure of transactions does not have a field for fees.
Instead, fees are implied as the difference between the sum of inputs
@ -164,55 +699,8 @@ the change as a transaction fee. "Keep the change!" might not be what
you intended.
====
((("use cases", "buying coffee")))Let's see how this works in practice,
by looking at Alice's coffee purchase again. Alice wants to spend 0.015
bitcoin to pay for coffee. To ensure this transaction is processed
promptly, she will want to include a transaction fee, say 0.001. That
will mean that the total cost of the transaction will be 0.016. Her
wallet must therefore source a set of UTXO that adds up to 0.016 bitcoin
or more and, if necessary, create change. Let's say her wallet has a
0.2-bitcoin UTXO available. It will therefore need to consume this UTXO,
create one output to Bob's Cafe for 0.015, and a second output with
0.184 bitcoin in change back to her own wallet, leaving 0.001 bitcoin
unallocated, as an implicit fee for the transaction.
((("use cases", "charitable donations")))((("charitable donations")))Now
let's look at a different scenario. Eugenia, our children's charity
director in the Philippines, has completed a fundraiser to purchase
schoolbooks for the children. She received several thousand small
donations from people all around the world, totaling 50 bitcoin, so her
wallet is full of very small payments (UTXO). Now she wants to purchase
hundreds of schoolbooks from a local publisher, paying in bitcoin.
As Eugenia's wallet application tries to construct a single larger
payment transaction, it must source from the available UTXO set, which
is composed of many smaller amounts. That means that the resulting
transaction will source from more than a hundred small-value UTXO as
inputs and only one output, paying the book publisher. A transaction
with that many inputs will be larger than one kilobyte, perhaps several
kilobytes in size. As a result, it will require a much higher fee than
the median-sized transaction.
Eugenia's wallet application will calculate the appropriate fee by
measuring the size of the transaction and multiplying that by the
per-kilobyte fee. Many wallets will overpay fees for larger transactions
to ensure the transaction is processed promptly. The higher fee is not
because Eugenia is spending more money, but because her transaction is
more complex and larger in size--the fee is independent of the
transaction's bitcoin value.((("", startref="Tout06")))
[[transaction-pinning]]
=== Transaction Pinning
FIXME
[[rbf]]
=== Replace by Fee (RBF)
FIXME
[[fee_sniping]]
==== Timelock Defense Against Fee Sniping
=== Timelock Defense Against Fee Sniping
((("scripting", "timelocks", "defense against
fee-sniping")))((("timelocks", "defense against
@ -234,14 +722,15 @@ They can include any transactions that were in the "old" block
Essentially they have the option to pull transactions from the "present"
into the rewritten "past" when they re-create block #100,000.
Today, this attack is not very lucrative, because block reward is much
Today, this attack is not very lucrative, because the block subsidy is much
higher than total fees per block. But at some point in the future,
transaction fees will be the majority of the reward (or even the
entirety of the reward). At that time, this scenario becomes inevitable.
To prevent "fee sniping," when Bitcoin Core creates transactions, it
uses +nLocktime+ to limit them to the "next block," by default. In our
scenario, Bitcoin Core would set +nLocktime+ to 100,001 on any
Several wallets discourage fee sniping by creating transactions with an
+nLocktime+ that limits those transactions to being included in the next
block or any later block. In our
scenario, our wallet would set +nLocktime+ to 100,001 on any
transaction it created. Under normal circumstances, this +nLocktime+ has
no effect&#x2014;the transactions could only be included in block
#100,001 anyway; it's the next block.
@ -252,7 +741,8 @@ would be timelocked to block #100,001. They can only remine #100,000
with whatever transactions were valid at that time, essentially gaining
no new fees.
To achieve this, Bitcoin Core sets the +nLocktime+ on all new
transactions to <current block # + 1> and sets the +nSequence+ on all
the inputs to 0xFFFFFFFE to enable +nLocktime+.((("",
startref="Stimelock07")))
This does not entirely prevent fee sniping, but it does make it less
profitable in some cases and so can help preserve the stability of the
Bitcoin network as the block subsidy declines. We recommend all wallets
implement anti fee sniping when it doesn't interfere with the wallet's
other uses of the nLockTime field.

3
chapters/transactions.adoc
View File

@ -141,7 +141,7 @@ being widely considered. That proposal does not seek to change the
consensus rules but only the policy that Bitcoin full nodes use to relay
transactions. Under the proposal, version 3 transactions would be
subject to additional constraints in order to prevent certain Denial of
Service (DoS) attacks that we'll discuss further in <<transaction-pinning>>
Service (DoS) attacks that we'll discuss further in <<transaction_pinning>>
=== Extended Marker and Flag
@ -372,6 +372,7 @@ byte (0x6b) indicating the scriptSig is 107 bytes. We'll cover parsing
and using scripts in detail in the next chapter,
<<c_authorization_authentication>>.
[[nsequence]]
==== nSequence
The final four bytes of an input are its sequence number, called