|
|
[[ch06]]
|
|
|
|
|
|
[[tx_fees]]
|
|
|
==== Transaction Fees
|
|
|
|
|
|
((("transactions", "outputs and inputs", "transaction fees")))((("fees",
|
|
|
"transaction fees")))((("mining and consensus", "rewards and
|
|
|
fees")))Most transactions include transaction fees, which compensate the
|
|
|
bitcoin miners for securing the network. Fees also serve as a security
|
|
|
mechanism themselves, by making it economically infeasible for attackers
|
|
|
to flood the network with transactions. Mining and the fees and rewards
|
|
|
collected by miners are discussed in more detail in <<mining>>.
|
|
|
|
|
|
This section examines how transaction fees are included in a typical
|
|
|
transaction. Most wallets calculate and include transaction fees
|
|
|
automatically. However, if you are constructing transactions
|
|
|
programmatically, or using a command-line interface, you must manually
|
|
|
account for and include these fees.
|
|
|
|
|
|
Transaction fees serve as an incentive to include (mine) a transaction
|
|
|
into the next block and also as a disincentive against abuse of the
|
|
|
system by imposing a small cost on every transaction. Transaction fees
|
|
|
are collected by the miner who mines the block that records the
|
|
|
transaction on the blockchain.
|
|
|
|
|
|
Transaction fees are calculated based on the size of the transaction in
|
|
|
kilobytes, not the value of the transaction in bitcoin. Overall,
|
|
|
transaction fees are set based on market forces within the Bitcoin
|
|
|
network. Miners prioritize transactions based on many different
|
|
|
criteria, including fees, and might even process transactions for free
|
|
|
under certain circumstances. Transaction fees affect the processing
|
|
|
priority, meaning that a transaction with sufficient fees is likely to
|
|
|
be included in the next block mined, whereas a transaction with
|
|
|
insufficient or no fees might be delayed, processed on a best-effort
|
|
|
basis after a few blocks, or not processed at all. Transaction fees are
|
|
|
not mandatory, and transactions without fees might be processed
|
|
|
eventually; however, including transaction fees encourages priority
|
|
|
processing.
|
|
|
|
|
|
Over time, the way transaction fees are calculated and the effect they
|
|
|
have on transaction prioritization has evolved. At first, transaction
|
|
|
fees were fixed and constant across the network. Gradually, the fee
|
|
|
structure relaxed and may be influenced by market forces, based on
|
|
|
network capacity and transaction volume. Since at least the beginning of
|
|
|
2016, capacity limits in bitcoin have created competition between
|
|
|
transactions, resulting in higher fees and effectively making free
|
|
|
transactions a thing of the past. Zero fee or very low fee transactions
|
|
|
rarely get mined and sometimes will not even be propagated across the
|
|
|
network.
|
|
|
|
|
|
((("fees", "fee relay policies")))((("minrelaytxfee option")))In Bitcoin
|
|
|
Core, fee relay policies are set by the +minrelaytxfee+ option. The
|
|
|
current default +minrelaytxfee+ is 0.00001 bitcoin or a hundredth of a
|
|
|
millibitcoin per kilobyte. Therefore, by default, transactions with a
|
|
|
fee less than 0.00001 bitcoin are treated as free and are only relayed
|
|
|
if there is space in the mempool; otherwise, they are dropped. Bitcoin
|
|
|
nodes can override the default fee relay policy by adjusting the value
|
|
|
of +minrelaytxfee+.
|
|
|
|
|
|
((("dynamic fees")))((("fees", "dynamic fees")))Any bitcoin service that
|
|
|
creates transactions, including wallets, exchanges, retail applications,
|
|
|
etc., _must_ implement dynamic fees. Dynamic fees can be implemented
|
|
|
through a third-party fee estimation service or with a built-in fee
|
|
|
estimation algorithm. If you're unsure, begin with a third-party service
|
|
|
and as you gain experience design and implement your own algorithm if
|
|
|
you wish to remove the third-party dependency.
|
|
|
|
|
|
Fee estimation algorithms calculate the appropriate fee, based on
|
|
|
capacity and the fees offered by "competing" transactions. These
|
|
|
algorithms range from simplistic (average or median fee in the last
|
|
|
block) to sophisticated (statistical analysis). They estimate the
|
|
|
necessary fee (in satoshis per byte) that will give a transaction a high
|
|
|
probability of being selected and included within a certain number of
|
|
|
blocks. Most services offer users the option of choosing high, medium,
|
|
|
or low priority fees. High priority means users pay higher fees but the
|
|
|
transaction is likely to be included in the next block. Medium and low
|
|
|
priority means users pay lower transaction fees but the transactions may
|
|
|
take much longer to confirm.
|
|
|
|
|
|
((("bitcoinfees (third-party service)")))Many wallet applications use
|
|
|
third-party services for fee calculations. One popular service is
|
|
|
http://bitcoinfees.21.co/[_http://bitcoinfees.21.co_], which provides an
|
|
|
API and a visual chart showing the fee in satoshi/byte for different
|
|
|
priorities.
|
|
|
|
|
|
[TIP]
|
|
|
====
|
|
|
((("static fees")))((("fees", "static fees")))Static fees are no longer
|
|
|
viable on the Bitcoin network. Wallets that set static fees will produce
|
|
|
a poor user experience as transactions will often get "stuck" and remain
|
|
|
unconfirmed. Users who don't understand bitcoin transactions and fees
|
|
|
are dismayed by "stuck" transactions because they think they've lost
|
|
|
their money.
|
|
|
====
|
|
|
|
|
|
The chart in <<bitcoinfees21co>> shows the real-time estimate of fees in
|
|
|
10 satoshi/byte increments and the expected confirmation time (in
|
|
|
minutes and number of blocks) for transactions with fees in each range.
|
|
|
For each fee range (e.g., 61–70 satoshi/byte), two horizontal
|
|
|
bars show the number of unconfirmed transactions (1405) and total number
|
|
|
of transactions in the past 24 hours (102,975), with fees in that range.
|
|
|
Based on the graph, the recommended high-priority fee at this time was
|
|
|
80 satoshi/byte, a fee likely to result in the transaction being mined
|
|
|
in the very next block (zero block delay). For perspective, the median
|
|
|
transaction size is 226 bytes, so the recommended fee for a transaction
|
|
|
size would be 18,080 satoshis (0.00018080 BTC).
|
|
|
|
|
|
The fee estimation data can be retrieved via a simple HTTP REST API, at
|
|
|
https://bitcoinfees.21.co/api/v1/fees/recommended[https://bitcoinfees.21.co/api/v1/fees/recommended].
|
|
|
For example, on the command line using the +curl+ command:
|
|
|
|
|
|
.Using the fee estimation API
|
|
|
----
|
|
|
$ curl https://bitcoinfees.21.co/api/v1/fees/recommended
|
|
|
|
|
|
{"fastestFee":80,"halfHourFee":80,"hourFee":60}
|
|
|
----
|
|
|
|
|
|
The API returns a JSON object with the current fee estimate for fastest
|
|
|
confirmation (+fastestFee+), confirmation within three blocks
|
|
|
(+halfHourFee+) and six blocks (+hourFee+), in satoshi per byte.
|
|
|
|
|
|
[[bitcoinfees21co]]
|
|
|
.Fee estimation service bitcoinfees.21.co
|
|
|
image::images/mbc2_0602.png[Fee Estimation Service bitcoinfees.21.co]
|
|
|
|
|
|
==== Adding Fees to Transactions
|
|
|
|
|
|
The data structure of transactions does not have a field for fees.
|
|
|
Instead, fees are implied as the difference between the sum of inputs
|
|
|
and the sum of outputs. Any excess amount that remains after all outputs
|
|
|
have been deducted from all inputs is the fee that is collected by the
|
|
|
miners:
|
|
|
|
|
|
[[tx_fee_equation]]
|
|
|
.Transaction fees are implied, as the excess of inputs minus outputs:
|
|
|
----
|
|
|
Fees = Sum(Inputs) – Sum(Outputs)
|
|
|
----
|
|
|
|
|
|
This is a somewhat confusing element of transactions and an important
|
|
|
point to understand, because if you are constructing your own
|
|
|
transactions you must ensure you do not inadvertently include a very
|
|
|
large fee by underspending the inputs. That means that you must account
|
|
|
for all inputs, if necessary by creating change, or you will end up
|
|
|
giving the miners a very big tip!
|
|
|
|
|
|
For example, if you consume a 20-bitcoin UTXO to make a 1-bitcoin
|
|
|
payment, you must include a 19-bitcoin change output back to your
|
|
|
wallet. Otherwise, the 19-bitcoin "leftover" will be counted as a
|
|
|
transaction fee and will be collected by the miner who mines your
|
|
|
transaction in a block. Although you will receive priority processing
|
|
|
and make a miner very happy, this is probably not what you intended.
|
|
|
|
|
|
[WARNING]
|
|
|
====
|
|
|
((("warnings and cautions", "change outputs")))If you forget to add a
|
|
|
change output in a manually constructed transaction, you will be paying
|
|
|
the change as a transaction fee. "Keep the change!" might not be what
|
|
|
you intended.
|
|
|
====
|
|
|
|
|
|
((("use cases", "buying coffee")))Let's see how this works in practice,
|
|
|
by looking at Alice's coffee purchase again. Alice wants to spend 0.015
|
|
|
bitcoin to pay for coffee. To ensure this transaction is processed
|
|
|
promptly, she will want to include a transaction fee, say 0.001. That
|
|
|
will mean that the total cost of the transaction will be 0.016. Her
|
|
|
wallet must therefore source a set of UTXO that adds up to 0.016 bitcoin
|
|
|
or more and, if necessary, create change. Let's say her wallet has a
|
|
|
0.2-bitcoin UTXO available. It will therefore need to consume this UTXO,
|
|
|
create one output to Bob's Cafe for 0.015, and a second output with
|
|
|
0.184 bitcoin in change back to her own wallet, leaving 0.001 bitcoin
|
|
|
unallocated, as an implicit fee for the transaction.
|
|
|
|
|
|
((("use cases", "charitable donations")))((("charitable donations")))Now
|
|
|
let's look at a different scenario. Eugenia, our children's charity
|
|
|
director in the Philippines, has completed a fundraiser to purchase
|
|
|
schoolbooks for the children. She received several thousand small
|
|
|
donations from people all around the world, totaling 50 bitcoin, so her
|
|
|
wallet is full of very small payments (UTXO). Now she wants to purchase
|
|
|
hundreds of schoolbooks from a local publisher, paying in bitcoin.
|
|
|
|
|
|
As Eugenia's wallet application tries to construct a single larger
|
|
|
payment transaction, it must source from the available UTXO set, which
|
|
|
is composed of many smaller amounts. That means that the resulting
|
|
|
transaction will source from more than a hundred small-value UTXO as
|
|
|
inputs and only one output, paying the book publisher. A transaction
|
|
|
with that many inputs will be larger than one kilobyte, perhaps several
|
|
|
kilobytes in size. As a result, it will require a much higher fee than
|
|
|
the median-sized transaction.
|
|
|
|
|
|
Eugenia's wallet application will calculate the appropriate fee by
|
|
|
measuring the size of the transaction and multiplying that by the
|
|
|
per-kilobyte fee. Many wallets will overpay fees for larger transactions
|
|
|
to ensure the transaction is processed promptly. The higher fee is not
|
|
|
because Eugenia is spending more money, but because her transaction is
|
|
|
more complex and larger in size--the fee is independent of the
|
|
|
transaction's bitcoin value.((("", startref="Tout06")))
|
|
|
|
|
|
[[tx_script]]
|
|
|
[role="pagebreak-before less_space_h1"]
|
|
|
=== Transaction Scripts and Script Language
|
|
|
|
|
|
((("transactions", "scripts and Script language",
|
|
|
id="Tsript06")))((("scripting", "transactions and",
|
|
|
id="Stransact06")))The bitcoin transaction script language, called
|
|
|
_Script_, is a Forth-like reverse-polish notation stack-based execution
|
|
|
language. If that sounds like gibberish, you probably haven't studied
|
|
|
1960s programming languages, but that's ok—we will explain it all
|
|
|
in this chapter. Both the locking script placed on an UTXO and the
|
|
|
unlocking script are written in this scripting language. When a
|
|
|
transaction is validated, the unlocking script in each input is executed
|
|
|
alongside the corresponding locking script to see if it satisfies the
|
|
|
spending condition.
|
|
|
|
|
|
Script is a very simple language that was designed to be limited in
|
|
|
scope and executable on a range of hardware, perhaps as simple as an
|
|
|
embedded device. It requires minimal processing and cannot do many of
|
|
|
the fancy things modern programming languages can do. For its use in
|
|
|
validating programmable money, this is a deliberate security feature.
|
|
|
|
|
|
((("Pay-to-Public-Key-Hash (P2PKH)")))Today, most transactions processed
|
|
|
through the Bitcoin network have the form "Payment to Bob's Bitcoin
|
|
|
address" and are based on a script called a Pay-to-Public-Key-Hash
|
|
|
script. However, bitcoin transactions are not limited to the "Payment
|
|
|
to Bob's Bitcoin address" script. In fact, locking scripts can be
|
|
|
written to express a vast variety of complex conditions. In order to
|
|
|
understand these more complex scripts, we must first understand the
|
|
|
basics of transaction scripts and script language.
|
|
|
|
|
|
In this section, we will demonstrate the basic components of the bitcoin
|
|
|
transaction scripting language and show how it can be used to express
|
|
|
simple conditions for spending and how those conditions can be satisfied
|
|
|
by unlocking scripts.
|
|
|
|
|
|
[TIP]
|
|
|
====
|
|
|
((("programmable money")))Bitcoin transaction validation is not based on
|
|
|
a static pattern, but instead is achieved through the execution of a
|
|
|
scripting language. This language allows for a nearly infinite variety
|
|
|
of conditions to be expressed. This is how bitcoin gets the power of
|
|
|
"programmable money."
|
|
|
====
|
|
|
|
|
|
==== Turing Incompleteness
|
|
|
|
|
|
((("Turing incompleteness")))The bitcoin transaction script language
|
|
|
contains many operators, but is deliberately limited in one important
|
|
|
way--there are no loops or complex flow control capabilities other than
|
|
|
conditional flow control. This ensures that the language is not _Turing
|
|
|
Complete_, meaning that scripts have limited complexity and predictable
|
|
|
execution times. Script is not a general-purpose language.
|
|
|
((("denial-of-service attacks")))((("denial-of-service attacks",
|
|
|
see="also security")))((("security", "denial-of-service attacks")))These
|
|
|
limitations ensure that the language cannot be used to create an
|
|
|
infinite loop or other form of "logic bomb" that could be embedded in a
|
|
|
transaction in a way that causes a denial-of-service attack against the
|
|
|
Bitcoin network. Remember, every transaction is validated by every full
|
|
|
node on the Bitcoin network. A limited language prevents the transaction
|
|
|
validation mechanism from being used as a vulnerability.
|
|
|
|
|
|
==== Stateless Verification
|
|
|
|
|
|
((("stateless verification")))The bitcoin transaction script language is
|
|
|
stateless, in that there is no state prior to execution of the script,
|
|
|
or state saved after execution of the script. Therefore, all the
|
|
|
information needed to execute a script is contained within the script. A
|
|
|
script will predictably execute the same way on any system. If your
|
|
|
system verifies a script, you can be sure that every other system in the
|
|
|
Bitcoin network will also verify the script, meaning that a valid
|
|
|
transaction is valid for everyone and everyone knows this. This
|
|
|
predictability of outcomes is an essential benefit of the Bitcoin
|
|
|
system.
|
|
|
|
|
|
[[tx_lock_unlock]]
|
|
|
==== Script Construction (Lock + Unlock)
|
|
|
|
|
|
Bitcoin's transaction validation engine relies on two types of scripts
|
|
|
to validate transactions: a locking script and an unlocking script.
|
|
|
|
|
|
((("locking scripts")))((("unlocking scripts")))((("scripting", "locking
|
|
|
scripts")))A locking script is a spending condition placed on an output:
|
|
|
it specifies the conditions that must be met to spend the output in the
|
|
|
future. ((("scriptPubKey")))Historically, the locking script was called
|
|
|
a _scriptPubKey_, because it usually contained a public key or Bitcoin
|
|
|
address (public key hash). In this book we refer to it as a "locking
|
|
|
script" to acknowledge the much broader range of possibilities of this
|
|
|
scripting technology. In most bitcoin applications, what we refer to as
|
|
|
a locking script will appear in the source code as +scriptPubKey+.
|
|
|
((("witnesses")))((("cryptographic puzzles")))You will also see the
|
|
|
locking script referred to as a _witness script_ (see <<segwit>>) or
|
|
|
more generally as a _cryptographic puzzle_. These terms all mean the
|
|
|
same thing, at different levels of abstraction.
|
|
|
|
|
|
An unlocking script is a script that "solves," or satisfies, the
|
|
|
conditions placed on an output by a locking script and allows the output
|
|
|
to be spent. Unlocking scripts are part of every transaction input. Most
|
|
|
of the time they contain a digital signature produced by the user's
|
|
|
wallet from his or her private key. ((("scriptSig")))Historically, the
|
|
|
unlocking script was called _scriptSig_, because it usually contained a
|
|
|
digital signature. In most bitcoin applications, the source code refers
|
|
|
to the unlocking script as +scriptSig+. You will also see the unlocking
|
|
|
script referred to as a _witness_ (see <<segwit>>). In this book, we
|
|
|
refer to it as an "unlocking script" to acknowledge the much broader
|
|
|
range of locking script requirements, because not all unlocking scripts
|
|
|
must contain signatures.
|
|
|
|
|
|
Every bitcoin validating node will validate transactions by executing
|
|
|
the locking and unlocking scripts together. Each input contains an
|
|
|
unlocking script and refers to a previously existing UTXO. The
|
|
|
validation software will copy the unlocking script, retrieve the UTXO
|
|
|
referenced by the input, and copy the locking script from that UTXO. The
|
|
|
unlocking and locking script are then executed in sequence. The input is
|
|
|
valid if the unlocking script satisfies the locking script conditions
|
|
|
(see <<script_exec>>). All the inputs are validated independently, as
|
|
|
part of the overall validation of the transaction.
|
|
|
|
|
|
Note that the UTXO is permanently recorded in the blockchain, and
|
|
|
therefore is invariable and is unaffected by failed attempts to spend it
|
|
|
by reference in a new transaction. Only a valid transaction that
|
|
|
correctly satisfies the conditions of the output results in the output
|
|
|
being considered as "spent" and removed from the set of unspent
|
|
|
transaction outputs (UTXO set).
|
|
|
|
|
|
<<scriptSig_and_scriptPubKey>> is an example of the unlocking and
|
|
|
locking scripts for the most common type of bitcoin transaction (a
|
|
|
payment to a public key hash), showing the combined script resulting
|
|
|
from the concatenation of the unlocking and locking scripts prior to
|
|
|
script validation.
|
|
|
|
|
|
[[scriptSig_and_scriptPubKey]]
|
|
|
.Combining scriptSig and scriptPubKey to evaluate a transaction script
|
|
|
image::images/mbc2_0603.png["scriptSig_and_scriptPubKey"]
|
|
|
|
|
|
===== The script execution stack
|
|
|
|
|
|
Bitcoin's scripting language is called a stack-based language because it
|
|
|
uses a data structure called a _stack_. A stack is a very simple data
|
|
|
structure that can be visualized as a stack of cards. A stack allows two
|
|
|
operations: push and pop. Push adds an item on top of the stack. Pop
|
|
|
removes the top item from the stack. Operations on a stack can only act
|
|
|
on the topmost item on the stack. A stack data structure is also called
|
|
|
a Last-In-First-Out, or "LIFO" queue.
|
|
|
|
|
|
The scripting language executes the script by processing each item from
|
|
|
left to right. Numbers (data constants) are pushed onto the stack.
|
|
|
Operators push or pop one or more parameters from the stack, act on
|
|
|
them, and might push a result onto the stack. For example, +OP_ADD+ will
|
|
|
pop two items from the stack, add them, and push the resulting sum onto
|
|
|
the stack.
|
|
|
|
|
|
Conditional operators evaluate a condition, producing a boolean result
|
|
|
of TRUE or FALSE. For example, +OP_EQUAL+ pops two items from the stack
|
|
|
and pushes TRUE (TRUE is represented by the number 1) if they are equal
|
|
|
or FALSE (represented by zero) if they are not equal. Bitcoin
|
|
|
transaction scripts usually contain a conditional operator, so that they
|
|
|
can produce the TRUE result that signifies a valid transaction.
|
|
|
|
|
|
===== A simple script
|
|
|
|
|
|
Now let's apply what we've learned about scripts and stacks to some simple examples.
|
|
|
|
|
|
In <<simplemath_script>>, the script +2 3 OP_ADD 5 OP_EQUAL+
|
|
|
demonstrates the arithmetic addition operator +OP_ADD+, adding two
|
|
|
numbers and putting the result on the stack, followed by the conditional
|
|
|
operator +OP_EQUAL+, which checks that the resulting sum is equal to
|
|
|
+5+. For brevity, the +OP_+ prefix is omitted in the step-by-step
|
|
|
example. For more details on the available script operators and
|
|
|
functions, see <<tx_script_ops>>.
|
|
|
|
|
|
Although most locking scripts refer to a public key hash (essentially, a
|
|
|
Bitcoin address), thereby requiring proof of ownership to spend the
|
|
|
funds, the script does not have to be that complex. Any combination of
|
|
|
locking and unlocking scripts that results in a TRUE value is valid. The
|
|
|
simple arithmetic we used as an example of the scripting language is
|
|
|
also a valid locking script that can be used to lock a transaction
|
|
|
output.
|
|
|
|
|
|
Use part of the arithmetic example script as the locking script:
|
|
|
|
|
|
----
|
|
|
3 OP_ADD 5 OP_EQUAL
|
|
|
----
|
|
|
|
|
|
which can be satisfied by a transaction containing an input with the
|
|
|
unlocking script:
|
|
|
|
|
|
----
|
|
|
2
|
|
|
----
|
|
|
|
|
|
The validation software combines the locking and unlocking scripts and
|
|
|
the resulting script is:
|
|
|
|
|
|
----
|
|
|
2 3 OP_ADD 5 OP_EQUAL
|
|
|
----
|
|
|
|
|
|
As we saw in the step-by-step example in <<simplemath_script>>, when
|
|
|
this script is executed, the result is +OP_TRUE+, making the transaction
|
|
|
valid. Not only is this a valid transaction output locking script, but
|
|
|
the resulting UTXO could be spent by anyone with the arithmetic skills
|
|
|
to know that the number 2 satisfies the script.
|
|
|
|
|
|
[TIP]
|
|
|
====
|
|
|
((("transactions", "valid and invalid")))Transactions are valid if the
|
|
|
top result on the stack is +TRUE+ (noted as ++{0x01}++), any
|
|
|
other nonzero value, or if the stack is empty after script execution.
|
|
|
Transactions are invalid if the top value on the stack is +FALSE+ (a
|
|
|
zero-length empty value, noted as ++{}++) or if script
|
|
|
execution is halted explicitly by an operator, such as +OP_VERIFY+,
|
|
|
+OP_RETURN+, or a conditional terminator such as +OP_ENDIF+. See
|
|
|
<<tx_script_ops>> for details.
|
|
|
====
|
|
|
|
|
|
[[simplemath_script]]
|
|
|
.Bitcoin's script validation doing simple math
|
|
|
image::images/mbc2_0604.png["TxScriptSimpleMathExample"]
|
|
|
|
|
|
[role="pagebreak-before"]
|
|
|
The following is a slightly more complex script, which calculates ++2 +
|
|
|
7 -- 3 + 1++. Notice that when the script contains several operators in
|
|
|
a row, the stack allows the results of one operator to be acted upon by
|
|
|
the next operator:
|
|
|
|
|
|
----
|
|
|
2 7 OP_ADD 3 OP_SUB 1 OP_ADD 7 OP_EQUAL
|
|
|
----
|
|
|
|
|
|
Try validating the preceding script yourself using pencil and paper.
|
|
|
When the script execution ends, you should be left with the value +TRUE+
|
|
|
on the stack.
|
|
|
|
|
|
[[script_exec]]
|
|
|
===== Separate execution of unlocking and locking scripts
|
|
|
|
|
|
((("security", "locking and unlocking scripts")))In the original Bitcoin
|
|
|
client, the unlocking and locking scripts were concatenated and executed
|
|
|
in sequence. For security reasons, this was changed in 2010, because of
|
|
|
a vulnerability that allowed a malformed unlocking script to push data
|
|
|
onto the stack and corrupt the locking script. In the current
|
|
|
implementation, the scripts are executed separately with the stack
|
|
|
transferred between the two executions, as described next.
|
|
|
|
|
|
First, the unlocking script is executed, using the stack execution
|
|
|
engine. If the unlocking script is executed without errors (e.g., it has
|
|
|
no "dangling" operators left over), the main stack is copied and the
|
|
|
locking script is executed. If the result of executing the locking
|
|
|
script with the stack data copied from the unlocking script is "TRUE,"
|
|
|
the unlocking script has succeeded in resolving the conditions imposed
|
|
|
by the locking script and, therefore, the input is a valid authorization
|
|
|
to spend the UTXO. If any result other than "TRUE" remains after
|
|
|
execution of the combined script, the input is invalid because it has
|
|
|
failed to satisfy the spending conditions placed on the UTXO.
|
|
|
|
|
|
|
|
|
[[p2pkh]]
|
|
|
==== Pay-to-Public-Key-Hash (P2PKH)
|
|
|
|
|
|
((("Pay-to-Public-Key-Hash (P2PKH)")))The vast majority of transactions
|
|
|
processed on the Bitcoin network spend outputs locked with a
|
|
|
Pay-to-Public-Key-Hash or "P2PKH" script. These outputs contain a
|
|
|
locking script that locks the output to a public key hash, more commonly
|
|
|
known as a Bitcoin address. An output locked by a P2PKH script can be
|
|
|
unlocked (spent) by presenting a public key and a digital signature
|
|
|
created by the corresponding private key (see <<digital_sigs>>).
|
|
|
|
|
|
((("use cases", "buying coffee")))For example, let's look at Alice's
|
|
|
payment to Bob's Cafe again. Alice made a payment of 0.015 bitcoin to
|
|
|
the cafe's Bitcoin address. That transaction output would have a locking
|
|
|
script of the form:
|
|
|
|
|
|
----
|
|
|
OP_DUP OP_HASH160 <Cafe Public Key Hash> OP_EQUALVERIFY OP_CHECKSIG
|
|
|
----
|
|
|
|
|
|
The +Cafe Public Key Hash+ is equivalent to the Bitcoin address of the
|
|
|
cafe, without the Base58Check encoding. Most applications would show the
|
|
|
_public key hash_ in hexadecimal encoding and not the familiar Bitcoin
|
|
|
address Base58Check format that begins with a "1."
|
|
|
|
|
|
The preceding locking script can be satisfied with an unlocking script
|
|
|
of the form:
|
|
|
|
|
|
----
|
|
|
<Cafe Signature> <Cafe Public Key>
|
|
|
----
|
|
|
|
|
|
The two scripts together would form the following combined validation
|
|
|
script:
|
|
|
|
|
|
----
|
|
|
<Cafe Signature> <Cafe Public Key> OP_DUP OP_HASH160
|
|
|
<Cafe Public Key Hash> OP_EQUALVERIFY OP_CHECKSIG
|
|
|
----
|
|
|
|
|
|
When executed, this combined script will evaluate to TRUE if, and only
|
|
|
if, the unlocking script matches the conditions set by the locking
|
|
|
script. In other words, the result will be TRUE if the unlocking script
|
|
|
has a valid signature from the cafe's private key that corresponds to
|
|
|
the public key hash set as an encumbrance.
|
|
|
|
|
|
Figures pass:[<a data-type="xref" href="#P2PubKHash1"
|
|
|
data-xrefstyle="select: labelnumber">#P2PubKHash1</a>] and pass:[<a
|
|
|
data-type="xref" href="#P2PubKHash2" data-xrefstyle="select:
|
|
|
labelnumber">#P2PubKHash2</a>] show (in two parts) a step-by-step
|
|
|
execution of the combined script, which will prove this is a valid
|
|
|
transaction.((("", startref="Tsript06")))((("",
|
|
|
startref="Stransact06")))
|
|
|
|
|
|
[[P2PubKHash1]]
|
|
|
.Evaluating a script for a P2PKH transaction (part 1 of 2)
|
|
|
image::images/mbc2_0605.png["Tx_Script_P2PubKeyHash_1"]
|
|
|
|
|
|
[[P2PubKHash2]]
|
|
|
.Evaluating a script for a P2PKH transaction (part 2 of 2)
|
|
|
image::images/mbc2_0606.png["Tx_Script_P2PubKeyHash_2"]
|
|
|
|
|
|
[[digital_sigs]]
|
|
|
=== Digital Signatures (ECDSA)
|
|
|
|
|
|
((("transactions", "digital signatures and", id="Tdigsig06")))So far, we
|
|
|
have not delved into any detail about "digital signatures." In this
|
|
|
section we look at how digital signatures work and how they can present
|
|
|
proof of ownership of a private key without revealing that private key.
|
|
|
|
|
|
((("digital signatures", "algorithm used")))((("Elliptic Curve Digital
|
|
|
Signature Algorithm (ECDSA)")))The digital signature algorithm used in
|
|
|
bitcoin is the _Elliptic Curve Digital Signature Algorithm_, or _ECDSA_.
|
|
|
ECDSA is the algorithm used for digital signatures based on elliptic
|
|
|
curve private/public key pairs, as described in <<elliptic_curve>>.
|
|
|
ECDSA is used by the script functions +OP_CHECKSIG+,
|
|
|
+OP_CHECKSIGVERIFY+, +OP_CHECKMULTISIG+, and +OP_CHECKMULTISIGVERIFY+.
|
|
|
Any time you see those in a locking script, the unlocking script must
|
|
|
contain an ECDSA signature.
|
|
|
|
|
|
((("digital signatures", "purposes of")))A digital signature serves
|
|
|
three purposes in bitcoin (see the following sidebar). First, the
|
|
|
signature proves that the owner of the private key, who is by
|
|
|
implication the owner of the funds, has _authorized_ the spending of
|
|
|
those funds. Secondly, the proof of authorization is _undeniable_
|
|
|
(nonrepudiation). Thirdly, the signature proves that the transaction (or
|
|
|
specific parts of the transaction) have not and _cannot be modified_ by
|
|
|
anyone after it has been signed.
|
|
|
|
|
|
Note that each transaction input is signed independently. This is
|
|
|
critical, as neither the signatures nor the inputs have to belong to or
|
|
|
be applied by the same "owners." In fact, a specific transaction scheme
|
|
|
called "CoinJoin" uses this fact to create multi-party transactions for
|
|
|
privacy.
|
|
|
|
|
|
[NOTE]
|
|
|
====
|
|
|
Each transaction input and any signature it may contain is _completely_
|
|
|
independent of any other input or signature. Multiple parties can
|
|
|
collaborate to construct transactions and sign only one input each.
|
|
|
====
|
|
|
|
|
|
[[digital_signature_definition]]
|
|
|
.Wikipedia's Definition of a "Digital Signature"
|
|
|
****
|
|
|
((("digital signatures", "defined")))A digital signature is a
|
|
|
mathematical scheme for demonstrating the authenticity of a digital
|
|
|
message or documents. A valid digital signature gives a recipient reason
|
|
|
to believe that the message was created by a known sender
|
|
|
(authentication), that the sender cannot deny having sent the message
|
|
|
(nonrepudiation), and that the message was not altered in transit
|
|
|
(integrity).
|
|
|
|
|
|
_Source: https://en.wikipedia.org/wiki/Digital_signature_
|
|
|
****
|
|
|
|
|
|
==== How Digital Signatures Work
|
|
|
|
|
|
((("digital signatures", "how they work")))A digital signature is a
|
|
|
_mathematical scheme_ that consists of two parts. The first part is an
|
|
|
algorithm for creating a signature, using a private key (the signing
|
|
|
key), from a message (the transaction). The second part is an algorithm
|
|
|
that allows anyone to verify the signature, given also the message and a
|
|
|
public key.
|
|
|
|
|
|
===== Creating a digital signature
|
|
|
|
|
|
In bitcoin's implementation of the ECDSA algorithm, the "message" being
|
|
|
signed is the transaction, or more accurately a hash of a specific
|
|
|
subset of the data in the transaction (see <<sighash_types>>). The
|
|
|
signing key is the user's private key. The result is the signature:
|
|
|
|
|
|
latexmath:[\(Sig = F_{sig}(F_{hash}(m), dA)\)]
|
|
|
|
|
|
where:
|
|
|
|
|
|
* _dA_ is the signing private key
|
|
|
* _m_ is the transaction (or parts of it)
|
|
|
* _F_~_hash_~ is the hashing function
|
|
|
* _F_~_sig_~ is the signing algorithm
|
|
|
* _Sig_ is the resulting signature
|
|
|
|
|
|
More details on the mathematics of ECDSA can be found in <<ecdsa_math>>.
|
|
|
|
|
|
The function _F_~_sig_~ produces a signature +Sig+ that is composed of
|
|
|
two values, commonly referred to as +R+ and +S+:
|
|
|
|
|
|
----
|
|
|
Sig = (R, S)
|
|
|
----
|
|
|
|
|
|
((("Distinguished Encoding Rules (DER)")))Now that the two values +R+
|
|
|
and +S+ have been calculated, they are serialized into a byte-stream
|
|
|
using an international standard encoding scheme called the
|
|
|
_Distinguished Encoding Rules_, or _DER_.
|
|
|
|
|
|
[[seralization_of_signatures_der]]
|
|
|
===== Serialization of signatures (DER)
|
|
|
|
|
|
Let's look at the transaction Alice ((("use cases", "buying coffee",
|
|
|
id="alicesixtwo")))created again. In the transaction input there is an
|
|
|
unlocking script that contains the following DER-encoded signature from
|
|
|
Alice's wallet:
|
|
|
|
|
|
----
|
|
|
3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e381301
|
|
|
----
|
|
|
|
|
|
That signature is a serialized byte-stream of the +R+ and +S+ values
|
|
|
produced by Alice's wallet to prove she owns the private key authorized
|
|
|
to spend that output. The serialization format consists of nine elements
|
|
|
as follows:
|
|
|
|
|
|
* +0x30+—indicating the start of a DER sequence
|
|
|
* +0x45+—the length of the sequence (69 bytes)
|
|
|
* +0x02+—an integer value follows
|
|
|
* +0x21+—the length of the integer (33 bytes)
|
|
|
* +R+—++00884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb++
|
|
|
* +0x02+—another integer follows
|
|
|
* +0x20+—the length of the integer (32 bytes)
|
|
|
* +S+—++4b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813++
|
|
|
* A suffix (+0x01+) indicating the type of hash used (+SIGHASH_ALL+)
|
|
|
|
|
|
See if you can decode Alice's serialized (DER-encoded) signature using
|
|
|
this list. The important numbers are +R+ and +S+; the rest of the data
|
|
|
is part of the DER encoding scheme.
|
|
|
|
|
|
==== Verifying the Signature
|
|
|
|
|
|
((("digital signatures", "verifying")))To verify the signature, one must
|
|
|
have the signature (+R+ and +S+), the serialized transaction, and the
|
|
|
public key (that corresponds to the private key used to create the
|
|
|
signature). Essentially, verification of a signature means "Only the
|
|
|
owner of the private key that generated this public key could have
|
|
|
produced this signature on this transaction."
|
|
|
|
|
|
The signature verification algorithm takes the message (a hash of the
|
|
|
transaction or parts of it), the signer's public key and the signature
|
|
|
(+R+ and +S+ values), and returns TRUE if the signature is valid for
|
|
|
this message and public key.
|
|
|
|
|
|
[[sighash_types]]
|
|
|
==== Signature Hash Types (SIGHASH)
|
|
|
|
|
|
((("digital signatures", "signature hash
|
|
|
types")))((("commitment")))Digital signatures are applied to messages,
|
|
|
which in the case of bitcoin, are the transactions themselves. The
|
|
|
signature implies a _commitment_ by the signer to specific transaction
|
|
|
data. In the simplest form, the signature applies to the entire
|
|
|
transaction, thereby committing all the inputs, outputs, and other
|
|
|
transaction fields. However, a signature can commit to only a subset of
|
|
|
the data in a transaction, which is useful for a number of scenarios as
|
|
|
we will see in this section.
|
|
|
|
|
|
((("SIGHASH flags")))Bitcoin signatures have a way of indicating which
|
|
|
part of a transaction's data is included in the hash signed by the
|
|
|
private key using a +SIGHASH+ flag. The +SIGHASH+ flag is a single byte
|
|
|
that is appended to the signature. Every signature has a +SIGHASH+ flag
|
|
|
and the flag can be different from input to input. A transaction with
|
|
|
three signed inputs may have three signatures with different +SIGHASH+
|
|
|
flags, each signature signing (committing) different parts of the
|
|
|
transaction.
|
|
|
|
|
|
Remember, each input may contain a signature in its unlocking script. As
|
|
|
a result, a transaction that contains several inputs may have signatures
|
|
|
with different +SIGHASH+ flags that commit different parts of the
|
|
|
transaction in each of the inputs. Note also that bitcoin transactions
|
|
|
may contain inputs from different "owners," who may sign only one input
|
|
|
in a partially constructed (and invalid) transaction, collaborating with
|
|
|
others to gather all the necessary signatures to make a valid
|
|
|
transaction. Many of the +SIGHASH+ flag types only make sense if you
|
|
|
think of multiple participants collaborating outside the Bitcoin network
|
|
|
and updating a partially signed transaction.
|
|
|
|
|
|
[role="pagebreak-before"]
|
|
|
There are three +SIGHASH+ flags: +ALL+, +NONE+, and +SINGLE+, as shown
|
|
|
in <<sighash_types_and_their>>.
|
|
|
|
|
|
[[sighash_types_and_their]]
|
|
|
.SIGHASH types and their meanings
|
|
|
[options="header"]
|
|
|
|=======================
|
|
|
|+SIGHASH+ flag| Value | Description
|
|
|
| +ALL+ | 0x01 | Signature applies to all inputs and outputs
|
|
|
| +NONE+ | 0x02 | Signature applies to all inputs, none of the outputs
|
|
|
| +SINGLE+ | 0x03 | Signature applies to all inputs but only the one output with the same index number as the signed input
|
|
|
|=======================
|
|
|
|
|
|
In addition, there is a modifier flag +SIGHASH_ANYONECANPAY+, which can
|
|
|
be combined with each of the preceding flags. When +ANYONECANPAY+ is
|
|
|
set, only one input is signed, leaving the rest (and their sequence
|
|
|
numbers) open for modification. The +ANYONECANPAY+ has the value +0x80+
|
|
|
and is applied by bitwise OR, resulting in the combined flags as shown
|
|
|
in <<sighash_types_with_modifiers>>.
|
|
|
|
|
|
[[sighash_types_with_modifiers]]
|
|
|
.SIGHASH types with modifiers and their meanings
|
|
|
[options="header"]
|
|
|
|=======================
|
|
|
|SIGHASH flag| Value | Description
|
|
|
| ALL\|ANYONECANPAY | 0x81 | Signature applies to one input and all outputs
|
|
|
| NONE\|ANYONECANPAY | 0x82 | Signature applies to one input, none of the outputs
|
|
|
| SINGLE\|ANYONECANPAY | 0x83 | Signature applies to one input and the output with the same index number
|
|
|
|=======================
|
|
|
|
|
|
The way +SIGHASH+ flags are applied during signing and verification is
|
|
|
that a copy of the transaction is made and certain fields within are
|
|
|
truncated (set to zero length and emptied). The resulting transaction is
|
|
|
serialized. The +SIGHASH+ flag is added to the end of the serialized
|
|
|
transaction and the result is hashed. The hash itself is the "message"
|
|
|
that is signed. Depending on which +SIGHASH+ flag is used, different
|
|
|
parts of the transaction are truncated. The resulting hash depends on
|
|
|
different subsets of the data in the transaction. By including the
|
|
|
+SIGHASH+ as the last step before hashing, the signature commits the
|
|
|
+SIGHASH+ type as well, so it can't be changed (e.g., by a miner).
|
|
|
|
|
|
[NOTE]
|
|
|
====
|
|
|
All +SIGHASH+ types sign the transaction +nLocktime+ field (see
|
|
|
<<transaction_locktime_nlocktime>>). In addition, the +SIGHASH+ type
|
|
|
itself is appended to the transaction before it is signed, so that it
|
|
|
can't be modified once signed.
|
|
|
====
|
|
|
|
|
|
In the example of Alice's transaction (see the list in
|
|
|
<<seralization_of_signatures_der>>), we saw that the last part of the
|
|
|
DER-encoded signature was +01+, which is the +SIGHASH_ALL+ flag. This
|
|
|
locks the transaction data, so Alice's signature is committing the state
|
|
|
of all inputs and outputs. This is the most common signature form.
|
|
|
|
|
|
Let's look at some of the other +SIGHASH+ types and how they can be used
|
|
|
in practice:
|
|
|
|
|
|
+ALL|ANYONECANPAY+ :: ((("charitable donations")))((("use cases",
|
|
|
"charitable donations")))This construction can be used to make a
|
|
|
"crowdfunding”-style transaction. Someone attempting to raise
|
|
|
funds can construct a transaction with a single output. The single
|
|
|
output pays the "goal" amount to the fundraiser. Such a transaction is
|
|
|
obviously not valid, as it has no inputs. However, others can now amend
|
|
|
it by adding an input of their own, as a donation. They sign their own
|
|
|
input with +ALL|ANYONECANPAY+. Unless enough inputs are gathered to
|
|
|
reach the value of the output, the transaction is invalid. Each donation
|
|
|
is a "pledge," which cannot be collected by the fundraiser until the
|
|
|
entire goal amount is raised.
|
|
|
|
|
|
+NONE+ :: This construction can be used to create a "bearer check" or
|
|
|
"blank check" of a specific amount. It commits to the input, but allows
|
|
|
the output locking script to be changed. Anyone can write their own
|
|
|
Bitcoin address into the output locking script and redeem the
|
|
|
transaction. However, the output value itself is locked by the
|
|
|
signature.
|
|
|
|
|
|
+NONE|ANYONECANPAY+ :: This construction can be used to build a "dust
|
|
|
collector." Users who have tiny UTXO in their wallets can't spend these
|
|
|
without the cost in fees exceeding the value of the dust. With this type
|
|
|
of signature, the dust UTXO can be donated for anyone to aggregate and
|
|
|
spend whenever they want.
|
|
|
|
|
|
((("Bitmask Sighash Modes")))There are some proposals to modify or
|
|
|
expand the +SIGHASH+ system. One such proposal is _Bitmask Sighash
|
|
|
Modes_ by Blockstream's Glenn Willen, as part of the Elements project.
|
|
|
This aims to create a flexible replacement for +SIGHASH+ types that
|
|
|
allows "arbitrary, miner-rewritable bitmasks of inputs and outputs" that
|
|
|
can express "more complex contractual precommitment schemes, such as
|
|
|
signed offers with change in a distributed asset exchange."
|
|
|
|
|
|
[NOTE]
|
|
|
====
|
|
|
You will not see +SIGHASH+ flags presented as an option in a user's
|
|
|
wallet application. With few exceptions, wallets construct P2PKH scripts
|
|
|
and sign with +SIGHASH_ALL+ flags. To use a different +SIGHASH+ flag,
|
|
|
you would have to write software to construct and sign transactions.
|
|
|
More importantly, +SIGHASH+ flags can be used by special-purpose bitcoin
|
|
|
applications that enable novel uses.
|
|
|
====
|
|
|
|
|
|
[[ecdsa_math]]
|
|
|
==== ECDSA Math
|
|
|
|
|
|
((("Elliptic Curve Digital Signature Algorithm (ECDSA)")))As mentioned
|
|
|
previously, signatures are created by a mathematical function _F_~_sig_~
|
|
|
that produces a signature composed of two values _R_ and _S_. In this
|
|
|
section we look at the function _F_~_sig_~ in more detail.
|
|
|
|
|
|
((("public and private keys", "key pairs", "ephemeral")))The signature
|
|
|
algorithm first generates an _ephemeral_ (temporary) private public key
|
|
|
pair. This temporary key pair is used in the calculation of the _R_ and
|
|
|
_S_ values, after a transformation involving the signing private key and
|
|
|
the transaction hash.
|
|
|
|
|
|
The temporary key pair is based on a random number _k_, which is used as
|
|
|
the temporary private key. From _k_, we generate the corresponding
|
|
|
temporary public key _P_ (calculated as _P = k*G_, in the same way
|
|
|
bitcoin public keys are derived; see <<pubkey>>). The _R_ value of the
|
|
|
digital signature is then the x coordinate of the ephemeral public key
|
|
|
_P_.
|
|
|
|
|
|
From there, the algorithm calculates the _S_ value of the signature,
|
|
|
such that:
|
|
|
|
|
|
_S_ = __k__^-1^ (__Hash__(__m__) + __dA__ * __R__) _mod p_
|
|
|
|
|
|
where:
|
|
|
|
|
|
* _k_ is the ephemeral private key
|
|
|
* _R_ is the x coordinate of the ephemeral public key
|
|
|
* _dA_ is the signing private key
|
|
|
* _m_ is the transaction data
|
|
|
* _p_ is the prime order of the elliptic curve
|
|
|
|
|
|
Verification is the inverse of the signature generation function, using
|
|
|
the _R_, _S_ values and the public key to calculate a value _P_, which
|
|
|
is a point on the elliptic curve (the ephemeral public key used in
|
|
|
signature creation):
|
|
|
|
|
|
_P_ = __S__^-1^ * __Hash__(__m__) * _G_ + __S__^-1^ * _R_ * _Qa_
|
|
|
|
|
|
where:
|
|
|
|
|
|
- _R_ and _S_ are the signature values
|
|
|
- _Qa_ is Alice's public key
|
|
|
- _m_ is the transaction data that was signed
|
|
|
- _G_ is the elliptic curve generator point
|
|
|
|
|
|
If the x coordinate of the calculated point _P_ is equal to _R_, then
|
|
|
the verifier can conclude that the signature is valid.
|
|
|
|
|
|
Note that in verifying the signature, the private key is neither known
|
|
|
nor revealed.
|
|
|
|
|
|
[TIP]
|
|
|
====
|
|
|
ECDSA is necessarily a fairly complicated piece of math; a full
|
|
|
explanation is beyond the scope of this book. A number of great guides
|
|
|
online take you through it step by step: search for "ECDSA explained" or
|
|
|
try this one: http://bit.ly/2r0HhGB[].
|
|
|
====
|
|
|
|
|
|
==== The Importance of Randomness in Signatures
|
|
|
|
|
|
((("digital signatures", "randomness in")))As we saw in <<ecdsa_math>>,
|
|
|
the signature generation algorithm uses a random key _k_, as the basis
|
|
|
for an ephemeral private/public key pair. The value of _k_ is not
|
|
|
important, _as long as it is random_. If the same value _k_ is used to
|
|
|
produce two signatures on different messages (transactions), then the
|
|
|
signing _private key_ can be calculated by anyone. Reuse of the same
|
|
|
value for _k_ in a signature algorithm leads to exposure of the private
|
|
|
key!
|
|
|
|
|
|
[WARNING]
|
|
|
====
|
|
|
((("warnings and cautions", "digital signatures")))If the same value _k_
|
|
|
is used in the signing algorithm on two different transactions, the
|
|
|
private key can be calculated and exposed to the world!
|
|
|
====
|
|
|
|
|
|
This is not just a theoretical possibility. We have seen this issue lead
|
|
|
to exposure of private keys in a few different implementations of
|
|
|
transaction-signing algorithms in bitcoin. People have had funds stolen
|
|
|
because of inadvertent reuse of a _k_ value. The most common reason for
|
|
|
reuse of a _k_ value is an improperly initialized random-number
|
|
|
generator.
|
|
|
|
|
|
((("random numbers", "random number generation")))((("entropy", "random
|
|
|
number generation")))((("deterministic initialization")))To avoid this
|
|
|
vulnerability, the industry best practice is to not generate _k_ with a
|
|
|
random-number generator seeded with entropy, but instead to use a
|
|
|
deterministic-random process seeded with the transaction data itself.
|
|
|
This ensures that each transaction produces a different _k_. The
|
|
|
industry-standard algorithm for deterministic initialization of _k_ is
|
|
|
defined in https://tools.ietf.org/html/rfc6979[RFC 6979], published by
|
|
|
the Internet Engineering Task Force.
|
|
|
|
|
|
If you are implementing an algorithm to sign transactions in bitcoin,
|
|
|
you _must_ use RFC 6979 or a similarly deterministic-random algorithm to
|
|
|
ensure you generate a different _k_ for each transaction.((("",
|
|
|
startref="Tdigsig06")))
|
|
|
|
