bitcoinbook/chapters/signatures.adoc

[[c_signatures]]
== Digital Signatures

[[sighashes]]
=== Signature Hashes (Sighashes)
((("transactions", "digital signatures and", id="Tdigsig06")))So far, we
have not delved into any detail about "digital signatures." In this
section we look at how digital signatures work and how they can present
proof of ownership of a private key without revealing that private key.

FIXME
((("digital signatures", "algorithm used")))((("Elliptic Curve Digital
Signature Algorithm (ECDSA)")))Two signature algorithms are currently
used in Bitcoin, the _schnorr signature algorithm_ and the _Elliptic
Curve Digital Signature Algorithm_ (_ECDSA_).
These algorithms are used for digital signatures based on elliptic
curve private/public key pairs, as described in <<elliptic_curve>>.
They are used by the script functions +OP_CHECKSIG+,
+OP_CHECKSIGVERIFY+, +OP_CHECKMULTISIG+, and +OP_CHECKMULTISIGVERIFY+.
Any time one of those is executed in a script, a signature must be
provided.

((("digital signatures", "purposes of")))A digital signature serves
three purposes in bitcoin (see the following sidebar). First, the
signature proves that the owner of the private key, who is by
implication the owner of the funds, has _authorized_ the spending of
those funds. Secondly, the proof of authorization is _undeniable_
(nonrepudiation). Thirdly, the signature proves that the transaction (or
specific parts of the transaction) have not and _cannot be modified_ by
anyone after it has been signed.

Note that each transaction input is signed independently. This is
critical, as neither the signatures nor the inputs have to belong to or
be applied by the same "owners." In fact, a specific transaction scheme
called "CoinJoin" uses this fact to create multi-party transactions for
privacy.

[NOTE]
====
Each transaction input and any signature it may contain is _completely_
independent of any other input or signature. Multiple parties can
collaborate to construct transactions and sign only one input each.
====

[[digital_signature_definition]]
.Wikipedia's Definition of a "Digital Signature"
****
((("digital signatures", "defined")))A digital signature is a
mathematical scheme for demonstrating the authenticity of a digital
message or documents. A valid digital signature gives a recipient reason
to believe that the message was created by a known sender
(authentication), that the sender cannot deny having sent the message
(nonrepudiation), and that the message was not altered in transit
(integrity).

_Source: https://en.wikipedia.org/wiki/Digital_signature_
****

==== How Digital Signatures Work

((("digital signatures", "how they work")))A digital signature is a
_mathematical scheme_ that consists of two parts. The first part is an
algorithm for creating a signature, using a private key (the signing
key), from a message (the transaction). The second part is an algorithm
that allows anyone to verify the signature, given also the message and a
public key.

===== Creating a digital signature

In Bitcoin's implementation of digital signature algorithms, the "message" being
signed is the transaction, or more accurately a hash of a specific
subset of the data in the transaction (see <<sighash_types>>). The
signing key is the user's private key. The result is the signature:

latexmath:[\(Sig = F_{sig}(F_{hash}(m), dA)\)]

where:

* _dA_ is the signing private key
* _m_ is the transaction (or parts of it)
* _F_~_hash_~ is the hashing function
* _F_~_sig_~ is the signing algorithm
* _Sig_ is the resulting signature

More details on the mathematics of schnorr and ECDSA signatures can be found in <<signature_math>>.

In both schnorr and ECDSA signatures, the function _F_~_sig_~ produces a signature +Sig+ that is composed of
two values.  There are differences between the two values in the
different algorithms, which we'll explore later.

((("Distinguished Encoding Rules (DER)")))After the two values
are calculated, they are serialized into a byte-stream.  For ECDSA
signatures, the encoding uses an international standard encoding scheme
called the
_Distinguished Encoding Rules_, or _DER_.  For schnorr signatures, a
simpler serialization format is used.

[[serialization_of_signatures_der]]
===== Serialization of ECDSA signatures (DER)

Let's look at
the following DER-encoded signature:

----
3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e381301
----

That signature is a serialized byte-stream of the +R+ and +S+ values
produced by to prove control of the private key authorized
to spend an output. The serialization format consists of nine elements
as follows:

* +0x30+&#x2014;indicating the start of a DER sequence
* +0x45+&#x2014;the length of the sequence (69 bytes)
  * +0x02+&#x2014;an integer value follows
  * +0x21+&#x2014;the length of the integer (33 bytes)
  * +R+&#x2014;++00884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb++
  * +0x02+&#x2014;another integer follows
  * +0x20+&#x2014;the length of the integer (32 bytes)
  * +S+&#x2014;++4b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813++
* A suffix (+0x01+) indicating the type of hash used (+SIGHASH_ALL+)

See if you can decode Alice's serialized (DER-encoded) signature using
this list. The important numbers are +R+ and +S+; the rest of the data
is part of the DER encoding scheme.

==== Verifying the Signature

((("digital signatures", "verifying")))To verify the signature, one must
have the signature, the serialized transaction, some data about the
output being spend, and the public key (that corresponds to the private key used to create the
signature). Essentially, verification of a signature means "Only the
owner of the private key that generated this public key could have
produced this signature on this transaction."

The signature verification algorithm takes the message (a hash of the
transaction or parts of it), the signer's public key and the signature,
and returns TRUE if the signature is valid for
this message and public key.

[[sighash_types]]
==== Signature Hash Types (SIGHASH)

((("digital signatures", "signature hash
types")))((("commitment")))Digital signatures are applied to messages,
which in the case of bitcoin, are the transactions themselves. The
signature implies a _commitment_ by the signer to specific transaction
data. In the simplest form, the signature applies to almost the entire
transaction, thereby committing all the inputs, outputs, and other
transaction fields. However, a signature can commit to only a subset of
the data in a transaction, which is useful for a number of scenarios as
we will see in this section.

((("SIGHASH flags")))Bitcoin signatures have a way of indicating which
part of a transaction's data is included in the hash signed by the
private key using a +SIGHASH+ flag. The +SIGHASH+ flag is a single byte
that is appended to the signature. Every signature has either an
explicit or implicit +SIGHASH+ flag
and the flag can be different from input to input. A transaction with
three signed inputs may have three signatures with different +SIGHASH+
flags, each signature signing (committing) different parts of the
transaction.

Remember, each input may contain a signature in its unlocking script. As
a result, a transaction that contains several inputs may have signatures
with different +SIGHASH+ flags that commit different parts of the
transaction in each of the inputs. Note also that bitcoin transactions
may contain inputs from different "owners," who may sign only one input
in a partially constructed transaction, collaborating with
others to gather all the necessary signatures to make a valid
transaction. Many of the +SIGHASH+ flag types only make sense if you
think of multiple participants collaborating outside the Bitcoin network
and updating a partially signed transaction.

[role="pagebreak-before"]
There are three +SIGHASH+ flags: +ALL+, +NONE+, and +SINGLE+, as shown
in <<sighash_types_and_their>>.

[[sighash_types_and_their]]
.SIGHASH types and their meanings
[options="header"]
|=======================
|+SIGHASH+ flag| Value | Description
| +ALL+ | 0x01 | Signature applies to all inputs and outputs
| +NONE+ | 0x02 | Signature applies to all inputs, none of the outputs
| +SINGLE+ | 0x03 | Signature applies to all inputs but only the one output with the same index number as the signed input
|=======================

In addition, there is a modifier flag +SIGHASH_ANYONECANPAY+, which can
be combined with each of the preceding flags. When +ANYONECANPAY+ is
set, only one input is signed, leaving the rest (and their sequence
numbers) open for modification. The +ANYONECANPAY+ has the value +0x80+
and is applied by bitwise OR, resulting in the combined flags as shown
in <<sighash_types_with_modifiers>>.

[[sighash_types_with_modifiers]]
.SIGHASH types with modifiers and their meanings
[options="header"]
|=======================
|SIGHASH flag| Value | Description
| ALL\|ANYONECANPAY | 0x81 | Signature applies to one input and all outputs
| NONE\|ANYONECANPAY | 0x82 | Signature applies to one input, none of the outputs
| SINGLE\|ANYONECANPAY | 0x83 | Signature applies to one input and the output with the same index number
|=======================

The way +SIGHASH+ flags are applied during signing and verification is
that a copy of the transaction is made and certain fields within are
either omitted or truncated (set to zero length and emptied). The resulting transaction is
serialized. The +SIGHASH+ flag is included in the serialized
transaction data and the result is hashed. The hash digest itself is the "message"
that is signed. Depending on which +SIGHASH+ flag is used, different
parts of the transaction are included. The resulting hash depends on
different subsets of the data in the transaction. By including the
+SIGHASH+ flag itself, the signature commits the
+SIGHASH+ type as well, so it can't be changed (e.g., by a miner).

[NOTE]
====
All +SIGHASH+ types sign the transaction +nLocktime+ field (see
<<nlocktime>>). In addition, the +SIGHASH+ type
itself is appended to the transaction before it is signed, so that it
can't be modified once signed.
====

In
<<serialization_of_signatures_der>>, we saw that the last part of the
DER-encoded signature was +01+, which is the +SIGHASH_ALL+ flag for ECDSA signatures. This
locks the transaction data, so Alice's signature is committing to the state
of all inputs and outputs. This is the most common signature form.

Let's look at some of the other +SIGHASH+ types and how they can be used
in practice:

+ALL|ANYONECANPAY+ :: ((("charitable donations")))((("use cases",
"charitable donations")))This construction can be used to make a
"crowdfunding&#x201d;-style transaction. Someone attempting to raise
funds can construct a transaction with a single output. The single
output pays the "goal" amount to the fundraiser. Such a transaction is
obviously not valid, as it has no inputs. However, others can now amend
it by adding an input of their own, as a donation. They sign their own
input with +ALL|ANYONECANPAY+. Unless enough inputs are gathered to
reach the value of the output, the transaction is invalid. Each donation
is a "pledge," which cannot be collected by the fundraiser until the
entire goal amount is raised.

+NONE+ :: This construction can be used to create a "bearer check" or
"blank check" of a specific amount. It commits to the input, but allows
the output locking script to be changed. Anyone can write their own
Bitcoin address into the output scriptPubKey. However, the output value
itself cannot be changed.

+NONE|ANYONECANPAY+ :: This construction can be used to build a "dust
collector." Users who have tiny UTXOs in their wallets can't spend these
without the cost in fees exceeding the value of the UTXO, see
<<uneconomical_outputs>>. With this type
of signature, the uneconomical UTXOs can be donated for anyone to aggregate and
spend whenever they want.

((("Bitmask Sighash Modes")))There are some proposals to modify or
expand the +SIGHASH+ system. One such proposal is _Bitmask Sighash
Modes_ by Blockstream's Glenn Willen, as part of the Elements project.
This aims to create a flexible replacement for +SIGHASH+ types that
allows "arbitrary, miner-rewritable bitmasks of inputs and outputs" that
can express "more complex contractual precommitment schemes, such as
signed offers with change in a distributed asset exchange."

[NOTE]
====
You will not often see +SIGHASH+ flags presented as an option in a user's
wallet application.  Simple wallet applications
sign with +SIGHASH_ALL+ flags.  More sophisticated applications, such as
Lightning Network nodes, may use alternative +SIGHASH+ flags, but they
use protocols that have been extensively reviewed to understand the
influence of the alternative flags.
====

[[ecdsa_math]]
==== ECDSA Math

((("Elliptic Curve Digital Signature Algorithm (ECDSA)")))As mentioned
previously, signatures are created by a mathematical function _F_~_sig_~
that produces a signature composed of two values.  In ECDSA, those two
values are _R_ and _S_. In this
section we look at the function _F_~_sig_~ for ECDSA in more detail.

((("public and private keys", "key pairs", "ephemeral")))The signature
algorithm first generates an _ephemeral_ (temporary) private public key
pair. This temporary key pair is used in the calculation of the _R_ and
_S_ values, after a transformation involving the signing private key and
the transaction hash.

The temporary key pair is based on a random number _k_, which is used as
the temporary private key. From _k_, we generate the corresponding
temporary public key _P_ (calculated as _P = k*G_, in the same way
bitcoin public keys are derived; see <<public_key_derivation>>). The _R_ value of the
digital signature is then the x coordinate of the ephemeral public key
_P_.

From there, the algorithm calculates the _S_ value of the signature,
such that:

_S_ = __k__^-1^ (__Hash__(__m__) + __dA__ * __R__) _mod p_

where:

* _k_ is the ephemeral private key
* _R_ is the x coordinate of the ephemeral public key
* _dA_ is the signing private key
* _m_ is the transaction data
* _p_ is the prime order of the elliptic curve

Verification is the inverse of the signature generation function, using
the _R_, _S_ values and the public key to calculate a value _P_, which
is a point on the elliptic curve (the ephemeral public key used in
signature creation):

_P_ = __S__^-1^ * __Hash__(__m__) * _G_ + __S__^-1^ * _R_ * _Qa_

where:

- _R_ and _S_ are the signature values
- _Qa_ is Alice's public key
- _m_ is the transaction data that was signed
- _G_ is the elliptic curve generator point

If the x coordinate of the calculated point _P_ is equal to _R_, then
the verifier can conclude that the signature is valid.

Note that in verifying the signature, the private key is neither known
nor revealed.

[TIP]
====
ECDSA is necessarily a fairly complicated piece of math; a full
explanation is beyond the scope of this book. A number of great guides
online take you through it step by step: search for "ECDSA explained" or
try this one: http://bit.ly/2r0HhGB[].
====

==== The Importance of Randomness in Signatures

((("digital signatures", "randomness in")))As we saw in <<ecdsa_math>>,
the signature generation algorithm uses a random key _k_, as the basis
for an ephemeral private/public key pair. The value of _k_ is not
important, _as long as it is random_. If the same value _k_ is used to
produce two signatures on different messages (transactions), then the
signing _private key_ can be calculated by anyone. Reuse of the same
value for _k_ in a signature algorithm leads to exposure of the private
key!

[WARNING]
====
((("warnings and cautions", "digital signatures")))If the same value _k_
is used in the signing algorithm on two different transactions, the
private key can be calculated and exposed to the world!
====

This is not just a theoretical possibility. We have seen this issue lead
to exposure of private keys in a few different implementations of
transaction-signing algorithms in Bitcoin. People have had funds stolen
because of inadvertent reuse of a _k_ value. The most common reason for
reuse of a _k_ value is an improperly initialized random-number
generator.

((("random numbers", "random number generation")))((("entropy", "random
number generation")))((("deterministic initialization")))To avoid this
vulnerability, the industry best practice is to not generate _k_ with a
random-number generator seeded with entropy, but instead to use a
deterministic-random process seeded with the transaction data itself.
This ensures that each transaction produces a different _k_. The
industry-standard algorithm for deterministic initialization of _k_ is
defined in https://tools.ietf.org/html/rfc6979[RFC 6979], published by
the Internet Engineering Task Force.

If you are implementing an algorithm to sign transactions in bitcoin,
you _must_ use RFC 6979 or a similarly deterministic-random algorithm to
ensure you generate a different _k_ for each transaction.((("",
startref="Tdigsig06")))

==== Segregated Witness' New Signing Algorithm

Segregated Witness modified the semantics of the four signature
verification functions from legacy Bitcoin Script (+CHECKSIG+, +CHECKSIGVERIFY+, +CHECKMULTISIG+,
and +CHECKMULTISIGVERIFY+), changing the way a transaction commitment
hash is calculated.

Signatures in bitcoin transactions are applied on a _commitment hash_,
which is calculated from the transaction data, locking specific parts of
the data indicating the signer's commitment to those values. For
example, in a simple +SIGHASH_ALL+ type signature, the commitment hash
includes all inputs and outputs.

Unfortunately, the way the commitment hash was calculated introduced the
possibility that a node verifying the signature can be forced to perform
a significant number of hash computations. Specifically, the hash
operations increase in O(n^2^) with respect to the number of signature
operations in the transaction. An attacker could therefore create a
transaction with a very large number of signature operations, causing
the entire Bitcoin network to have to perform hundreds or thousands of
hash operations to verify the transaction.

Segwit represented an opportunity to address this problem by changing
the way the commitment hash is calculated. For segwit version 0 witness
programs, signature verification occurs using an improved commitment
hash algorithm as specified in BIP-143.

The new algorithm allows the number of
hash operations increases by a much more gradual O(n) to the number of
signature operations, reducing the opportunity to create
denial-of-service attacks with overly complex transactions.