From 1b2eb07bf16b9df5a69f42566aa4dc4a783d97d8 Mon Sep 17 00:00:00 2001 From: theStack Date: Tue, 30 Jan 2018 17:40:59 -0500 Subject: [PATCH 01/23] corrected standard multisignature tx limit to N=3 see BIP11: "A new standard transaction type (scriptPubKey) that is relayed by clients and included in mined blocks: m {pubkey}...{pubkey} n OP_CHECKMULTISIG But only for n less than or equal to 3." also see the code in function IsStandard() in Bitcoin Core since v0.6.0: "// Support up to x-of-3 multisig txns as standard if (n < 1 || n > 3) return false;" --- ch07.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index a6fae6c6..72da4d7e 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -12,7 +12,7 @@ First, we will look at _multisignature_ scripts. Next, we will examine the secon [[multisig]] === Multisignature -((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multisignature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multisignature or any combination within that range. The limitation to 15 listed keys might be lifted by the time this book is published, so check the +isStandard()+ function to see what is currently accepted by the network. +((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multisignature scripts are limited to at most 3 listed public keys, meaning you can do anything from a 1-of-1 to a 3-of-3 multisignature or any combination within that range. The limitation to 3 listed keys might be lifted by the time this book is published, so check the +isStandard()+ function to see what is currently accepted by the network. The general form of a locking script setting an M-of-N multisignature condition is: From 9511f12a3777d4cd05913ff61debd0e207053f73 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 15:41:21 -0800 Subject: [PATCH 02/23] Change heading level and index reference name --- appdx-segwit.asciidoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc index 37086cf9..5d69ede2 100644 --- a/appdx-segwit.asciidoc +++ b/appdx-segwit.asciidoc @@ -1,8 +1,7 @@ [[segwit]] -[appendix] -== Segregated Witness +=== Segregated Witness -((("segwit (Segregated Witness)", id="segwit16")))Segregated Witness (segwit) is an upgrade to the bitcoin consensus rules and network protocol, proposed and implemented as a BIP-9 soft-fork that is currently (mid-2017) pending activation. +((("segwit (Segregated Witness)", id="Ssegwit07")))Segregated Witness (segwit) is an upgrade to the bitcoin consensus rules and network protocol, proposed and implemented as a BIP-9 soft-fork that was activated on bitcoin's mainnet on August 1st, 2017. In cryptography, the term "witness" is used to describe a solution to a cryptographic puzzle. In bitcoin terms, the witness satisfies a cryptographic condition placed on a unspent transaction output (UTXO). From 8bdfb3b7da1979920fe38d9ba042995ea222e277 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 15:42:28 -0800 Subject: [PATCH 03/23] Remove end-of-chapter index reference --- ch07.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index a6fae6c6..f60f3485 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -598,4 +598,4 @@ A few more things to consider when reading this example. See if you can find the * How do the partners "reset" the clock every 29 or 89 days to prevent the lawyer from accessing the funds? -* Why do some +CHECKSIG+ opcodes in this script have the +VERIFY+ suffix while others don't?((("", startref="Tadv07")))((("", startref="Scomplex07")))((("", startref="mohamseventwo"))) +* Why do some +CHECKSIG+ opcodes in this script have the +VERIFY+ suffix while others don't?((("", startref="Scomplex07")))((("", startref="mohamseventwo"))) From 14cde546261180c55bd725a7a6527220837ada70 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 15:43:48 -0800 Subject: [PATCH 04/23] Add segwit to chapter 7 intro --- ch07.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index f60f3485..19ef296f 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -7,7 +7,7 @@ In the previous chapter, we introduced the basic elements of bitcoin transactions and looked at the most common type of transaction script, the P2PKH script. In this chapter we will look at more advanced scripting and how we can use it to build transactions with complex conditions. -First, we will look at _multisignature_ scripts. Next, we will examine the second most common transaction script, _Pay-to-Script-Hash_, which opens up a whole world of complex scripts. Then, we will examine new script operators that add a time dimension to bitcoin, through _timelocks_. +First, we will look at _multisignature_ scripts. Next, we will examine the second most common transaction script, _Pay-to-Script-Hash_, which opens up a whole world of complex scripts. Then, we will examine new script operators that add a time dimension to bitcoin, through _timelocks_. Finally, we will look at _Segregated Witness_, an architectural change to the structure of transactions. [[multisig]] === Multisignature From 5924b349ec02a54d4c78f1c08397105591150887 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 15:46:00 -0800 Subject: [PATCH 05/23] Add chapter 7 ending index reference to segwit --- appdx-segwit.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc index 5d69ede2..cb3a34bf 100644 --- a/appdx-segwit.asciidoc +++ b/appdx-segwit.asciidoc @@ -351,4 +351,4 @@ Let’s look at an example of what incentives are created by the transaction fee Both transactions are less expensive when segregated witness is implemented. But comparing the costs between the two transactions, we see that before Segregated Witness, the fee is higher for the transaction that has a negative Net-new-UTXO. After Segregated Witness, the transaction fees align with the incentive to minimize new UTXO creation by not inadvertently penalizing transactions with many inputs. -Segregated Witness therefore has two main effects on the fees paid by bitcoin users. Firstly, segwit reduces the overall cost of transactions by discounting witness data and increasing the capacity of the bitcoin blockchain. Secondly, segwit’s discount on witness data corrects a misalignment of incentives that may have inadvertently created more bloat in the UTXO set.((("", startref="segwit16"))) +Segregated Witness therefore has two main effects on the fees paid by bitcoin users. Firstly, segwit reduces the overall cost of transactions by discounting witness data and increasing the capacity of the bitcoin blockchain. Secondly, segwit’s discount on witness data corrects a misalignment of incentives that may have inadvertently created more bloat in the UTXO set.((("", startref="Tadv07")))((("", startref="Ssegwit07"))) From 452bf49740f122fc1fec2c0ad29baabde94d5b13 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 15:46:22 -0800 Subject: [PATCH 06/23] Correct wording to Segwit outputs --- appdx-segwit.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc index cb3a34bf..7b4ceeb4 100644 --- a/appdx-segwit.asciidoc +++ b/appdx-segwit.asciidoc @@ -43,7 +43,7 @@ Offline Signing Improvement :: Segregated Witness signatures incorporate the val At first glance, Segregated Witness appears to be a change to how transactions are constructed and therefore a transaction-level feature, but it is not. In fact, Segregated Witness is also a change to how individual UTXO are spent and therefore is a per-output feature. -A transaction can spend Segregated Witness outputs or traditional (inline-witness) outputs or both. Therefore, it does not make much sense to refer to a transaction as a “Segregated Witness transaction.” Rather we should refer to specific transaction inputs as “Segregated Witness inputs." +A transaction can spend Segregated Witness outputs or traditional (inline-witness) outputs or both. Therefore, it does not make much sense to refer to a transaction as a “Segregated Witness transaction.” Rather we should refer to specific transaction outputs as “Segregated Witness outputs." When a transaction spends an UTXO, it must provide a witness. In a traditional UTXO, the locking script requires that witness data be provided _inline_ in the input part of the transaction that spends the UTXO. A Segregated Witness UTXO, however, specifies a locking script that can be satisfied with witness data outside of the input (segregated). From 00851c44a811559355e02d166116e135b1ff9b2c Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 15:46:46 -0800 Subject: [PATCH 07/23] Improve clarity of P2WSH to P2SH comparison --- appdx-segwit.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc index 7b4ceeb4..f127a6a2 100644 --- a/appdx-segwit.asciidoc +++ b/appdx-segwit.asciidoc @@ -148,7 +148,7 @@ Again, as with the example of P2WPKH, you can see that the Segregated Witness eq [TIP] ==== -While P2SH uses the 20-byte +RIPEMD160(SHA256(script))+ hash, the P2WSH witness program uses a 32-byte +SHA256(script)+ hash. This difference in the selection of the hashing algorithm is deliberate and is used to differentiate between the two types of witness programs (P2WPKH and P2WSH) by the length of the hash and to provide stronger security to P2WSH (128 bits versus 80 bits of P2SH). +While P2SH uses the 20-byte +RIPEMD160(SHA256(script))+ hash, the P2WSH witness program uses a 32-byte +SHA256(script)+ hash. This difference in the selection of the hashing algorithm is deliberate and is used to differentiate between the two types of witness programs (P2WPKH and P2WSH) by the length of the hash and to provide stronger security to P2WSH (128 bits of security in P2WSH versus 80 bits of security in P2SH). ==== From ecd13d3523cd4485c3150d2f39c3fc0c13d09b91 Mon Sep 17 00:00:00 2001 From: cclauss Date: Sat, 3 Feb 2018 01:33:46 +0100 Subject: [PATCH 08/23] whitespace fixes to placate flake8 --- code/get-utxo.py | 4 ++-- code/hash_example.py | 10 +++++----- code/pycoin_example.py | 2 ++ code/rpc_block.py | 7 ++----- 4 files changed, 11 insertions(+), 12 deletions(-) diff --git a/code/get-utxo.py b/code/get-utxo.py index ebce0083..2b3ffef1 100644 --- a/code/get-utxo.py +++ b/code/get-utxo.py @@ -8,7 +8,7 @@ address = '1Dorian4RoXcnBv9hnQ4Y2C1an6NJ4UrjX' # The API URL is https://blockchain.info/unspent?active=
# It returns a JSON object with a list "unspent_outputs", containing UTXO, like this: -#{ "unspent_outputs":[ +# { "unspent_outputs":[ # { # "tx_hash":"ebadfaa92f1fd29e2fe296eda702c48bd11ffd52313e986e99ddad9084062167", # "tx_index":51919767, @@ -19,7 +19,7 @@ address = '1Dorian4RoXcnBv9hnQ4Y2C1an6NJ4UrjX' # "confirmations":28691 # }, # ... -#]} +# ]} resp = requests.get('https://blockchain.info/unspent?active=%s' % address) utxo_set = json.loads(resp.text)["unspent_outputs"] diff --git a/code/hash_example.py b/code/hash_example.py index 14bb5a43..d5e69b2b 100644 --- a/code/hash_example.py +++ b/code/hash_example.py @@ -9,10 +9,10 @@ text = "I am Satoshi Nakamoto" for nonce in range(20): # add the nonce to the end of the text - input_data = text + str(nonce) - + input_data = text + str(nonce) + # calculate the SHA-256 hash of the input (text+nonce) - hash_data = hashlib.sha256(input_data).hexdigest() - + hash_data = hashlib.sha256(input_data).hexdigest() + # show the input and hash result - print(input_data, '=>', hash_data) + print(input_data, '=>', hash_data) diff --git a/code/pycoin_example.py b/code/pycoin_example.py index 0791732c..05761caa 100755 --- a/code/pycoin_example.py +++ b/code/pycoin_example.py @@ -8,6 +8,7 @@ from pycoin.key.validate import is_address_valid, is_wif_valid from pycoin.services import spendables_for_address from pycoin.tx.tx_utils import create_signed_tx + def get_address(which): while 1: print("enter the %s address=> " % which, end='') @@ -17,6 +18,7 @@ def get_address(which): return address print("invalid address, please try again") + src_address = get_address("source") spendables = spendables_for_address(src_address) print(spendables) diff --git a/code/rpc_block.py b/code/rpc_block.py index bd7899ab..52fc28ff 100644 --- a/code/rpc_block.py +++ b/code/rpc_block.py @@ -27,11 +27,8 @@ for txid in transactions: for output in decoded_tx['vout']: # Add up the value of each output tx_value = tx_value + output['value'] - - # Add the value of this transaction to the total + + # Add the value of this transaction to the total block_value = block_value + tx_value print("Total value in block: ", block_value) - - - From 6d1f105049ebce4b63908742c995443355b60cec Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 16:57:41 -0800 Subject: [PATCH 09/23] Added BIP-173 bech32 address section --- appdx-segwit.asciidoc | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc index f127a6a2..f76c085b 100644 --- a/appdx-segwit.asciidoc +++ b/appdx-segwit.asciidoc @@ -24,6 +24,9 @@ https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki[BIP-144] :: Peer https://github.com/bitcoin/bips/blob/master/bip-0145.mediawiki[BIP-145] :: getblocktemplate Updates for Segregated Witness (for mining) +https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki[BIP-173] +:: Base32 address format for native v0-16 witness outputs + ==== Why Segregated Witness? @@ -274,9 +277,37 @@ Now, Mohammed's clients can make payments to this address without any need to su ===== Segregated Witness addresses -After segwit is deployed on the bitcoin network, it will take some time until wallets are upgraded. It is quite likely therefore that segwit will mostly be used embedded in P2SH, as we saw in the previous section, at least for several months. +Even after segwit activation, it will take some time until most wallets are upgraded. At first, segwit will be embedded in P2SH, as we saw in the previous section, to ease compatibility between segit-aware and unaware wallets. -Eventually, however, almost all wallets will be able to support segwit payments. At that time it will no longer be necessary to embed segwit in P2SH. It is therefore likely that a new form of bitcoin address will be created, one that indicates the recipient is segwit-aware and that directly encodes a witness program. There have been a number of proposals for a Segregated Witness address scheme, but none have been actively pursued. +However, once wallets are broadly supporting segwit, it makes sense to encode witness scripts directly in a native address format designed for segwit, rather than embed it in P2SH. + +The native segwit address format is defined in BIP-173: + +https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki[BIP-173] +:: Base32 address format for native v0-16 witness outputs + +BIP-173 only encodes witness (P2WPKH and P2WSH) scripts. It is not compatible with non-segwit P2PKH or P2SH scripts. BIP-173 is a checksummed Base32 encoding, as compared to the Base58 encoding of a "traditional" bitcoin address. BIP-173 addesses are also called _bech32_ addresses, pronounced "beh-ch thirty two", alluding to the use of a "BCH" error detection algorithm and 32-character encoding set. + +BIP-173 addresses use 32 lower-case-only alphanumeric character set, carefully selected to reduce errors from misreading or mistyping. By choosing a lower-case-only character set, bech32 is easier to read, speak, and 45% more efficient to encode in QR codes. + +The BCH error detection algorithm is a vast improvement over the previous checksum algorithm (from Base58Check), allowing not only detection but also _correction_ of errors. Address-input interfaces (such as text-fields in forms) can detect and highlight which character was most likely mistyped when they detect an error. + +From the BIP-173 specification, here are some examples of bech32 addresses: + +Mainnet P2WPKH:: bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4 +Testnet P2WPKH:: tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx +Mainnet P2WSH:: bc1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3qccfmv3 +Testnet P2WSH:: tb1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3q0sl5k7 + +As you can see in these examples, a segwit bech32 string is up to 90 characters long and consists of three parts: + +The human readable part:: This prefix "bc" or "tb" identifying mainnet or testnet. + +The separator:: The digit "1", which is not part of the 32-character encoding set and can only appear in this position as a separator + +The data part:: A minimum of 6 alphanumeric characters, the checksum encoded witness script + +At this time, only a few wallets accept or produce native segwit bech32 addresses, but as segwit adoption increases, you will see these more and more often. [[segwit_txid]] ===== Transaction identifiers From 89ff9a6e82bcdad4a3d8abe981b3177fb9729a53 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 16:57:57 -0800 Subject: [PATCH 10/23] Improved wording --- appdx-segwit.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc index f76c085b..8f325abf 100644 --- a/appdx-segwit.asciidoc +++ b/appdx-segwit.asciidoc @@ -44,7 +44,7 @@ Offline Signing Improvement :: Segregated Witness signatures incorporate the val ==== How Segregated Witness Works -At first glance, Segregated Witness appears to be a change to how transactions are constructed and therefore a transaction-level feature, but it is not. In fact, Segregated Witness is also a change to how individual UTXO are spent and therefore is a per-output feature. +At first glance, Segregated Witness appears to be a change to how transactions are constructed and therefore a transaction-level feature, but it is not. Rather, Segregated Witness is a change to how individual UTXO are spent and therefore is a per-output feature. A transaction can spend Segregated Witness outputs or traditional (inline-witness) outputs or both. Therefore, it does not make much sense to refer to a transaction as a “Segregated Witness transaction.” Rather we should refer to specific transaction outputs as “Segregated Witness outputs." From 7dfaf91542115984482a9f56a3289c967e564730 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 16:58:14 -0800 Subject: [PATCH 11/23] Missing period at end of sentence --- appdx-segwit.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc index 8f325abf..c7901e44 100644 --- a/appdx-segwit.asciidoc +++ b/appdx-segwit.asciidoc @@ -192,7 +192,7 @@ Segregated Witness will not be implemented simultaneously across the entire netw * Ability of a sender's wallet that is not segwit-aware to make a payment to a recipient's wallet that can process segwit transactions -* Ability of a sender's wallet that is segwit-aware to recognize and distinguish between recipients that are segwit-aware and ones that are not, by their _addresses_ +* Ability of a sender's wallet that is segwit-aware to recognize and distinguish between recipients that are segwit-aware and ones that are not, by their _addresses_. ===== Embedding Segregated Witness inside P2SH From 32464c86dd578388a4d949d46b48ba2948a7d62d Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 16:59:08 -0800 Subject: [PATCH 12/23] Improved P2SH example Showing use of bx to produce script-hash --- ch07.asciidoc | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index a6fae6c6..35d72e65 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -128,7 +128,26 @@ If the placeholders are replaced by actual public keys (shown here as 520-bit nu 04C16B8698A9ABF84250A7C3EA7EEDEF9897D1C8C6ADF47F06CF73370D74DCCA01CDCA79DCC5C395D7EEC6984D83F1F50C900A24DD47F569FD4193AF5DE762C58704A2192968D8655D6A935BEAF2CA23E3FB87A3495E7AF308EDF08DAC3C1FCBFC2C75B4B0F4D0B1B70CD2423657738C0C2B1D5CE65C97D78D0E34224858008E8B49047E63248B75DB7379BE9CDA8CE5751D16485F431E46117B9D0C1837C9D5737812F393DA7D4420D7E1A9162F0279CFC10F1E8E8F3020DECDBC3C0DD389D99779650421D65CBD7149B255382ED7F78E946580657EE6FDA162A187543A9D85BAAA93A4AB3A8F044DADA618D087227440645ABE8A35DA8C5B73997AD343BE5C2AFD94A5043752580AFA1ECED3C68D446BCAB69AC0BA7DF50D56231BE0AABF1FDEEC78A6A45E394BA29A1EDF518C022DD618DA774D207D137AAB59E0B000EB7ED238F4D800 5 CHECKMULTISIG ---- -This entire script can instead be represented by a 20-byte cryptographic hash, by first applying the SHA256 hashing algorithm and then applying the RIPEMD160 algorithm on the result. The 20-byte hash of the preceding script is: +This entire script can instead be represented by a 20-byte cryptographic hash, by first applying the SHA256 hashing algorithm and then applying the RIPEMD160 algorithm on the result. + +We use +libbitcoin-explorer+ (+bx+) to produce the script hash, as follows: + +---- +echo \ +2 \ +[04C16B8698A9ABF84250A7C3EA7EEDEF9897D1C8C6ADF47F06CF73370D74DCCA01CDCA79DCC5C395D7EEC6984D83F1F50C900A24DD47F569FD4193AF5DE762C587] \ +[04A2192968D8655D6A935BEAF2CA23E3FB87A3495E7AF308EDF08DAC3C1FCBFC2C75B4B0F4D0B1B70CD2423657738C0C2B1D5CE65C97D78D0E34224858008E8B49] \ +[047E63248B75DB7379BE9CDA8CE5751D16485F431E46117B9D0C1837C9D5737812F393DA7D4420D7E1A9162F0279CFC10F1E8E8F3020DECDBC3C0DD389D9977965] \ +[0421D65CBD7149B255382ED7F78E946580657EE6FDA162A187543A9D85BAAA93A4AB3A8F044DADA618D087227440645ABE8A35DA8C5B73997AD343BE5C2AFD94A5] \ +[043752580AFA1ECED3C68D446BCAB69AC0BA7DF50D56231BE0AABF1FDEEC78A6A45E394BA29A1EDF518C022DD618DA774D207D137AAB59E0B000EB7ED238F4D800] \ +5 CHECKMULTISIG \ +| bx script-encode | bx sha256 | bx ripemd160 +54c557e07dde5bb6cb791c7a540e0a4796f5e97e +---- + + +The 20-byte hash of Mohammed's redeem script is: + ---- 54c557e07dde5bb6cb791c7a540e0a4796f5e97e @@ -471,7 +490,7 @@ So, when do we use +VERIFY+ and when do we use +IF+? If all we are trying to do [TIP] ==== -((("EQUAL opcode")))((("opcodes", "EQUAL")))((("EQUALVERIFY opcode")))((("opcodes", "EQUALVERIFY")))An opcode such as +EQUAL+ will push the result (+TRUE+/+FALSE+) onto the stack, leaving it there for evaluation by subsequent opcodes. In contrast, the opcode +EQUALVERIFY+ suffix does not leave anything on the stack. Opcodes that end in +VERIFY+ do not leave the result on the stack. +((("EQUAL opcode")))((("opcodes", "EQUAL")))((("EQUALVERIFY opcode")))((("opcodes", "EQUALVERIFY")))An opcode such as +EQUAL+ will push the result (+TRUE+/+FALSE+) onto the stack, leaving it there for evaluation by subsequent opcodes. In contrast, the opcode +EQUALVERIFY+ suffix does not leave anything on the stack. Opcodes that end in +VERIFY+ do not leave the result on the stack. ==== ==== Using Flow Control in Scripts From 06158f95bf3790a30a3ce6e663b2dd6e1b8b20ff Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 17:08:49 -0800 Subject: [PATCH 13/23] Moved segwit appendix to ch07 --- appdx-segwit.asciidoc | 385 ----------------------------------------- ch07.asciidoc | 388 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 387 insertions(+), 386 deletions(-) delete mode 100644 appdx-segwit.asciidoc diff --git a/appdx-segwit.asciidoc b/appdx-segwit.asciidoc deleted file mode 100644 index c7901e44..00000000 --- a/appdx-segwit.asciidoc +++ /dev/null @@ -1,385 +0,0 @@ -[[segwit]] -=== Segregated Witness - -((("segwit (Segregated Witness)", id="Ssegwit07")))Segregated Witness (segwit) is an upgrade to the bitcoin consensus rules and network protocol, proposed and implemented as a BIP-9 soft-fork that was activated on bitcoin's mainnet on August 1st, 2017. - -In cryptography, the term "witness" is used to describe a solution to a cryptographic puzzle. In bitcoin terms, the witness satisfies a cryptographic condition placed on a unspent transaction output (UTXO). - -In the context of bitcoin, a digital signature is _one type of witness_, but a witness is more broadly any solution that can satisfy the conditions imposed on an UTXO and unlock that UTXO for spending. The term “witness” is a more general term for an “unlocking script” or “scriptSig.” - -Before segwit’s introduction, every input in a transaction was followed by the witness data that unlocked it. The witness data was embedded in the transaction as part of each input. The term _segregated witness_, or _segwit_ for short, simply means separating the signature or unlocking script of a specific output. Think "separate scriptSig," or “separate signature” in the simplest form. - -Segregated Witness therefore is an architectural change to bitcoin that aims to move the witness data from the +scriptSig+ (unlocking script) field of a transaction into a separate _witness_ data structure that accompanies a transaction. Clients may request transaction data with or without the accompanying witness data. - -In this section we will look at some of the benefits of Segregated Witness, describe the mechanism used to deploy and implement this architecture change, and demonstrate the use of Segregated Witness in transactions and addresses. - -[role="pagebreak-before"] -Segregated Witness is defined by the following BIPs: - -https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki[BIP-141] :: The main definition of Segregated Witness. - -https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki[BIP-143] :: Transaction Signature Verification for Version 0 Witness Program - -https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki[BIP-144] :: Peer Services—New network messages and serialization formats - -https://github.com/bitcoin/bips/blob/master/bip-0145.mediawiki[BIP-145] :: getblocktemplate Updates for Segregated Witness (for mining) - -https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki[BIP-173] -:: Base32 address format for native v0-16 witness outputs - - -==== Why Segregated Witness? - -Segregated Witness is an architectural change that has several effects on the scalability, security, economic incentives, and performance of bitcoin: - -Transaction Malleability :: By moving the witness outside the transaction, the transaction hash used as an identifier no longer includes the witness data. Since the witness data is the only part of the transaction that can be modified by a third party (see <>), removing it also removes the opportunity for transaction malleability attacks. With Segregated Witness, transaction hashes become immutable by anyone other than the creator of the transaction, which greatly improves the implementation of many other protocols that rely on advanced bitcoin transaction construction, such as payment channels, chained transactions, and lightning networks. - -Script Versioning :: With the introduction of Segregated Witness scripts, every locking script is preceded by a _script version_ number, similar to how transactions and blocks have version numbers. The addition of a script version number allows the scripting language to be upgraded in a backward-compatible way (i.e., using soft fork upgrades) to introduce new script operands, syntax, or semantics. The ability to upgrade the scripting language in a nondisruptive way will greatly accelerate the rate of innovation in bitcoin. - -Network and Storage Scaling :: The witness data is often a big contributor to the total size of a transaction. More complex scripts such as those used for multisig or payment channels are very large. In some cases these scripts account for the majority (more than 75%) of the data in a transaction. By moving the witness data outside the transaction, Segregated Witness improves bitcoin’s scalability. Nodes can prune the witness data after validating the signatures, or ignore it altogether when doing simplified payment verification. The witness data doesn’t need to be transmitted to all nodes and does not need to be stored on disk by all nodes. - -Signature Verification Optimization :: Segregated Witness upgrades the signature functions (+CHECKSIG+, +CHECKMULTISIG+, etc.) to reduce the algorithm's computational complexity. Before segwit, the algorithm used to produce a signature required a number of hash operations that was proportional to the size of the transaction. Data-hashing computations increased in O(n^2^) with respect to the number of signature operations, introducing a substantial computational burden on all nodes verifying the signature. With segwit, the algorithm is changed to reduce the complexity to O(n). - -Offline Signing Improvement :: Segregated Witness signatures incorporate the value (amount) referenced by each input in the hash that is signed. Previously, an offline signing device, such as a hardware wallet, would have to verify the amount of each input before signing a transaction. This was usually accomplished by streaming a large amount of data about the previous transactions referenced as inputs. Since the amount is now part of the commitment hash that is signed, an offline device does not need the previous transactions. If the amounts do not match (are misrepresented by a compromised online system), the signature will be invalid. - -==== How Segregated Witness Works - -At first glance, Segregated Witness appears to be a change to how transactions are constructed and therefore a transaction-level feature, but it is not. Rather, Segregated Witness is a change to how individual UTXO are spent and therefore is a per-output feature. - -A transaction can spend Segregated Witness outputs or traditional (inline-witness) outputs or both. Therefore, it does not make much sense to refer to a transaction as a “Segregated Witness transaction.” Rather we should refer to specific transaction outputs as “Segregated Witness outputs." - -When a transaction spends an UTXO, it must provide a witness. In a traditional UTXO, the locking script requires that witness data be provided _inline_ in the input part of the transaction that spends the UTXO. A Segregated Witness UTXO, however, specifies a locking script that can be satisfied with witness data outside of the input (segregated). - -==== Soft Fork (Backward Compatibility) - -Segregated Witness is a significant change to the way outputs and transactions are architected. Such a change would normally require a simultaneous change in every bitcoin node and wallet to change the consensus rules—what is known as a hard fork. Instead, segregated witness is introduced with a much less disruptive change, which is backward compatible, known as a soft fork. This type of upgrade allows nonupgraded software to ignore the changes and continue to operate without any disruption. - -Segregated Witness outputs are constructed so that older systems that are not segwit-aware can still validate them. To an old wallet or node, a Segregated Witness output looks like an output that _anyone can spend_. Such outputs can be spent with an empty signature, therefore the fact that there is no signature inside the transaction (it is segregated) does not invalidate the transaction. Newer wallets and mining nodes, however, see the Segregated Witness output and expect to find a valid witness for it in the transaction’s witness data. - -==== Segregated Witness Output and Transaction Examples - -Let’s look at some of our example transactions and see how they would change with Segregated Witness. We’ll first look at how a Pay-to-Public-Key-Hash (P2PKH) payment is transformed with the Segregated Witness program. Then, we’ll look at the Segregated Witness equivalent for Pay-to-Script-Hash (P2SH) scripts. Finally, we’ll look at how both of the preceding Segregated Witness programs can be embedded inside a P2SH script. - -[[p2wpkh]] -===== Pay-to-Witness-Public-Key-Hash (P2WPKH) - -In <>, ((("use cases", "buying coffee", id="aliced")))Alice created a transaction to pay Bob for a cup of coffee. That transaction created a P2PKH output with a value of 0.015 BTC that was spendable by Bob. The output’s script looks like this: - -.Example P2PKH output script ----- -DUP HASH160 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 EQUALVERIFY CHECKSIG ----- - -With Segregated Witness, Alice would create a Pay-to-Witness-Public-Key-Hash (P2WPKH) script, which looks like this: - -.Example P2WPKH output script ----- -0 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 ----- - -As you can see, a Segregated Witness output’s locking script is much simpler than a traditional output. It consists of two values that are pushed on to the script evaluation stack. To an old (nonsegwit-aware) bitcoin client, the two pushes would look like an output that anyone can spend and does not require a signature (or rather, can be spent with an empty signature). To a newer, segwit-aware client, the first number (0) is interpreted as a version number (the _witness version_) and the second part (20 bytes) is the equivalent of a locking script known as a _witness program_. The 20-byte witness program is simply the hash of the public key, as in a P2PKH script - -Now, let’s look at the corresponding transaction that Bob uses to spend this output. For the original script (nonsegwit), Bob’s transaction would have to include a signature within the transaction input: - -.Decoded transaction showing a P2PKH output being spent with a signature ----- -[...] -“Vin” : [ -"txid": "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2", -"vout": 0, - "scriptSig": “”, -] -[...] ----- - -However, to spend the Segregated Witness output, the transaction has no signature on that input. Instead, Bob’s transaction has an empty +scriptSig+ and includes a Segregated Witness, outside the transaction itself: - -.Decoded transaction showing a P2WPKH output being spent with separate witness data ----- -[...] -“Vin” : [ -"txid": "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2", -"vout": 0, - "scriptSig": “”, -] -[...] -“witness”: “” -[...] ----- - -===== Wallet construction of P2WPKH - -It is extremely important to note that P2WPKH should only be created by the payee (recipient) and not converted by the sender from a known public key, P2PKH script, or address. The sender has no way of knowing if the recipient's wallet has the ability to construct segwit transactions and spend P2WPKH outputs. - -Additionally, P2WPKH outputs must be constructed from the hash of a _compressed_ public key. Uncompressed public keys are nonstandard in segwit and may be explicitly disabled by a future soft fork. If the hash used in the P2WPKH came from an uncompressed public key, it may be unspendable and you may lose funds. P2WPKH outputs should be created by the payee's wallet by deriving a compressed public key from their private key. - -[WARNING] -==== -P2WPKH should be constructed by the payee (recipient) by converting a compressed public key to a P2WPKH hash. You should never transform a P2PKH script, bitcoin address, or uncompressed public key to a P2WPKH witness script. -==== - -[[p2wsh]] -===== Pay-to-Witness-Script-Hash (P2WSH) - -The ((("use cases", "import/export", id="mohamappd")))second type of witness program corresponds to a Pay-to-Script-Hash (P2SH) script. We saw this type of script in <>. In that example, P2SH was used by Mohammed's company to express a multisignature script. Payments to Mohammed's company were encoded with a locking script like this: - -.Example P2SH output script ----- -HASH160 54c557e07dde5bb6cb791c7a540e0a4796f5e97e EQUAL ----- - -This P2SH script references the hash of a _redeem script_ that defines a 2-of-3 multisignature requirement to spend funds. To spend this output, Mohammed's company would present the redeem script (whose hash matches the script hash in the P2SH output) and the signatures necessary to satisfy that redeem script, all inside the transaction input: - -.Decoded transaction showing a P2SH output being spent ----- -[...] -“Vin” : [ -"txid": "abcdef12345...", -"vout": 0, - "scriptSig": “ <2 PubA PubB PubC PubD PubE 5 CHECKMULTISIG>”, -] ----- - -Now, let's look at how this entire example would be upgraded to segwit. If Mohammed's customers were using a segwit-compatible wallet, they would make a payment, creating a Pay-to-Witness-Script-Hash (P2WSH) output that would look like this: - -.Example P2WSH output script ----- -0 9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73 ----- - -Again, as with the example of P2WPKH, you can see that the Segregated Witness equivalent script is a lot simpler and omits the various script operands that you see in P2SH scripts. Instead, the Segregated Witness program consists of two values pushed to the stack: a witness version (0) and the 32-byte SHA256 hash of the redeem script. - -[TIP] -==== -While P2SH uses the 20-byte +RIPEMD160(SHA256(script))+ hash, the P2WSH witness program uses a 32-byte +SHA256(script)+ hash. This difference in the selection of the hashing algorithm is deliberate and is used to differentiate between the two types of witness programs (P2WPKH and P2WSH) by the length of the hash and to provide stronger security to P2WSH (128 bits of security in P2WSH versus 80 bits of security in P2SH). - -==== - -Mohammed's company can spend outputs the P2WSH output by presenting the correct redeem script and sufficient signatures to satisfy it. Both the redeem script and the signatures would be segregated _outside_ the spending transaction as part of the witness data. Within the transaction input, Mohammed's ((("", startref="mohamappd")))wallet would put an empty +scriptSig+: - -.Decoded transaction showing a P2WSH output being spent with separate witness data ----- -[...] -“Vin” : [ -"txid": "abcdef12345...", -"vout": 0, - "scriptSig": “”, -] -[...] -“witness”: “ <2 PubA PubB PubC PubD PubE 5 CHECKMULTISIG>” -[...] ----- - -===== Differentiating between P2WPKH and P2WSH - -In the previous two sections, we demonstrated two types of witness programs: <> and <>. Both types of witness programs consist of a single byte version number followed by a longer hash. They look very similar, but are interpreted very differently: one is interpreted as a public key hash, which is satisfied by a signature and the other as a script hash, which is satisfied by a redeem script. The critical difference between them is the length of the hash: - -* The public key hash in P2WPKH is 20 bytes -* The script hash in P2WSH is 32 bytes - -This is the one difference that allows a wallet to differentiate between the two types of witness programs. By looking at the length of the hash, a wallet can determine what type of witness program it is, P2WPKH or P2WSH. - -==== Upgrading to Segregated Witness - -As we can see from the previous examples, upgrading to Segregated Witness is a two-step process. First, wallets must create special segwit type outputs. Then, these outputs can be spent by wallets that know how to construct Segregated Witness transactions. In the examples, Alice's wallet was segwit-aware and able to create special outputs with Segregated Witness scripts. Bob's wallet is also segwit-aware and able to spend those outputs. What may not be obvious from the example is that in practice, Alice's wallet needs to _know_ that Bob uses a segwit-aware wallet and can spend these outputs. Otherwise, if Bob's wallet is not upgraded and Alice tries to make segwit payments to Bob, Bob's wallet will not be able to detect these payments. - -[TIP] -==== -For P2WPKH and P2WSH payment types, both the sender and the recipient wallets need to be upgraded to be able to use segwit. Furthermore, the sender's wallet needs to know that the recipient's wallet is segwit-aware. -==== - -Segregated Witness will not be implemented simultaneously across the entire network. Rather, Segregated Witness is implemented as a backward-compatible upgrade, where _old and new clients can coexist_. Wallet developers will independently upgrade wallet software to add segwit capabilities. The P2WPKH and P2WSH payment types are used when both sender and recipient are segwit-aware. The traditional P2PKH and P2SH will continue to work for nonupgraded wallets. That leaves two important scenarios, which are addressed in the next section: - -* Ability of a sender's wallet that is not segwit-aware to make a payment to a recipient's wallet that can process segwit transactions - -* Ability of a sender's wallet that is segwit-aware to recognize and distinguish between recipients that are segwit-aware and ones that are not, by their _addresses_. - -===== Embedding Segregated Witness inside P2SH - -Let's assume, for example, that Alice's wallet is not upgraded to segwit, but Bob's wallet is upgraded and can handle segwit transactions. Alice and Bob can use "old" non-segwit transactions. But Bob would likely want to use segwit to reduce transaction fees, taking advantage of the discount that applies to witness data. - -In this case Bob's wallet can construct a P2SH address that contains a segwit script inside it. Alice's wallet sees this as a "normal" P2SH address and can make payments to it without any knowledge of segwit. Bob's wallet can then spend this payment with a segwit transaction, taking full advantage of segwit and reducing transaction fees. - -Both forms of witness scripts, P2WPKH and P2WSH, can be embedded in a P2SH address. The first is noted as P2SH(P2WPKH) and the second is noted as P2SH(P2WSH). - -===== Pay-to-Witness-Public-Key-Hash inside Pay-to-Script-Hash - -The first form of witness script we will examine is P2SH(P2WPKH). This is a Pay-to-Witness-Public-Key-Hash witness program, embedded inside a Pay-to-Script-Hash script, so that it can be used by a wallet that is not aware of segwit. - -Bob's wallet constructs a P2WPKH witness program with Bob's public key. This witness program is then hashed and the resulting hash is encoded as a P2SH script. The P2SH script is converted to a bitcoin address, one that starts with a "3," as we saw in the <> section. - -Bob's wallet starts with the P2WPKH witness program we saw earlier: - -.Bob's P2WPKH witness program ----- -0 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 ----- - -The P2WPKH witness program consists of the witness version and Bob's 20-byte public key hash. - -Bob's wallet then hashes the preceding witness program, first with SHA256, then with RIPEMD160, producing another 20-byte hash: - -.HASH160 of the P2WPKH witness program ----- -3e0547268b3b19288b3adef9719ec8659f4b2b0b ----- - -[role="pagebreak-before"] -The hash of the witness program is then embedded in a P2SH script: - ----- -HASH160 3e0547268b3b19288b3adef9719ec8659f4b2b0b EQUAL ----- - -Finally, the P2SH script is converted to a P2SH bitcoin address: - -.P2SH address ----- -37Lx99uaGn5avKBxiW26HjedQE3LrDCZru ----- - -Now, Bob can display this address for customers to pay for their coffee. Alice's wallet can make a payment to +37Lx99uaGn5avKBxiW26HjedQE3LrDCZru+, just as it would to any other bitcoin address. Even though Alice's wallet has no support for segwit, the payment it creates can be spent by Bob with a segwit transaction.((("", startref="aliced"))) - -===== Pay-to-Witness-Script-Hash inside Pay-to-Script-Hash - -Similarly, a P2WSH witness program for a multisig script or other complicated script can be embedded inside a P2SH script and address, making it possible for any wallet to make payments that are segwit compatible. - -As we saw in <>, Mohammed's ((("use cases", "import/export")))company is using Segregated Witness payments to multisignature scripts. To make it possible for any client to pay his company, regardless of whether their wallets are upgraded for segwit, Mohammed's wallet can embed the P2WSH witness program inside a P2SH script. - -First, Mohammed's wallet creates the P2WSH witness program that corresponds to the multisignature script, hashed with SHA256: - -.Mohammed's wallet creates a P2WSH witness program ----- -0 9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73 ----- - -Then, the witness program itself is hashed with SHA256 and RIPEMD160, producing a new 20-byte hash, as used in traditional P2SH: - -.The HASH160 of the P2WSH witness program ----- -86762607e8fe87c0c37740cddee880988b9455b2 ----- - -Next, Mohammed's wallet puts the hash into a P2SH script: - ----- -HASH160 86762607e8fe87c0c37740cddee880988b9455b2 EQUAL ----- - -Finally, the wallet constructs a bitcoin address from this script: - -.P2SH bitcoin address ----- -3Dwz1MXhM6EfFoJChHCxh1jWHb8GQqRenG ----- - -Now, Mohammed's clients can make payments to this address without any need to support segwit. Mohammed's company can then construct segwit transactions to spend these payments, taking advantage of segwit features including lower transaction fees. - -===== Segregated Witness addresses - -Even after segwit activation, it will take some time until most wallets are upgraded. At first, segwit will be embedded in P2SH, as we saw in the previous section, to ease compatibility between segit-aware and unaware wallets. - -However, once wallets are broadly supporting segwit, it makes sense to encode witness scripts directly in a native address format designed for segwit, rather than embed it in P2SH. - -The native segwit address format is defined in BIP-173: - -https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki[BIP-173] -:: Base32 address format for native v0-16 witness outputs - -BIP-173 only encodes witness (P2WPKH and P2WSH) scripts. It is not compatible with non-segwit P2PKH or P2SH scripts. BIP-173 is a checksummed Base32 encoding, as compared to the Base58 encoding of a "traditional" bitcoin address. BIP-173 addesses are also called _bech32_ addresses, pronounced "beh-ch thirty two", alluding to the use of a "BCH" error detection algorithm and 32-character encoding set. - -BIP-173 addresses use 32 lower-case-only alphanumeric character set, carefully selected to reduce errors from misreading or mistyping. By choosing a lower-case-only character set, bech32 is easier to read, speak, and 45% more efficient to encode in QR codes. - -The BCH error detection algorithm is a vast improvement over the previous checksum algorithm (from Base58Check), allowing not only detection but also _correction_ of errors. Address-input interfaces (such as text-fields in forms) can detect and highlight which character was most likely mistyped when they detect an error. - -From the BIP-173 specification, here are some examples of bech32 addresses: - -Mainnet P2WPKH:: bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4 -Testnet P2WPKH:: tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx -Mainnet P2WSH:: bc1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3qccfmv3 -Testnet P2WSH:: tb1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3q0sl5k7 - -As you can see in these examples, a segwit bech32 string is up to 90 characters long and consists of three parts: - -The human readable part:: This prefix "bc" or "tb" identifying mainnet or testnet. - -The separator:: The digit "1", which is not part of the 32-character encoding set and can only appear in this position as a separator - -The data part:: A minimum of 6 alphanumeric characters, the checksum encoded witness script - -At this time, only a few wallets accept or produce native segwit bech32 addresses, but as segwit adoption increases, you will see these more and more often. - -[[segwit_txid]] -===== Transaction identifiers - -((("transaction IDs (txid)")))One of the greatest benefits of Segregated Witness is that it eliminates third-party transaction malleability. - -Before segwit, transactions could have their signatures subtly modified by third parties, changing their transaction ID (hash) without changing any fundamental properties (inputs, outputs, amounts). This created opportunities for denial-of-service attacks as well as attacks against poorly written wallet software that assumed unconfirmed transaction hashes were immutable. - -With the introduction of Segregated Witness, transactions have two identifiers, +txid+ and +wtxid+. The traditional transaction ID +txid+ is the double-SHA256 hash of the serialized transaction, without the witness data. A transaction +wtxid+ is the double-SHA256 hash of the new serialization format of the transaction with witness data. - -The traditional +txid+ is calculated in exactly the same way as with a nonsegwit transaction. However, since the segwit transaction has empty ++scriptSig++s in every input, there is no part of the transaction that can be modified by a third party. Therefore, in a segwit transaction, the +txid+ is immutable by a third party, even when the transaction is unconfirmed. - -The +wtxid+ is like an "extended" ID, in that the hash also incorporates the witness data. If a transaction is transmitted without witness data, then the +wtxid+ and +txid+ are identical. Note than since the +wtxid+ includes witness data (signatures) and since witness data may be malleable, the +wtxid+ should be considered malleable until the transaction is confirmed. Only the +txid+ of a segwit transaction can be considered immutable by third parties and only if _all_ the inputs of the transaction are segwit inputs. - -[TIP] -==== -Segregated Witness transactions have two IDs: +txid+ and +wtxid+. The +txid+ is the hash of the transaction without the witness data and the +wtxid+ is the hash inclusive of witness data. The +txid+ of a transaction where all inputs are segwit inputs is not susceptible to third-party transaction malleability. -==== - -==== Segregated Witness' New Signing Algorithm - -Segregated Witness modifies the semantics of the four signature verification functions (+CHECKSIG+, +CHECKSIGVERIFY+, +CHECKMULTISIG+, and +CHECKMULTISIGVERIFY+), changing the way a transaction commitment hash is calculated. - -Signatures in bitcoin transactions are applied on a _commitment hash_, which is calculated from the transaction data, locking specific parts of the data indicating the signer's commitment to those values. For example, in a simple +SIGHASH_ALL+ type signature, the commitment hash includes all inputs and outputs. - -Unfortunately, the way the commitment hash was calculated introduced the possibility that a node verifying the signature can be forced to perform a significant number of hash computations. Specifically, the hash operations increase in O(n^2^) with respect to the number of signature operations in the transaction. An attacker could therefore create a transaction with a very large number of signature operations, causing the entire bitcoin network to have to perform hundreds or thousands of hash operations to verify the transaction. - -Segwit represented an opportunity to address this problem by changing the way the commitment hash is calculated. For segwit version 0 witness programs, signature verification occurs using an improved commitment hash algorithm as specified in BIP-143. - -The new algorithm achieves two important goals. Firstly, the number of hash operations increases by a much more gradual O(n) to the number of signature operations, reducing the opportunity to create denial-of-service attacks with overly complex transactions. Secondly, the commitment hash now also includes the value (amounts) of each input as part of the commitment. This means that a signer can commit to a specific input value without needing to "fetch" and check the previous transaction referenced by the input. In the case of offline devices, such as hardware wallets, this greatly simplifies the communication between the host and the hardware wallet, removing the need to stream previous transactions for validation. A hardware wallet can accept the input value "as stated" by an untrusted host. Since the signature is invalid if that input value is not correct, the hardware wallet doesn't need to validate the value before signing the input. - -==== Economic Incentives for Segregated Witness - -Bitcoin mining nodes and full nodes incur costs for the resources used to support the bitcoin network and the blockchain. As the volume of bitcoin transactions increases, so does the cost of resources (CPU, network bandwidth, disk space, memory). Miners are compensated for these costs through fees that are proportional to the size (in bytes) of each transaction. Nonmining full nodes are not compensated, so they incur these costs because they have a need to run an authoritative fully validating full-index node, perhaps because they use the node to operate a bitcoin business. - -Without transaction fees, the growth in bitcoin data would arguably increase dramatically. Fees are intended to align the needs of bitcoin users with the burden their transactions impose on the network, through a market-based price discovery mechanism. - -The calculation of fees based on transaction size treats all the data in the transaction as equal in cost. But from the perspective of full nodes and miners, some parts of a transaction carry much higher costs. Every transaction added to the bitcoin network affects the consumption of four resources on nodes: - -Disk Space :: Every transaction is stored in the blockchain, adding to the total size of the blockchain. The blockchain is stored on disk, but the storage can be optimized by “pruning” older transactions. - -CPU :: Every transaction must be validated, which requires CPU time. - -Bandwidth :: Every transaction is transmitted (through flood propagation) across the network at least once. Without any optimization in the block propagation protocol, transactions are transmitted again as part of a block, doubling the impact on network capacity. - -Memory :: Nodes that validate transactions keep the UTXO index or the entire UTXO set in memory to speed up validation. Because memory is at least one order of magnitude more expensive than disk, growth of the UTXO set contributes disproportionately to the cost of running a node. - -As you can see from the list, not every part of a transaction has an equal impact on the cost of running a node or on the ability of bitcoin to scale to support more transactions. The most expensive part of a transaction are the newly created outputs, as they are added to the in-memory UTXO set. By comparison, signatures (aka witness data) add the least burden to the network and the cost of running a node, because witness data are only validated once and then never used again. Furthermore, immediately after receiving a new transaction and validating witness data, nodes can discard that witness data. If fees are calculated on transaction size, without discriminating between these two types of data, then the market incentives of fees are not aligned with the actual costs imposed by a transaction. In fact, the current fee structure actually encourages the opposite behavior, because witness data is the largest part of a transaction. - -The incentives created by fees matter because they affect the behavior of wallets. All wallets must implement some strategy for assembling transactions that takes into consideration a number of factors, such as privacy (reducing address reuse), fragmentation (making lots of loose change), and fees. If the fees are overwhelmingly motivating wallets to use as few inputs as possible in transactions, this can lead to UTXO picking and change address strategies that inadvertently bloat the UTXO set. - -Transactions consume UTXO in their inputs and create new UTXO with their outputs. A transaction, therefore, that has more inputs than outputs will result in a decrease in the UTXO set, whereas a transaction that has more outputs than inputs will result in an increase in the UTXO set. Let’s consider the _difference_ between inputs and outputs and call that the “Net-new-UTXO.” That’s an important metric, as it tells us what impact a transaction will have on the most expensive network-wide resource, the in-memory UTXO set. A transaction with positive Net-new-UTXO adds to that burden. A transaction with a negative Net-new-UTXO reduces the burden. We would therefore want to encourage transactions that are either negative Net-new-UTXO or neutral with zero Net-new-UTXO. - -Let’s look at an example of what incentives are created by the transaction fee calculation, with and without Segregated Witness. We will look at two different transactions. Transaction A is a 3-input, 2-output transaction, which has a Net-new-UTXO metric of –1, meaning it consumes one more UTXO than it creates, reducing the UTXO set by one. Transaction B is a 2-input, 3-output transaction, which has a Net-new-UTXO metric of 1, meaning it adds one UTXO to the UTXO set, imposing additional cost on the entire bitcoin network. Both transactions use multisignature (2-of-3) scripts to demonstrate how complex scripts increase the impact of segregated witness on fees. Let’s assume a transaction fee of 30 satoshi per byte and a 75% fee discount on witness data: - -++++ -
-
Without Segregated Witness
-
-

Transaction A fee: 25,710 satoshi

-

Transaction B fee: 18,990 satoshi

-
- -
With Segregated Witness
-
-

Transaction A fee: 8,130 satoshi

-

Transaction B fee: 12,045 satoshi

-
-
-++++ - - -Both transactions are less expensive when segregated witness is implemented. But comparing the costs between the two transactions, we see that before Segregated Witness, the fee is higher for the transaction that has a negative Net-new-UTXO. After Segregated Witness, the transaction fees align with the incentive to minimize new UTXO creation by not inadvertently penalizing transactions with many inputs. - -Segregated Witness therefore has two main effects on the fees paid by bitcoin users. Firstly, segwit reduces the overall cost of transactions by discounting witness data and increasing the capacity of the bitcoin blockchain. Secondly, segwit’s discount on witness data corrects a misalignment of incentives that may have inadvertently created more bloat in the UTXO set.((("", startref="Tadv07")))((("", startref="Ssegwit07"))) diff --git a/ch07.asciidoc b/ch07.asciidoc index 19ef296f..05d8f421 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -471,7 +471,7 @@ So, when do we use +VERIFY+ and when do we use +IF+? If all we are trying to do [TIP] ==== -((("EQUAL opcode")))((("opcodes", "EQUAL")))((("EQUALVERIFY opcode")))((("opcodes", "EQUALVERIFY")))An opcode such as +EQUAL+ will push the result (+TRUE+/+FALSE+) onto the stack, leaving it there for evaluation by subsequent opcodes. In contrast, the opcode +EQUALVERIFY+ suffix does not leave anything on the stack. Opcodes that end in +VERIFY+ do not leave the result on the stack. +((("EQUAL opcode")))((("opcodes", "EQUAL")))((("EQUALVERIFY opcode")))((("opcodes", "EQUALVERIFY")))An opcode such as +EQUAL+ will push the result (+TRUE+/+FALSE+) onto the stack, leaving it there for evaluation by subsequent opcodes. In contrast, the opcode +EQUALVERIFY+ suffix does not leave anything on the stack. Opcodes that end in +VERIFY+ do not leave the result on the stack. ==== ==== Using Flow Control in Scripts @@ -599,3 +599,389 @@ A few more things to consider when reading this example. See if you can find the * How do the partners "reset" the clock every 29 or 89 days to prevent the lawyer from accessing the funds? * Why do some +CHECKSIG+ opcodes in this script have the +VERIFY+ suffix while others don't?((("", startref="Scomplex07")))((("", startref="mohamseventwo"))) + +[[segwit]] +=== Segregated Witness + +((("segwit (Segregated Witness)", id="Ssegwit07")))Segregated Witness (segwit) is an upgrade to the bitcoin consensus rules and network protocol, proposed and implemented as a BIP-9 soft-fork that was activated on bitcoin's mainnet on August 1st, 2017. + +In cryptography, the term "witness" is used to describe a solution to a cryptographic puzzle. In bitcoin terms, the witness satisfies a cryptographic condition placed on a unspent transaction output (UTXO). + +In the context of bitcoin, a digital signature is _one type of witness_, but a witness is more broadly any solution that can satisfy the conditions imposed on an UTXO and unlock that UTXO for spending. The term “witness” is a more general term for an “unlocking script” or “scriptSig.” + +Before segwit’s introduction, every input in a transaction was followed by the witness data that unlocked it. The witness data was embedded in the transaction as part of each input. The term _segregated witness_, or _segwit_ for short, simply means separating the signature or unlocking script of a specific output. Think "separate scriptSig," or “separate signature” in the simplest form. + +Segregated Witness therefore is an architectural change to bitcoin that aims to move the witness data from the +scriptSig+ (unlocking script) field of a transaction into a separate _witness_ data structure that accompanies a transaction. Clients may request transaction data with or without the accompanying witness data. + +In this section we will look at some of the benefits of Segregated Witness, describe the mechanism used to deploy and implement this architecture change, and demonstrate the use of Segregated Witness in transactions and addresses. + +[role="pagebreak-before"] +Segregated Witness is defined by the following BIPs: + +https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki[BIP-141] :: The main definition of Segregated Witness. + +https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki[BIP-143] :: Transaction Signature Verification for Version 0 Witness Program + +https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki[BIP-144] :: Peer Services—New network messages and serialization formats + +https://github.com/bitcoin/bips/blob/master/bip-0145.mediawiki[BIP-145] :: getblocktemplate Updates for Segregated Witness (for mining) + +https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki[BIP-173] +:: Base32 address format for native v0-16 witness outputs + + +==== Why Segregated Witness? + +Segregated Witness is an architectural change that has several effects on the scalability, security, economic incentives, and performance of bitcoin: + +Transaction Malleability :: By moving the witness outside the transaction, the transaction hash used as an identifier no longer includes the witness data. Since the witness data is the only part of the transaction that can be modified by a third party (see <>), removing it also removes the opportunity for transaction malleability attacks. With Segregated Witness, transaction hashes become immutable by anyone other than the creator of the transaction, which greatly improves the implementation of many other protocols that rely on advanced bitcoin transaction construction, such as payment channels, chained transactions, and lightning networks. + +Script Versioning :: With the introduction of Segregated Witness scripts, every locking script is preceded by a _script version_ number, similar to how transactions and blocks have version numbers. The addition of a script version number allows the scripting language to be upgraded in a backward-compatible way (i.e., using soft fork upgrades) to introduce new script operands, syntax, or semantics. The ability to upgrade the scripting language in a nondisruptive way will greatly accelerate the rate of innovation in bitcoin. + +Network and Storage Scaling :: The witness data is often a big contributor to the total size of a transaction. More complex scripts such as those used for multisig or payment channels are very large. In some cases these scripts account for the majority (more than 75%) of the data in a transaction. By moving the witness data outside the transaction, Segregated Witness improves bitcoin’s scalability. Nodes can prune the witness data after validating the signatures, or ignore it altogether when doing simplified payment verification. The witness data doesn’t need to be transmitted to all nodes and does not need to be stored on disk by all nodes. + +Signature Verification Optimization :: Segregated Witness upgrades the signature functions (+CHECKSIG+, +CHECKMULTISIG+, etc.) to reduce the algorithm's computational complexity. Before segwit, the algorithm used to produce a signature required a number of hash operations that was proportional to the size of the transaction. Data-hashing computations increased in O(n^2^) with respect to the number of signature operations, introducing a substantial computational burden on all nodes verifying the signature. With segwit, the algorithm is changed to reduce the complexity to O(n). + +Offline Signing Improvement :: Segregated Witness signatures incorporate the value (amount) referenced by each input in the hash that is signed. Previously, an offline signing device, such as a hardware wallet, would have to verify the amount of each input before signing a transaction. This was usually accomplished by streaming a large amount of data about the previous transactions referenced as inputs. Since the amount is now part of the commitment hash that is signed, an offline device does not need the previous transactions. If the amounts do not match (are misrepresented by a compromised online system), the signature will be invalid. + +==== How Segregated Witness Works + +At first glance, Segregated Witness appears to be a change to how transactions are constructed and therefore a transaction-level feature, but it is not. Rather, Segregated Witness is a change to how individual UTXO are spent and therefore is a per-output feature. + +A transaction can spend Segregated Witness outputs or traditional (inline-witness) outputs or both. Therefore, it does not make much sense to refer to a transaction as a “Segregated Witness transaction.” Rather we should refer to specific transaction outputs as “Segregated Witness outputs." + +When a transaction spends an UTXO, it must provide a witness. In a traditional UTXO, the locking script requires that witness data be provided _inline_ in the input part of the transaction that spends the UTXO. A Segregated Witness UTXO, however, specifies a locking script that can be satisfied with witness data outside of the input (segregated). + +==== Soft Fork (Backward Compatibility) + +Segregated Witness is a significant change to the way outputs and transactions are architected. Such a change would normally require a simultaneous change in every bitcoin node and wallet to change the consensus rules—what is known as a hard fork. Instead, segregated witness is introduced with a much less disruptive change, which is backward compatible, known as a soft fork. This type of upgrade allows nonupgraded software to ignore the changes and continue to operate without any disruption. + +Segregated Witness outputs are constructed so that older systems that are not segwit-aware can still validate them. To an old wallet or node, a Segregated Witness output looks like an output that _anyone can spend_. Such outputs can be spent with an empty signature, therefore the fact that there is no signature inside the transaction (it is segregated) does not invalidate the transaction. Newer wallets and mining nodes, however, see the Segregated Witness output and expect to find a valid witness for it in the transaction’s witness data. + +==== Segregated Witness Output and Transaction Examples + +Let’s look at some of our example transactions and see how they would change with Segregated Witness. We’ll first look at how a Pay-to-Public-Key-Hash (P2PKH) payment is transformed with the Segregated Witness program. Then, we’ll look at the Segregated Witness equivalent for Pay-to-Script-Hash (P2SH) scripts. Finally, we’ll look at how both of the preceding Segregated Witness programs can be embedded inside a P2SH script. + +[[p2wpkh]] +===== Pay-to-Witness-Public-Key-Hash (P2WPKH) + +In <>, ((("use cases", "buying coffee", id="aliced")))Alice created a transaction to pay Bob for a cup of coffee. That transaction created a P2PKH output with a value of 0.015 BTC that was spendable by Bob. The output’s script looks like this: + +.Example P2PKH output script +---- +DUP HASH160 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 EQUALVERIFY CHECKSIG +---- + +With Segregated Witness, Alice would create a Pay-to-Witness-Public-Key-Hash (P2WPKH) script, which looks like this: + +.Example P2WPKH output script +---- +0 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 +---- + +As you can see, a Segregated Witness output’s locking script is much simpler than a traditional output. It consists of two values that are pushed on to the script evaluation stack. To an old (nonsegwit-aware) bitcoin client, the two pushes would look like an output that anyone can spend and does not require a signature (or rather, can be spent with an empty signature). To a newer, segwit-aware client, the first number (0) is interpreted as a version number (the _witness version_) and the second part (20 bytes) is the equivalent of a locking script known as a _witness program_. The 20-byte witness program is simply the hash of the public key, as in a P2PKH script + +Now, let’s look at the corresponding transaction that Bob uses to spend this output. For the original script (nonsegwit), Bob’s transaction would have to include a signature within the transaction input: + +.Decoded transaction showing a P2PKH output being spent with a signature +---- +[...] +“Vin” : [ +"txid": "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2", +"vout": 0, + "scriptSig": “”, +] +[...] +---- + +However, to spend the Segregated Witness output, the transaction has no signature on that input. Instead, Bob’s transaction has an empty +scriptSig+ and includes a Segregated Witness, outside the transaction itself: + +.Decoded transaction showing a P2WPKH output being spent with separate witness data +---- +[...] +“Vin” : [ +"txid": "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2", +"vout": 0, + "scriptSig": “”, +] +[...] +“witness”: “” +[...] +---- + +===== Wallet construction of P2WPKH + +It is extremely important to note that P2WPKH should only be created by the payee (recipient) and not converted by the sender from a known public key, P2PKH script, or address. The sender has no way of knowing if the recipient's wallet has the ability to construct segwit transactions and spend P2WPKH outputs. + +Additionally, P2WPKH outputs must be constructed from the hash of a _compressed_ public key. Uncompressed public keys are nonstandard in segwit and may be explicitly disabled by a future soft fork. If the hash used in the P2WPKH came from an uncompressed public key, it may be unspendable and you may lose funds. P2WPKH outputs should be created by the payee's wallet by deriving a compressed public key from their private key. + +[WARNING] +==== +P2WPKH should be constructed by the payee (recipient) by converting a compressed public key to a P2WPKH hash. You should never transform a P2PKH script, bitcoin address, or uncompressed public key to a P2WPKH witness script. +==== + +[[p2wsh]] +===== Pay-to-Witness-Script-Hash (P2WSH) + +The ((("use cases", "import/export", id="mohamappd")))second type of witness program corresponds to a Pay-to-Script-Hash (P2SH) script. We saw this type of script in <>. In that example, P2SH was used by Mohammed's company to express a multisignature script. Payments to Mohammed's company were encoded with a locking script like this: + +.Example P2SH output script +---- +HASH160 54c557e07dde5bb6cb791c7a540e0a4796f5e97e EQUAL +---- + +This P2SH script references the hash of a _redeem script_ that defines a 2-of-3 multisignature requirement to spend funds. To spend this output, Mohammed's company would present the redeem script (whose hash matches the script hash in the P2SH output) and the signatures necessary to satisfy that redeem script, all inside the transaction input: + +.Decoded transaction showing a P2SH output being spent +---- +[...] +“Vin” : [ +"txid": "abcdef12345...", +"vout": 0, + "scriptSig": “ <2 PubA PubB PubC PubD PubE 5 CHECKMULTISIG>”, +] +---- + +Now, let's look at how this entire example would be upgraded to segwit. If Mohammed's customers were using a segwit-compatible wallet, they would make a payment, creating a Pay-to-Witness-Script-Hash (P2WSH) output that would look like this: + +.Example P2WSH output script +---- +0 9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73 +---- + +Again, as with the example of P2WPKH, you can see that the Segregated Witness equivalent script is a lot simpler and omits the various script operands that you see in P2SH scripts. Instead, the Segregated Witness program consists of two values pushed to the stack: a witness version (0) and the 32-byte SHA256 hash of the redeem script. + +[TIP] +==== +While P2SH uses the 20-byte +RIPEMD160(SHA256(script))+ hash, the P2WSH witness program uses a 32-byte +SHA256(script)+ hash. This difference in the selection of the hashing algorithm is deliberate and is used to differentiate between the two types of witness programs (P2WPKH and P2WSH) by the length of the hash and to provide stronger security to P2WSH (128 bits of security in P2WSH versus 80 bits of security in P2SH). + +==== + +Mohammed's company can spend outputs the P2WSH output by presenting the correct redeem script and sufficient signatures to satisfy it. Both the redeem script and the signatures would be segregated _outside_ the spending transaction as part of the witness data. Within the transaction input, Mohammed's ((("", startref="mohamappd")))wallet would put an empty +scriptSig+: + +.Decoded transaction showing a P2WSH output being spent with separate witness data +---- +[...] +“Vin” : [ +"txid": "abcdef12345...", +"vout": 0, + "scriptSig": “”, +] +[...] +“witness”: “ <2 PubA PubB PubC PubD PubE 5 CHECKMULTISIG>” +[...] +---- + +===== Differentiating between P2WPKH and P2WSH + +In the previous two sections, we demonstrated two types of witness programs: <> and <>. Both types of witness programs consist of a single byte version number followed by a longer hash. They look very similar, but are interpreted very differently: one is interpreted as a public key hash, which is satisfied by a signature and the other as a script hash, which is satisfied by a redeem script. The critical difference between them is the length of the hash: + +* The public key hash in P2WPKH is 20 bytes +* The script hash in P2WSH is 32 bytes + +This is the one difference that allows a wallet to differentiate between the two types of witness programs. By looking at the length of the hash, a wallet can determine what type of witness program it is, P2WPKH or P2WSH. + +==== Upgrading to Segregated Witness + +As we can see from the previous examples, upgrading to Segregated Witness is a two-step process. First, wallets must create special segwit type outputs. Then, these outputs can be spent by wallets that know how to construct Segregated Witness transactions. In the examples, Alice's wallet was segwit-aware and able to create special outputs with Segregated Witness scripts. Bob's wallet is also segwit-aware and able to spend those outputs. What may not be obvious from the example is that in practice, Alice's wallet needs to _know_ that Bob uses a segwit-aware wallet and can spend these outputs. Otherwise, if Bob's wallet is not upgraded and Alice tries to make segwit payments to Bob, Bob's wallet will not be able to detect these payments. + +[TIP] +==== +For P2WPKH and P2WSH payment types, both the sender and the recipient wallets need to be upgraded to be able to use segwit. Furthermore, the sender's wallet needs to know that the recipient's wallet is segwit-aware. +==== + +Segregated Witness will not be implemented simultaneously across the entire network. Rather, Segregated Witness is implemented as a backward-compatible upgrade, where _old and new clients can coexist_. Wallet developers will independently upgrade wallet software to add segwit capabilities. The P2WPKH and P2WSH payment types are used when both sender and recipient are segwit-aware. The traditional P2PKH and P2SH will continue to work for nonupgraded wallets. That leaves two important scenarios, which are addressed in the next section: + +* Ability of a sender's wallet that is not segwit-aware to make a payment to a recipient's wallet that can process segwit transactions + +* Ability of a sender's wallet that is segwit-aware to recognize and distinguish between recipients that are segwit-aware and ones that are not, by their _addresses_. + +===== Embedding Segregated Witness inside P2SH + +Let's assume, for example, that Alice's wallet is not upgraded to segwit, but Bob's wallet is upgraded and can handle segwit transactions. Alice and Bob can use "old" non-segwit transactions. But Bob would likely want to use segwit to reduce transaction fees, taking advantage of the discount that applies to witness data. + +In this case Bob's wallet can construct a P2SH address that contains a segwit script inside it. Alice's wallet sees this as a "normal" P2SH address and can make payments to it without any knowledge of segwit. Bob's wallet can then spend this payment with a segwit transaction, taking full advantage of segwit and reducing transaction fees. + +Both forms of witness scripts, P2WPKH and P2WSH, can be embedded in a P2SH address. The first is noted as P2SH(P2WPKH) and the second is noted as P2SH(P2WSH). + +===== Pay-to-Witness-Public-Key-Hash inside Pay-to-Script-Hash + +The first form of witness script we will examine is P2SH(P2WPKH). This is a Pay-to-Witness-Public-Key-Hash witness program, embedded inside a Pay-to-Script-Hash script, so that it can be used by a wallet that is not aware of segwit. + +Bob's wallet constructs a P2WPKH witness program with Bob's public key. This witness program is then hashed and the resulting hash is encoded as a P2SH script. The P2SH script is converted to a bitcoin address, one that starts with a "3," as we saw in the <> section. + +Bob's wallet starts with the P2WPKH witness program we saw earlier: + +.Bob's P2WPKH witness program +---- +0 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 +---- + +The P2WPKH witness program consists of the witness version and Bob's 20-byte public key hash. + +Bob's wallet then hashes the preceding witness program, first with SHA256, then with RIPEMD160, producing another 20-byte hash: + +.HASH160 of the P2WPKH witness program +---- +3e0547268b3b19288b3adef9719ec8659f4b2b0b +---- + +[role="pagebreak-before"] +The hash of the witness program is then embedded in a P2SH script: + +---- +HASH160 3e0547268b3b19288b3adef9719ec8659f4b2b0b EQUAL +---- + +Finally, the P2SH script is converted to a P2SH bitcoin address: + +.P2SH address +---- +37Lx99uaGn5avKBxiW26HjedQE3LrDCZru +---- + +Now, Bob can display this address for customers to pay for their coffee. Alice's wallet can make a payment to +37Lx99uaGn5avKBxiW26HjedQE3LrDCZru+, just as it would to any other bitcoin address. Even though Alice's wallet has no support for segwit, the payment it creates can be spent by Bob with a segwit transaction.((("", startref="aliced"))) + +===== Pay-to-Witness-Script-Hash inside Pay-to-Script-Hash + +Similarly, a P2WSH witness program for a multisig script or other complicated script can be embedded inside a P2SH script and address, making it possible for any wallet to make payments that are segwit compatible. + +As we saw in <>, Mohammed's ((("use cases", "import/export")))company is using Segregated Witness payments to multisignature scripts. To make it possible for any client to pay his company, regardless of whether their wallets are upgraded for segwit, Mohammed's wallet can embed the P2WSH witness program inside a P2SH script. + +First, Mohammed's wallet creates the P2WSH witness program that corresponds to the multisignature script, hashed with SHA256: + +.Mohammed's wallet creates a P2WSH witness program +---- +0 9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73 +---- + +Then, the witness program itself is hashed with SHA256 and RIPEMD160, producing a new 20-byte hash, as used in traditional P2SH: + +.The HASH160 of the P2WSH witness program +---- +86762607e8fe87c0c37740cddee880988b9455b2 +---- + +Next, Mohammed's wallet puts the hash into a P2SH script: + +---- +HASH160 86762607e8fe87c0c37740cddee880988b9455b2 EQUAL +---- + +Finally, the wallet constructs a bitcoin address from this script: + +.P2SH bitcoin address +---- +3Dwz1MXhM6EfFoJChHCxh1jWHb8GQqRenG +---- + +Now, Mohammed's clients can make payments to this address without any need to support segwit. Mohammed's company can then construct segwit transactions to spend these payments, taking advantage of segwit features including lower transaction fees. + +===== Segregated Witness addresses + +Even after segwit activation, it will take some time until most wallets are upgraded. At first, segwit will be embedded in P2SH, as we saw in the previous section, to ease compatibility between segit-aware and unaware wallets. + +However, once wallets are broadly supporting segwit, it makes sense to encode witness scripts directly in a native address format designed for segwit, rather than embed it in P2SH. + +The native segwit address format is defined in BIP-173: + +https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki[BIP-173] +:: Base32 address format for native v0-16 witness outputs + +BIP-173 only encodes witness (P2WPKH and P2WSH) scripts. It is not compatible with non-segwit P2PKH or P2SH scripts. BIP-173 is a checksummed Base32 encoding, as compared to the Base58 encoding of a "traditional" bitcoin address. BIP-173 addesses are also called _bech32_ addresses, pronounced "beh-ch thirty two", alluding to the use of a "BCH" error detection algorithm and 32-character encoding set. + +BIP-173 addresses use 32 lower-case-only alphanumeric character set, carefully selected to reduce errors from misreading or mistyping. By choosing a lower-case-only character set, bech32 is easier to read, speak, and 45% more efficient to encode in QR codes. + +The BCH error detection algorithm is a vast improvement over the previous checksum algorithm (from Base58Check), allowing not only detection but also _correction_ of errors. Address-input interfaces (such as text-fields in forms) can detect and highlight which character was most likely mistyped when they detect an error. + +From the BIP-173 specification, here are some examples of bech32 addresses: + +Mainnet P2WPKH:: bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4 +Testnet P2WPKH:: tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx +Mainnet P2WSH:: bc1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3qccfmv3 +Testnet P2WSH:: tb1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3q0sl5k7 + +As you can see in these examples, a segwit bech32 string is up to 90 characters long and consists of three parts: + +The human readable part:: This prefix "bc" or "tb" identifying mainnet or testnet. + +The separator:: The digit "1", which is not part of the 32-character encoding set and can only appear in this position as a separator + +The data part:: A minimum of 6 alphanumeric characters, the checksum encoded witness script + +At this time, only a few wallets accept or produce native segwit bech32 addresses, but as segwit adoption increases, you will see these more and more often. + +[[segwit_txid]] +===== Transaction identifiers + +((("transaction IDs (txid)")))One of the greatest benefits of Segregated Witness is that it eliminates third-party transaction malleability. + +Before segwit, transactions could have their signatures subtly modified by third parties, changing their transaction ID (hash) without changing any fundamental properties (inputs, outputs, amounts). This created opportunities for denial-of-service attacks as well as attacks against poorly written wallet software that assumed unconfirmed transaction hashes were immutable. + +With the introduction of Segregated Witness, transactions have two identifiers, +txid+ and +wtxid+. The traditional transaction ID +txid+ is the double-SHA256 hash of the serialized transaction, without the witness data. A transaction +wtxid+ is the double-SHA256 hash of the new serialization format of the transaction with witness data. + +The traditional +txid+ is calculated in exactly the same way as with a nonsegwit transaction. However, since the segwit transaction has empty ++scriptSig++s in every input, there is no part of the transaction that can be modified by a third party. Therefore, in a segwit transaction, the +txid+ is immutable by a third party, even when the transaction is unconfirmed. + +The +wtxid+ is like an "extended" ID, in that the hash also incorporates the witness data. If a transaction is transmitted without witness data, then the +wtxid+ and +txid+ are identical. Note than since the +wtxid+ includes witness data (signatures) and since witness data may be malleable, the +wtxid+ should be considered malleable until the transaction is confirmed. Only the +txid+ of a segwit transaction can be considered immutable by third parties and only if _all_ the inputs of the transaction are segwit inputs. + +[TIP] +==== +Segregated Witness transactions have two IDs: +txid+ and +wtxid+. The +txid+ is the hash of the transaction without the witness data and the +wtxid+ is the hash inclusive of witness data. The +txid+ of a transaction where all inputs are segwit inputs is not susceptible to third-party transaction malleability. +==== + +==== Segregated Witness' New Signing Algorithm + +Segregated Witness modifies the semantics of the four signature verification functions (+CHECKSIG+, +CHECKSIGVERIFY+, +CHECKMULTISIG+, and +CHECKMULTISIGVERIFY+), changing the way a transaction commitment hash is calculated. + +Signatures in bitcoin transactions are applied on a _commitment hash_, which is calculated from the transaction data, locking specific parts of the data indicating the signer's commitment to those values. For example, in a simple +SIGHASH_ALL+ type signature, the commitment hash includes all inputs and outputs. + +Unfortunately, the way the commitment hash was calculated introduced the possibility that a node verifying the signature can be forced to perform a significant number of hash computations. Specifically, the hash operations increase in O(n^2^) with respect to the number of signature operations in the transaction. An attacker could therefore create a transaction with a very large number of signature operations, causing the entire bitcoin network to have to perform hundreds or thousands of hash operations to verify the transaction. + +Segwit represented an opportunity to address this problem by changing the way the commitment hash is calculated. For segwit version 0 witness programs, signature verification occurs using an improved commitment hash algorithm as specified in BIP-143. + +The new algorithm achieves two important goals. Firstly, the number of hash operations increases by a much more gradual O(n) to the number of signature operations, reducing the opportunity to create denial-of-service attacks with overly complex transactions. Secondly, the commitment hash now also includes the value (amounts) of each input as part of the commitment. This means that a signer can commit to a specific input value without needing to "fetch" and check the previous transaction referenced by the input. In the case of offline devices, such as hardware wallets, this greatly simplifies the communication between the host and the hardware wallet, removing the need to stream previous transactions for validation. A hardware wallet can accept the input value "as stated" by an untrusted host. Since the signature is invalid if that input value is not correct, the hardware wallet doesn't need to validate the value before signing the input. + +==== Economic Incentives for Segregated Witness + +Bitcoin mining nodes and full nodes incur costs for the resources used to support the bitcoin network and the blockchain. As the volume of bitcoin transactions increases, so does the cost of resources (CPU, network bandwidth, disk space, memory). Miners are compensated for these costs through fees that are proportional to the size (in bytes) of each transaction. Nonmining full nodes are not compensated, so they incur these costs because they have a need to run an authoritative fully validating full-index node, perhaps because they use the node to operate a bitcoin business. + +Without transaction fees, the growth in bitcoin data would arguably increase dramatically. Fees are intended to align the needs of bitcoin users with the burden their transactions impose on the network, through a market-based price discovery mechanism. + +The calculation of fees based on transaction size treats all the data in the transaction as equal in cost. But from the perspective of full nodes and miners, some parts of a transaction carry much higher costs. Every transaction added to the bitcoin network affects the consumption of four resources on nodes: + +Disk Space :: Every transaction is stored in the blockchain, adding to the total size of the blockchain. The blockchain is stored on disk, but the storage can be optimized by “pruning” older transactions. + +CPU :: Every transaction must be validated, which requires CPU time. + +Bandwidth :: Every transaction is transmitted (through flood propagation) across the network at least once. Without any optimization in the block propagation protocol, transactions are transmitted again as part of a block, doubling the impact on network capacity. + +Memory :: Nodes that validate transactions keep the UTXO index or the entire UTXO set in memory to speed up validation. Because memory is at least one order of magnitude more expensive than disk, growth of the UTXO set contributes disproportionately to the cost of running a node. + +As you can see from the list, not every part of a transaction has an equal impact on the cost of running a node or on the ability of bitcoin to scale to support more transactions. The most expensive part of a transaction are the newly created outputs, as they are added to the in-memory UTXO set. By comparison, signatures (aka witness data) add the least burden to the network and the cost of running a node, because witness data are only validated once and then never used again. Furthermore, immediately after receiving a new transaction and validating witness data, nodes can discard that witness data. If fees are calculated on transaction size, without discriminating between these two types of data, then the market incentives of fees are not aligned with the actual costs imposed by a transaction. In fact, the current fee structure actually encourages the opposite behavior, because witness data is the largest part of a transaction. + +The incentives created by fees matter because they affect the behavior of wallets. All wallets must implement some strategy for assembling transactions that takes into consideration a number of factors, such as privacy (reducing address reuse), fragmentation (making lots of loose change), and fees. If the fees are overwhelmingly motivating wallets to use as few inputs as possible in transactions, this can lead to UTXO picking and change address strategies that inadvertently bloat the UTXO set. + +Transactions consume UTXO in their inputs and create new UTXO with their outputs. A transaction, therefore, that has more inputs than outputs will result in a decrease in the UTXO set, whereas a transaction that has more outputs than inputs will result in an increase in the UTXO set. Let’s consider the _difference_ between inputs and outputs and call that the “Net-new-UTXO.” That’s an important metric, as it tells us what impact a transaction will have on the most expensive network-wide resource, the in-memory UTXO set. A transaction with positive Net-new-UTXO adds to that burden. A transaction with a negative Net-new-UTXO reduces the burden. We would therefore want to encourage transactions that are either negative Net-new-UTXO or neutral with zero Net-new-UTXO. + +Let’s look at an example of what incentives are created by the transaction fee calculation, with and without Segregated Witness. We will look at two different transactions. Transaction A is a 3-input, 2-output transaction, which has a Net-new-UTXO metric of –1, meaning it consumes one more UTXO than it creates, reducing the UTXO set by one. Transaction B is a 2-input, 3-output transaction, which has a Net-new-UTXO metric of 1, meaning it adds one UTXO to the UTXO set, imposing additional cost on the entire bitcoin network. Both transactions use multisignature (2-of-3) scripts to demonstrate how complex scripts increase the impact of segregated witness on fees. Let’s assume a transaction fee of 30 satoshi per byte and a 75% fee discount on witness data: + +++++ +
+
Without Segregated Witness
+
+

Transaction A fee: 25,710 satoshi

+

Transaction B fee: 18,990 satoshi

+
+ +
With Segregated Witness
+
+

Transaction A fee: 8,130 satoshi

+

Transaction B fee: 12,045 satoshi

+
+
+++++ + + +Both transactions are less expensive when segregated witness is implemented. But comparing the costs between the two transactions, we see that before Segregated Witness, the fee is higher for the transaction that has a negative Net-new-UTXO. After Segregated Witness, the transaction fees align with the incentive to minimize new UTXO creation by not inadvertently penalizing transactions with many inputs. + +Segregated Witness therefore has two main effects on the fees paid by bitcoin users. Firstly, segwit reduces the overall cost of transactions by discounting witness data and increasing the capacity of the bitcoin blockchain. Secondly, segwit’s discount on witness data corrects a misalignment of incentives that may have inadvertently created more bloat in the UTXO set.((("", startref="Tadv07")))((("", startref="Ssegwit07"))) From 3bf14a5e5d9c6ec1ee933b924de9a3d43f58cdc9 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 17:09:29 -0800 Subject: [PATCH 14/23] Remove segwit appendix from book Renumbered index references in subsequent chapters --- appdx-bitcore.asciidoc | 4 ++-- appdx-bx.asciidoc | 10 +++++----- appdx-pycoin.asciidoc | 6 +++--- book.asciidoc | 2 -- 4 files changed, 10 insertions(+), 12 deletions(-) diff --git a/appdx-bitcore.asciidoc b/appdx-bitcore.asciidoc index e0e5b7e8..4fbc2850 100644 --- a/appdx-bitcore.asciidoc +++ b/appdx-bitcore.asciidoc @@ -4,7 +4,7 @@ == Bitcore -((("Bitcore", id="bitcore17")))Bitcore is a suite of tools provided by BitPay. Its goal is to provide easy-to-use tools for Bitcoin developers. Almost all of Bitcore's code is written in JavaScript. There are some modules written specifically for NodeJS. Finally, the "node" module of Bitcore includes Bitcoin Core's C++ code. Please see https://bitcore.io for more information. +((("Bitcore", id="bitcore16")))Bitcore is a suite of tools provided by BitPay. Its goal is to provide easy-to-use tools for Bitcoin developers. Almost all of Bitcore's code is written in JavaScript. There are some modules written specifically for NodeJS. Finally, the "node" module of Bitcore includes Bitcoin Core's C++ code. Please see https://bitcore.io for more information. === Bitcore's Feature List @@ -91,7 +91,7 @@ Broadcasting a transaction to the Bitcoin network 1. Copy the code below into a file called _broadcast.js_. 2. The +tx+ and +rbfTx+ variables are the output of +tx.serialize()+ and +rbfTx.serialize()+, respectively. 3. In order to replace-by-fee, the peer must support bitcoind option +mempoolreplace+ and have it set to +1+. -4. Run the file node _broadcast.js_((("", startref="bitcore17"))): +4. Run the file node _broadcast.js_((("", startref="bitcore16"))): ---- var p2p = require('bitcore-p2p'); diff --git a/appdx-bx.asciidoc b/appdx-bx.asciidoc index de789b69..b4fd2a55 100644 --- a/appdx-bx.asciidoc +++ b/appdx-bx.asciidoc @@ -3,7 +3,7 @@ == Bitcoin Explorer (bx) Commands -((("Bitcoin Explorer (bx) commands", id="BX19_1", range="startofrange")))Bitcoin Explorer (bx) is a command-line tool that offers a variety of commands for key management and transaction construction. It is part of the libbitcoin bitcoin library. +((("Bitcoin Explorer (bx) commands", id="BX18_1", range="startofrange")))Bitcoin Explorer (bx) is a command-line tool that offers a variety of commands for key management and transaction construction. It is part of the libbitcoin bitcoin library. ---- Usage: bx COMMAND [--help] @@ -95,7 +95,7 @@ Generate a random "seed" value using the +seed+ command, which uses the operatin ---- $ bx seed | bx ec-new > private_key -$ cat private_key +$ cat private_key 73096ed11ab9f1db6135857958ece7d73ea7c30862145bcc4bbc7649075de474 ---- @@ -103,14 +103,14 @@ Now, generate the public key from that private key using the +ec-to-public+ comm ---- $ bx ec-to-public < private_key > public_key -$ cat public_key +$ cat public_key 02fca46a6006a62dfdd2dbb2149359d0d97a04f430f12a7626dd409256c12be500 ---- We can reformat the +public_key+ as an address using the +ec-to-address+ command. We pass the _public_key_ into standard input: ---- -$ bx ec-to-address < public_key +$ bx ec-to-address < public_key 17re1S4Q8ZHyCP8Kw7xQad1Lr6XUzWUnkG ---- @@ -178,4 +178,4 @@ $ bx mnemonic-decode < words eb68ee9f3df6bd4441a9feadec179ff1 ---- -Mnemonic encoding can make the seed easier to record and even remember.(((range="endofrange", startref="BX19_1"))) +Mnemonic encoding can make the seed easier to record and even remember.(((range="endofrange", startref="BX18_1"))) diff --git a/appdx-pycoin.asciidoc b/appdx-pycoin.asciidoc index 43a3078d..9fd32208 100644 --- a/appdx-pycoin.asciidoc +++ b/appdx-pycoin.asciidoc @@ -9,7 +9,7 @@ The pycoin library supports both Python 2 (2.7.x) and Python 3 (after 3.3) and c === Key Utility (KU) -((("key utility (ku)", id="keyutil18")))The command-line utility +ku+ ("key utility") is a Swiss Army knife for manipulating keys. It supports BIP-32 keys, WIF, and addresses (bitcoin and alt coins). Following are some examples. +((("key utility (ku)", id="keyutil17")))The command-line utility +ku+ ("key utility") is a Swiss Army knife for manipulating keys. It supports BIP-32 keys, WIF, and addresses (bitcoin and alt coins). Following are some examples. Create a BIP-32 key using the default entropy sources of GPG and _/dev/random_: @@ -329,7 +329,7 @@ Dogecoin address : DFpN6QqFfUm3gKNaxN6tNcab1FArL9cZLE ==== Transaction Utility (TX) -((("transaction utility (TX)", id="TX18"))) +((("transaction utility (TX)", id="TX17"))) The command-line utility +tx+ will display transactions in human-readable form, fetch base transactions from pycoin's transaction cache or from web services (blockchain.info, blockcypher.com, blockr.io and chain.so are currently supported), merge transactions, add or delete inputs or outputs, and sign transactions. Following are some examples. @@ -405,7 +405,7 @@ Total fees 0.00000 mBTC all incoming transaction values validated ---- -((("", startref="TX18")))Now, let's look at unspent outputs for a specific address (UTXO). In block #1, we see a coinbase transaction to +12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX+. Let's use +fetch_unspent+ to find all coins in this address: +((("", startref="TX17")))Now, let's look at unspent outputs for a specific address (UTXO). In block #1, we see a coinbase transaction to +12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX+. Let's use +fetch_unspent+ to find all coins in this address: ---- $ fetch_unspent 12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX diff --git a/book.asciidoc b/book.asciidoc index 5ae3e541..4ee73e54 100644 --- a/book.asciidoc +++ b/book.asciidoc @@ -34,8 +34,6 @@ include::appdx-scriptops.asciidoc[] include::appdx-bips.asciidoc[] -include::appdx-segwit.asciidoc[] - include::appdx-bitcore.asciidoc[] include::appdx-pycoin.asciidoc[] From 9f674c3cfe5ab21f00a0d02f4fccad3e8b747120 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Fri, 2 Feb 2018 21:29:27 -0600 Subject: [PATCH 15/23] Link book.asciidoc in README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 272a470d..831a64f7 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ If you know how to make a pull request to contribute a fix, please write the cor # Reading this book (Where is the PDF?) -To read this book, see book.asciidoc. Click on each of the chapters to read in your browser. This is not as convenient as reading a PDF or an ebook on your e-reader. Convenience costs money (see below). +To read this book, see [book.asciidoc](https://github.com/bitcoinbook/bitcoinbook/blob/develop/book.asciidoc). Click on each of the chapters to read in your browser. This is not as convenient as reading a PDF or an ebook on your e-reader. Convenience costs money (see below). The 2nd edition of "Mastering Bitcoin" is available under a CC-BY-NC-ND license, not a CC-BY-SA license. From 41014a8c3a554ceab12c99735590b4fbe90631ec Mon Sep 17 00:00:00 2001 From: theStack Date: Sat, 3 Feb 2018 05:28:31 -0500 Subject: [PATCH 16/23] Revert "corrected standard multisignature tx limit to N=3" This reverts commit 1b2eb07bf16b9df5a69f42566aa4dc4a783d97d8. --- ch07.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index 72da4d7e..a6fae6c6 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -12,7 +12,7 @@ First, we will look at _multisignature_ scripts. Next, we will examine the secon [[multisig]] === Multisignature -((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multisignature scripts are limited to at most 3 listed public keys, meaning you can do anything from a 1-of-1 to a 3-of-3 multisignature or any combination within that range. The limitation to 3 listed keys might be lifted by the time this book is published, so check the +isStandard()+ function to see what is currently accepted by the network. +((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multisignature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multisignature or any combination within that range. The limitation to 15 listed keys might be lifted by the time this book is published, so check the +isStandard()+ function to see what is currently accepted by the network. The general form of a locking script setting an M-of-N multisignature condition is: From 2f1015c23f63d5936478c1d4b350f299e8435f39 Mon Sep 17 00:00:00 2001 From: theStack Date: Sat, 3 Feb 2018 05:41:05 -0500 Subject: [PATCH 17/23] s/isStandard/IsStandard/ (ref. to Bitcoin Core) --- ch07.asciidoc | 6 +++--- ch10.asciidoc | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index 781c4378..735ad427 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -12,7 +12,7 @@ First, we will look at _multisignature_ scripts. Next, we will examine the secon [[multisig]] === Multisignature -((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multisignature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multisignature or any combination within that range. The limitation to 15 listed keys might be lifted by the time this book is published, so check the +isStandard()+ function to see what is currently accepted by the network. +((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multisignature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multisignature or any combination within that range. The limitation to 15 listed keys might be lifted by the time this book is published, so check the +IsStandard()+ function to see what is currently accepted by the network. The general form of a locking script setting an M-of-N multisignature condition is: @@ -197,7 +197,7 @@ P2SH addresses hide all of the complexity, so that the person making a payment d ==== Redeem Script and Validation -((("scripting", "Pay-to-Script-Hash", "redeem scripts and validation")))((("Pay-to-Script-Hash (P2SH)", "redeem scripts and validation")))((("redeem scripts")))((("validation")))Prior to version 0.9.2 of the Bitcoin Core client, Pay-to-Script-Hash was limited to the standard types of bitcoin transaction scripts, by the +isStandard()+ function. That means that the redeem script presented in the spending transaction could only be one of the standard types: P2PK, P2PKH, or multisig nature, excluding +RETURN+ and P2SH itself. +((("scripting", "Pay-to-Script-Hash", "redeem scripts and validation")))((("Pay-to-Script-Hash (P2SH)", "redeem scripts and validation")))((("redeem scripts")))((("validation")))Prior to version 0.9.2 of the Bitcoin Core client, Pay-to-Script-Hash was limited to the standard types of bitcoin transaction scripts, by the +IsStandard()+ function. That means that the redeem script presented in the spending transaction could only be one of the standard types: P2PK, P2PKH, or multisig nature, excluding +RETURN+ and P2SH itself. As of version 0.9.2 of the Bitcoin Core client, P2SH transactions can contain any valid script, making the P2SH standard much more flexible and allowing for experimentation with many novel and complex types of transactions. @@ -231,7 +231,7 @@ RETURN Keep in mind that there is no "unlocking script" that corresponds to +RETURN+ that could possibly be used to "spend" a +RETURN+ output. The whole point of +RETURN+ is that you can't spend the money locked in that output, and therefore it does not need to be held in the UTXO set as potentially spendable—+RETURN+ is _provably unspendable_. +RETURN+ is usually an output with a zero bitcoin amount, because any bitcoin assigned to such an output is effectively lost forever. If a +RETURN+ is referenced as an input in a transaction, the script validation engine will halt the execution of the validation script and mark the transaction as invalid. The execution of +RETURN+ essentially causes the script to "RETURN" with a +FALSE+ and halt. Thus, if you accidentally reference a +RETURN+ output as an input in a transaction, that transaction is invalid. -A standard transaction (one that conforms to the +isStandard()+ checks) can have only one +RETURN+ output. However, a single +RETURN+ output can be combined in a transaction with outputs of any other type. +A standard transaction (one that conforms to the +IsStandard()+ checks) can have only one +RETURN+ output. However, a single +RETURN+ output can be combined in a transaction with outputs of any other type. Two new command-line options have been added in Bitcoin Core as of version 0.10. The option +datacarrier+ controls relay and mining of +RETURN+ transactions, with the default set to "1" to allow them. The option +datacarriersize+ takes a numeric argument specifying the maximum size in bytes of the +RETURN+ script, 83 bytes by default, which, allows for a maximum of 80 bytes of +RETURN+ data plus one byte of +RETURN+ opcode and two bytes of +PUSHDATA+ opcode. diff --git a/ch10.asciidoc b/ch10.asciidoc index 6459529f..8216a090 100644 --- a/ch10.asciidoc +++ b/ch10.asciidoc @@ -113,7 +113,7 @@ Each node verifies every transaction against a long checklist of criteria: * +nLocktime+ is equal to +INT_MAX+, or +nLocktime+ and +nSequence+ values are satisfied according to +MedianTimePast+. * The transaction size in bytes is greater than or equal to 100. * The number of signature operations (SIGOPS) contained in the transaction is less than the signature operation limit. -* The unlocking script (+scriptSig+) can only push numbers on the stack, and the locking script (+scriptPubkey+) must match +isStandard+ forms (this rejects "nonstandard" transactions). +* The unlocking script (+scriptSig+) can only push numbers on the stack, and the locking script (+scriptPubkey+) must match +IsStandard+ forms (this rejects "nonstandard" transactions). * A matching transaction in the pool, or in a block in the main branch, must exist. * For each input, if the referenced output exists in any other transaction in the pool, the transaction must be rejected. * For each input, look in the main branch and the transaction pool to find the referenced output transaction. If the output transaction is missing for any input, this will be an orphan transaction. Add to the orphan transactions pool, if a matching transaction is not already in the pool. From 4ebfce27d23614e42c843d1e43979e5d0ba01612 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 3 Feb 2018 15:35:51 -0600 Subject: [PATCH 18/23] wording --- ch07.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index 781c4378..e8ef0dfb 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -130,7 +130,7 @@ If the placeholders are replaced by actual public keys (shown here as 520-bit nu This entire script can instead be represented by a 20-byte cryptographic hash, by first applying the SHA256 hashing algorithm and then applying the RIPEMD160 algorithm on the result. -We use +libbitcoin-explorer+ (+bx+) to produce the script hash, as follows: +We use +libbitcoin-explorer+ (+bx+) on the command-line to produce the script hash, as follows: ---- echo \ From 2fcc1db70f180e7b6135a8178b6f5f150be01bd8 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 3 Feb 2018 15:36:16 -0600 Subject: [PATCH 19/23] wording --- ch07.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index e8ef0dfb..7d1c3990 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -145,15 +145,15 @@ echo \ 54c557e07dde5bb6cb791c7a540e0a4796f5e97e ---- +The series of commands above first encodes Mohammed's multisig redeem script as a serialized hex-encoded bitcoin Script. The next +bx+ command calculates the SHA256 hash of that. The next +bx+ command hashes again with RIPEMD160, producing the final script-hash: The 20-byte hash of Mohammed's redeem script is: - ---- 54c557e07dde5bb6cb791c7a540e0a4796f5e97e ---- -A P2SH transaction locks the output to this hash instead of the longer script, using the locking script: +A P2SH transaction locks the output to this hash instead of the longer redeem script, using the locking script: ---- HASH160 54c557e07dde5bb6cb791c7a540e0a4796f5e97e EQUAL From 240b5b0c1c3244a43eb3f72ec018baebf74f40b5 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 3 Feb 2018 15:36:44 -0600 Subject: [PATCH 20/23] Better P2SH example with bx --- ch07.asciidoc | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index 7d1c3990..b5f4b7d0 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -180,7 +180,19 @@ Almost all the scripts described in this chapter can only be implemented as P2SH ==== P2SH Addresses -((("scripting", "Pay-to-Script-Hash", "addresses")))((("Pay-to-Script-Hash (P2SH)", "addresses")))((("bitcoin improvement proposals", "Address Format for P2SH (BIP-13)")))Another important part of the P2SH feature is the ability to encode a script hash as an address, as defined in BIP-13. P2SH addresses are Base58Check encodings of the 20-byte hash of a script, just like bitcoin addresses are Base58Check encodings of the 20-byte hash of a public key. P2SH addresses use the version prefix "5," which results in Base58Check-encoded addresses that start with a "3." For example, Mohammed's complex script, hashed and Base58Check-encoded as a P2SH address, becomes +39RF6JqABiHdYHkfChV6USGMe6Nsr66Gzw+. Now, Mohammed can give this "address" to his customers and they can use almost any bitcoin wallet to make a simple payment, as if it were a bitcoin address. The 3 prefix gives them a hint that this is a special type of address, one corresponding to a script instead of a public key, but otherwise it works in exactly the same way as a payment to a bitcoin address. +((("scripting", "Pay-to-Script-Hash", "addresses")))((("Pay-to-Script-Hash (P2SH)", "addresses")))((("bitcoin improvement proposals", "Address Format for P2SH (BIP-13)")))Another important part of the P2SH feature is the ability to encode a script hash as an address, as defined in BIP-13. P2SH addresses are Base58Check encodings of the 20-byte hash of a script, just like bitcoin addresses are Base58Check encodings of the 20-byte hash of a public key. P2SH addresses use the version prefix "5," which results in Base58Check-encoded addresses that start with a "3." + +For example, Mohammed's complex script, hashed and Base58Check-encoded as a P2SH address, becomes +39RF6JqABiHdYHkfChV6USGMe6Nsr66Gzw+. We can confirm that with the +bx+ command: + +---- +echo \ +'54c557e07dde5bb6cb791c7a540e0a4796f5e97e'\ + | bx address-encode -v 5 +39RF6JqABiHdYHkfChV6USGMe6Nsr66Gzw +---- + + +Now, Mohammed can give this "address" to his customers and they can use almost any bitcoin wallet to make a simple payment, as if it were a bitcoin address. The 3 prefix gives them a hint that this is a special type of address, one corresponding to a script instead of a public key, but otherwise it works in exactly the same way as a payment to a bitcoin address. P2SH addresses hide all of the complexity, so that the person making a payment does not see the script. From 1ddd89b65841dea02b5c4ab1e31cf1f4de23ba52 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 3 Feb 2018 15:38:34 -0600 Subject: [PATCH 21/23] Several fixes to Segwit/P2SH #440 There were several errors in the address and hash calculations. Fixed errors and added command line examples to clarify the process --- ch07.asciidoc | 76 +++++++++++++++++++++++++++++++++++---------------- 1 file changed, 53 insertions(+), 23 deletions(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index b5f4b7d0..01b4bf63 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -777,7 +777,7 @@ Now, let's look at how this entire example would be upgraded to segwit. If Moham .Example P2WSH output script ---- -0 9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73 +0 a9b7b38d972cabc7961dbfbcb841ad4508d133c47ba87457b4a0e8aae86dbb89 ---- Again, as with the example of P2WPKH, you can see that the Segregated Witness equivalent script is a lot simpler and omits the various script operands that you see in P2SH scripts. Instead, the Segregated Witness program consists of two values pushed to the stack: a witness version (0) and the 32-byte SHA256 hash of the redeem script. @@ -850,28 +850,37 @@ Bob's wallet starts with the P2WPKH witness program we saw earlier: The P2WPKH witness program consists of the witness version and Bob's 20-byte public key hash. -Bob's wallet then hashes the preceding witness program, first with SHA256, then with RIPEMD160, producing another 20-byte hash: +Bob's wallet then hashes the preceding witness program, first with SHA256, then with RIPEMD160, producing another 20-byte hash. + +Let's use +bx+ on the command-line to replicate that: .HASH160 of the P2WPKH witness program ---- +echo \ +'0 [ab68025513c3dbd2f7b92a94e0581f5d50f654e7]'\ + | bx script-encode | bx sha256 | bx ripemd160 3e0547268b3b19288b3adef9719ec8659f4b2b0b ---- -[role="pagebreak-before"] -The hash of the witness program is then embedded in a P2SH script: +Next, the redeem script hash is converted to a bitcoin address. Let's use +bx+ on the command-line again: + +.P2SH address +---- +echo \ +'3e0547268b3b19288b3adef9719ec8659f4b2b0b' \ +| bx address-encode -v 5 +37Lx99uaGn5avKBxiW26HjedQE3LrDCZru +---- + +Now, Bob can display this address for customers to pay for their coffee. Alice's wallet can make a payment to +37Lx99uaGn5avKBxiW26HjedQE3LrDCZru+, just as it would to any other bitcoin address. + +To pay Bob, Alice's wallet would lock the output with a P2SH script: ---- HASH160 3e0547268b3b19288b3adef9719ec8659f4b2b0b EQUAL ---- -Finally, the P2SH script is converted to a P2SH bitcoin address: - -.P2SH address ----- -37Lx99uaGn5avKBxiW26HjedQE3LrDCZru ----- - -Now, Bob can display this address for customers to pay for their coffee. Alice's wallet can make a payment to +37Lx99uaGn5avKBxiW26HjedQE3LrDCZru+, just as it would to any other bitcoin address. Even though Alice's wallet has no support for segwit, the payment it creates can be spent by Bob with a segwit transaction.((("", startref="aliced"))) +Even though Alice's wallet has no support for segwit, the payment it creates can be spent by Bob with a segwit transaction.((("", startref="aliced"))) ===== Pay-to-Witness-Script-Hash inside Pay-to-Script-Hash @@ -879,34 +888,55 @@ Similarly, a P2WSH witness program for a multisig script or other complicated sc As we saw in <>, Mohammed's ((("use cases", "import/export")))company is using Segregated Witness payments to multisignature scripts. To make it possible for any client to pay his company, regardless of whether their wallets are upgraded for segwit, Mohammed's wallet can embed the P2WSH witness program inside a P2SH script. -First, Mohammed's wallet creates the P2WSH witness program that corresponds to the multisignature script, hashed with SHA256: +First, Mohammed's wallet hashes the redeem script with SHA256 (just once). Let's use +bx+ to do that on the command-line: .Mohammed's wallet creates a P2WSH witness program +---- +echo \ +2 \ [04C16B8698A9ABF84250A7C3EA7EEDEF9897D1C8C6ADF47F06CF73370D74DCCA01CDCA79DCC5C395D7EEC6984D83F1F50C900A24DD47F569FD4193AF5DE762C587] \ +[04A2192968D8655D6A935BEAF2CA23E3FB87A3495E7AF308EDF08DAC3C1FCBFC2C75B4B0F4D0B1B70CD2423657738C0C2B1D5CE65C97D78D0E34224858008E8B49] \ +[047E63248B75DB7379BE9CDA8CE5751D16485F431E46117B9D0C1837C9D5737812F393DA7D4420D7E1A9162F0279CFC10F1E8E8F3020DECDBC3C0DD389D9977965] \ +[0421D65CBD7149B255382ED7F78E946580657EE6FDA162A187543A9D85BAAA93A4AB3A8F044DADA618D087227440645ABE8A35DA8C5B73997AD343BE5C2AFD94A5] \ +[043752580AFA1ECED3C68D446BCAB69AC0BA7DF50D56231BE0AABF1FDEEC78A6A45E394BA29A1EDF518C022DD618DA774D207D137AAB59E0B000EB7ED238F4D800] \ +5 CHECKMULTISIG \ +| bx script-encode | bx sha256 +9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73 +---- + +Next, the hashed redeem script is turned into a P2WSH witness program: + ---- 0 9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73 ---- -Then, the witness program itself is hashed with SHA256 and RIPEMD160, producing a new 20-byte hash, as used in traditional P2SH: +Then, the witness program itself is hashed with SHA256 and RIPEMD160, producing a new 20-byte hash, as used in traditional P2SH. Let's use +bx+ on the command-line to do that: .The HASH160 of the P2WSH witness program ---- + echo \ +'0 [9592d601848d04b172905e0ddb0adde59f1590f1e553ffc81ddc4b0ed927dd73]'\ + | bx script-encode | bx sha256 | bx ripemd160 86762607e8fe87c0c37740cddee880988b9455b2 ---- -Next, Mohammed's wallet puts the hash into a P2SH script: +Next, the wallet constructs a P2SH bitcoin address from this hash. Again, we use +bx+ to calculate on the command-line: +.P2SH bitcoin address +---- +echo \ +'86762607e8fe87c0c37740cddee880988b9455b2'\ + | bx address-encode -v 5 +3Dwz1MXhM6EfFoJChHCxh1jWHb8GQqRenG +---- + +Now, Mohammed's clients can make payments to this address without any need to support segwit. To send a payment to Mohammed, a wallet would lock the output with the following P2SH script: + +.P2SH script used to lock payments to Mohammed's multisig ---- HASH160 86762607e8fe87c0c37740cddee880988b9455b2 EQUAL ---- -Finally, the wallet constructs a bitcoin address from this script: - -.P2SH bitcoin address ----- -3Dwz1MXhM6EfFoJChHCxh1jWHb8GQqRenG ----- - -Now, Mohammed's clients can make payments to this address without any need to support segwit. Mohammed's company can then construct segwit transactions to spend these payments, taking advantage of segwit features including lower transaction fees. +Mohammed's company can then construct segwit transactions to spend these payments, taking advantage of segwit features including lower transaction fees. ===== Segregated Witness addresses From 76e44f226ebf37f2d4cef0f270ab9423b011ca73 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 3 Feb 2018 16:02:41 -0600 Subject: [PATCH 22/23] Rewrite 3-key limit information --- ch07.asciidoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index 735ad427..26b14cfb 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -12,7 +12,9 @@ First, we will look at _multisignature_ scripts. Next, we will examine the secon [[multisig]] === Multisignature -((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multisignature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multisignature or any combination within that range. The limitation to 15 listed keys might be lifted by the time this book is published, so check the +IsStandard()+ function to see what is currently accepted by the network. +((("transactions", "advanced", "multisignature scripts")))((("transactions", "advanced", id="Tadv07")))((("scripting", "multisignature scripts", id="Smulti07")))((("multisignature scripts")))Multisignature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to unlock the funds. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multisignature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. + +At this time, _standard_ multisignature scripts are limited to at most 3 listed public keys, meaning you can do anything from a 1-of-1 to a 3-of-3 multisignature or any combination within that range. The limitation to 3 listed keys might be lifted by the time this book is published, so check the +IsStandard()+ function to see what is currently accepted by the network. Note that the limit of 3 keys applies only to standard (also known as "bare") multisignature scripts, not to multisignature scripts wrapped in a Pay-to-Script-Hash (P2SH) script. P2SH multisignature scripts are limited to 15 keys, allowing for up to 15-of-15 multisignature. We will learn about P2SH in <>. The general form of a locking script setting an M-of-N multisignature condition is: From bd730b5570db9748ebf0717ee39f324c530d3a51 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 3 Feb 2018 16:04:39 -0600 Subject: [PATCH 23/23] wording --- ch07.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.asciidoc b/ch07.asciidoc index 26b14cfb..67e98380 100644 --- a/ch07.asciidoc +++ b/ch07.asciidoc @@ -199,7 +199,7 @@ P2SH addresses hide all of the complexity, so that the person making a payment d ==== Redeem Script and Validation -((("scripting", "Pay-to-Script-Hash", "redeem scripts and validation")))((("Pay-to-Script-Hash (P2SH)", "redeem scripts and validation")))((("redeem scripts")))((("validation")))Prior to version 0.9.2 of the Bitcoin Core client, Pay-to-Script-Hash was limited to the standard types of bitcoin transaction scripts, by the +IsStandard()+ function. That means that the redeem script presented in the spending transaction could only be one of the standard types: P2PK, P2PKH, or multisig nature, excluding +RETURN+ and P2SH itself. +((("scripting", "Pay-to-Script-Hash", "redeem scripts and validation")))((("Pay-to-Script-Hash (P2SH)", "redeem scripts and validation")))((("redeem scripts")))((("validation")))Prior to version 0.9.2 of the Bitcoin Core client, Pay-to-Script-Hash was limited to the standard types of bitcoin transaction scripts, by the +IsStandard()+ function. That means that the redeem script presented in the spending transaction could only be one of the standard types: P2PK, P2PKH, or multisig, excluding +RETURN+ and P2SH itself. As of version 0.9.2 of the Bitcoin Core client, P2SH transactions can contain any valid script, making the P2SH standard much more flexible and allowing for experimentation with many novel and complex types of transactions.