arno/bitcoinbook
1
0
Fork 0
mirror of https://github.com/bitcoinbook/bitcoinbook synced 2025-01-22 05:31:15 +00:00
Code Issues Releases Wiki Activity

Merge branch 'develop' into chapter/Chapter10

Browse Source
This commit is contained in:
Andreas M. Antonopoulos 2014-10-01 08:15:51 -04:00
commit 6fc6b4f70d
36 changed files with 1933 additions and 748 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md Normal file
View File

@ -0,0 +1,13 @@
# Mastering Bitcoin
Mastering Bitcoin is a book for developers, but the first two chapters cover bitcoin at a level that is approachable to non-programmers. Anyone with a basic understanding of technology can read the first two chapters and get a great understanding of bitcoin.
# Contributing
To contribute to this book, please fork and make changes to your forked copy, then submit a pull request. Or alternatively, if you cannot use the github.com pull request system, submit an Issue from the menu on the right. Major contributions will receive an acknowledgment in the preface of the book.
# Purchasing
"Mastering Bitcoin" will be available in print and e-book format in Fall 2014. To pre-order your copy, please visit:
http://bitcoinbook.info

163
appdx-scriptops.asciidoc Normal file
View File

@ -0,0 +1,163 @@
[[tx_script_ops]]
== Appendix: Transaction Script Language Operators, Constants and Symbols
[[tx_script_ops_table_pushdata]]
.Push Value onto Stack
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_0 or OP_FALSE | 0x00 | An empty array is pushed on to the stack
| 1-75 | 0x01-0x4b | Push the next N bytes onto the stack, where N is 1 to 75 bytes
| OP_PUSHDATA1 | 0x4c | The next script byte contains N, push the following N bytes onto the stack
| OP_PUSHDATA2 | 0x4d | The next two script bytes contain N, push the following N bytes onto the stack
| OP_PUSHDATA4 | 0x4e | The next four script bytes contain N, push the following N bytes onto the stack
| OP_1NEGATE | 0x4f | Push the value "-1" onto the stack
| OP_RESERVED | 0x50 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_1 or OP_TRUE| 0x51 | Push the value "1" onto the stack
| OP_2 to OP_16 | 0x52 to 0x60 | For OP_N, push the value "N" onto the stack. E.g., OP_2 pushes "2"
|=======
[[tx_script_ops_table_control]]
.Conditional Flow Control
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP | 0x61 | Do nothing
| OP_VER | 0x62 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_IF | 0x63 | Execute the statements following if top of stack is not 0
| OP_NOTIF | 0x64 | Execute the statements following if top of stack is 0
| OP_VERIF | 0x65 | Halt - Invalid transaction
| OP_VERNOTIF | 0x66 | Halt - Invalid transaction
| OP_ELSE | 0x67 | Execute only if the previous statements were not executed
| OP_ENDIF | 0x68 | Ends the OP_IF, OP_NOTIF, OP_ELSE block
| OP_VERIFY | 0x69 | Check the top of the stack, Halt and Invalidate transaction if not TRUE
| OP_RETURN | 0x6a | Halt and invalidate transaction
|=======
[[tx_script_ops_table_stack]]
.Stack Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_TOALTSTACK | 0x6b | Pop top item from stack and push to alternative stack
| OP_FROMALTSTACK | 0x6c | Pop top item from alternative stack and push to stack
| OP_2DROP | 0x6d | Pop top two stack items
| OP_2DUP | 0x6e | Duplicate top two stack items
| OP_3DUP | 0x6f | Duplicate top three stack items
| OP_2OVER | 0x70 | Copies the third and fourth items in the stack to the top
| OP_2ROT | 0x71 | Moves the fifth and sixth items in the stack to the top
| OP_2SWAP | 0x72 | Swap the two top pairs of items in the stack
| OP_IFDUP | 0x73 | Duplicate the top item in the stack if it is not 0
| OP_DEPTH | 0x74 | Count the items on the stack and push the resulting count
| OP_DROP | 0x75 | Pop the top item in the stack
| OP_DUP | 0x76 | Duplicate the top item in the stack
| OP_NIP | 0x77 | Pop the second item in the stack
| OP_OVER | 0x78 | Copy the second item in the stack and push it on to the top
| OP_PICK | 0x79 | Pop value N from top, then copy the Nth item to the top of the stack
| OP_ROLL | 0x7a | Pop value N from top, then move the Nth item to the top of the stack
| OP_ROT | 0x7b | Rotate the top three items in the stack
| OP_SWAP | 0x7c | Swap the top three items in the stack
| OP_TUCK | 0x7d | Copy the top item and insert it between the top and second item.
|=======
[[tx_script_ops_table_splice]]
.String Splice Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_CAT_ | 0x7e | Disabled (Concatenates top two items)
| _OP_SUBSTR_ | 0x7f | Disabled (Returns substring)
| _OP_LEFT_ | 0x80 | Disabled (Returns left substring)
| _OP_RIGHT_ | 0x81 | Disabled (Returns right substring)
| OP_SIZE | 0x82 | Calculate string length of top item and push the result
|=======
[[tx_script_ops_table_binmath]]
.Binary Arithmetic and Conditionals
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_INVERT_ | 0x83 | Disabled (Flip the bits of the top item)
| _OP_AND_ | 0x84 | Disabled (Boolean AND of two top items)
| _OP_OR_ | 0x85 | Disabled (Boolean OR of two top items)
| _OP_XOR_ | 0x86 | Disabled (Boolean XOR of two top items)
| OP_EQUAL | 0x87 | Push TRUE (1) if top two items are exactly equal, push FALSE (0) otherwise
| OP_EQUALVERIFY | 0x88 | Same as OP_EQUAL, but run OP_VERIFY after to halt if not TRUE
| OP_RESERVED1 | 0x89 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_RESERVED2 | 0x8a | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
|=======
[[tx_script_ops_table_numbers]]
.Numeric Operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_1ADD | 0x8b | Add 1 to the top item
| OP_1SUB | 0x8c | Subtract 1 from the top item
| _OP_2MUL_ | 0x8d | Disabled (Multiply top item by 2)
| _OP_2DIV_ | 0x8e | Disabled (Divide top item by 2)
| OP_NEGATE | 0x8f | Flip the sign of top item
| OP_ABS | 0x90 | Change the sign of the top item to positive
| OP_NOT | 0x91 | If top item is 0 or 1 boolean flip it, otherwise return 0
| OP_0NOTEQUAL | 0x92 | If top item is 0 return 0, otherwise return 1
| OP_ADD | 0x93 | Pop top two items, add them and push result
| OP_SUB | 0x94 | Pop top two items, subtract first form second, push result
| OP_MUL | 0x95 | Disabled (Multiply top two items)
| OP_DIV | 0x96 | Disabled (Divide second item by first item)
| OP_MOD | 0x97 | Disabled (Remainder divide second item by first item)
| OP_LSHIFT | 0x98 | Disabled (Shift second item left by first item number of bits)
| OP_RSHIFT | 0x99 | Disabled (Shift second item right by first item number of bits)
| OP_BOOLAND | 0x9a | Boolean AND of top two items
| OP_BOOLOR | 0x9b | Boolean OR of top two items
| OP_NUMEQUAL | 0x9c | Return TRUE if top two items are equal numbers
| OP_NUMEQUALVERIFY | 0x9d | Same as NUMEQUAL, then OP_VERIFY to halt if not TRUE
| OP_NUMNOTEQUAL | 0x9e | Return TRUE if top two items are not equal numbers
| OP_LESSTHAN | 0x9f | Return TRUE if second item is less than top item
| OP_GREATERTHAN | 0xa0 | Return TRUE if second item is greater than top item
| OP_LESSTHANOREQUAL | 0xa1 | Return TRUE if second item is less than or equal to top item
| OP_GREATERTHANOREQUAL | 0xa2 | Return TRUE if second item is great than or equal to top item
| OP_MIN | 0xa3 | Return the smaller of the two top items
| OP_MAX | 0xa4 | Return the larger of the two top items
| OP_WITHIN | 0xa5 | Return TRUE if the third item is between the second item (or equal) and first item
|=======
[[tx_script_ops_table_crypto]]
.Cryptographic and Hashing Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_RIPEMD160 | 0xa6 | Return RIPEMD160 hash of top item
| OP_SHA1 | 0xa7 | Return SHA1 hash of top item
| OP_SHA256 | 0xa8 | Return SHA256 hash of top item
| OP_HASH160 | 0xa9 | Return RIPEMD160(SHA256(x)) hash of top item
| OP_HASH256 | 0xaa | Return SHA256(SHA256(x)) hash of top item
| OP_CODESEPARATOR | 0xab | Mark the beginning of signature-checked data
| OP_CHECKSIG | 0xac | Pop a public key and signature and validate the signature for the transaction's hashed data, return TRUE if matching
| OP_CHECKSIGVERIFY | 0xad | Same as CHECKSIG, then OP_VEIRFY to halt if not TRUE
| OP_CHECKMULTISIG | 0xae | Run CHECKSIG for each pair of signature and public key provided. All must match. Bug in implementation pops an extra value, prefix with OP_NOP as workaround
| OP_CHECKMULTISIGVERIFY | 0xaf | Same as CHECKMULTISIG, then OP_VERIFY to halt if not TRUE
|=======
[[tx_script_ops_table_nop]]
.Non-Operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP1-OP_NOP10 | 0xb0-0xb9 | Does nothing, ignored.
|=======
[[tx_script_ops_table_internal]]
.Reserved OP codes for internal use by the parser
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_SMALLDATA | 0xf9 | Represents small data field
| OP_SMALLINTEGER | 0xfa | Represents small integer data field
| OP_PUBKEYS | 0xfb | Represents public key fields
| OP_PUBKEYHASH | 0xfd | Represents a public key hash field
| OP_PUBKEY | 0xfe | Represents a public key field
| OP_INVALIDOPCODE | 0xff | Represents any OP code not currently assigned
|=======

133
appdx-sx.asciidoc Normal file
View File

@ -0,0 +1,133 @@
[[sx_cmds]]
== Appendix: Available commands with sx tools
----
The sx commands are:
DEPRECATED
ELECTRUM STYLE DETERMINISTIC KEYS AND ADDRESSES
genaddr Generate a Bitcoin address deterministically from a wallet
seed or master public key.
genpriv Generate a private key deterministically from a seed.
genpub Generate a public key deterministically from a wallet
seed or master public key.
mpk Extract a master public key from a deterministic wallet seed.
newseed Create a new deterministic wallet seed.
EXPERIMENTAL
APPS
wallet Experimental command line wallet.
OFFLINE BLOCKCHAIN
HEADERS
showblkhead Show the details of a block header.
OFFLINE KEYS AND ADDRESSES
BASIC
addr See Bitcoin address of a public or private key.
embed-addr Generate an address used for embedding record of data into the blockchain
get-pubkey Get the pubkey of an address if available
newkey Create a new private key.
pubkey See the public part of a private key.
validaddr Validate an address.
BRAIN STORAGE
brainwallet Make 256 bit bitcoin private key from an arbitrary passphrase.
mnemonic Make 12 word mnemonic out of 128 bit electrum or bip32 seed.
HD / BIP32
hd-priv Create an private HD key from another HD private key.
hd-pub Create an HD public key from another HD private or public key.
hd-seed Create a random new HD key.
hd-to-address Convert an HD public or private key to a Bitcoin address.
hd-to-wif Convert an HD private key to a WIF private key.
MULTISIG ADDRESSES
scripthash Create BIP 16 script hash address from raw script hex.
STEALTH
stealth-addr See a stealth address from given input.
stealth-initiate Initiate a new stealth payment.
stealth-newkey Generate new stealth keys and an address.
stealth-show-addr Show details for a stealth address.
stealth-uncover Uncover a stealth address.
stealth-uncover-secret Uncover a stealth secret.
OFFLINE TRANSACTIONS
SCRIPTING
mktx Create an unsigned tx.
rawscript Create the raw hex representation from a script.
set-input Set a transaction input.
showscript Show the details of a raw script.
showtx Show the details of a transaction.
sign-input Sign a transaction input.
unwrap Validates checksum and recovers version byte and original data from hexstring.
validsig Validate a transaction input's signature.
wrap Adds version byte and checksum to hexstring.
ONLINE (BITCOIN P2P)
BLOCKCHAIN UPDATES
sendtx-node Send transaction to a single node.
sendtx-p2p Send tx to bitcoin network.
ONLINE (BLOCKCHAIN.INFO)
BLOCKCHAIN QUERIES (blockchain.info)
bci-fetch-last-height Fetch the last block height using blockchain.info.
bci-history Get list of output points, values, and their spends
from blockchain.info
BLOCKCHAIN UPDATES
sendtx-bci Send tx to blockchain.info/pushtx.
ONLINE (BLOCKEXPLORER.COM)
BLOCKCHAIN QUERIES (blockexplorer.com)
blke-fetch-transaction Fetches a transaction from blockexplorer.com
ONLINE (OBELISK)
BLOCKCHAIN QUERIES
balance Show balance of a Bitcoin address in satoshis.
fetch-block-header Fetch raw block header.
fetch-last-height Fetch the last block height.
fetch-stealth Fetch a stealth information using a network connection to
make requests against the obelisk load balancer backend.
fetch-transaction Fetch a raw transaction using a network connection to
make requests against the obelisk load balancer backend.
fetch-transaction-index
Fetch block height and index in block of transaction.
get-utxo Get enough unspent transaction outputs from a given set of
addresses to pay a given number of satoshis
history Get list of output points, values, and their spends for an
address. grep can filter for just unspent outputs which can
be fed into mktx.
validtx Validate a transaction.
BLOCKCHAIN UPDATES
sendtx-obelisk Send tx to obelisk server.
BLOCKCHAIN WATCHING
monitor Monitor an address.
watchtx Watch transactions from the network searching for a certain hash.
OBELISK ADMIN
initchain Initialize a new blockchain.
UTILITY
EC MATH
ec-add-modp Calculate the result of INTEGER + INTEGER.
ec-multiply Multiply an integer and a point together.
ec-tweak-add Calculate the result of POINT + INTEGER * G.
FORMAT (BASE 58)
base58-decode Convert from base58 to hex
base58-encode Convert from hex to base58
FORMAT (BASE58CHECK)
base58check-decode Convert from base58check to hex
base58check-encode Convert from hex to base58check
decode-addr Decode a address from base58check form to internal RIPEMD representation
encode-addr Encode an address from internal RIPEMD representation to base58check form
FORMAT (WIF)
secret-to-wif Convert a secret exponent value to Wallet Import Format
wif-to-secret Convert a Wallet Import Format to secret exponent value.
HASHES
ripemd-hash RIPEMD hash data from STDIN.
sha256 Perform SHA256 hash of data.
MISC
qrcode Generate Bitcoin QR codes offline.
SATOSHI MATH
btc Convert Satoshis into Bitcoins.
satoshi Convert Bitcoins into Satoshis.
See 'sx help COMMAND' for more information on a specific command.
----

8
book.asciidoc
View File

@ -18,4 +18,10 @@ include::ch06.asciidoc[]
include::ch07.asciidoc[]
include::ch08.asciidoc[]
include::ch08.asciidoc[]
include::ch09.asciidoc[]
include::appdx-scriptops.asciidoc[]
include::appdx-sx.asciidoc[]

80
ch01.asciidoc
View File

@ -11,13 +11,14 @@ Unlike traditional currencies, bitcoins are entirely virtual. There are no physi
Bitcoin is a fully-distributed, peer-to-peer system. As such there is no "central" server or point of control. Bitcoins are created through a process called "mining", which involves looking for a solution to a difficult problem. Any participant in the bitcoin network (i.e., any device running the full bitcoin protocol stack) may operate as a miner, using their computer's processing power to attempt to find solutions to this problem. Every 10 minutes on average, a new solution is found by someone who then is able to validate the transactions of the past 10 minutes and is rewarded with brand new bitcoins. Essentially, bitcoin mining de-centralizes the currency-issuance and clearing functions of a central bank and replaces the need for any central bank with this global competition.
The bitcoin protocol includes built-in algorithms that regulate the mining function across the network. The difficulty of the problem that miners must solve is adjusted dynamically so that, on average, someone finds a correct answer every 10 minutes regardless of how many miners (and CPUs) are working on the problem at any moment. The protocol also halves the rate at which new bitcoins are created every 4 years, and limits the total number of bitcoins that will be created to a fixed total of 21 million coins. The result is that the number of bitcoins in circulation closely follows an easily predictable curve that reaches 21 million by the year 2140. As a result, the bitcoin currency is deflationary and cannot be inflated by "printing" new money above and beyond the expected issuance rate.
The bitcoin protocol includes built-in algorithms that regulate the mining function across the network. The difficulty of the problem that miners must solve is adjusted dynamically so that, on average, someone finds a correct answer every 10 minutes regardless of how many miners (and CPUs) are working on the problem at any moment. The protocol also halves the rate at which new bitcoins are created every 4 years, and limits the total number of bitcoins that will be created to a fixed total of 21 million coins. The result is that the number of bitcoins in circulation closely follows an easily predictable curve that reaches 21 million by the year 2140. Due to bitcoin's diminishing rate of issuance, over the long term, the bitcoin currency is deflationary. Furthermore, bitcoin cannot be inflated by "printing" new money above and beyond the expected issuance rate.
Behind the scenes, bitcoin is also the name of the protocol, a network and a distributed computing innovation. The bitcoin currency is really only the first application of this invention. As a developer, I see bitcoin as akin to the Internet of money, a network for propagating value and securing the ownership of digital assets via distributed computation. There's a lot more to bitcoin than first meets the eye.
In this chapter we'll get started by explaining some of the main concepts and terms, getting the necessary software and using bitcoin for simple transactions. In following chapters we'll start unwrapping the layers of technology that make bitcoin possible and examine the inner workings of the bitcoin network and protocol.
=== History of Bitcoin
.Digital Currencies Before Bitcoin
****
The emergence of viable digital money is closely linked to developments in cryptography. This is not surprising when one considers the fundamental challenges involved with using bits to represent value that can be exchanged for goods and services. Two fundamental questions for anyone accepting digital money are:
@ -28,7 +29,7 @@ Issuers of paper money are constantly battling the counterfeiting problem by usi
When cryptography started becoming more broadly available and understood in the late 1980s, many researchers began trying to use cryptography to build digital currencies. These early digital currency projects issued digital money, usually backed by a national currency or precious metal such as gold.
While these earlier digital currencies worked, they were centralized and as a result they were easy to attack by governments and hackers. Early digital currencies used a central clearinghouse to settle all transactions at regular intervals, just like a traditional banking system. Unfortunately, in most cases these nascent digital currencies were targeted by worried governments and eventually litigated out of existence. Some failed in spectacular crashes when the parent company liquidated abruptly. To be robust against intervention by antagonists, whether they are legitimate governments or criminal elements, a new digital currency was needed to avoid the use of a central currency issuer or transaction clearing authority that could be a single point of attack. Bitcoin is such a system, completely de-centralized by design, and free of any central authority or point of control that can be attacked or corrupted.
While these earlier digital currencies worked, they were centralized and as a result they were easy to attack by governments and hackers. Early digital currencies used a central clearinghouse to settle all transactions at regular intervals, just like a traditional banking system. Unfortunately, in most cases these nascent digital currencies were targeted by worried governments and eventually litigated out of existence. Some failed in spectacular crashes when the parent company liquidated abruptly. To be robust against intervention by antagonists, whether legitimate governments or criminal elements, a de-centralized digital currency was needed to avoid a single point of attack. Bitcoin is such a system, completely de-centralized by design, and free of any central authority or point of control that can be attacked or corrupted.
Bitcoin represents the culmination of decades of research in cryptography and distributed systems and includes four key innovations brought together in a unique and powerful combination. Bitcoin consists of:
@ -36,10 +37,14 @@ Bitcoin represents the culmination of decades of research in cryptography and di
* A public transaction ledger (the blockchain);
* A de-centralized mathematical and deterministic currency issuance (distributed mining), and;
* A de-centralized transaction verification system (transaction script).
Bitcoin was invented in 2008 by Satoshi Nakamoto with the publication of a paper titled "Bitcoin: A Peer-to-Peer Electronic Cash System". Satoshi Nakamoto combined several prior inventions such as b-money and HashCash to create a completely de-centralized electronic cash system that does not rely on a central authority for currency issuance or settlement and validation of transactions. The key innovation was to use a Proof-Of-Work algorithm to conduct a global "election" every 10 minutes, allowing the de-centralized network to arrive at _consensus_ about the state of transactions. This elegantly solves the issue of double-spend where a single currency unit can be spent twice. Previously, the double-spend problem was a weakness of digital currency and was addressed by clearing all transactions through a central clearinghouse.
The bitcoin network started in 2009, based on a reference implementation published by Nakamoto and since revised by many other programmers. During the first four years of operation, the network has grown to include an enormous amount of Proof-Of-Work computation, thereby increasing its security and resilience. In 2013, the total market value of bitcoin's primary monetary supply measure (M0) is estimated at more than 10 billion US dollars. The largest transaction processed by the network was $150 million US dollars, transmitted instantly and processed without any fees.
****
=== History of Bitcoin
Bitcoin was invented in 2008 by Satoshi Nakamoto with the publication of a paper titled "Bitcoin: A Peer-to-Peer Electronic Cash System". Satoshi Nakamoto combined several prior inventions such as b-money and HashCash to create a completely de-centralized electronic cash system that does not rely on a central authority for currency issuance or settlement and validation of transactions. The key innovation was to use a distributed computation system (called a "Proof-Of-Work" algorithm) to conduct a global "election" every 10 minutes, allowing the de-centralized network to arrive at _consensus_ about the state of transactions. This elegantly solves the issue of double-spend where a single currency unit can be spent twice. Previously, the double-spend problem was a weakness of digital currency and was addressed by clearing all transactions through a central clearinghouse.
The bitcoin network started in 2009, based on a reference implementation published by Nakamoto and since revised by many other programmers. The distributed computation that provides security and resilience for bitcoin has increased exponentially and now exceeds that combined processing capacity of the world's top super-computers. Bitcoin's total market value is estimated at between 5 and 10 billion US dollars, depending on the dollar/bitcoin exchange rate. The largest transaction processed so far by the network was $150 million US dollars, transmitted instantly and processed without any fees.
Satoshi Nakamoto withdrew from the public in April of 2011, leaving the responsibility of developing the code and network to a thriving group of volunteers. The name Satoshi Nakamoto is an alias and the identity of the person or people behind this invention is currently unknown. However, neither Satoshi Nakamoto nor anyone else exerts control over the bitcoin system, which operates based on fully transparent mathematical principles. The invention itself is groundbreaking and has already spawned new science in the fields of distributed computing, economics and econometrics.
@ -55,27 +60,24 @@ Satoshi Nakamoto's invention is also a practical solution to a previously unsolv
Bitcoin is a technology, but it expresses money which is fundamentally a language for exchanging value between people. Let's look at the people who are using bitcoin and some of the most common uses of the currency and protocol through their stories. We will re-use these stories throughout the book to illustrate the real-life uses of digital money and how they are made possible by the various technologies that are part of bitcoin.
North American Retail::
North American Low Value Retail::
Alice lives in Northern California's Bay Area. She has heard about bitcoin from her techie friends and wants to start using it. We will follow her story as she learns about bitcoin, acquires some and then spends some of her bitcoin to buy a cup of coffee at Bob's Cafe in Palo Alto. This story will introduce us to the software, the exchanges and basic transactions from the perspective of a retail consumer.
North American High Value Retail::
Carol is an art gallery owner in San Francisco. She sells expensive paintings for bitcoin. This story will introduce the risks of a "51%" consensus attack for retailers of high-value items.
Offshore Contract Services::
Bob, the cafe owner in Palo Alto is building a new website. He has contracted with an Indian web developer, Gopesh, who lives in Bangalore, India. Gopesh has agreed to be paid in bitcoin. This story will examine the use of bitcoin for outsourcing, contract services and international wire transfers.
Charitable Donations::
Eugenia is the director of a children's charity in the Philippines. Recently she has discovered bitcoin and wants to use it to reach a whole new group of foreign and domestic donors to fundraise for her charity. She's also investigating ways to use bitcoin to distribute funds quickly to areas of need. This story will show the use of bitcoin for global fundraising across currencies and borders and the use of an open ledger for transparency in charitable organizations.
Remittances and Reverse Remittances::
Gopesh, the Indian web developer, is supporting his daughter Radhika who is a student in Essex, England. Gopesh is now considering sending Radhika bitcoin, eliminating the fees he used to pay for remittances. This story will demonstrate the use of local exchange and peer-to-peer exchanges for international remittances with bitcoin.
Import/Export::
Mohammed is an electronics importer in Dubai. He's trying to use bitcoin to buy electronics from the USA and China for import into the U.A.E. to accelerate the process of payments for imports. This story will show how bitcoin can be used for large business-to-business international payments tied to physical goods.
Mining for Bitcoin::
Jing is a computer engineering student in Shanghai. He has built a "mining" rig to mine for bitcoins, using his engineering skills to supplement his income. This story will examine the "industrial" base of bitcoin, the specialized equipment used to secure the bitcoin network and issue new currency.
Peer Lending::
Zenab is a shopkeeper in Kisumu, Kenya and needs a loan to buy new inventory for her shop. With the assistance of a micro-lending organization, she is financing a micro-loan in bitcoin from individual lenders all across the world. This story will demonstrate the potential for bitcoin to offer peer-to-peer micro-lending by aggregating small investments, matching them with borrowers in developing nations.
Each of the stories above is based on real people and real industries that are currently using bitcoin to create new markets, new industries and innovative solutions to global economic issues.
=== Getting Started
@ -84,7 +86,7 @@ To join the bitcoin network and start using the currency, all a user has to do i
The three primary forms of bitcoin clients are:
Full Client:: A full client, or "full node" is a client that stores the entire history of bitcoin transactions, manages the user's wallets and can initiate transactions directly on the bitcoin network. This is similar to a standalone email server, in that it handles all aspects of the protocol without relying on any other servers or third party services.
Full Client:: A full client, or "full node" is a client that stores the entire history of bitcoin transactions (every transaction by every user, ever), manages the user's wallets and can initiate transactions directly on the bitcoin network. This is similar to a standalone email server, in that it handles all aspects of the protocol without relying on any other servers or third party services.
Light Client:: A lightweight client stores the user's wallet but relies on third-party owned servers for access to the bitcoin transactions and network. The light client does not store a full copy of all transactions and therefore must trust the third party servers for transaction validation. This is similar to a standalone email client that connects to a mail server for access to a mailbox, in that it relies on a third party for interactions with the network.
@ -92,50 +94,53 @@ Web Client:: Web-clients are accessed through a web browser and store the user's
.Mobile Bitcoin
****
Mobile clients for smartphones, such as those based on the Android system, can either operate as full clients, light clients or web clients. Some mobile clients are synchronized with a web or desktop client, providing a multi-platform wallet across multiple devices but with a common source of funds. See <<mobile_bitcoin>>
Mobile clients for smartphones, such as those based on the Android system, can either operate as full clients, light clients or web clients. Some mobile clients are synchronized with a web or desktop client, providing a multi-platform wallet across multiple devices but with a common source of funds.
****
The choice of bitcoin client depends on how much control the user wants over funds. A full client will offer the highest level of control and independence for the user, but in turn puts the burden of backups and security on the user. On the other end of the range of choices, a web client is the easiest to set up and use, but the tradeoff with a web client is that counterparty risk is introduced because security and control is shared by the user and the owner of the web service. If a web-wallet service is compromised, as many have been, the users can lose all their funds. Conversely, if a user has a full client without adequate backups, they may lose their funds through a computer mishap.
For the purposes of this book, we will be demonstrating the use of a variety of bitcoin clients, from the reference implementation (the Satoshi client) to web-wallets. Some of the examples will require the use of the reference client which exposes APIs to the wallet, network and transaction services. If you are planning to explore the programmatic interfaces into the bitcoin system, you will need the reference client.
==== Quick Start - Web Wallet
==== Quick Start
A web-wallet is the easiest way to start using bitcoin, and is the choice of Alice who we introduced in <<user-stories>>. Alice is not a technical user and only recently heard about bitcoin from a friend. She starts her journey by visiting the official website bitcoin.org, where she finds a broad selection of bitcoin clients. Following the advice on the bitcoin.org site, she chooses a web-wallet by blockchain.info, a popular hosted-wallet service. Following a link from bitcoin.org, she opens the blockchain.info wallet page at https://blockchain.info/wallet and selects "Start a New Wallet". To register her new wallet, she must enter an email address, enter a password and prove that she is a human by completing a CAPTCHA test.
Alice, who we introduced in <<user-stories>>, is not a technical user and only recently heard about bitcoin from a friend. She starts her journey by visiting the official website bitcoin.org, where she finds a broad selection of bitcoin clients. Following the advice on the bitcoin.org site, she chooses the lightweight bitcoin client _Multibit_.
Alice follows a link from the bitcoin.org site to download and install Multibit on her desktop. Multibit is available for Windows, Mac OS and Linux desktops.
[WARNING]
====
When creating a bitcoin wallet you will need to provide a password or passphrase to protect your wallet. There are many bad actors attempting to break weak passwords, so take care to select one that cannot be easily broken. Use a combination of upper and lower-case characters, numbers and symbols. Avoid personal information such as birthdates or names of sports teams. Avoid any words commonly found in dictionaries, in any language. If you can, use a password generator to create a completely random password that is at least 12 characters in length. Remember: bitcoin is money and can be instantly moved anywhere in the world. If it is not well protected, it can be easily stolen.
A bitcoin wallet must be protected by a password or passphrase. There are many bad actors attempting to break weak passwords, so take care to select one that cannot be easily broken. Use a combination of upper and lower-case characters, numbers and symbols. Avoid personal information such as birth-dates or names of sports teams. Avoid any words commonly found in dictionaries, in any language. If you can, use a password generator to create a completely random password that is at least 12 characters in length. Remember: bitcoin is money and can be instantly moved anywhere in the world. If it is not well protected, it can be easily stolen.
====
Once Alice has completed the registration form, she is presented with a Wallet Recovery Mnemonic. This is a series of words that can be used to reconstruct her wallet in case she loses the password or account details. Following the instructions on screen, Alice copies the words onto paper, locking it away in a secure location.
Once Alice has downloaded and installed the Multibit application, she runs it and is greeted by a "welcome" screen:
[[blockchain-mnemonic]]
.Blockchain.info - Wallet Recovery Mnemonic
image::images/blockchain-mnemonic.png["wallet recovery mnemonic"]
[[multibit-welcome]]
.The Multibit Bitcoin Client - Welcome Screen
image::images/MultibitWelcome.png["MultibitWelcome"]
A few seconds later, Alice can start using her new bitcoin web-wallet by logging in with her account ID and password. In her web browser, she sees the web-wallet home screen:
Multibit automatically creates a wallet and a new bitcoin address for Alice, which Alice can see by clicking on the "Request" tab:
[[multibit-request]]
.Alice's new bitcoin address, in the "Request" tab of the Multibit client
image::images/MultibitReceive.png["MultibitReceive"]
[[blockchain-home]]
.Blockchain.info - Wallet Home Screen
image::images/blockchain-home.png["wallet home screen"]
The most important part of this screen is Alice's _bitcoin address_. Like an email address, Alice can share this address and anyone can use it to send money directly to her new wallet. On the screen it appears as a long string of letters and numbers: +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. Next to the wallet's bitcoin address, there is a QR code, a form of barcode that contains the same information in a format that can be easily scanned by a smartphone's camera. The QR code is the black and white square on the right side of the window. Alice can copy the bitcoin address or the QR code onto her clipboard by clicking on the copy button adjacent to each of them. Clicking on the QR code itself will magnify it, so that it can be easily scanned by a smartphone camera.
The most important part of this screen is Alice's _bitcoin address_. Like an email address, Alice can share this address and anyone can use it to send money directly to her new web-wallet. On the screen it appears as a long string of letters and numbers: +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. Next to the wallet's bitcoin address, there is a QR code, a form of barcode that contains the same information in a format that can be easily scanned by a smartphone's camera. Alice can print the QR code as a way to easily give her address to others without them having to type the long string of letters and numbers.
Alice can also print the QR code as a way to easily give her address to others without them having to type the long string of letters and numbers.
[TIP]
====
Bitcoin addresses start with the digit "1" or "3". Like email addresses, they can be shared with other bitcoin users who can use them to send bitcoin directly to your wallet. Unlike email addresses, you can create new addresses as often as you like, all of which will direct funds to your wallet. A wallet is simply a collection of addresses and the keys that unlock the funds within. There is practically no limit to the number of addresses a user can create.
====
Alice is now ready to start using her new bitcoin web-wallet.
Alice is now ready to start using her new bitcoin wallet.
[[getting_first_bitcoin]]
==== Getting your first bitcoins
It is not possible to buy bitcoins at a bank or foreign exchange kiosks at this time. It is not possible to use a credit card to buy bitcoins, either. As of 2014, it is still quite difficult to acquire bitcoins in most countries. There are a number of specialized currency exchanges where you can buy and sell bitcoin in exchange for a local currency. These operate as web-based currency markets and include:
It is not possible to buy bitcoins at a bank or foreign exchange kiosks at this time. As of 2014, it is still quite difficult to acquire bitcoins in most countries. There are a number of specialized currency exchanges where you can buy and sell bitcoin in exchange for a local currency. These operate as web-based currency markets and include:
* Bitstamp (bitstamp.net), a European currency market that supports several currencies including euros (EUR) and US dollars (USD) via wire transfer
* Coinbase (coinbase.com), a US-based currency market in California that supports US dollar exchange to and from bitcoin. Coinbase can connect to US checking accounts via the ACH system.
* Coinbase (coinbase.com), a US-based bitcoin wallet and platform where merchants and consumers can transact in bitcoin. Coinbase makes it easy to buy and sell bitcoin, allowing users to connect to US checking accounts via the ACH system.
Crypto-currency exchanges such as these operate at the intersection of national currencies and crypto-currencies. As such, they are subject to national and international regulations and are often specific to a single country or economic area and specialize in the national currencies of that area. Your choice of currency exchange will be specific to the national currency you use and limited to the exchanges that operate within the legal jurisdiction of your country. Similar to opening a bank account, it takes several days or weeks to set up the necessary accounts with the above services because they require various forms of identification to comply with KYC (Know Your Customer) and AML (Anti-Money Laundering) banking regulations. Once you have an account on a bitcoin exchange, you can then buy or sell bitcoins quickly just as you could with foreign currency with a brokerage account.
@ -146,20 +151,25 @@ There are three other methods for getting bitcoins as a new user:
* Find a friend who has bitcoins and buy some from them directly. Many bitcoin users started this way.
* Use a classified service like localbitcoins.com to find a seller in your area to buy bitcoins for cash in an in-person transaction.
* Sell a product or service for bitcoin. If you're a programmer, sell your programming skills. If you have an online store, see <<bitcoin-commerce>> to sell in bitcoin.
* Use a bitcoin ATM in your city. A map of bitcoin ATMs can be found at http://www.coindesk.com/bitcoin-atm-map/
Alice was introduced to bitcoin by a friend and so she has an easy way of getting her first bitcoin while she waits for her account on a California currency market to be verified and activated.
==== Sending and receiving bitcoins
[[sending_receiving]]
==== Sending and receiving bitcoins
Alice has created her bitcoin web-wallet and she is now ready to receive funds. Her web-wallet application randomly generated a bitcoin address together with its corresponding key (an elliptic curve private key, described in more detail in <<private keys>>). At this point, her bitcoin address is not known to the bitcoin network or "registered" with any part of the bitcoin system. Her bitcoin address is simply a number that corresponds to a key that she can use to control access to the funds. There is no account or association between that address and an account. Until the moment this address is referenced as the recipient of value in a transaction posted on the bitcoin ledger (the blockchain), it is simply part of the vast number of possible addresses that are "valid" in bitcoin. Once it has been associated with a transaction, it becomes part of the known addresses in the network and anyone can check its balance on the public ledger.
Alice has created her bitcoin wallet and she is now ready to receive funds. Her wallet application randomly generated a private key (described in more detail in <<private_keys>>) together with its corresponding bitcoin address. At this point, her bitcoin address is not known to the bitcoin network or "registered" with any part of the bitcoin system. Her bitcoin address is simply a number that corresponds to a key that she can use to control access to the funds. There is no account or association between that address and an account. Until the moment this address is referenced as the recipient of value in a transaction posted on the bitcoin ledger (the blockchain), it is simply part of the vast number of possible addresses that are "valid" in bitcoin. Once it has been associated with a transaction, it becomes part of the known addresses in the network and Alice can check its balance on the public ledger.
Alice meets her friend Joe who introduced her to bitcoin at a local restaurant so they can exchange some US dollars and put some bitcoins into her account. She has brought a print out of her address and the QR code as shown on the home page of her web-wallet. There is nothing sensitive from a security perspective about the bitcoin address. It can be posted anywhere without risking the security of her account and it can be changed by creating a new address at any time. Alice wants to convert just $10 US dollars into bitcoin, so as not to risk too much money on this new technology. She gives Joe a $10 bill and the printout of her address so that Joe can send her the equivalent amount of bitcoin.
Alice meets her friend Joe who introduced her to bitcoin at a local restaurant so they can exchange some US dollars and put some bitcoins into her account. She has brought a printout of her address and the QR code as displayed in her bitcoin wallet. There is nothing sensitive, from a security perspective, about the bitcoin address. It can be posted anywhere without risking the security of her account.
First, Joe has to figure out the exchange rate so that he can give the correct amount of bitcoin to Alice. There are hundreds of applications and web sites that can provide the current market rate, here are some of the most popular:
Alice wants to convert just $10 US dollars into bitcoin, so as not to risk too much money on this new technology. She gives Joe a $10 bill and the printout of her address so that Joe can send her the equivalent amount of bitcoin.
Next, Joe has to figure out the exchange rate so that he can give the correct amount of bitcoin to Alice. There are hundreds of applications and web sites that can provide the current market rate, here are some of the most popular:
* bitcoincharts.com, a market data listing service that shows the market rate of bitcoin across many exchanges around the globe, denominated in different local currencies
* bitcoinaverage.com, a site that provides a simple view of the volume-weighted-average for each currency
* ZeroBlock, a free Android and iOS application that can display a bitcoin price from different exchanges
* bitcoinwisdom.com, another market data listing service
[[zeroblock-android]]
.ZeroBlock - A bitcoin market-rate application for Android and iOS
@ -184,7 +194,7 @@ If Alice has a smartphone or laptop with her, she will also be able to see the t
.Confirmations
****
At first, Alice's address will show the transaction from Joe as "Unconfirmed". This means that the transaction has been propagated to the network but has not yet been included in the bitcoin transaction ledger, known as the blockchain. To be included, the transaction must be "picked up" by a miner and included in a block of transactions. Once a miner has discovered a solution to the Proof-of-Work algorithm for this block (in approximately 10 minutes), the transactions within the block will be accepted as "confirmed" by the network and can be spent. The transaction is seen by all instantly, but it is only "trusted" by all when it is included in a newly mined block. The more blocks mined after that block, the more trusted it is, as more and more computation is "piled" on top of it.
At first, Alice's address will show the transaction from Joe as "Unconfirmed". This means that the transaction has been propagated to the network but has not yet been included in the bitcoin transaction ledger, known as the blockchain. To be included, the transaction must be "picked up" by a miner and included in a block of transactions. Once a new block is created, in approximately 10 minutes, the transactions within the block will be accepted as "confirmed" by the network and can be spent. The transaction is seen by all instantly, but it is only "trusted" by all when it is included in a newly mined block.
****
Alice is now the proud owner of 0.10 bitcoin which she can spend. In the next chapter we will look at her first purchase with bitcoin and examine the underlying transaction and propagation technologies in more detail.

51
ch02.asciidoc
View File

@ -1,7 +1,7 @@
[[ch02_bitcoin_overview]]
== How Bitcoin Works
=== Transactions, Blocks, Mining and the Blockchain
=== Transactions, Blocks, Mining, and the Blockchain
The bitcoin system, unlike traditional banking and payment systems, is based on de-centralized trust. Instead of a central trusted authority, in bitcoin, trust is achieved as an emergent property from the interactions of different participants in the bitcoin system. In this chapter we will examine bitcoin from a high-level by tracking a single transaction through the bitcoin system and watch as it becomes "trusted" and accepted by the bitcoin mechanism of distributed consensus and is finally recorded on the blockchain, the distributed ledger of all transactions.
@ -11,22 +11,24 @@ Popular blockchain explorers include:
* blockchain.info
* blockexplorer.com
* biteasy.com
* insight.bitpay.com
* blockr.io
Each of these has a search function that can take an address, transaction hash or block number and find the equivalent data on the bitcoin network and blockchain. With each example, we will provide a URL that takes you directly to the relevant entry, so you can study it in detail.
==== Bitcoin Overview
In the overview diagram below, we see that the bitcoin system consists of users with wallets containing keys, transactions which are propagated across the network and miners who produce (through competitive computation) the consensus blockchain, the authoritative ledger of all transactions. In this chapter, we will trace a single transaction as it travels across the network and examine the interactions between each part of the bitcoin system, at a high level. Subsequent chapters will delve deeper into the technology behind wallets, mining and merchant systems.
In the overview diagram below, we see that the bitcoin system consists of users with wallets containing keys, transactions which are propagated across the network and miners who produce (through competitive computation) the consensus blockchain, the authoritative ledger of all transactions. In this chapter, we will trace a single transaction as it travels across the network and examine the interactions between each part of the bitcoin system, at a high level. Subsequent chapters will delve into the technology behind wallets, mining and merchant systems.
[[blockchain-mnemonic]]
[[bitcoin-overview]]
.Bitcoin Overview
image::images/Bitcoin_Overview.png["Bitcoin Overview"]
[[cup_of_coffee]]
==== Buying a cup of coffee
Alice, introduced in the previous chapter, is a new user who has just acquired her first bitcoin. In <<getting_first_bitcoin>>, Alice met with her friend Joe to exchange some cash for bitcoin. The transaction created by Joe, funded Alice's wallet with 0.10 BTC. Now Alice will make her first retail transaction, buying a cup of coffee at Bob's coffee shop in Palo Alto, California. Bob's coffee shop recently started accepting bitcoin payments, by adding a bitcoin option to his point-of-sale system (see <<bitcoin_for_merchants>> for information on using bitcoin for merchants/retail). The prices at Bob's Cafe are listed in the local currency (US dollars) but at the register, customers have the option of paying in either dollars or bitcoin. Alice places her order for a cup of coffee and Bob enters the transaction at the register. The point-of-sale system will convert the total price from US dollars to bitcoins at the prevailing market rate and display the prices in both currencies, as well as showing a QR code containing a _payment request_ for this transaction:
Alice, introduced in the previous chapter, is a new user who has just acquired her first bitcoin. In <<getting_first_bitcoin>>, Alice met with her friend Joe to exchange some cash for bitcoin. The transaction created by Joe, funded Alice's wallet with 0.10 BTC. Now Alice will make her first retail transaction, buying a cup of coffee at Bob's coffee shop in Palo Alto, California. Bob's coffee shop recently started accepting bitcoin payments, by adding a bitcoin option to his point-of-sale system. The prices at Bob's Cafe are listed in the local currency (US dollars) but at the register, customers have the option of paying in either dollars or bitcoin. Alice places her order for a cup of coffee and Bob enters the transaction at the register. The point-of-sale system will convert the total price from US dollars to bitcoins at the prevailing market rate and display the prices in both currencies, as well as showing a QR code containing a _payment request_ for this transaction:
.Displayed on Bob's cash register
----
@ -58,7 +60,7 @@ A description for the payment: "Purchase at Bob's Cafe"
[TIP]
====
Unlike a QR code that simply contains a destination bitcoin address, a "payment request" is a QR encoded URL that contains a destination address, a payment amount and a generic description such as "Bob's Cafe". This allows a bitcoin wallet application to pre-fill the information used to send the payment while showing a human-readable description to the user. See <<payment request URL>>, for more details. You can scan the QR code above with a bitcoin wallet application to see what Alice would see.
Unlike a QR code that simply contains a destination bitcoin address, a "payment request" is a QR encoded URL that contains a destination address, a payment amount and a generic description such as "Bob's Cafe". This allows a bitcoin wallet application to pre-fill the information used to send the payment while showing a human-readable description to the user. You can scan the QR code above with a bitcoin wallet application to see what Alice would see.
====
Bob says "That's one-dollar-fifty, or fifteen milliBits".
@ -69,7 +71,7 @@ In the following sections we will examine this transaction in more detail, see h
[NOTE]
====
The bitcoin network can transact in fractional values, e.g. from millibitcoins (1/1000th of a bitcoin) down to 1/100,000,000th of a bitcoin, which is known as a Satoshi. Throughout this book we’ll use the term “bitcoins” to refer to any quantity of bitcoin currency, from the smallest unit (1 Satoshi) to the total number (21,000,000) of all bitcoins that will ever be mined.
The bitcoin network can transact in fractional values, e.g. from milli-bitcoins (1/1000th of a bitcoin) down to 1/100,000,000th of a bitcoin, which is known as a Satoshi. Throughout this book we’ll use the term “bitcoins” to refer to any quantity of bitcoin currency, from the smallest unit (1 Satoshi) to the total number (21,000,000) of all bitcoins that will ever be mined.
====
@ -83,7 +85,7 @@ Transactions are like lines in a double-entry bookkeeping ledger. In simple term
.Transaction as Double-Entry Bookkeeping
image::images/Transaction_Double_Entry.png["Transaction Double-Entry"]
The transaction also contains proof of ownership for each amount of bitcoin (inputs) whose value is transferred, in the form of a digital signature from the owner, which can be independently validated by anyone. In bitcoin terms, "spending" is signing the value of a previous transaction for which you have the keys over to a new owner identified by a bitcoin address.
The transaction also contains proof of ownership for each amount of bitcoin (inputs) whose value is transferred, in the form of a digital signature from the owner, which can be independently validated by anyone. In bitcoin terms, "spending" is signing a transaction which transfers value from a previous transaction over to a new owner identified by a bitcoin address.
[TIP]
@ -93,7 +95,7 @@ _Transactions_ move value *from* _transaction inputs_ *to* _transaction outputs_
[[blockchain-mnemonic]]
.Transaction Chain
.A chain of transactions, where the output of one transaction is the input of the next transaction
image::images/Transaction_Chain.png["Transaction chain"]
Alice's payment to Bob's Cafe utilizes a previous transaction as its input. In the previous chapter Alice received bitcoin from her friend Joe in return for cash. That transaction has a number of bitcoins locked (encumbered) against Alice's key. Her new transaction to Bob's Cafe references the previous transaction as an input and creates new outputs to pay for the cup of coffee and receive change. The transactions form a chain, where the inputs from the latest transaction correspond to outputs from previous transactions. Alice's key provides the signature which unlocks those previous transaction outputs, thereby proving to the bitcoin network that she owns the funds. She attaches the payment for coffee to Bob's address, thereby "encumbering" that output with the requirement that Bob produces a signature in order to spend that amount. This represents a transfer of value between Alice and Bob.
@ -124,14 +126,17 @@ Alice's wallet application contains all the logic for selecting appropriate inpu
==== Getting the right inputs
Alice's wallet application will first have to find inputs that can pay for the amount she wants to send to Bob. Most wallet applications keep a small database of "unspent transaction outputs" that are locked (encumbered) with the wallet's own keys. Therefore, Alice's wallet would contain a copy of the transaction output from Joe's transaction which was created in exchange for cash (see <<getting bitcoin>>). A bitcoin wallet application that runs as a full-index client actually contains a copy of *every unspent output* from every transaction in the blockchain. This allows a wallet to construct transaction inputs as well as to quickly verify incoming transactions as having correct inputs.
Alice's wallet application will first have to find inputs that can pay for the amount she wants to send to Bob. Most wallet applications keep a small database of "unspent transaction outputs" that are locked (encumbered) with the wallet's own keys. Therefore, Alice's wallet would contain a copy of the transaction output from Joe's transaction which was created in exchange for cash (see <<getting_first_bitcoin>>). A bitcoin wallet application that runs as a full-index client actually contains a copy of every unspent output from every transaction in the blockchain. This allows a wallet to construct transaction inputs as well as to quickly verify incoming transactions as having correct inputs. However, since a full-index client takes up a lot of disk space, most user wallets run "lightweight" clients that track only the user's own unspent outputs.
If the wallet application does not maintain a copy of unspent transaction outputs, it can query the bitcoin network to retrieve this information, using a variety of APIs available by different providers, or by asking a full-index node using the bitcoin JSON RPC API. Below we see an example of a RESTful API request, constructed as an HTTP GET command to a specific URL. This URL will return all the unspent transaction outputs for an address, giving any application the information it needs to construct transaction inputs for spending. We use the simple command-line HTTP client _cURL_ to retrieve the response:
.Look up all the unspent outputs for Alice's address 1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK
.Look up all the unspent outputs for Alice's bitcoin address
[source,bash]
----
$ curl https://blockchain.info/unspent?active=1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK
----
[source,json]
----
{
"unspent_outputs":[
@ -150,11 +155,11 @@ $ curl https://blockchain.info/unspent?active=1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK
}
----
The response above shows that the bitcoin network knows of one unspent output (one that has not been redeemed yet) under the ownership of Alice's address _+1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+_. The response includes the reference to the transaction in which this unspent output is contained (the payment from Joe) and its value in Satoshis, at 10 million, equivalent to 0.10 bitcoin. With this information, Alice's wallet application can construct a transaction to transfer that value to new owner addresses.
The response above shows that the bitcoin network knows of one unspent output (one that has not been redeemed yet) under the ownership of Alice's address +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. The response includes the reference to the transaction in which this unspent output is contained (the payment from Joe) and its value in Satoshis, at 10 million, equivalent to 0.10 bitcoin. With this information, Alice's wallet application can construct a transaction to transfer that value to new owner addresses.
[TIP]
====
Look up the transaction from Joe to Alice to see the information referenced above as it is stored in the bitcoin blockchain. Using the blockchain explorer web application, follow the URL below:
Use the following link to look up the transaction from Joe to Alice:
https://blockchain.info/tx/7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18
====
@ -176,13 +181,13 @@ The resulting transaction can be seen using a blockchain explorer web applicatio
.Alice's transaction to Bob's Cafe
image::images/AliceCoffeeTransaction.png["Alice Coffee Transaction"]
Use the following link to see the transaction on the bitcoin blockchain:
[[transaction-alice-url]]
.Link to Alice's transaction on the bitcoin blockchain
----
[TIP]
====
Use the following link to look up the transaction from Alice to Bob's Cafe:
https://blockchain.info/tx/0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2
----
====
==== Adding the transaction to the ledger
@ -214,7 +219,7 @@ The bitcoin system of trust is based on computation. Transactions are bundled in
* Mining creates new bitcoins in each block, almost like a central bank printing new money. The amount of bitcoin created per block is fixed and diminishes with time.
* Mining creates trust by ensuring that transactions are only confirmed if enough computational power was devoted to the block that contains them. More blocks mean more computation which means more trust.
A good way to describe mining is like a giant competitive game of sudoku that resets every time someone finds a solution and whose difficulty automatically adjusts so that it takes approximately 10 minutes to find a solution. Imagine a giant sudoku puzzle, several thousand rows and columns in size. If I show you a completed puzzle you can verify it quite quickly. If it is empty, however, it takes a lot of work to solve! The difficulty of the sudoku can be adjusted by changing its size (more or fewer rows and columns), but it can still be verified quite easily even if it is very large. The "puzzle" used in bitcoin is based on a cryptographic hash and exhibits similar characteristics: it is asymmetrically hard to solve but easy to verify, and its difficulty can be adjusted.
A good way to describe mining is like a giant competitive game of sudoku that resets every time someone finds a solution and whose difficulty automatically adjusts so that it takes approximately 10 minutes to find a solution. Imagine a giant sudoku puzzle, several thousand rows and columns in size. If I show you a completed puzzle you can verify it quite quickly. However, if the puzzle has a few squares filled and the rest is empty, it takes a lot of work to solve! The difficulty of the sudoku can be adjusted by changing its size (more or fewer rows and columns), but it can still be verified quite easily even if it is very large. The "puzzle" used in bitcoin is based on a cryptographic hash and exhibits similar characteristics: it is asymmetrically hard to solve but easy to verify, and its difficulty can be adjusted.
In <<user-stories>> we introduced Jing, a computer engineering student in Shanghai. Jing is participating in the bitcoin network as a miner. Every 10 minutes or so, Jing joins thousands of other miners in a global race to find a solution to a block of transactions. Finding such a solution, the so-called "Proof-of-Work", requires quadrillions of hashing operations per second across the entire bitcoin network. The algorithm for "Proof-of-Work" involves repeatedly hashing the header of the block and a random number with the SHA256 cryptographic algorithm until a solution matching a pre-determined pattern emerges. The first miner to find such a solution wins the round of competition and publishes that block into the blockchain.
@ -222,7 +227,7 @@ Jing started mining in 2010 using a very fast desktop computer to find a suitabl
=== Mining transactions in blocks
A transaction transmitted across the network is not verified until it becomes part of the global distributed ledger, the blockchain. Every ten minutes, miners generate a new block that contains all the transactions since the last block. New transactions are constantly flowing into the network from user wallets and other applications. As these are seen by the bitcoin network nodes, they get added to a temporary "pool" of unverified transactions maintained by each node. As miners build a new block, they add unverified transactions from this pool to a new block and then attempt to solve a very hard problem (aka Proof-of-Work) to prove the validity of that new block. The process of mining is explained in detail in <<mining>>.
A transaction transmitted across the network is not verified until it becomes part of the global distributed ledger, the blockchain. Every ten minutes on average, miners generate a new block that contains all the transactions since the last block. New transactions are constantly flowing into the network from user wallets and other applications. As these are seen by the bitcoin network nodes, they get added to a temporary "pool" of unverified transactions maintained by each node. As miners build a new block, they add unverified transactions from this pool to a new block and then attempt to solve a very hard problem (aka Proof-of-Work) to prove the validity of that new block. The process of mining is explained in detail in <<mining>>.
Transactions are added to the new block, prioritized by the highest-fee transactions first and a few other criteria. Each miner starts the process of mining a new block of transactions as soon as they receive the previous block from the network, knowing they have lost that previous round of competition. They immediately create a new block, fill it with transactions and the fingerprint of the previous block and start calculating the Proof-of-Work for the new block. Each miner includes a special transaction in their block, one that pays their own bitcoin address a reward of newly created bitcoins (currently 25 BTC per block). If they find a solution that makes that block valid, they "win" this reward because their successful block is added to the global blockchain and the reward transaction they included becomes spendable. Jing, who participates in a mining pool, has set up his software to create new blocks that assign the reward to a pool address. From there, a share of the reward is distributed to Jing and other miners in proportion to the amount of work they contributed in the last round.
@ -233,7 +238,7 @@ https://blockchain.info/block-height/277316
A few minutes later, a new block, #277317 is mined by another miner. As this new block is based on the previous block (#277316) that contained Alice's transaction, it added even more computation on top of that block, thereby strengthening the trust in those transactions. One block mined on top of the one containing the transaction is called "one confirmation" for that transaction. As the blocks pile on top of each other, it becomes exponentially harder to reverse the transaction, thereby making it more and more trusted by the network.
In the diagram below we can see block #277316, which contains Alice's transaction. Below it are 277,315 blocks, linked to each other in a chain of blocks (blockchain) all the way back to block #0, the genesis block. Over time, as the "height" in blocks increases, so does the computation difficulty for each block and the chain as a whole. The blocks mined after the one that contains Alice's transaction act as further assurance, as they pile on more computation in a longer and longer chain. The blocks above count as "confirmations". By convention, any block with more than 6 confirmations is considered irrevocable, as it would require an immense amount of computation to invalidate and re-calculate six blocks. We will examine the process of mining and the way it builds trust in more detail in <<mining>>.
In the diagram below we can see block #277316, which contains Alice's transaction. Below it are 277,316 blocks (including block #0), linked to each other in a chain of blocks (blockchain) all the way back to block #0, the genesis block. Over time, as the "height" in blocks increases, so does the computation difficulty for each block and the chain as a whole. The blocks mined after the one that contains Alice's transaction act as further assurance, as they pile on more computation in a longer and longer chain. The blocks above count as "confirmations". By convention, any block with more than 6 confirmations is considered irrevocable, as it would require an immense amount of computation to invalidate and re-calculate six blocks. We will examine the process of mining and the way it builds trust in more detail in <<mining>>.
[[block-alice]]
.Alice's transaction included in block #277,316
@ -241,7 +246,7 @@ image::images/Blockchain_height_and_depth.png["Alice's transaction included in a
=== Spending the transaction
Now that Alice's transaction has been embedded in the blockchain as part of a block, it is part of the distributed ledger of bitcoin and visible to all bitcoin applications. Each bitcoin client can independently verify the transaction as valid and spendable. Full-index clients can track the source of the funds from the moment the bitcoins were first generated in a block, incrementally from transaction to transaction, until they reach Bob's address. Lightweight clients can do a Simple Payment Verification (See SPV:<<spv>>) by confirming that the transaction is in the blockchain and has several blocks mined after it, thus providing assurance that the network accepts it as valid.
Now that Alice's transaction has been embedded in the blockchain as part of a block, it is part of the distributed ledger of bitcoin and visible to all bitcoin applications. Each bitcoin client can independently verify the transaction as valid and spendable. Full-index clients can track the source of the funds from the moment the bitcoins were first generated in a block, incrementally from transaction to transaction, until they reach Bob's address. Lightweight clients can do a Simplified Payment Verification (See <<spv_nodes>>) by confirming that the transaction is in the blockchain and has several blocks mined after it, thus providing assurance that the network accepts it as valid.
Bob can now spend the output from this and other transactions, by creating his own transactions that reference these outputs as their inputs and assign them new ownership. For example, Bob can pay a contractor or supplier by transferring value from Alice's coffee cup payment to these new owners. Most likely, Bob's bitcoin software will aggregate many small payments into a larger payment, perhaps concentrating all the day's bitcoin revenue into a single transaction. This would move the various payments into a single address, utilized as the store's general "checking" account. For a diagram of an aggregating transaction, see <<transaction-aggregating>>.

443
ch03.asciidoc
View File

@ -1,9 +1,9 @@
[[ch03_bitcoin_client]]
== The Bitcoin Client
=== Bitcoin Core - The Reference Implementation, aka Satoshi Client
=== Bitcoin Core - The reference implementation
You can download the Reference Client, also known as _Bitcoin Core_ from bitcoin.org. The reference client implements all aspects of the bitcoin system, including wallets, a transaction verification engine with a full copy of the entire transaction ledger (blockchain) and a full network node in the peer-to-peer bitcoin network.
You can download the Reference Client _Bitcoin Core_, also known as the "Satoshi client", from bitcoin.org. The reference client implements all aspects of the bitcoin system, including wallets, a transaction verification engine with a full copy of the entire transaction ledger (blockchain) and a full network node in the peer-to-peer bitcoin network.
Go to http://bitcoin.org/en/choose-your-wallet and select "Bitcoin Core" to download the reference client. Depending on your operating system, you will download an executable installer. For Windows, this is either a ZIP archive or an EXE executable. For Mac OS it is a DMG disk image. Linux versions include a PPA package for Ubuntu or a TAR.GZ archive.
@ -11,7 +11,7 @@ Go to http://bitcoin.org/en/choose-your-wallet and select "Bitcoin Core" to down
.Bitcoin - Choose A Bitcoin Client
image::images/bitcoin-choose-client.png["bitcoin choose client"]
==== Bitcoin Core - Running the client for the first time
==== Running Bitcoin Core for the first time
If you download an installable package, such as an EXE, DMG or PPA, you can install it the same way as any application on your operating system. For Windows, run the EXE and follow the step-by-step instructions. For Mac OS, launch the DMG and drag the Bitcoin-QT icon into your Applications folder. For Ubuntu, double-click on the PPA in your File Explorer and it will open the package manager to install the package. Once you have completed installation you should have a new application "Bitcoin-Qt" in your application list. Double-click on the icon to start the bitcoin client.
@ -27,10 +27,11 @@ Bitcoin Core keeps a full copy of the transaction ledger (blockchain), with ever
image::images/bitcoin-qt-firstload.png["bitcoin-qt first run"]
==== Bitcoin Core - Compiling the client from the source code
==== Compiling Bitcoin Core from the source code
For developers, there is also the option to download the full source code as a ZIP archive or by cloning the authoritative source repository from Github. Go to https://github.com/bitcoin/bitcoin and select "Download ZIP" from the sidebar. Alternatively, use the git command line to create a local copy of the source code on your system. In the example below, we are cloning the source code from a unix-like command-line, in Linux or Mac OS:
[source,bash]
----
$ git clone https://github.com/bitcoin/bitcoin.git
Cloning into 'bitcoin'...
@ -49,12 +50,14 @@ The instructions and resulting output may vary from version to version. Follow t
When the git cloning operation has completed, you will have a complete local copy of the source code repository in the directory _bitcoin_. Change to this directory by typing +cd bitcoin+ at the prompt:
[source,bash]
----
$ cd bitcoin
----
By default, the local copy will be synchronized with the most recent code, which may be an unstable or "beta" version of bitcoin. Before compiling the code, we want to select a specific version by checking out a release _tag_. This will synchronize the local copy with a specific snapshot of the code repository identified by a keyword tag. Tags are used by the developers to mark specific releases of the code by version number. First, to find the available tags, we use the +git tag+ command:
[source,bash]
----
$ git tag
v0.1.5
@ -75,6 +78,7 @@ v0.9.0rc1
The list of tags shows all the released versions of bitcoin. By convention, _release candidates_, which are intended for testing, have the suffix "rc". Stable releases that can be run on production systems have no suffix. From the list above, we select the highest version release, which at this time is v0.9.0rc1. To synchronize the local code with this version, we use the +git checkout+ command:
[source,bash]
----
$ git checkout v0.9.0rc1
Note: checking out 'v0.9.0rc1'.
@ -90,9 +94,10 @@ Carefully review the build pre-requisites which are in the first part of the bui
[TIP]
====
The bitcoind build process was changed to use the autogen/configure/make system starting with version 0.9. Older versions use a simple Makefile and work slightly differently from the example below. Follow the instructions for the version you want to compile. The autogen/configure/make introduced in 0.9 is likely to be the build system used for all future versions of the code and is the system demonstrated in the examples below.
The Bitcoin Core build process was changed to use the autogen/configure/make system starting with version 0.9. Older versions use a simple Makefile and work slightly differently from the example below. Follow the instructions for the version you want to compile. The autogen/configure/make introduced in 0.9 is likely to be the build system used for all future versions of the code and is the system demonstrated in the examples below.
====
[source,bash]
----
$ ./autogen.sh
configure.ac:12: installing `src/build-aux/config.guess'
@ -105,6 +110,7 @@ $
The +autogen.sh+ script creates a set of automatic configuration scripts that will interrogate your system to discover the correct settings and ensure you have all the necessary libraries to compile the code. The most important of these is the +configure+ script that offers a number of different options to customize the build process. Type +./configure --help+ to see the various options:
[source,bash]
----
$ ./configure --help
@ -142,6 +148,7 @@ $
The +configure+ script allows you to enable or disable certain features of bitcoind through the use of the +--enable-FEATURE+ and +--disable-FEATURE+ flags, where +FEATURE+ is replaced by the feature name, as listed in the help output above. In this chapter, we will build the bitcoind client with all the default features. We won't be using the configuration flags, but you should review them to understand what optional features are part of the client. Next, we run the +configure+ script to automatically discover all the necessary libraries and create a customized build script for our system:
[source,bash]
----
$ ./configure
checking build system type... x86_64-unknown-linux-gnu
@ -172,6 +179,7 @@ $
If all goes well, the +configure+ command will end by creating the customized build scripts that will allow us to compile bitcoind. If there are any missing libraries or errors, the +configure+ command will terminate with an error instead of creating the build scripts as shown above. If an error occurs, it is most likely a missing or incompatible library. Review the build documentation again and make sure you install the missing pre-requisites. Then run +configure+ again and see if that fixes the error. Next, we will compile the source code, a process that can take up to an hour to complete. During the compilation process you should see output every few seconds or every few minutes, or an error if something goes wrong. The compilation process can be resumed at any time if interrupted. Type +make+ to start compiling:
[source,bash]
----
$ make
Making all in src
@ -203,6 +211,7 @@ $
If all goes well, bitcoind is now compiled. The final step is to install the bitcoind executable into the system path using the +make+ command:
[source,bash]
----
$ sudo make install
Making install in src
@ -216,15 +225,20 @@ make install-am
$
----
We can confirm that bitcoin is correctly installed, as follows:
We can confirm that bitcoin is correctly installed, by asking the system for the path of the two executables, as follows:
[source,bash]
----
$ which bitcoind
/usr/local/bin/bitcoind
$ which bitcoin-cli
/usr/local/bin/bitcoin-cli
----
The default installation of bitcoind puts it in +/usr/local/bin+. When we first run bitcoind it will remind us to create a configuration file with a strong password for the JSON-RPC interface. We run it by typing +bitcoind+ into the terminal:
[source,bash]
----
$ bitcoind
Error: To use the "-server" option, you must set a rpcpassword in the configuration file:
@ -241,19 +255,50 @@ for example: alertnotify=echo %s | mail -s "Bitcoin Alert" admin@foo.com
Edit the configuration file in your preferred editor and set the parameters, replacing the password with a strong password as recommended by bitcoind. Do *not* use the password shown below. Create a file inside the +.bitcoin+ directory so that it is named +.bitcoin/bitcoin.conf+ and enter a username and password:
[source,ini]
----
rpcuser=bitcoinrpc
rpcpassword=2XA4DuKNCbtZXsBQRRNDEwEY2nM6M4H9Tx5dFjoAVVbK
----
=== Using bitcoind from the command line
While you're editing this configuration file, you may want to set a few other options, such as +txindex+ (See <<txindex>>). For a full listing of the available options type +bitcoind --help+.
The reference client bitcoind offers a number of commands that can be run from the command line. These are the same commands as those offered via the JSON-RPC API, so the command line allows us to experiment interactively with the capabilities that are also available programmatically. To start, we can invoke the +help+ command to see a list of the available bitcoin commands:
Now, run the Bitcoin Core client. The first time you run it, it will rebuild the bitcoin blockchain by downloading all the blocks. This is a multi-gigabyte file and will take on average 2 days to download in full. You can shorten the blockchain initialization time by downloading a partial copy of the blockchain using a bittorrent client from +http://sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/+.
Run bitcoind in the background with the option +-daemon+:
[source,bash]
----
$ bitcoind -daemon
Bitcoin version v0.9.0rc1-beta (2014-01-31 09:30:15 +0100)
Using OpenSSL version OpenSSL 1.0.1c 10 May 2012
Default data directory /home/bitcoin/.bitcoin
Using data directory /bitcoin/
Using at most 4 connections (1024 file descriptors available)
init message: Verifying wallet...
dbenv.open LogDir=/bitcoin/database ErrorFile=/bitcoin/db.log
Bound to [::]:8333
Bound to 0.0.0.0:8333
init message: Loading block index...
Opening LevelDB in /bitcoin/blocks/index
Opened LevelDB successfully
Opening LevelDB in /bitcoin/chainstate
Opened LevelDB successfully
[... more startup messages ...]
----
=== Using Bitcoin Core's JSON-RPC API from the command line
The Bitcoin Core client implements a JSON-RPC interface that can also be accessed using the command line helper +bitcoin-cli+. The command line allows us to experiment interactively with the capabilities that are also available programmatically via the API. To start, we can invoke the +help+ command to see a list of the available bitcoin commands:
[[bitcoind_commands]]
.Bitcoind command list
.Bitcoin Core RPC commands
[source,bash]
----
$ bitcoin-cli help
addmultisigaddress nrequired ["key",...] ( "account" )
addnode "node" "add|remove|onetry"
backupwallet "destination"
@ -263,7 +308,6 @@ decoderawtransaction "hexstring"
decodescript "hex"
dumpprivkey "bitcoinaddress"
dumpwallet "filename"
encryptwallet "passphrase"
getaccount "bitcoinaddress"
getaccountaddress "account"
getaddednodeinfo dns ( "node" )
@ -271,6 +315,7 @@ getaddressesbyaccount "account"
getbalance ( "account" minconf )
getbestblockhash
getblock "hash" ( verbose )
getblockchaininfo
getblockcount
getblockhash index
getblocktemplate ( "jsonrequestobject" )
@ -282,6 +327,7 @@ getinfo
getmininginfo
getnettotals
getnetworkhashps ( blocks height )
getnetworkinfo
getnewaddress ( "account" )
getpeerinfo
getrawchangeaddress
@ -293,6 +339,7 @@ gettransaction "txid"
gettxout "txid" n ( includemempool )
gettxoutsetinfo
getunconfirmedbalance
getwalletinfo
getwork ( "data" )
help ( "command" )
importprivkey "bitcoinprivkey" ( "label" rescan )
@ -323,43 +370,24 @@ submitblock "hexdata" ( "jsonparametersobject" )
validateaddress "bitcoinaddress"
verifychain ( checklevel numblocks )
verifymessage "bitcoinaddress" "signature" "message"
walletlock
walletpassphrase "passphrase" timeout
walletpassphrasechange "oldpassphrase" "newpassphrase"
----
==== Running bitcoind
==== Getting information on the Bitcoin Core client status
Commands: -daemon, getinfo
Commands: getinfo
Now, run the bitcoin client. The first time you run it the bitcoin blockchain will be rebuilt. This is a multi-gigabyte file and will take on average 2 days to download in full. You can shorten the blockchain initialization time by downloading a partial copy of the blockchain using bittorrent from +http://sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/+.
Run bitcoind in the background with the option +-daemon+:
Bitcoin's +getinfo+ RPC command shows us basic information about the status of the bitcoin network node, the wallet and the blockchain database. We use bitcoin-cli to run it:
[source,bash]
----
$ bitcoind -daemon
$
Bitcoin version v0.9.0rc1-beta (2014-01-31 09:30:15 +0100)
Using OpenSSL version OpenSSL 1.0.1c 10 May 2012
Default data directory /home/bitcoin/.bitcoin
Using data directory /bitcoin/
Using at most 4 connections (1024 file descriptors available)
init message: Verifying wallet...
dbenv.open LogDir=/bitcoin/database ErrorFile=/bitcoin/db.log
Bound to [::]:8333
Bound to 0.0.0.0:8333
init message: Loading block index...
Opening LevelDB in /bitcoin/blocks/index
Opened LevelDB successfully
Opening LevelDB in /bitcoin/chainstate
Opened LevelDB successfully
[... more startup messages ...]
$ bitcoin-cli getinfo
----
Bitcoin's +getinfo+ command shows us basic information about the status of the bitcoin network node, the wallet and the blockchain database:
[source,json]
----
$ bitcoind getinfo
{
"version" : 90000,
"protocolversion" : 70002,
@ -387,24 +415,28 @@ It will take some time, perhaps more than a day, for the bitcoind client to "cat
==== Wallet setup and encryption
Commands: bitcoind encryptwallet, walletpassphrase
Commands: encryptwallet, walletpassphrase
Before we proceed with creating keys and other commands, we will first encrypt the wallet with a password. For this example, we use the +encryptwallet+ command with the password "foo". Obviously, replace "foo" with a strong and complex password!
----
$ bitcoind encryptwallet foo
$ bitcoin-cli encryptwallet foo
wallet encrypted; Bitcoin server stopping, restart to run with encrypted wallet. The keypool has been flushed, you need to make a new backup.
$
----
We can verify the wallet has been encrypted by running +getinfo+ again. This time you will notice a new entry +unlocked_until+ that is a counter showing how long the wallet decryption password will be stored in memory, keeping the wallet unlocked. At first this will be set to zero, meaning the wallet is locked:
[source,bash]
----
$ bitcoin-cli getinfo
----
[source,json]
----
$ bitcoind getinfo
{
"version" : 90000,
[... other information...]
#[... other information...]
"unlocked_until" : 0,
"errors" : ""
@ -415,18 +447,22 @@ $
To unlock the wallet, we issue the +walletpassphrase+ command that takes two parameters, the password and a number of seconds until the wallet is locked again automatically (a time counter):
----
$ bitcoind walletpassphrase foo 360
$ bitcoin-cli walletpassphrase foo 360
$
----
Confirm the wallet is unlocked and see the timeout by running +getinfo+ again:
[source,bash]
----
$ bitcoin-cli getinfo
----
[source,json]
----
$ bitcoind getinfo
{
"version" : 90000,
[... other information ...]
#[... other information ...]
"unlocked_until" : 1392580909,
"errors" : ""
@ -439,22 +475,25 @@ Commands: backupwallet, importwallet, dumpwallet
Next, we will practice creating a wallet backup file and then restoring the wallet from the backup file. Use the +backupwallet+ command to backup, providing the file name as the parameter. Here we backup the wallet to the file +wallet.backup+:
[source,bash]
----
$ bitcoind backupwallet wallet.backup
$ bitcoin-cli backupwallet wallet.backup
$
----
Now, to restore the backup file, use the +importwallet+ command. If your wallet is locked, you will need to unlock it first (see +walletpassphrase+ above) in order to import the backup file:
[source,bash]
----
$ bitcoind importwallet wallet.backup
$ bitcoin-cli importwallet wallet.backup
$
----
The +dumpwallet+ command can be used to dump the wallet into a text file that is human-readable:
[source,bash]
----
$ bitcoind dumpwallet wallet.txt
$ bitcoin-cli dumpwallet wallet.txt
$ more wallet.txt
# Wallet dump created by Bitcoin v0.9.0rc1-beta (2014-01-31 09:30:15 +0100)
# * Created on 2014-02- 8dT20:34:55Z
@ -474,8 +513,9 @@ Commands: getnewaddress, getreceivedbyaddress, listtransactions, getaddressesbya
The bitcoin reference client maintains a pool of addresses, the size of which is displayed by +keypoolsize+ when you use the command +getinfo+. These addresses are generated automatically and can then be used as public receiving addresses or change addresses. To get one of these addresses, you can use the +getnewaddress+ command:
[source,bash]
----
$ bitcoind getnewaddress
$ bitcoin-cli getnewaddress
1hvzSofGwT8cjb8JU7nBsCSfEVQX5u9CL
----
@ -483,22 +523,28 @@ Now, we can use this address to send a small amount of bitcoin to our bitcoind w
We can now query the bitcoind client for the amount received by this address, and specify how many confirmations are required before an amount is counted in that balance. For this example, we will specify zero confirmations. A few seconds after sending the bitcoin from another wallet, we will see it reflected in the wallet. We use +getreceivedbyaddress+ with the address and the number of confirmations set to zero (0):
[source,bash]
----
$ bitcoind getreceivedbyaddress 1hvzSofGwT8cjb8JU7nBsCSfEVQX5u9CL 0
$ bitcoin-cli getreceivedbyaddress 1hvzSofGwT8cjb8JU7nBsCSfEVQX5u9CL 0
0.05000000
----
If we omit the zero from the end of this command, we will only see the amounts that have at least +minconf+ confirmations, where +minconf+ is the setting for the minimum number of confirmations before a transaction is listed in the balance. The +minconf+ setting is specified in the bitcoind configuration file. Since the transaction sending this bitcoin was only sent in the last few seconds, it has still not confirmed and therefore we will see it list a zero balance:
[source,bash]
----
$ bitcoind getreceivedbyaddress 1hvzSofGwT8cjb8JU7nBsCSfEVQX5u9CL
$ bitcoin-cli getreceivedbyaddress 1hvzSofGwT8cjb8JU7nBsCSfEVQX5u9CL
0.00000000
----
The transactions received by the entire wallet can also be displayed using the +listtransactions+ command:
[source,bash]
----
$ bitcoin-cli listtransactions
----
[source,json]
----
$ bitcoind listtransactions
[
{
"account" : "",
@ -516,8 +562,12 @@ $ bitcoind listtransactions
We can list all addresses in the entire wallet using the +getaddressesbyaccount+ command:
[source,bash]
----
$ bitcoin-cli getaddressesbyaccount ""
----
[source,json]
----
$ bitcoind getaddressesbyaccount ""
[
"1LQoTPYy1TyERbNV4zZbhEmgyfAipC6eqL",
"17vrg8uwMQUibkvS2ECRX4zpcVJ78iFaZS",
@ -537,8 +587,9 @@ $ bitcoind getaddressesbyaccount ""
Finally, the command +getbalance+ will show the total balance of the wallet, adding up all transactions confirmed with at least +minconf+ confirmations:
[source,bash]
----
$ bitcoind getbalance
$ bitcoin-cli getbalance
0.05000000
----
@ -553,8 +604,12 @@ Commands: gettransaction, getrawtransaction, decoderawtransaction
We'll now explore the incoming transaction that was listed above using the +gettransaction+ command. We can retrieve a transaction by its transaction hash, shown at +txid+, above with the +gettransaction+ command:
[source,bash]
----
$ bitcoin-cli gettransaction 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3
----
[source,json]
----
$ bitcoind gettransaction 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3
{
"amount" : 0.05000000,
"confirmations" : 0,
@ -579,15 +634,20 @@ Transaction IDs are not authoritative until a transaction has been confirmed. Ab
The transaction form shown above with the command +gettransaction+ is the simplified form. To retrieve the full transaction code and decode it we will use two commands, +getrawtransaction+ and +decoderawtransaction+. First, +getrawtransaction+ takes the _transaction hash (txid)_ as a parameter and returns the full transaction as a "raw" hex string, exactly as it exists on the bitcoin network:
[source,bash]
----
$ bitcoind getrawtransaction 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3
$ bitcoin-cli getrawtransaction 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3
0100000001d717279515f88e2f56ce4e8a31e2ae3e9f00ba1d0add648e80c480ea22e0c7d3000000008b483045022100a4ebbeec83225dedead659bbde7da3d026c8b8e12e61a2df0dd0758e227383b302203301768ef878007e9ef7c304f70ffaf1f2c975b192d34c5b9b2ac1bd193dfba2014104793ac8a58ea751f9710e39aad2e296cc14daa44fa59248be58ede65e4c4b884ac5b5b6dede05ba84727e34c8fd3ee1d6929d7a44b6e111d41cc79e05dbfe5ceaffffffff02404b4c00000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac1f312906000000001976a914107b7086b31518935c8d28703d66d09b3623134388ac00000000
----
To decode this hex string, we can use the +decoderawtransaction+ command. Copy and paste the hex as the first parameter of +decoderawtransaction+ to get the full contents interpreted as a JSON data structure (for formatting reasons the hex string is shortened in the example below):
[source,bash]
----
$ bitcoin-cli decoderawtransaction 0100000001d717...388ac00000000
----
[source,json]
----
$ bitcoind decoderawtransaction 0100000001d717...388ac00000000
{
"txid" : "9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3",
"version" : 1,
@ -597,8 +657,14 @@ $ bitcoind decoderawtransaction 0100000001d717...388ac00000000
"txid" : "d3c7e022ea80c4808e64dd0a1dba009f3eaee2318a4ece562f8ef815952717d7",
"vout" : 0,
"scriptSig" : {
"asm" : "3045022100a4ebbeec83225dedead659bbde7da3d026c8b8e12e61a2df0dd0758e227383b302203301768ef878007e9ef7c304f70ffaf1f2c975b192d34c5b9b2ac1bd193dfba201 04793ac8a58ea751f9710e39aad2e296cc14daa44fa59248be58ede65e4c4b884ac5b5b6dede05ba84727e34c8fd3ee1d6929d7a44b6e111d41cc79e05dbfe5cea",
"hex" : "483045022100a4ebbeec83225dedead659bbde7da3d026c8b8e12e61a2df0dd0758e227383b302203301768ef878007e9ef7c304f70ffaf1f2c975b192d34c5b9b2ac1bd193dfba2014104793ac8a58ea751f9710e39aad2e296cc14daa44fa59248be58ede65e4c4b884ac5b5b6dede05ba84727e34c8fd3ee1d6929d7a44b6e111d41cc79e05dbfe5cea"
"asm" : "3045022100a4ebbeec83225dedead659bbde7da3d026c8b8e12e61a2df0dd0\
758e227383b302203301768ef878007e9ef7c304f70ffaf1f2c975b192d34c5b9b2ac1b\
d193dfba201 04793ac8a58ea751f9710e39aad2e296cc14daa44fa59248be58ede65e4\
c4b884ac5b5b6dede05ba84727e34c8fd3ee1d6929d7a44b6e111d41cc79e05dbfe5cea",
"hex" : "483045022100a4ebbeec83225dedead659bbde7da3d026c8b8e12e61a2df0d\
d0758e227383b302203301768ef878007e9ef7c304f70ffaf1f2c975b192d34c5b9b2ac\
1bd193dfba2014104793ac8a58ea751f9710e39aad2e296cc14daa44fa59248be58ede6\
5e4c4b884ac5b5b6dede05ba84727e34c8fd3ee1d6929d7a44b6e111d41cc79e05dbfe5cea"
},
"sequence" : 4294967295
}
@ -608,7 +674,8 @@ $ bitcoind decoderawtransaction 0100000001d717...388ac00000000
"value" : 0.05000000,
"n" : 0,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 07bdb518fa2e6089fd810235cf1100c9c13d1fd2 OP_EQUALVERIFY OP_CHECKSIG",
"asm" : "OP_DUP OP_HASH160 07bdb518fa2e6089fd810235cf1100c9c13d1fd2\
OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
@ -621,7 +688,8 @@ $ bitcoind decoderawtransaction 0100000001d717...388ac00000000
"value" : 1.03362847,
"n" : 1,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 107b7086b31518935c8d28703d66d09b36231343 OP_EQUALVERIFY OP_CHECKSIG",
"asm" : "OP_DUP OP_HASH160 107b7086b31518935c8d28703d66d09b36231343\
OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a914107b7086b31518935c8d28703d66d09b3623134388ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
@ -640,8 +708,12 @@ We can further explore the blockchain by examining the previous transaction refe
Once the transaction we received has been confirmed by inclusion in a block, the +gettransaction+ command will return additional information, showing the _block hash (identifier)_ in which the transaction was included:
[source,bash]
----
$ bitcoin-cli gettransaction 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3
----
[source,json]
----
$ bitcoind gettransaction 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3
{
"amount" : 0.05000000,
"confirmations" : 1,
@ -664,14 +736,24 @@ $ bitcoind gettransaction 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309ac
Above, we see the new information in the entries +blockhash+, the hash of the block in which the transaction was included, and +blockindex+ with value 18, indicating that our transaction was the 18th transaction in that block.
[[txindex]]
.Transaction database index and txindex option
****
By default, Bitcoin Core builds a database containing _only_ the transactions related to the user's wallet. If you want to be able to access _any_ transaction with commands like +gettransaction+, you need to configure Bitcoin Core to build a complete transaction index, which can be achieved with the +txindex+ option. Set +txindex=1+ in the Bitcoin Core configuration file (usually found in your home directory under +.bitcoin/bitcoin.conf+). Once you change this parameter, you need to restart bitcoind and wait for it to rebuild the index.
****
==== Exploring blocks
Commands: getblock, getblockhash
Now that we know which block our transaction was included in, we can query that block. We use the +getblock+ command with the block hash as the parameter:
[source,bash]
----
$ bitcoin-cli getblock 000000000000000051d2e759c63a26e247f185ecb7926ed7a6624bc31c2a717b true
----
[source,bash]
----
$ bitcoind getblock 000000000000000051d2e759c63a26e247f185ecb7926ed7a6624bc31c2a717b true
{
"hash" : "000000000000000051d2e759c63a26e247f185ecb7926ed7a6624bc31c2a717b",
"confirmations" : 2,
@ -700,7 +782,7 @@ $ bitcoind getblock 000000000000000051d2e759c63a26e247f185ecb7926ed7a6624bc31c2a
"e57a4c70f91c8d9ba0ff0a55987ea578affb92daaa59c76820125f31a9584dfc",
"9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3",
[... many more transactions ...]
#[... many more transactions ...]
],
"time" : 1392660808,
@ -717,15 +799,20 @@ The block contains 367 transactions and as you see above, the 18th transaction l
We can also retrieve a block by its block height using the +getblockhash+ command, which takes the block height as the parameter and returns the block hash for that block:
[source,bash]
----
$ bitcoind getblockhash 0
$ bitcoin-cli getblockhash 0
000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
----
Above, we retrieve the block hash of the "genesis block", the first block mined by Satoshi Nakamoto, at height zero. Retrieving this block shows:
[source,bash]
----
$ bitcoin-cli getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
----
[source,json]
----
$ bitcoind getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
{
"hash" : "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
"confirmations" : 286388,
@ -756,8 +843,12 @@ Bitcoin's transactions are based on the concept of spending "outputs", which are
First, we use the +listunspent+ command to show all the unspent *confirmed* outputs in our wallet:
[source,bash]
----
$ bitcoin-cli listunspent
----
[source,json]
----
$ bitcoind listunspent
[
{
"txid" : "9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3",
@ -775,8 +866,12 @@ We see that the transaction +9ca8f9...+ created an output (with vout index 0) as
First, let's look at the specific output in more detail. We use the +gettxout+ to get the details of this unspent output above. Transaction outputs are always referenced by txid and vout, and these are the parameters we pass to +gettxout+:
[source,bash]
----
$ bitcoin-cli gettxout 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3 0
----
[source,json]
----
$ bitcoind gettxout 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3 0
{
"bestblock" : "0000000000000001405ce69bd4ceebcdfdb537749cebe89d371eb37e13899fd9",
"confirmations" : 7,
@ -797,8 +892,9 @@ $ bitcoind gettxout 9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c1
What we see above is the output that assigned 50 millibits to our address +1hvz...+. To spend this output we will create a new transaction. First, let's make an address to which we will send the money:
[source,bash]
----
$ bitcoind getnewaddress
$ bitcoin-cli getnewaddress
1LnfTndy3qzXGN19Jwscj1T8LR3MVe3JDb
----
@ -806,19 +902,31 @@ We will send 25 millibits to the new address +1LnfTn...+ we just created in our
We use the +createrawtransaction+ to create the transaction described above. As parameters to +createrawtransaction+ we provide the transaction input (the 50 millibit unspent output from our confirmed transaction) and the two transaction outputs (money sent to the new address and change sent back to the previous address):
[source,bash]
----
bitcoind createrawtransaction
'[{"txid" : "9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3", "vout" : 0}]'
'{"1LnfTndy3qzXGN19Jwscj1T8LR3MVe3JDb": 0.025,
$ bitcoin-cli createrawtransaction \
'[{"txid" : "9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3", "vout" : 0}]' \
'{"1LnfTndy3qzXGN19Jwscj1T8LR3MVe3JDb": 0.025, \
"1hvzSofGwT8cjb8JU7nBsCSfEVQX5u9CL": 0.0245}'
0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c0000000000ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac00000000
0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c\
0000000000ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd6810756\
4a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d\
1fd288ac00000000
----
The +createrawtransaction+ command produces a raw hex string that encodes the transaction details we supplied. Let's confirm everything is correct by decoding this raw string using the +decoderawtransaction+ command:
[source,bash]
----
$ bitcoin-cli decoderawtransaction \
0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c\
0000000000ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd6810756\
4a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d\
1fd288ac00000000
----
[source,json]
----
$ bitcoind decoderawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c0000000000ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac00000000
{
"txid" : "0793299cb26246a8d24e468ec285a9520a1c30fcb5b6125a102e3fc05d4f3cba",
"version" : 1,
@ -839,7 +947,8 @@ $ bitcoind decoderawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db
"value" : 0.02500000,
"n" : 0,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 d90d36e98f62968d2bc9bbd68107564a156a9bcf OP_EQUALVERIFY OP_CHECKSIG",
"asm" : "OP_DUP OP_HASH160 d90d36e98f62968d2bc9bbd68107564a156a9bcf\
OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
@ -852,7 +961,8 @@ $ bitcoind decoderawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db
"value" : 0.02450000,
"n" : 1,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 07bdb518fa2e6089fd810235cf1100c9c13d1fd2 OP_EQUALVERIFY OP_CHECKSIG",
"asm" : "OP_DUP OP_HASH160 07bdb518fa2e6089fd810235cf1100c9c13d1fd2\
OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
@ -874,9 +984,10 @@ As you may notice, the transaction contains an empty +scriptSig+ because we have
An encrypted wallet must be unlocked before a transaction is signed because signing requires access to the secret keys in the wallet.
====
[source,bash]
----
$ bitcoind walletpassphrase foo 360
$ bitcoind signrawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c0000000000ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac00000000
$ bitcoin-cli walletpassphrase foo 360
$ bitcoin-cli signrawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c0000000000ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac00000000
{
"hex" : "0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c000000006a47304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af52051a06012103c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71dc5127ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac00000000",
"complete" : true
@ -885,8 +996,19 @@ $ bitcoind signrawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f
The +signrawtransaction+ command returns another hex encoded raw transaction. We decode it to see what changed, with +decoderawtransaction+:
[source,bash]
----
$ bitcoin-cli decoderawtransaction\
0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c\
000000006a47304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76\
d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af52051\
a06012103c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71dc5127ff\
ffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88a\
c50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac000000\
00
----
[source,json]
----
$ bitcoind decoderawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c000000006a47304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af52051a06012103c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71dc5127ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac00000000
{
"txid" : "ae74538baa914f3799081ba78429d5d84f36a0127438e9f721dff584ac17b346",
"version" : 1,
@ -896,8 +1018,14 @@ $ bitcoind decoderawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db
"txid" : "9ca8f969bd3ef5ec2a8685660fdbf7a8bd365524c2e1fc66c309acbae2c14ae3",
"vout" : 0,
"scriptSig" : {
"asm" : "304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af52051a0601 03c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71dc5127",
"hex" : "47304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af52051a06012103c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71dc5127"
"asm" : "304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76\
d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af5\
2051a0601 03c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71d\
c5127",
"hex" : "47304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b\
76d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4a\
f52051a06012103c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c\
71dc5127"
},
"sequence" : 4294967295
}
@ -907,7 +1035,8 @@ $ bitcoind decoderawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db
"value" : 0.02500000,
"n" : 0,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 d90d36e98f62968d2bc9bbd68107564a156a9bcf OP_EQUALVERIFY OP_CHECKSIG",
"asm" : "OP_DUP OP_HASH160 d90d36e98f62968d2bc9bbd68107564a156a9bcf\
OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
@ -920,7 +1049,8 @@ $ bitcoind decoderawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db
"value" : 0.02450000,
"n" : 1,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 07bdb518fa2e6089fd810235cf1100c9c13d1fd2 OP_EQUALVERIFY OP_CHECKSIG",
"asm" : "OP_DUP OP_HASH160 07bdb518fa2e6089fd810235cf1100c9c13d1fd2\
OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
@ -937,16 +1067,29 @@ Now, the inputs used in the transaction contain a +scriptSig+, which is a digita
Now it's time to submit the newly created transaction to the network. We do that with the command +sendrawtransaction+ which takes the raw hex string produced by +signrawtransaction+. This is the same string we just decoded above:
[source,bash]
----
$ bitcoind sendrawtransaction 0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c000000006a47304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af52051a06012103c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71dc5127ffffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88ac50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac00000000
$ bitcoin-cli sendrawtransaction\
0100000001e34ac1e2baac09c366fce1c2245536bda8f7db0f6685862aecf53ebd69f9a89c\
000000006a47304402203e8a16522da80cef66bacfbc0c800c6d52c4a26d1d86a54e0a1b76\
d661f020c9022010397f00149f2a8fb2bc5bca52f2d7a7f87e3897a273ef54b277e4af52051\
a06012103c9700559f690c4a9182faa8bed88ad8a0c563777ac1d3f00fd44ea6c71dc5127ff\
ffffff02a0252600000000001976a914d90d36e98f62968d2bc9bbd68107564a156a9bcf88a\
c50622500000000001976a91407bdb518fa2e6089fd810235cf1100c9c13d1fd288ac000000\
00
ae74538baa914f3799081ba78429d5d84f36a0127438e9f721dff584ac17b346
----
The command +sendrawtransaction+ returns a _transaction hash (txid)_ as it submits the transaction on the network. We can now query that transaction id with +gettransaction+:
[source,bash]
----
$ bitcoin-cli gettransaction \
ae74538baa914f3799081ba78429d5d84f36a0127438e9f721dff584ac17b346
----
[source,json]
----
$ bitcoind gettransaction ae74538baa914f3799081ba78429d5d84f36a0127438e9f721dff584ac17b346
{
"amount" : 0.00000000,
"fee" : -0.00050000,
@ -987,7 +1130,7 @@ $ bitcoind gettransaction ae74538baa914f3799081ba78429d5d84f36a0127438e9f721dff5
As before, we can also examine this in more detail using the +getrawtransaction+ and +decodetransaction+ commands. These commands will return the exact same hex string that we produced and decoded previously just before we sent it on the network.
[[alt_libraries]]
=== Alternative clients, libraries and toolkits
Beyond the reference client, bitcoind, there are other clients and libraries that can be used to interact with the bitcoin network and data structures. These are implemented in a variety of programming languages, offering programmers native interfaces in their own language.
@ -999,9 +1142,12 @@ Alternative implementations include:
* btcd, a Go language full node bitcoin client (https://opensource.conformal.com/wiki/btcd)
* Bits of Proof (BOP), a Java enterprise-class implementation of bitcoin (https://bitsofproof.com)
* picocoin, a C implementation of a light-weight client library for bitcoin (https://github.com/jgarzik/picocoin)
* pybitcointools, a Python bitcoin library (https://github.com/vbuterin/pybitcointools)
* pycoin, another Python bitcoin library (https://github.com/richardkiss/pycoin)
Many more libraries exist in a variety of other programming languages and more are created all the time.
[[sx_tools]]
==== Libbitcoin and sx tools
The libbitcoin library is a C++ scalable multi-threaded and modular implementation that supports a full-node client and a command-line toolset named "sx", which offers many of the same capabilities as the bitcoind client commands we illustrated in this chapter. The sx tools also offer some key management and manipulation tools that are not offered by bitcoind, including type-2 deterministic keys and key mnemonics.
@ -1010,122 +1156,19 @@ The libbitcoin library is a C++ scalable multi-threaded and modular implementati
To install sx and the supporting library libbitcoin, download and run the online installer on a Linux system:
[source,bash]
----
$ wget http://sx.dyne.org/install-sx.sh
$ sudo bash ./install-sx.sh
----
You should now have the sx tools installed. Type +sx+ with no parameters to display the help text, which lists all the available commands:
You should now have the sx tools installed. Type +sx+ with no parameters to display the help text, which lists all the available commands (See <<sx_cmds_>>)
----
Usage: sx COMMAND [ARGS]...
-c, --config Specify a config file
The sx commands are:
DETERMINISTIC KEYS AND ADDRESSES
genaddr Generate a Bitcoin address deterministically from a wallet
seed or master public key.
genpriv Generate a private key deterministically from a seed.
genpub Generate a public key deterministically from a wallet
seed or master public key.
mpk Extract a master public key from a deterministic wallet seed.
newseed Create a new deterministic wallet seed.
TRANSACTION PARSING
showscript Show the details of a raw script.
showtx Show the details of a transaction.
BLOCKCHAIN QUERIES (blockexplorer.com)
blke-fetch-transaction Fetches a transaction from blockexplorer.com
FORMAT
base58-decode Convert from base58 to hex
base58-encode Convert from hex to base58
base58check-decode Convert from base58check to hex
base58check-encode Convert from hex to base58check
decode-addr Decode an address to its internal RIPEMD representation.
embed-addr Generate an address used for embedding record of data into the blockchain.
encode-addr Encode an address to base58check form.
ripemd-hash RIPEMD hash data from STDIN.
unwrap Validates checksum and recovers version byte and original data from hexstring.
validaddr Validate an address.
wrap Adds version byte and checksum to hexstring.
BRAINWALLET
brainwallet Make a private key from a brainwallet
mnemonic Work with Electrum compatible mnemonics (12 words wallet seed).
BLOCKCHAIN WATCHING
monitor Monitor an address.
watchtx Watch transactions from the network searching for a certain hash.
BLOCKCHAIN QUERIES (blockchain.info)
bci-fetch-last-height Fetch the last block height using blockchain.info.
bci-history Get list of output points, values, and their spends
from blockchain.info
MISC
btc Convert Satoshis into Bitcoins.
initchain Initialize a new blockchain.
qrcode Generate Bitcoin QR codes offline.
satoshi Convert Bitcoins into Satoshis.
showblkhead Show the details of a block header.
wallet Experimental command line wallet.
MULTISIG ADDRESSES
scripthash Create BIP 16 script hash address from raw script hex.
LOOSE KEYS AND ADDRESSES
addr See Bitcoin address of a public or private key.
get-pubkey Get the pubkey of an address if available
newkey Create a new private key.
pubkey See the public part of a private key.
STEALTH
secret-to-wif Convert a secret exponent value to Wallet. Import. Format.
stealth-new Generate a new master stealth secret.
stealth-recv Regenerate the secret from your master secret and provided nonce.
stealth-send Generate a new sending address and a stealth nonce.
CREATE TRANSACTIONS
mktx Create an unsigned tx.
rawscript Create the raw hex representation from a script.
set-input Set a transaction input.
sign-input Sign a transaction input.
VALIDATE
validsig Validate a transaction input's signature.
BLOCKCHAIN QUERIES
balance Show balance of a Bitcoin address in satoshis.
fetch-block-header Fetch raw block header.
fetch-last-height Fetch the last block height.
fetch-transaction Fetch a raw transaction using a network connection to make requests against the obelisk load balancer backend.
fetch-transaction-index Fetch block height and index in block of transaction.
get-utxo Get enough unspent transaction outputs from a given set of
addresses to pay a given number of satoshis
history Get list of output points, values, and their spends for an
address. grep can filter for just unspent outputs which can
be fed into mktx.
validtx Validate a transaction.
BLOCKCHAIN UPDATES
sendtx-bci Send tx to blockchain.info/pushtx.
sendtx-node Send transaction to a single node.
sendtx-obelisk Send tx to obelisk server.
sendtx-p2p Send tx to bitcoin network.
See 'sx help COMMAND' for more information on a specific command.
SpesmiloXchange home page: <http://sx.dyne.org/>
----
===== Generating and manipulating keys with sxBitcoin Core
===== Generating and manipulating keys with sx
Generate a new private key with the operating system's random number generator by using the +newkey+ command. We save the standard output into the file +private_key+:
[source,bash]
----
$ sx newkey > private_key
$ cat private_key
@ -1134,6 +1177,7 @@ $ cat private_key
Now, generate the public key from that private key using the +pubkey+ command. Pass the +private_key+ file into the standard input and save the standard output of the command into a new file +public_key+:
[source,bash]
----
$ sx pubkey < private_key > public_key
$ cat public_key
@ -1141,6 +1185,8 @@ $ cat public_key
----
We can re-format the public_key as an address using the +addr+ command. We pass the +public_key+ into standard input:
[source,bash]
----
$ sx addr < public_key
17re1S4Q8ZHyCP8Kw7xQad1Lr6XUzWUnkG
@ -1152,6 +1198,7 @@ The keys generated above are so called type-0 non-deterministic keys. That means
First, we generate a "seed" that will be used as the basis to derive a chain of keys, compatible with the Electrum wallet and other similar implementations. We use the +newseed+ command to produce a seed value:
[source,bash]
----
$ sx newseed > seed
$ cat seed
@ -1161,6 +1208,7 @@ eb68ee9f3df6bd4441a9feadec179ff1
The seed value can also be exported as a word mnemonic that is human readable and easier to store and type than a hexadecimal string
using the +mnemonic+ command:
[source,bash]
----
$ sx mnemonic < seed > words
$ cat words
@ -1169,6 +1217,7 @@ adore repeat vision worst especially veil inch woman cast recall dwell appreciat
The mnemonic words can be used to reproduce the seed using the +mnemonic+ command again:
[source,bash]
----
$ sx mnemonic < words
eb68ee9f3df6bd4441a9feadec179ff1
@ -1176,6 +1225,7 @@ eb68ee9f3df6bd4441a9feadec179ff1
With the seed, we can now generate a sequence of private and public keys, a key chain. We use the +genpriv+ command to generate a sequence of private keys from a seed and the +addr+ command to generate the corresponding public key.
[source,bash]
----
$ sx genpriv 0 < seed
5JzY2cPZGViPGgXZ4Syb9Y4eUGjJpVt6sR8noxrpEcqgyj7LK7i
@ -1204,12 +1254,14 @@ One key difference between btcd and bitcoind is that btcd does not include walle
To install btcd, for Windows, download and run the msi available at https://github.com/conformal/btcd/releases or run the following command on Linux, assuming you already have installed the Go language:
[source,bash]
----
$ go get github.com/conformal/btcd/...
----
To update btcd to the latest version, just run:
[source,bash]
----
$ go get -u -v github.com/conformal/btcd/...
----
@ -1218,6 +1270,7 @@ $ go get -u -v github.com/conformal/btcd/...
btcd has a number of configuration options, which can be viewed by running:
[source,bash]
----
$ btcd --help
----
@ -1225,12 +1278,14 @@ $ btcd --help
btcd comes prepacked with some goodies such as btcctl, a command line utility that can be used to both control and query btcd via RPC. btcd does not enable its RPC server by default; you must configure at minimum both an RPC username and password:
* btcd.conf configuration file
[source,ini]
----
[Application Options]
rpcuser=myuser
rpcpass=SomeDecentp4ssw0rd
----
* btcctl.conf configuration file
[source,ini]
----
[Application Options]
rpcuser=myuser
@ -1239,6 +1294,7 @@ rpcpass=SomeDecentp4ssw0rd
Or if you want to override the configuration files from the command line:
[source,bash]
----
$ btcd -u myuser -P SomeDecentp4ssw0rd
$ btcctl -u myuser -P SomeDecentp4ssw0rd
@ -1246,6 +1302,7 @@ $ btcctl -u myuser -P SomeDecentp4ssw0rd
For a list of available options, run:
[source,bash]
----
$ btcctl --help
----

515
ch04.asciidoc
View File

@ -5,61 +5,16 @@
Ownership of bitcoin is established through _digital keys_, _bitcoin addresses_ and _digital signatures_. The digital keys are not actually stored in the network, but are instead created and stored by end-users in a file, or simple database, called a _wallet_. The digital keys in a user's wallet are completely independent of the bitcoin protocol and can be generated and managed by the user's wallet software without reference to the blockchain or access to the Internet. Keys enable many of the interesting properties of bitcoin, including de-centralized trust and control, ownership attestation and the cryptographic-proof security model.
The digital keys within each user's wallet allow the user to sign transactions, thereby providing cryptographic proof of the ownership of the bitcoins sourced by the transaction. Keys come in pairs consisting of a private (secret) and public key. Think of the public key as similar to a bank account number and the private key as similar to the secret PIN number, or signature on a cheque that provides control over the account. These digital keys are very rarely seen by the users of bitcoin. For the most part, they are stored inside the wallet file and managed by the bitcoin wallet software.
HNFUzKACoVNLnyx4PlW7hJg3dpko2UlDwttaxaUPan4afMmK9ZAtHMKB4PqJgYFe8VDIp0x7OpruTFugxA5hBTE=
Every bitcoin transaction requires a valid signature to be included in the blockchain, which can only be generated with valid digital keys, therefore anyone with a copy of those keys has control of the bitcoin in that account. Keys come in pairs consisting of a private (secret) and public key. Think of the public key as similar to a bank account number and the private key as similar to the secret PIN number, or signature on a cheque that provides control over the account. These digital keys are very rarely seen by the users of bitcoin. For the most part, they are stored inside the wallet file and managed by the bitcoin wallet software.
In the payment portion of a bitcoin transaction, the recipient's public key is represented by its digital fingerprint, called a _bitcoin address_, which is used in the same way as the beneficiary name on a cheque (i.e. "Pay to the order of"). In most cases, a bitcoin address is generated from and corresponds to a public key. However, not all bitcoin addresses represent public keys; they can also represent other beneficiaries such as scripts, as we will see later in this chapter. This way, bitcoin addresses abstract the recipient of funds, making transaction destinations flexible, similar to paper cheques: a single payment instrument that can be used to pay into people's accounts, company accounts, pay for bills or pay to cash. The bitcoin address is the only representation of the keys that users will routinely see, as this is the part they need to share with the world.
In this chapter we will introduce wallets, which contain cryptographic keys. We will look at how keys are generated, stored and managed. We will review the various encoding formats used to represent private and public keys, addresses and script addresses. Finally we will look at special uses of keys: to sign messages, to prove ownership and to create vanity addresses and paper wallets.
=== ECDSA Arithmetic
Just as RSA uses integer arithmetic operators over a prime modulus to provide cryptographic functions, ECDSA defines an arithmetic that has some features in common with traditional arithmetic.
Wikipedia has a good article that explains the details of how arithmetic operations work on an elliptic curve. See http://en.wikipedia.org/wiki/Elliptic_curve_cryptography for more information.
Glossing over details, here are the fundamental facts:
An elliptic curve field is a set of points (x, y) each of which that satisfies the equation
y^2^ = x^3^ + ax + b (mod P)
for some constants a, b and P (where P is prime). Bitcoin uses a standard curve known as secp256, where a=0, b=7, and P = 2^256^ - 2^32^ - 2^9^ - 2^8^ - 2^7^ - 2^6^ - 2^4^ - 1.
So for example, (55066263022277343669578718895168534326250603453777594175500187360389116729240, 32670510020758816978083085130507043184471273380659243275938904335757337482424) is a point on the secp256 curve. You can check this yourself using Python.
----
Python 3.4.0 (default, Mar 30 2014, 19:23:13)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> p = 115792089237316195423570985008687907853269984665640564039457584007908834671663
>>> x = 55066263022277343669578718895168534326250603453777594175500187360389116729240
>>> y = 32670510020758816978083085130507043184471273380659243275938904335757337482424
>>> (x ** 3 + 7 - y**2) % p
0
----
In addition, there is also a "point at infinity", which roughly corresponds to the role of 0 in addition. On computers, it's sometimes represented by x = y = 0 (which doesn't satisfy the elliptic curve equation -- but it's an easy separate case that can be checked).
There is also an operator "+", called "addition" which has some properties similar to the traditional addition of real numbers that grade school children learn. Given two points P~1~ and P~2~ on the elliptic curve, there is a third point P~3~ = P~1~ + P~2~, also on the elliptic curve.
Geometrically, this third point P~3~ is calculated by drawing a line between P~1~ and P~2~. This line will intersect the elliptic curve in exactly one additional place. Call this point P~3~' = (x, y). Then reflect in the X axis to get P~3~ = (x, -y).
There are a couple of special cases which explain the need for the "point at infinity".
If P~1~ and P~2~ are the same point, the line "between" P~1~ and P~2~ should extend to be the tangent on the curve at this point P~1~. This tangent will intersect the curve in exactly one new point. You can use techniques from calculus to determine the slope of the tangent line. These techniques curiously work even though we are restricting our interest to points on the curve with two integer coordinates!
In some cases (i.e., if P~1~ and P~2~ have the same x values but different y values), the tangent line will be exactly vertical, in which case P3 = "point at infinity".
If one of P~1~ is the "point at infinity", then the sum P~1~ + P~2~ = P~2~. Similary, if P~2~ is the point at infinity, then P~1~ + P~2~ = P~1~. This shows how the point at infinity plays the roll of 0.
It turns out that + is associative, which means that `(A+B)+C = A+(B+C)`. That means we can write A+B+C without parentheses without any ambiguity.
Now that we have defined addition, we can define multiplication in the standard way that extends addition. For a point P on the elliptic curve, if k is a whole number, then kP = P + P + P + ... + P (k times). Note that k is sometimes confusingly called an "exponent" in this case. (It would make a lot more sense to call it this if we used an operator that looked like multiplication rather than "+".)
=== Keys
==== Public key cryptography and crypto-currency
((("public key")))
Public key cryptography was invented in the 1970s and is a mathematical foundation for computer and information security.
@ -85,7 +40,7 @@ A bitcoin wallet contains a collection of key pairs, each consisting of a privat
.Private Key, Public Key and Bitcoin Address
image::images/privk_to_pubK_to_addressA.png["privk_to_pubK_to_addressA"]
[[private_keys]]
==== Private Keys
A +private key+ is simply a number, picked at random. Ownership and control over the private key is the root of user control over all funds associated with the corresponding bitcoin address. The private key is used to create signatures that are required to spend bitcoins by proving ownership of funds used in a transaction. The private key must remain secret at all times, as revealing it to a third party is equivalent to giving them control over the bitcoins secured by that key. The private key must also be backed up and protected from accidental loss, since if lost it cannot be recovered and the funds secured by it are forever lost too.
@ -103,7 +58,7 @@ More accurately, the private key can be any number between +1+ and +n - 1+, wher
[TIP]
====
Do not try and design your own pseudo random number generator (PRNG). Use a cryptographically-secure pseudo-random number generator (CSPRNG) with a seed from a source of sufficient entropy, the choice of which depends on the operating-system. Correct implementation of the CSPRNG is critical to the security of the keys. DIY is highly discouraged unless you are a professional cryptographer.
Do not write your own code to create a random number or use a "simple" random number generator offered by your programming language. Use a cryptographically-secure pseudo-random number generator (CSPRNG) with a seed from a source of sufficient entropy. Study the documentation of the random number generator library you choose to make sure it is cryptographically secure. Correct implementation of the CSPRNG is critical to the security of the keys.
====
Below is a randomly generated private key shown in hexadecimal format (256 binary digits shown as 64 hexadecimal digits, each 4 bits):
@ -141,7 +96,8 @@ $ sx newkey
[[pubkey]]
==== Public Keys
The public key is calculated from the private key using elliptic curve multiplication, which is irreversible: latexmath:[\(K = k * G\)]+ where +k+ is the private key, +G+ is a constant point called the _Generator Point_ and +K+ is the resulting public key. The reverse operation, known as "finding the discrete logarithm" -- calculating +k+ if you know +K+ -- is as difficult as trying all possible values of +k+, i.e., a brute-force search. Before we demonstrate how to generate a public key from a private key, let's look at Elliptic Curve Cryptography in a bit more detail.
The public key is calculated from the private key using elliptic curve multiplication, which is irreversible: latexmath:[\(K = k * G\)] where +k+ is the private key, +G+ is a constant point called the _Generator Point_ and +K+ is the resulting public key. The reverse operation, known as "finding the discrete logarithm" -- calculating +k+ if you know +K+ -- is as difficult as trying all possible values of +k+, i.e., a brute-force search. Before we demonstrate how to generate a public key from a private key, let's look at Elliptic Curve Cryptography in a bit more detail.
[[elliptic_curve]]
==== Elliptic Curve Cryptography Explained
@ -160,7 +116,7 @@ Bitcoin uses a specific elliptic curve and set of mathematical constants, as def
[latexmath]
++++
\begin{equation}
{y^2 = (x^3 \+ 7)} \text{over} \mathbb{F}_p
{y^2 = (x^3 \+ 7)} \text{over} \(\mathbb{F}_p\)
\end{equation}
++++
or
@ -172,7 +128,7 @@ or
\end{equation}
++++
The +mod p+ (modulo prime number p) indicates that this curve is over a finite field of prime order +p+, also written as latexmath:[\(\mathbb{F}_p\)], where p = 2^256^ - 2^32^ - 2^9^ - 2^8^ - 2^7^ - 2^6^ - 2^4^ - 1\)], a very large prime number.
The +mod p+ (modulo prime number p) indicates that this curve is over a finite field of prime order +p+, also written as latexmath:[\(\mathbb{F}_p\)], where p = 2^256^ - 2^32^ - 2^9^ - 2^8^ - 2^7^ - 2^6^ - 2^4^ - 1, a very large prime number.
Because this curve is defined over a finite field of prime order instead of over the real numbers it looks like a pattern of dots scattered in two dimensions, which makes it difficult to visualize. However, the math is identical as that of an elliptic curve over the real numbers shown above. As an example, below is the same elliptic curve over a much smaller finite field of prime order 17, showing a pattern of dots on a grid. The +secp256k1+ bitcoin elliptic curve can be thought of as a much more complex pattern of dots on a unfathomably large grid.
@ -180,6 +136,44 @@ Because this curve is defined over a finite field of prime order instead of over
.Elliptic Curve Cryptography: Visualizing an elliptic curve over F(p), with p=17
image::images/ecc-over-F17-math.png["ecc-over-F17-math"]
So for example, below is a point P with coordinates (x,y) that is a point on the secp256k1 curve. You can check this yourself using Python:
----
P = (55066263022277343669578718895168534326250603453777594175500187360389116729240,\
32670510020758816978083085130507043184471273380659243275938904335757337482424)
----
====
[source, pycon]
----
Python 3.4.0 (default, Mar 30 2014, 19:23:13)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> p = 115792089237316195423570985008687907853269984665640564039457584007908834671663
>>> x = 55066263022277343669578718895168534326250603453777594175500187360389116729240
>>> y = 32670510020758816978083085130507043184471273380659243275938904335757337482424
>>> (x ** 3 + 7 - y**2) % p
0
----
====
In elliptic curve math, there is a point called the "point at infinity", which roughly corresponds to the role of 0 in addition. On computers, it's sometimes represented by x = y = 0 (which doesn't satisfy the elliptic curve equation -- but it's an easy separate case that can be checked).
There is also an operator "+", called "addition" which has some properties similar to the traditional addition of real numbers that grade school children learn. Given two points P~1~ and P~2~ on the elliptic curve, there is a third point P~3~ = P~1~ + P~2~, also on the elliptic curve.
Geometrically, this third point P~3~ is calculated by drawing a line between P~1~ and P~2~. This line will intersect the elliptic curve in exactly one additional place. Call this point P~3~' = (x, y). Then reflect in the X axis to get P~3~ = (x, -y).
There are a couple of special cases which explain the need for the "point at infinity".
If P~1~ and P~2~ are the same point, the line "between" P~1~ and P~2~ should extend to be the tangent on the curve at this point P~1~. This tangent will intersect the curve in exactly one new point. You can use techniques from calculus to determine the slope of the tangent line. These techniques curiously work even though we are restricting our interest to points on the curve with two integer coordinates!
In some cases (i.e., if P~1~ and P~2~ have the same x values but different y values), the tangent line will be exactly vertical, in which case P3 = "point at infinity".
If P~1~ is the "point at infinity", then the sum P~1~ + P~2~ = P~2~. Similary, if P~2~ is the point at infinity, then P~1~ + P~2~ = P~1~. This shows how the point at infinity plays the roll of 0.
It turns out that + is associative, which means that `(A+B)+C = A+(B+C)`. That means we can write A+B+C without parentheses without any ambiguity.
Now that we have defined addition, we can define multiplication in the standard way that extends addition. For a point P on the elliptic curve, if k is a whole number, then kP = P + P + P + ... + P (k times). Note that k is sometimes confusingly called an "exponent" in this case.
[[public_key_derivation]]
==== Generating a public key
@ -192,7 +186,12 @@ Starting with a private key in the form of a randomly generated number +k+, we m
\end{equation}
++++
where +k+ is the private key, +G+ is a fixed point on the curve called the _generator point_, ((("generator point"))) and +K+ is the resulting public key, another point on the curve. Since the generator point is always the same, a private key k multiplied with G will always produce the same public key K.
where +k+ is the private key, +G+ is the generator point, ((("generator point"))) and +K+ is the resulting public key, a point on the curve. Since the generator point is always the same for all bitcoin users, a private key k multiplied with G will always result in the same public key K. The relationship between k and K is fixed, but can only be calculated in one direction, from k to K. That's why a bitcoin address (derived from K) can be shared with anyone and does not reveal the user's private key (k).
[TIP]
====
A private key can be converted into a public key, but a public key cannot be converted back into a private key because the math only works one way.
====
Implementing the elliptic curve multiplication above, we take the private key generated previously and multiply it by G:
@ -212,9 +211,7 @@ y = 07CF33DA18...505BDB
To visualize multiplication of a point with an integer, we will use the simpler elliptic curve over the real numbers -- remember, the math is the same. Our goal is to find the multiple kG of the generator point G. That is the same as adding G to itself, k times in a row. In elliptic curves, adding a point to itself is the equivalent of drawing a tangent line on the point and finding where it intersects the curve again, then reflecting that point on the x-axis.
Starting with the generator point G, we take the tangent of the curve at G until it crosses the curve again at another point. This new point is -2G. Reflecting that point across the x-axis gives us 2G. If we take the tangent at 2G, it crosses the curve at -4G, which again we reflect on the x-axis to find G. Continuing this process, we can bounce around the curve finding the multiples of G, 2G, 4G, 8G, etc. As you can see, a randomly selected large number k, when multiplied against the generator point G is like bouncing around the curve k times, until we land on the point kG which is the public key. This process is irreversible, meaning that it is infeasible to find the factor k (the secret k) in any way other than trying all multiples of G (1G, 2G, 4G, etc) in a brute-force search for k. Since k can be an enormous number, that brute-force search would take an infeasible amount of computation and time.
The diagram below shows the process for deriving G, 2G, 4G, as a geometric operation on the curve.
[[ecc_illustrated]]
.Elliptic Curve Cryptography: Visualizing the multiplication of a point G by an integer k on an elliptic curve
@ -222,14 +219,21 @@ image::images/ecc_illustrated.png["ecc_illustrated"]
[TIP]
====
A private key can be converted into a public key, but a public key cannot be converted back into a private key because the math only works one way.
Most bitcoin implementations use the OpenSSL cryptographic library to do the elliptic curve math. For example, to derive the public key, the function +EC_POINT_mul()+ is used. See http://wiki.openssl.org/index.php/Elliptic_Curve_Cryptography
====
=== Bitcoin Addresses
An address is a string of digits and characters that can be shared with anyone who wants to send you money. In bitcoin, addresses produced from public keys begin with the digit "1". The bitcoin address is what appears most commonly in a transaction as the "recipient" of the funds. If we were to compare a bitcoin transaction to a paper cheque, the bitcoin address is the beneficiary, which is what we write on the line after "Pay to the order of". On a paper cheque, that beneficiary can sometimes be the name of a bank account holder, but can also include corporations, institutions or even cash. Because paper cheques do not need to specify an account, but rather use an abstract name as the recipient of funds, that makes paper cheques very flexible as payment instruments. Bitcoin transactions use a similar abstraction, the bitcoin address, to make them very flexible. A bitcoin address can represent the owner of a private/public key pair, or it can represent something else, such as a payment script, as we will see in <<p2sh>>. For now, let's examine the simple case, a bitcoin address that represents, and is derived from, a public key.
A bitcoin address derived from a public key is a string of numbers and letters that begins with the number one, such as +1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy+. The bitcoin address is derived from the public key through the use of one-way cryptographic hashing; a "hashing algorithm" or simply "hash algorithm" is a one-way function that produces a fingerprint or "hash" of an arbitrary sized input. Cryptographic hash functions are used extensively in bitcoin: in bitcoin addresses, in script addresses and in the mining "Proof-of-Work" algorithm. The algorithms used to make a bitcoin address from a public key are the Secure Hash Algorithm (SHA) and the RACE Integrity Primitives Evaluation Message Digest (RIPEMD), specifically SHA256 and RIPEMD160.
A bitcoin address is a string of digits and characters that can be shared with anyone who wants to send you money. Addresses produced from public keys consist of a string of numbers and letters, beginning with the digit "1". Here's an example of a bitcoin address:
----
1thMirt546nngXqyPEz532S8fLwbozud8
----
The bitcoin address is what appears most commonly in a transaction as the "recipient" of the funds. If we were to compare a bitcoin transaction to a paper cheque, the bitcoin address is the beneficiary, which is what we write on the line after "Pay to the order of". On a paper cheque, that beneficiary can sometimes be the name of a bank account holder, but can also include corporations, institutions or even cash. Because paper cheques do not need to specify an account, but rather use an abstract name as the recipient of funds, that makes paper cheques very flexible as payment instruments. Bitcoin transactions use a similar abstraction, the bitcoin address, to make them very flexible. A bitcoin address can represent the owner of a private/public key pair, or it can represent something else, such as a payment script, as we will see in <<p2sh>>. For now, let's examine the simple case, a bitcoin address that represents, and is derived from, a public key.
The bitcoin address is derived from the public key through the use of one-way cryptographic hashing; a "hashing algorithm" or simply "hash algorithm" is a one-way function that produces a fingerprint or "hash" of an arbitrary sized input. Cryptographic hash functions are used extensively in bitcoin: in bitcoin addresses, in script addresses and in the mining "Proof-of-Work" algorithm. The algorithms used to make a bitcoin address from a public key are the Secure Hash Algorithm (SHA) and the RACE Integrity Primitives Evaluation Message Digest (RIPEMD), specifically SHA256 and RIPEMD160.
Starting with the public key K, we compute the SHA256 hash and then compute the RIPEMD160 hash of the result, producing a 160 bit (20 byte) number:
[latexmath]
@ -240,6 +244,12 @@ Starting with the public key K, we compute the SHA256 hash and then compute the
++++
where K is the public key and A is the resulting bitcoin address.
[TIP]
====
A bitcoin address is *not* the same as a public key. Bitcoin addresses are derived from a public key using a one-way function.
====
Bitcoin addresses are almost always presented to users in an encoding called "Base58Check" (see <<base58check>> below), which uses 58 characters (a base-58 number system) and a checksum to help human readability, avoid ambiguity and protect against errors in address transcription and entry. Base58Check is also used in many other ways in bitcoin, whenever there is a need for a user to read and correctly transcribe a number, such as a bitcoin address, a private key, an encrypted key, or a script hash. In the next section we will examine the mechanics of Base58Check encoding and decoding, and the resulting representations.
[[pubkey_to_adddress]]
@ -266,9 +276,15 @@ To add extra security against typos or transcription errors, Base58Check is a Ba
To convert data (a number) into a Base58Check format, we first add a prefix to the data, called the "version byte", which serves to easily identify the type of data that is encoded. For example, in the case of a bitcoin address the prefix is zero (0x00 in hex), whereas the prefix used when encoding a private key is 128 (0x80 in hex). A list of common version prefixes is shown below in <<base58check_versions>>.
Next compute the "double-SHA" checksum, meaning we apply the SHA256 hash-algorithm twice on the previous result (prefix and data): +checksum = SHA256(SHA256(prefix\+data))+ From the resulting 32-byte hash (hash-of-a-hash), we take only the first four bytes. These four bytes serve as the error-checking code, or checksum. The checksum is concatenated (appended) to the end.
Next compute the "double-SHA" checksum, meaning we apply the SHA256 hash-algorithm twice on the previous result (prefix and data):
----
checksum = SHA256(SHA256(prefix+data))
----
The result of the above is now a prefix, the data and a checksum, concatenated (bytewise). This result is encoded using the base-58 alphabet described in the section above.
From the resulting 32-byte hash (hash-of-a-hash), we take only the first four bytes. These four bytes serve as the error-checking code, or checksum. The checksum is concatenated (appended) to the end.
The result of the above is now a prefix, the data and a checksum. This result is encoded using the base-58 alphabet described in the section above.
[[base58check_encoding]]
.Base58Check Encoding: A base-58, versioned and checksummed format for unambiguously encoding bitcoin data
@ -289,6 +305,35 @@ In bitcoin, most of the data presented to the user is Base58Check encoded to mak
| BIP32 Extended Public Key | 0x0488B21E | xpub
|=======
===== Creating a bitcoin address from a private key
Let's look at the complete process of creating a bitcoin address, from a private key, to a public key (a point on the elliptic curve), to a double-hashed address and finally the Base58Check encoding. The C++ code in <<addr_example>> shows the complete process, from private key, to Base58Check encoded bitcoin address, step-by-step. The code example uses the libbitcoin library introduced in <<alt_libraries>> for some helper functions:
[[addr_example]]
.Creating a Base58Check encoded bitcoin address from a private key
====
[source, cpp]
----
include::code/addr.cpp[]
----
====
The code uses a pre-defined private key, so that it produces the same bitcoin address every time it is run:
[[addr_example_run]]
.Compiling and running the addr code
====
[source,bash]
----
# Compile the addr.cpp code
$ g++ -o addr addr.cpp $(pkg-config --cflags --libs libbitcoin)
# Run the addr executable
$ ./addr
Public key: 0202a406624211f2abbdc68da3df929f938c3399dd79fac1b51b0e4ad1d26a47aa
Address: 1PRTTaJesdNovgne6Ehcdu1fpEdX7913CK
----
====
==== Key Formats
[[priv_formats]]
@ -318,10 +363,11 @@ The private key we generated earlier can be represented as:
All of the above representations are different ways of showing the same number, the same private key. They look different, but any one format can easily be converted to any other format.
===== Decode from Base58Check to Hex
The sx-tools package (See <<sx_tools>>) makes Base58Check format decoding easy on the command line. We use the base58check-decode command:
The sx-tools package (See <<sx_tools>>) makes it easy to write shell-scripts and command-line "pipes" that manipulate bitcoin keys, addresses, and transactions. You can use sx-tools to decode the Base58Check format on the command line:
We use the base58check-decode command:
----
$ sx base58check-decode 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd 128
@ -331,7 +377,7 @@ The result is the hexadecimal key, followed by the Wallet Import Format (WIF) ve
===== Encode from Hex to Base58Check
To encode into Base58Check, we provide the hex private key, followed by the Wallet Import Format (WIF) version prefix 128:
To encode into Base58Check (the opposite of the previous command), we provide the hex private key, followed by the Wallet Import Format (WIF) version prefix 128:
----
$ sx base58check-encode 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd 128
5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn
@ -425,28 +471,80 @@ Here's the same key, encoded in WIF and WIF-compressed formats
The most comprehensive bitcoin library in Python is "pybitcointools"by Vitalik Buterin (https://github.com/vbuterin/pybitcointools). In the following code example, we use the pybitcointools library (imported as "bitcoin") to generate and display keys and addresses in various formats:
[[key-to-address_script]]
.Key and Address generation and formatting with the pybitcointools library
====
[source,python]
----
include::code/key-to-address-ecc-example.py[]
----
====
Here's the output from running this code:
[[key-to-address_script_run]]
.Running key-to-address-ecc-example.py
====
[source,bash]
----
$ python key-to-address-ecc-example.py
Private Key (hex) is: 3aba4162c7251c891207b747840551a71939b0de081f85c4e44cf7c13e41daa6
Private Key (decimal) is: 26563230048437957592232553826663696440606756685920117476832299673293013768870
Private Key (WIF) is: 5JG9hT3beGTJuUAmCQEmNaxAuMacCTfXuw1R3FCXig23RQHMr4K
Private Key Compressed (hex) is: 3aba4162c7251c891207b747840551a71939b0de081f85c4e44cf7c13e41daa601
Private Key (WIF-Compressed) is: KyBsPXxTuVD82av65KZkrGrWi5qLMah5SdNq6uftawDbgKa2wv6S
Public Key (x,y) coordinates is: (41637322786646325214887832269588396900663353932545912953362782457239403430124L, 16388935128781238405526710466724741593761085120864331449066658622400339362166L)
Public Key (hex) is: 045c0de3b9c8ab18dd04e3511243ec2952002dbfadc864b9628910169d9b9b00ec243bcefdd4347074d44bd7356d6a53c495737dd96295e2a9374bf5f02ebfc176
Compressed Public Key (hex) is: 025c0de3b9c8ab18dd04e3511243ec2952002dbfadc864b9628910169d9b9b00ec
Bitcoin Address (b58check) is: 1thMirt546nngXqyPEz532S8fLwbozud8
Compressed Bitcoin Address (b58check) is: 14cxpo3MBCYYWCgF74SWTdcmxipnGUsPw3
Private Key (hex) is:
3aba4162c7251c891207b747840551a71939b0de081f85c4e44cf7c13e41daa6
Private Key (decimal) is:
26563230048437957592232553826663696440606756685920117476832299673293013768870
Private Key (WIF) is:
5JG9hT3beGTJuUAmCQEmNaxAuMacCTfXuw1R3FCXig23RQHMr4K
Private Key Compressed (hex) is:
3aba4162c7251c891207b747840551a71939b0de081f85c4e44cf7c13e41daa601
Private Key (WIF-Compressed) is:
KyBsPXxTuVD82av65KZkrGrWi5qLMah5SdNq6uftawDbgKa2wv6S
Public Key (x,y) coordinates is:
(41637322786646325214887832269588396900663353932545912953362782457239403430124L,
16388935128781238405526710466724741593761085120864331449066658622400339362166L)
Public Key (hex) is:
045c0de3b9c8ab18dd04e3511243ec2952002dbfadc864b9628910169d9b9b00ec
243bcefdd4347074d44bd7356d6a53c495737dd96295e2a9374bf5f02ebfc176
Compressed Public Key (hex) is:
025c0de3b9c8ab18dd04e3511243ec2952002dbfadc864b9628910169d9b9b00ec
Bitcoin Address (b58check) is:
1thMirt546nngXqyPEz532S8fLwbozud8
Compressed Bitcoin Address (b58check) is:
14cxpo3MBCYYWCgF74SWTdcmxipnGUsPw3
----
====
Here's another example, using the Python ECDSA library for the Elliptic Curve math and without using any specialized bitcoin libraries:
[[ec_math]]
.A script demonstrating Elliptic Curve math used for bitcoin keys
====
[source, python]
----
include::code/ec-math.py[]
----
====
Running the script:
[[ec_math_run]]
.Installing the Python ECDSA library and running the ec_math.py script
====
[source,bash]
----
$ # Install Python PIP package manager
$ sudo apt-get install python-pip
$ # Install the Python ECDSA library
$ sudo pip install ecdsa
$ # Run the script
$ python ec-math.py
Secret: 38090835015954358862481132628887443905906204995912378278060168703580660294000
EC point: (70048853531867179489857750497606966272382583471322935454624595540007269312627,\
105262206478686743191060800263479589329920209527285803935736021686045542353380)
BTC public key: 029ade3effb0a67d5c8609850d797366af428f4a0d5194cb221d807770a1522873
----
====
=== Wallets
Wallets are containers for private keys, usually implemented as structured files or simple databases.
@ -457,9 +555,10 @@ Another method for making keys is _deterministic key generation_. Here you deriv
Wallets contain keys, not coins. The coins are stored on the blockchain in the form of transaction-outputs (often noted as _vout_ or _txout_). Each user has a wallet containing keys. Wallets are really keychains containing pairs of private/public keys (See <<public key>>). Users sign transactions with the keys, thereby proving they own the transaction outputs (their coins).
====
[[random_wallet]]
==== Non-Deterministic (Random) Wallets
In the first implementations of bitcoin clients, wallets were simply collections of randomly generated private keys. For example, the Bitcoin Core Client pre-generates 100 random private keys when first started and generates more keys as needed, trying to use each key only once. This type of wallet is nicknamed "Just a Bunch Of Keys", or JBOK, and such wallets are being replaced with deterministic wallets because they are cumbersome to manage, backup and import. The disadvantage of random keys is that if you generate many of them you must keep copies of all of them, meaning that the wallet must be backed-up frequently. Each key must be backed-up, or the funds it controls are irrevocably lost if the wallet becomes inaccessible. This conflicts directly with the principle of avoiding address re-use, by using each bitcoin address for only one transaction. Address re-use reduces privacy by associating multiple transactions and addresses with each other. A Type-0 wallet is a poor choice of wallet, especially if you want to avoid address re-use as that means managing many keys, which creates the need for very frequent backups. The Bitcoin Core Client includes a wallet that is implemented as a Type-0 wallet, but the use of this wallet is actively discouraged by the Bitcoin Core developers.
In the first implementations of bitcoin clients, wallets were simply collections of randomly generated private keys. This type of wallet is called a _Type-0 Non-Deterministic Wallets_. For example, the Bitcoin Core Client pre-generates 100 random private keys when first started and generates more keys as needed, using each key only once. This type of wallet is nicknamed "Just a Bunch Of Keys", or JBOK, and such wallets are being replaced with deterministic wallets because they are cumbersome to manage, backup and import. The disadvantage of random keys is that if you generate many of them you must keep copies of all of them, meaning that the wallet must be backed-up frequently. Each key must be backed-up, or the funds it controls are irrevocably lost if the wallet becomes inaccessible. This conflicts directly with the principle of avoiding address re-use, by using each bitcoin address for only one transaction. Address re-use reduces privacy by associating multiple transactions and addresses with each other. A Type-0 non-deterministic wallet is a poor choice of wallet, especially if you want to avoid address re-use as that means managing many keys, which creates the need for frequent backups. While the Bitcoin Core Client includes a wallet that is implemented as a Type-0 wallet, the use of this wallet is actively discouraged by developers of the Bitcoin Core.
[[Type0_wallet]]
.Type-0 Non-Deterministic (Random) Wallet: A Collection of Randomly Generated Keys
@ -469,13 +568,14 @@ image::images/non-deterministic_wallet.png["non-deterministic wallet"]
Deterministic, or "seeded" wallets are wallets that contain private keys which are all derived from a common seed, through the use of a one-way hash function. The seed is a randomly generated number which is combined with other data, such as an index number or "chain code" (see <<hd_wallets>>) to derive the private keys. In a deterministic wallet, the seed is sufficient to recover all the derived keys and therefore a single backup at creation time is sufficient. The seed is also sufficient for a wallet export or import, allowing for easy migration of all the user's keys between different wallet implementations.
==== Mnemonic Code Words (BIP0039)
[[mnemonic_code_words]]
==== Mnemonic Code Words
Mnemonic codes are English word sequences that are generated from a random sequence and used to produce a seed for use in deterministic wallets. The sequence of words is sufficient to re-create the seed and from there re-create the wallet and all the derived keys. A wallet application that implements deterministic wallets with mnemonic code will show the user a sequence of 12-24 words when first creating a wallet. That sequence of words is the wallet backup and can be used to recover and re-create all the keys in the same or any compatible wallet application.
Mnemonic codes are English word sequences that represent (encode) a random number used as a seed to derive a deterministic wallet. The sequence of words is sufficient to re-create the seed and from there re-create the wallet and all the derived keys. A wallet application that implements deterministic wallets with mnemonic code will show the user a sequence of 12-24 words when first creating a wallet. That sequence of words is the wallet backup and can be used to recover and re-create all the keys in the same or any compatible wallet application. Mnemonic code words make it easier for users to back up wallets, as they are easy to read and correctly transcribe, as compared to a random sequence of numbers.
The common standard for mnemonic codes is defined in Bitcoin Improvement Proposal 39 (see <<bip0039>>), currently in Draft status.
Mnemonic codes are defined in Bitcoin Improvement Proposal 39 (see <<bip0039>>), currently in Draft status. Note that BIP0039 is a draft proposal and not a standard. Specifically, there is a different standard, with a different set of words used by the Electrum wallet and _predating_ BIP0039. BIP0039 is used by the Trezor wallet and a few other wallets but is incompatible with Electrum's implementation.
The standard defines the creation of a mnemonic code and seed as a follows:
BIP0039 defines the creation of a mnemonic code and seed as a follows:
1. Create a random sequence (entropy) of 128 to 256 bits
2. Create a checksum of the random sequence by taking the first few bits of its SHA256 hash
@ -502,7 +602,8 @@ Here are some examples of mnemonic codes and the seeds they produce:
|=======
| entropy input (128 bits) | 0c1e24e5917779d297e14d45f14e1a1a
| mnemonic (12 words) | army van defense carry jealous true garbage claim echo media make crunch
| seed (512 bits) | 3338a6d2ee71c7f28eb5b882159634cd46a898463e9d2d0980f8e80dfbba5b0fa0291e5fb888a599b44b93187be6ee3ab5fd3ead7dd646341b2cdb8d08d13bf7
| seed (512 bits) | 3338a6d2ee71c7f28eb5b882159634cd46a898463e9d2d0980f8e80dfbba5b0fa0291e5fb88\
8a599b44b93187be6ee3ab5fd3ead7dd646341b2cdb8d08d13bf7
|=======
@ -511,23 +612,205 @@ Here are some examples of mnemonic codes and the seeds they produce:
| entropy input (256 bits) | 2041546864449caff939d32d574753fe684d3c947c3346713dd8423e74abcf8c
| mnemonic (24 words) | cake apple borrow silk endorse fitness top denial coil riot stay wolf
luggage oxygen faint major edit measure invite love trap field dilemma oblige
| seed (512 bits) | 3972e432e99040f75ebe13a660110c3e29d131a2c808c7ee5f1631d0a977
fcf473bee22fce540af281bf7cdeade0dd2c1c795bd02f1e4049e205a0158906c343
| seed (512 bits) | 3972e432e99040f75ebe13a660110c3e29d131a2c808c7ee5f1631d0a977fcf473bee22\
fce540af281bf7cdeade0dd2c1c795bd02f1e4049e205a0158906c343
|=======
==== Deterministic Chains (Electrum Key Chains)
[[Type1_wallet]]
.Type-1 Deterministic Wallet: A Chain of Keys Generated from a Seed
image::images/chained_wallet.png["chained wallet"]
[[hd_wallets]]
==== Deterministic Trees (BIP0032)
==== Hierarchical Deterministic Wallets (BIP0032/BIP0044)
Deterministic wallets were developed to make it easy to derive many keys from a single "seed". The most advanced form of deterministic wallets is the _Hierarchical Deterministic Wallet_ or _HD Wallet_ defined by the BIP0032 standard. Hierarchical deterministic wallets contain keys derived in a tree structure, such that a parent key can derive a sequence of children keys, each of which can derive a sequence of grandchildren keys and so on to an infinite depth. This tree structure is illustrated below:
[[Type2_wallet]]
.Type-2 Hierarchical Deterministic Wallet: A Tree of Keys Generated from a Seed
image::images/HD_wallet.png["HD wallet"]
[TIP]
====
If you are implementing a bitcoin wallet, it should be built as an HD wallet following the BIP0032 and BIP0044 standards.
====
HD wallets offer two major advantages over random (non-deterministic) keys. First, the tree structure can be used to express additional organizational meaning, such as when a specific branch of sub-keys is used to receive incoming payments and a different branch is used to receive change from outgoing payments. Branches of keys can also be used in a corporate setting, allocating different branches to departments, subsidiaries, specific functions or accounting categories.
The second advantage of HD wallets is that users can create a sequence of public keys without having access to the corresponding private keys. This allows HD wallets to be used on an insecure server or in a receive-only capacity, issuing a different public key for each transaction. The public keys do not need to be pre-loaded or derived in advance, yet the server doesn't have the private keys that can spend the funds.
===== HD wallet creation from a seed
HD wallets are created from a single _root seed_, which is a 128, 256 or 512 bit random number. Everything else in the HD wallet is deterministically derived from this root seed, which makes it possible to re-create the entire HD wallet from that seed in any compatible HD wallet. This makes it easy to backup, restore, export and import HD wallets containing thousands or even millions of keys by simply transferring only the root seed. The root seed is most often represented by a _mnemonic word sequence_, as described in the previous section <<mnemonic_code_words>>, to make it easier for people to transcribe and store it.
The process of creating the master keys and master chain code for an HD wallet is shown below:
[[HDWalletFromSeed]]
.Creating master keys and chain code from a root seed
image::images/HDWalletFromRootSeed.png["HDWalletFromRootSeed"]
The root seed is input into the HMAC-SHA512 algorithm and the resulting hash is used to create a _master private key_ (m) and a _master chain code_. The master private key (m) then generates a corresponding master public key (M), using the normal elliptic curve multiplication process +m * G+ that we saw previously in this chapter. The chain code is used to introduce entropy in the function that creates child keys from parent keys, as we will see in the next section.
===== Private child key derivation
Hierarchical Deterministic wallets use a _child key derivation_ (CKD) function to derive children keys from parent keys.
The child key derivation functions are based on one-way hash functions that combines:
* A parent private or public key (ECDSA uncompressed key)
* A seed called a chain code (256 bits)
* An index number (32 bits)
The chain code is used to introduce seemingly random data to the process, so that the index is not sufficient to derive other child keys. Thus, having a child key does not make it possible to find its siblings, unless you also have the chain code. The initial chain code seed (at the root of the tree) is made from random data, while subsequent chain codes are derived from each parent chain code.
These three items are combined and hashed to generate children keys, as follows:
The parent public key, chain code and the index number are combined and hashed with the HMAC-SHA512 algorithm to produce a 512 bit hash. The resulting hash is split into two halfs. The right-half 256 bits of the hash output become the chain code for the child. The left-half 256 bits of the hash and the index number are added to the parent private key to produce the child private key. In the diagram below, we see this illustrated with the index set to 0 to produce the 0'th (first by index) child of the parent.
[[CKDpriv]]
.Extending a parent private key to create a child private key
image::images/ChildPrivateDerivation.png["ChildPrivateDerivation"]
Changing the index allows us to extend the parent and create the other children in the sequence, e.g. Child 0, Child 1, Child 2 etc. Each parent key can have 2 billion children keys.
Repeating the process one level down the tree, each child can in turn become a parent and create its own children, in an infinite number of generations.
===== Using derived child keys
Child private keys are indistinguishable from non-deterministic (random) keys. Because the derivation function is a one way function, the child key cannot be used to find the parent key. The child key can also not be used to find any siblings. If you have the n~th~ child, you cannot find its siblings, such as the n-1 child or the n+1 child, or any other children that are part of the sequence. Only the parent key and chain code can derive all the children. Without the child chain code, the child key cannot be used to derive any grandchildren either. You need both the child private key and the child chain code to start a new branch and derive grandchildren.
So what can the child private key be used for on its own? It can be used to make a public key and a bitcoin address. Then, it can be used to sign transactions to spend anything paid to that address.
[TIP]
====
A child private key, the corresponding public key and the bitcoin address are all indistinguishable from keys and addresses created randomly. The fact that they are part of a sequence is not visible, outside of the HD wallet function that created them. Once created, they operate exactly as "normal" keys.
====
===== Extended keys
As we saw above, the key derivation function can be used to create children at any level of the tree, based on the three inputs: a key, a chain code and the index of the desired child. The two essential ingredients are the key and chain code and combined these are called an _extended key_. The term "extended key" could also be thought of as "extensible key" because such a key can be used to derive children.
Extended keys are stored and represented simply as the concatenation of the 256 bit key and 256 bit chain code into a 512 bit sequence. There are two types of extended keys: An extended private key is the combination of a private key and chain code and can be used to derive child private keys (and from them, child public keys). An extended public key is a public key and chain code, which can be used to create child public keys, as described in <<public_key_derivation>>.
Think of an extended key as the root of a branch in the tree structure of the HD wallet. With the root of the branch, you can derive the rest of the branch. The extended private key can create a complete branch, whereas the extended public key can only create a branch of public keys.
[TIP]
====
An extended key consists of a private or public key and chain code. An extended key can create children generating its own branch in the tree structure. Sharing an extended key gives access to the entire branch.
====
Extended keys are encoded using Base58Check, to easily export and import between different BIP0032 compatible wallets. The Base58Check coding for extended keys uses a special version number that results in the prefix "xprv" and "xpub" when encoded in base 58 characters, to make them easily recognizable. Since the extended key is 512 or 513 bits, it is also much longer than other Base58Check encoded strings we have seen previously.
Here's an example of an extended private key, encoded in Base58Check:
----
xprv9tyUQV64JT5qs3RSTJkXCWKMyUgoQp7F3hA1xzG6ZGu6u6Q9VMNjGr67Lctvy5P8oyaYAL9CAWrUE9i6GoNMKUga5biW6Hx4tws2six3b9c
----
Here's the corresponding extended public key, also encoded in Base58Check:
----
xpub67xpozcx8pe95XVuZLHXZeG6XWXHpGq6Qv5cmNfi7cS5mtjJ2tgypeQbBs2UAR6KECeeMVKZBPLrtJunSDMstweyLXhRgPxdp14sk9tJPW9
----
[[public_key_derivation]]
===== Public child key derivation
As mentioned above, a very useful characteristic of hierarchical deterministic wallets is the ability to derive public child keys from public parent keys, _without_ having the private keys. This gives us two ways to derive a child public key: either from the child private key, or directly from the parent public key.
An extended public key can be used, therefore, to derive all of the _public_ keys (and only the public keys) in that branch of the HD wallet structure.
This shortcut can be used to create very secure public-key-only deployments where a server or application has a copy of an extended public key and no private keys whatsoever. That kind of deployment can produce an infinite number of public keys and bitcoin addresses but cannot spend any of the money sent to those addresses. Meanwhile, on another more secure server, the extended private key can derive all the corresponding private keys to sign transactions and spend the money.
One common application of this solution is to install an extended public key on a web server that serves an e-commerce application. The web server can use the public key derivation function to create a new bitcoin address for every transaction (e.g. for a customer shopping cart). The web server will not have any private keys that would be vulnerable to theft. Without HD wallets, the only way to do this is to generate thousands of bitcoin addresses on a separate secure server and then pre-load them on the e-commerce server. That approach is cumbersome and requires constant maintenance to ensure that the e-commerce server doesn't "run out" of keys.
Another common application of this solution is for cold-storage or hardware wallets. In that scenario, the extended private key can be stored on a paper wallet or hardware device (such as a Trezor hardware wallet), while the extended public key can be kept online. The user can create "receive" addresses at will, while the private keys are safely stored offline. To spend the funds, they can use the extended private key on an offline signing bitcoin client or sign transactions on the hardware wallet device (e.g. Trezor).
[[CKDpub]]
.Extending a parent public key to create a child public key
image::images/ChildPublicDerivation.png["ChildPublicDerivation"]
===== Hardened child key derivation
The ability to derive a branch of public keys from an extended public key is very useful, but it comes with a potential risk. Access to an extended public key does not give access to child private keys. However, because the extended public key contains the chain code, if a child private key is known, or somehow leaked, it can be used with the chain code to derive all the other child private keys. A single leaked child private key, together with a parent chain code, reveals all the private keys of all the children. Worse, the child private key together with a parent chain code can be used to deduce the parent private key.
To counter this risk, HD wallets use an alternative derivation function called _hardened derivation_, which "breaks" the relationship between parent public key and child chain code. The hardened derivation function uses the parent private key to derive the child chain code, instead of the parent public key. This creates a "firewall" in the parent/child sequence, with a chain code that cannot be used to compromise a parent or sibling private key. The hardened derivation function looks almost identical to the normal child private key derivation, except that the parent private key is used as input to the hash function, instead of the parent public key, as shown in the diagram below:
[[CKDprime]]
.Hardened derivation of a child key, omits the parent public key
image::images/ChildHardPrivateDerivation.png["ChildHardPrivateDerivation"]
When the hardened private derivation function is used, the resulting child private key and chain code are completely different from what would result from the normal derivation function. The resulting "branch" of keys can be used to produce extended public keys which are not vulnerable, since the chain code they contain cannot be exploited to reveal any private keys. Hardened derivation is therefore used to create a "gap" in the tree above the level where extended public keys are used.
In simple terms, if you want to use the convenience of an extended public key to derive branches of public keys, without exposing yourself to the risk of a leaked chain code, you should derive it from a hardened parent, rather than a normal parent. As a best practice, the level-1 children of the master keys are always derived through the hardened derivation, to prevent compromise of the master keys.
===== Index numbers for normal and hardened derivation
The index number used in the derivation function is a 32-bit integer. To easily distinguish between keys derived through the normal derivation function versus keys derived through hardened derivation, this index number is split into two ranges. Index numbers between 0 and 2^31^-1 (0x0 to 0x7FFFFFFF) are used _only_ for normal derivation. Index numbers between 2^31^ and 2^32^-1 (0x80000000 to 0xFFFFFFFF) are used _only_ for hardened derivation. Therefore, if the index number is less than 2^31^, that means the child is normal, whereas if the index number is equal or above 2^31^, the child is hardened.
To make the index number easier to read and display, the index number for hardened children is displayed starting from zero, but with a prime symbol. The first normal child key is therefore displayed as 0, whereas the first hardened child (index 0x80000000) is displayed as 0'. In sequence then, the second hardened key would have index 0x80000001 and would be displayed as 1', and so on. When you see an HD wallet index i', that means 2^31^+i.
===== HD wallet key identifier (path)
Keys in an HD wallet are identified using a "path" naming convention, with each level of the tree separated by a slash (/) character. Private keys derived from the master private key start with "m". Public keys derived from the master public key start with "M". Therefore, the first child private key of the master private key is m/0. The first child public key is M/0. The second grandchild of the first child is m/0/1 and so on.
The "ancestry" of a key is read from right to left, until you reach the master key from which it was derived. For example, identifier m/x/y/z describes the key which is the z-th child of key m/x/y, which is the y-th child of key m/x, which is the x-th child of m.
.HD wallet path examples
|=======
| m/0 | The first (0) child private key from the master private key (m)
| m/0/0 | The first grandchild private key of the first child (m/0)
| m/0'/0 | The first normal grandchild of the first _hardened_ child (m/0')
| m/1/0 | The first grandchild private key of the second child (m/1)
| M/23/17/0/0 | The first great-great-grandchild public key of the first great-grandchild of the 18th grandchild of the 24th child
|=======
===== Navigating the HD wallet tree structure
The HD wallet tree structure offers tremendous flexibility. Each parent extended key can have 4 billion children: 2 billion normal children and 2 billion hardened children. Each of those children can have another 4 billion children and so on. The tree can be as deep as you want, with an infinite number of generations. With all that flexibility, however, it becomes quite difficult to navigate this infinite tree. It is especially difficult to transfer HD wallets between implementations, because the possibilities for internal organization into branches and sub-branches are endless.
Two Bitcoin Improvement Proposals (BIPs) offer a solution to this complexity, by creating some proposed standards for the structure of HD wallet trees. BIP0043 proposes the use of the first hardened child index as a special identifier that signifies the "purpose" of the tree structure. Based on BIP0043, an HD wallet should use only one level-1 branch of the tree, with the index number identifying the structure and namespace of the rest of the tree by defining its purpose. For example, an HD wallet using only branch m/i'/, is intended to signify a specific purpose and that purpose is identified by index number "i".
Extending that specification, BIP0044 proposes a multi-account structure as "purpose" number +44'+ under BIP0043. All HD wallets following the BIP0044 structure are identified by the fact that they only used one branch of the tree: m/44'/.
BIP0044 specifies the structure as consisting of five pre-defined tree levels:
+m / purpose' / coin_type' / account' / change / address_index+
The first level "purpose" is always set to +44'+. The second level "coin_type" specifies the type of crypto-currency coin, allowing for multi-currency HD wallets where each currency has its own subtree under the second level. There are three currencies defined for now: Bitcoin is m/44'/0', Bitcoin Testnet is m/44'/1' and Litecoin is m/44'/2'.
The third level of the tree is "account", which allows users to subdivide their wallets into separate logical sub-accounts, for accounting or organizational purposes. For example, an HD wallet might contain two bitcoin "accounts": m/44'/0'/0' and m/44'/0'/1'. Each account is the root of its own sub-tree.
On the fourth level "change", an HD wallet has two sub-trees, one for creating receiving addresses and one for creating change addresses. Note that whereas the previous levels used hardened derivation, this level uses normal derivation. This is to allow this level of the tree to export extended public keys for use in an non-secured environment. Usable addresses are derived by the HD wallet as children of the fourth level, making the fifth level of the tree the "address_index". For example, the third receiving address for bitcoin payments in the primary account would be M/44'/0'/0'/0/2. Here are a few more examples:
.BIP0044 HD wallet structure examples
|=======
| M/44'/0'/0'/0/2 | The third receiving public key for the primary bitcoin account
| M/44'/0'/3'/1/14 | The fifteenth change-address public key for the fourth bitcoin account
| m/44'/2'/0'/0/1 | The second private key in the Litecoin main account, for signing transactions
|=======
===== Experimenting with HD wallets using sx-tools
Using the command line tool +sx+, introduced in chapter 3, you can experiment with generating and extending BIP0032 deterministic keys, as well as displaying them in different formats:
====
[source, bash]
----
$ sx hd-seed > m # create a new master private key from a seed and store in file "m"
$ cat m # show the master extended private key
xprv9s21ZrQH143K38iQ9Y5p6qoB8C75TE71NfpyQPdfGvzghDt39DHPFpovvtWZaRgY5uPwV7RpEgHs7cvd\
gfiSjLjjbuGKGcjRyU7RGGSS8Xa
$ cat m | sx hd-pub 0 # generate the M/0 extended public key
xpub67xpozcx8pe95XVuZLHXZeG6XWXHpGq6Qv5cmNfi7cS5mtjJ2tgypeQbBs2UAR6KECeeMVKZBPLrtJun\
SDMstweyLXhRgPxdp14sk9tJPW9
$ cat m | sx hd-priv 0 # generate the m/0 extended private key
xprv9tyUQV64JT5qs3RSTJkXCWKMyUgoQp7F3hA1xzG6ZGu6u6Q9VMNjGr67Lctvy5P8oyaYAL9CAWrUE9i6\
GoNMKUga5biW6Hx4tws2six3b9c
$ cat m | sx hd-priv 0 | sx hd-to-wif # show the private key of m/0 as a WIF
L1pbvV86crAGoDzqmgY85xURkz3c435Z9nirMt52UbnGjYMzKBUN
$ cat m | sx hd-pub 0 | sx hd-to-address # show the bitcoin address of M/0
1CHCnCjgMNb6digimckNQ6TBVcTWBAmPHK
$ cat m | sx hd-priv 0 | sx hd-priv 12 --hard | sx hd-priv 4 # generate m/0/12'/4
xprv9yL8ndfdPVeDWJenF18oiHguRUj8jHmVrqqD97YQHeTcR3LCeh53q5PXPkLsy2kRaqgwoS6YZBLatRZR\
yUeAkRPe1kLR1P6Mn7jUrXFquUt
----
====
=== Advanced Keys and Addresses
==== Encrypted Private Keys (BIP0038)
@ -550,14 +833,14 @@ Test the following encrypted keys using bitaddress.org to see how you can get th
|=======
[[p2sh]]
[[p2sh_addresses]]
==== Pay To Script Hash (P2SH) and Multi-Sig Addresses
As we know, traditional bitcoin addresses begin with the number “1” and are derived from the public key, which is derived from the private key. While anyone can send bitcoin to a “1” address, that bitcoin can only be spent by presenting the corresponding private key signature and public key hash.
Bitcoin addresses that begin with the number “3” are pay-to-script-hash (P2SH) addresses, sometimes erroneously called multi-signature or multi-sig addresses. They designate the beneficiary of a bitcoin transaction as the hash of a script, instead of the owner of a public key. The feature was introduced in January 2012 with Bitcoin Improvement Proposal 16 or BIP0016 (see <<bip0016>>) and is being widely adopted because it provides the opportunity to add functionality to the address itself. Unlike transactions that "send" funds to traditional “1” bitcoin addresses, also known as pay-to-public-key-hash (P2PKH), funds sent to “3” addresses require something more than the presentation of one public key hash and one private key signature as proof of ownership. The requirements are designated at the time the address is created, within the script, and all inputs to this address will be encumbered with the same requirements.
A pay-to-script-hash address is created from a transaction script, which defines who can spend a transaction output (for more detail, see <<transactions>>). Encoding a pay-to-script hash address involves using the same double-hash function as used during creation of a bitcoin address, only applied on the script instead of the public key.
A pay-to-script-hash address is created from a transaction script, which defines who can spend a transaction output (for more detail, see <<p2sh>>). Encoding a pay-to-script hash address involves using the same double-hash function as used during creation of a bitcoin address, only applied on the script instead of the public key.
----
script hash = RIPEMD160(SHA256(script))
@ -619,6 +902,47 @@ As you can see, Eugenia won't be creating the vanity address "1KidsCharity" any
Another way to find a vanity address is to outsource the work to a pool of vanity-miners, such as the pool at vanitypool.appspot.com. A pool is a service that allows those with GPU hardware to earn bitcoin searching for vanity addresses for others. For a small payment (0.01 bitcoin or approximately $5 when this was written), Eugenia can outsource the search for a 7-character pattern vanity address and get results in a few hours instead of having to run a CPU search for months.
Generating a vanity address is a brute-force exercise: try a random key, check the resulting address to see if it matches the desired pattern, repeat until successful. Here's an example of a "vanity miner", a program designed to find vanity addresses, written in C++. The example uses the libbitcoin library, which we introduced in <<alt_libraries>>.
[[vanity_miner_code]]
.Vanity Address Miner
====
[source,cpp]
----
include::code/vanity-miner.cpp[]
----
====
The example code must be compiled using a C++ compiler and linked against the libbitcoin library (which must be first installed on that system). To run the example, run the +vanity-minder+ executable with no parameters and it will attempt to find a vanity address starting with "1kid":
[[vanity_miner_run]]
.Compiling and running the vanity-miner example
====
[source,bash]
----
$ # Compile the code with g++
$ g++ -o vanity-miner vanity-miner.cpp $(pkg-config --cflags --libs libbitcoin)
$ # Run the example
$ ./vanity-miner
Found vanity address! 1KiDzkG4MxmovZryZRj8tK81oQRhbZ46YT
Secret: 57cc268a05f83a23ac9d930bc8565bac4e277055f4794cbd1a39e5e71c038f3f
$ # Run it again for a different result
$ ./vanity-miner
Found vanity address! 1Kidxr3wsmMzzouwXibKfwTYs5Pau8TUFn
Secret: 7f65bbbbe6d8caae74a0c6a0d2d7b5c6663d71b60337299a1a2cf34c04b2a623
# Use "time" to see how long it takes to find a result
$ time ./vanity-miner
Found vanity address! 1KidPWhKgGRQWD5PP5TAnGfDyfWp5yceXM
Secret: 2a802e7a53d8aa237cd059377b616d2bfcfa4b0140bc85fa008f2d3d4b225349
real 0m8.868s
user 0m8.828s
sys 0m0.035s
----
====
The example code will take a few seconds to find a match for the three-character pattern "kid", as we can see when we use the +time+ Unix command to measure the execution time. Change the +search+ pattern in the source code and see how much longer it takes for four- or five-character patterns!
===== Vanity Address Security
Vanity addresses can be used to enhance _and_ to defeat security measures, they are truly a double-edged sword. Used to improve security, a distinctive address makes it harder for adversaries to substitute their own address and fool your customers into paying them instead of you. Unfortunately, vanity addresses also make it possible for anyone to create an address that _resembles_ any random address, or even another vanity address, thereby fooling your customers.
@ -638,6 +962,7 @@ In both cases, one of the risks of using a single fixed address (rather than a s
So does a vanity address increase security? If Eugenia generates the vanity address "1Kids33q44erFfpeXrmDSz7zEqG2FesZEN",
users are likely to look at the vanity pattern word _and a few characters beyond_, for example noticing the "1Kids33" part of the address. That would force an attacker to generate a vanity address matching at least 6 characters (2 more), expending an effort that is 3,364 times (58 x 58) higher than the effort Eugenia expended for her 4 character vanity. Essentially, the effort Eugenia expends (or pays a vanity pool for) "pushes" the attacker into having to produce a longer pattern vanity. If Eugenia pays a pool to generate an 8 character vanity address, the attacker would be pushed into the realm of 10 characters which is infeasible on a personal computer and expensive even with a custom vanity-mining rig or vanity pool. What is affordable for Eugenia becomes unaffordable for the attacker, especially if the potential reward of fraud is not high enough to cover the cost of the vanity address generation.
[[paper_wallets]]
==== Paper Wallets
Paper wallets are bitcoin private keys printed on paper. Often the paper wallet also includes the corresponding bitcoin address, for convenience, but this is not necessary since it can be derived from the private key. Paper wallets are a very effective way to create backups or offline bitcoin storage, also known as "cold storage". As a backup mechanism, a paper wallet can provide security against the loss of key due to a computer mishap such as a hard drive failure, theft, or accidental deletion. As a "cold storage" mechanism, if the paper wallet keys are generated offline and never stored on a computer system, they are much more secure against hackers, key-loggers and other online computer threats.

347
ch05.asciidoc
View File

@ -1,47 +1,50 @@
[[ch5]]
== Chapter 5 - Transactions
[[transactions]]
== Transactions
[[ch5_intro]]
=== Introduction
Transactions are the most important part of the bitcoin system. Everything else in bitcoin is designed to ensure that transactions can be created, propagated on the network, validated, and finally added to the global ledger of transactions, the blockchain. Transactions are data structures that encode the transfer of value between participants in the bitcoin system. Each transaction is an public entry in bitcoin's global double-entry bookkeeping ledger, the blockchain.
Transactions are the most important part of the bitcoin system. Everything else in bitcoin is designed to ensure that transactions can be created, propagated on the network, validated, and finally added to the global ledger of transactions, the blockchain. Transactions are data structures that encode the transfer of value between participants in the bitcoin system. Each transaction is a public entry in bitcoin's global double-entry bookkeeping ledger, the blockchain.
In this chapter we will examine all the various forms of transactions, what do they contain, how to create them, how they are verified, and how they become part of the permanent record of all transactions.
In this chapter we will examine all the various forms of transactions, what they contain, how to create them, how they are verified, and how they become part of the permanent record of all transactions.
[[tx_lifecycle]]
=== Transaction Lifecycle
A transaction's lifecycle starts with the transaction's creation, also known as origination. The transaction is then signed, with one or more signatures indicating the authorization to spend the funds referenced by the transaction. The transaction is then broadcast on the bitcoin network, where each network node (participant) validates and propagates the transaction until it reaches (almost) every node in the network. Finally, the transaction is verified by a mining node and included in a block of transactions that is recorded on the blockchain. Once recorded on the blockchain and confirmed by sufficient subsequent blocks (confirmations), the transaction is a permanent part of the bitcoin ledger and is accepted as valid by all participants. The funds allocated to a new owner by the transaction can then be spent in a new transaction, extending the chain of ownership and beginning the lifecycle of a transaction again.
A transaction's lifecycle starts with the transaction's creation, also known as origination. The transaction is then signed, with one or more signatures indicating the authorization to spend the funds referenced by the transaction. The transaction is then broadcast on the bitcoin network, where each network node (participant) validates and propagates the transaction until it reaches (almost) every node in the network. Finally, the transaction is verified by a mining node and included in a block of transactions that is recorded on the blockchain.
Once recorded on the blockchain and confirmed by sufficient subsequent blocks (confirmations), the transaction is a permanent part of the bitcoin ledger and is accepted as valid by all participants. The funds allocated to a new owner by the transaction can then be spent in a new transaction, extending the chain of ownership and beginning the lifecycle of a transaction again.
[[tx_origination]]
==== Creating Transactions
In some ways it helps to think of a transaction in the same way as a paper cheque. Like a cheque, a transaction is an instrument that expresses the intent to transfer money and is not visible to the financial system until it is submitted for execution. Like a cheque, the originator of the transaction does not have to be the one signing the transaction. Transactions can be created online or offline by anyone, even if the person creating the transaction is not an authorized signer on the account. For example an accounts payable clerk might process payable cheques for signature by the CEO. Similarly, an accounts payable clerk can create bitcoin transactions and then have the CEO apply digital signatures to make them valid. While a cheque references a specific account as the source of the funds, a bitcoin transaction references a specific previous transaction as its source, rather than an account.
In some ways it helps to think of a transaction in the same way as a paper cheque. Like a cheque, a transaction is an instrument that expresses the intent to transfer money and is not visible to the financial system until it is submitted for execution. Like a cheque, the originator of the transaction does not have to be the one signing the transaction.
Transactions can be created online or offline by anyone, even if the person creating the transaction is not an authorized signer on the account. For example an accounts payable clerk might process payable cheques for signature by the CEO. Similarly, an accounts payable clerk can create bitcoin transactions and then have the CEO apply digital signatures to make them valid. While a cheque references a specific account as the source of the funds, a bitcoin transaction references a specific previous transaction as its source, rather than an account.
Once a transaction has been created, it is signed by the owner (or owners) of the source funds. If it was properly formed and signed, the signed transaction is now valid and contains all the information needed to execute the transfer of funds. Finally, the valid transaction has to reach the bitcoin network so that it can be propagated until it reaches a miner for inclusion in the pubic ledger, the blockchain.
[[tx_bcast]]
==== Broadcasting Transactions to the Bitcoin Network
First, a transaction needs to be delivered to the bitcoin network so that it can be propagated and be included in the blockchain. In essence, a bitcoin transaction is just 300-400 bytes of data and has to reach any one of tens of thousands of bitcoin nodes. The sender does not need to trust the nodes they use to broadcast the transaction, as long as they use more than one to ensure that it propagates. The nodes don't need to trust the sender or establish the sender's "identity". Since the transaction is signed and contains no confidential information, private keys or credentials, it can be publicly broadcast using any underlying network transport that is convenient. Unlike credit card transactions, for example, which contain sensitive information and can only be transmitted on encrypted networks, a bitcoin transaction can be sent over any network. As long as the transaction can reach a bitcoin node that will propagate it into the bitcoin network, it doesn't matter how it is transported to the first node. Bitcoin transactions can therefore be transmitted to the bitcoin network over insecure networks such as Wifi, Bluetooth, NFC, Chirp, barcodes or by copying and pasting into a web form. In extreme cases, a bitcoin transaction could be transmitted over packet radio, satellite relay or shortwave using burst transmission, spread spectrum or frequency hopping to evade detection and jamming. A bitcoin transaction could even be encoded as smileys (emoticons) and posted in a public forum or sent as a text message or Skype chat message. Bitcoin has turned money into a data structure, making it virtually impossible to stop anyone from creating and executing a bitcoin transaction.
First, a transaction needs to be delivered to the bitcoin network so that it can be propagated and be included in the blockchain. In essence, a bitcoin transaction is just 300-400 bytes of data and has to reach any one of tens of thousands of bitcoin nodes. The sender does not need to trust the nodes they use to broadcast the transaction, as long as they use more than one to ensure that it propagates. The nodes don't need to trust the sender or establish the sender's "identity". Since the transaction is signed and contains no confidential information, private keys or credentials, it can be publicly broadcast using any underlying network transport that is convenient. Unlike credit card transactions, for example, which contain sensitive information and can only be transmitted on encrypted networks, a bitcoin transaction can be sent over any network. As long as the transaction can reach a bitcoin node that will propagate it into the bitcoin network, it doesn't matter how it is transported to the first node.
Bitcoin transactions can therefore be transmitted to the bitcoin network over insecure networks such as Wifi, Bluetooth, NFC, Chirp, barcodes or by copying and pasting into a web form. In extreme cases, a bitcoin transaction could be transmitted over packet radio, satellite relay or shortwave using burst transmission, spread spectrum or frequency hopping to evade detection and jamming. A bitcoin transaction could even be encoded as smileys (emoticons) and posted in a public forum or sent as a text message or Skype chat message. Bitcoin has turned money into a data structure, making it virtually impossible to stop anyone from creating and executing a bitcoin transaction.
[[tx_propagation]]
==== Propagating Transactions on the Bitcoin Network
Once a bitcoin transaction is sent to any node connected to the bitcoin network, the transaction will be validated by that node. If valid, that node will propagate it to the other nodes to which it is connected and a success message will be returned synchronously to the originator. If the transaction is invalid, the node will reject it and synchronously return a rejection message to the originator. The bitcoin network is a peer-to-peer network meaning that each bitcoin node is connected to a few other bitcoin nodes that it discovers during startup through the peer-to-peer protocol. The entire network forms a loosely connected mesh without a fixed topology or any structure making all nodes equal peers. Messages, including transactions and blocks, are propagated from each node to the peers to which it is connected. A new validated transaction injected into any node on the network will be sent to 3 to 4 of the neighboring nodes, each of which will send it to 3 to 4 more nodes and so on. In this way, within a few seconds a valid transaction will propagate in an exponentially expanding ripple across the network until all connected nodes have received it. The bitcoin network is designed to propagate transactions and blocks to all nodes in an efficient and resilient manner that is resistant to attacks. To prevent spamming, denial of service attacks, or other nuisance attacks against the bitcoin system, every node will independently validate every transaction before propagating it further. A malformed transaction will not get beyond one node. The rules by which transactions are validated are explained in more detail in <<tx_validation>>.
Once a bitcoin transaction is sent to any node connected to the bitcoin network, the transaction will be validated by that node. If valid, that node will propagate it to the other nodes to which it is connected and a success message will be returned synchronously to the originator. If the transaction is invalid, the node will reject it and synchronously return a rejection message to the originator.
[[tx_mining]]
==== Mining Transactions into Blocks
Some of the nodes in the bitcoin network participate in "mining". Mining is the process creating new blocks of transactions that will become part of the blockchain. Miners collect transactions and group them into blocks, they then attempt to prove each block with the proof-of-work algorithm. Blocks with a valid proof of work are added to and extend the linked chain of blocks called the blockchain. Once a transaction is added to the blockchain, the new owner of the funds can reference it in a new transaction and spend the funds.
The blockchain forms the authoritative ledger of all transactions since bitcoin's beginning in 2009. The blockchain is the subject of the next chapter, where we will examine the formation of the authoritative record through the competitive process of proof-of-work, also known as mining.
The bitcoin network is a peer-to-peer network meaning that each bitcoin node is connected to a few other bitcoin nodes that it discovers during startup through the peer-to-peer protocol. The entire network forms a loosely connected mesh without a fixed topology or any structure making all nodes equal peers. Messages, including transactions and blocks, are propagated from each node to the peers to which it is connected. A new validated transaction injected into any node on the network will be sent to 3 to 4 of the neighboring nodes, each of which will send it to 3 to 4 more nodes and so on. In this way, within a few seconds a valid transaction will propagate in an exponentially expanding ripple across the network until all connected nodes have received it.
The bitcoin network is designed to propagate transactions and blocks to all nodes in an efficient and resilient manner that is resistant to attacks. To prevent spamming, denial of service attacks, or other nuisance attacks against the bitcoin system, every node will independently validate every transaction before propagating it further. A malformed transaction will not get beyond one node. The rules by which transactions are validated are explained in more detail in <<tx_verification>>.
[[tx_structure]]
=== Transaction Structure
A transaction is a data structure that encodes a transfer of value from a source of funds, called an "input", to a destination, called an "output". Transaction inputs and outputs are not related to accounts or identities. Instead you should think of them as bitcoin amounts, chunks of bitcoin, being locked with a specific secret which only the owner, or person who knows the secret, can unlock.
A transaction is a _data structure_ that encodes a transfer of value from a source of funds, called an _input_, to a destination, called an _output_. Transaction inputs and outputs are not related to accounts or identities. Instead you should think of them as bitcoin amounts, chunks of bitcoin, being locked with a specific secret which only the owner, or person who knows the secret, can unlock.
A transaction contains a number of fields, as follows:
@ -58,7 +61,10 @@ A transaction contains a number of fields, as follows:
| 4 bytes | Locktime | A unix timestamp or block number
|=======
Note: Locktime defines the earliest time that a transaction can be added to the blockchain. It is set to zero in most transactions to indicate immediate execution. If locktime is non-zero and below 500 million, it is interpreted as a block height, meaning the transaction is not included in the blockchain prior to the specified block height. If it is above 500 million, it is interpreted as a Unix Epoch timestamp (seconds since Jan-1-1970) and the transaction is not included in the blockchain prior to the specified time.
.Transaction Locktime
****
Locktime defines the earliest time that a transaction can be added to the blockchain. It is set to zero in most transactions to indicate immediate execution. If locktime is non-zero and below 500 million, it is interpreted as a block height, meaning the transaction is not included in the blockchain prior to the specified block height. If it is above 500 million, it is interpreted as a Unix Epoch timestamp (seconds since Jan-1-1970) and the transaction is not included in the blockchain prior to the specified time. The use of locktime is equivalent to post-dating a paper cheque.
****
[[tx_inputs_outputs]]
=== Transaction Outputs and Inputs
@ -70,15 +76,18 @@ The fundamental building block of a bitcoin transaction is an _unspent transacti
There are no accounts or balances in bitcoin, there are only _unspent transaction outputs_ (UTXO) scattered in the blockchain.
====
Unlike cash, which exists in specific denominations (one dollar, five dollars, ten dollars), a UTXO can have any arbitrary value denominated as a multiple of satoshis (the smallest bitcoin unit equal to 100 millionth of a bitcoin). While UTXO can be any arbitrary value, once created it is indivisible just like a coin that cannot be cut in half. If a UTXO is larger than the desired value of a transaction, it must still be consumed in its entirety and change must be generated in the transaction. In other words, if you have a 20 bitcoin UTXO and want to pay 1 bitcoin, your transaction must consume the entire 20 bitcoin UTXO and produce two outputs: one paying 1 bitcoin to your desired recipient and another paying 19 bitcoin in change back to your wallet. As a result, most bitcoin transactions will generate change.
A UTXO can have an arbitrary value denominated as a multiple of satoshis. Just like dollars can be divided down to two decimal places as cents, bitcoins can be divided down to eight decimal places as satoshis. While UTXO can be any arbitrary value, once created it is indivisible just like a coin that cannot be cut in half. If a UTXO is larger than the desired value of a transaction, it must still be consumed in its entirety and change must be generated in the transaction. In other words, if you have a 20 bitcoin UTXO and want to pay 1 bitcoin, your transaction must consume the entire 20 bitcoin UTXO and produce two outputs: one paying 1 bitcoin to your desired recipient and another paying 19 bitcoin in change back to your wallet. As a result, most bitcoin transactions will generate change.
In simple terms, transactions consume the sender's available UTXO and create new UTXO locked to the recipient's bitcoin address. Imagine a shopper buying a $1.50 beverage, reaching into their wallet and trying to find a combination of coins and bank notes to cover the $1.50 cost. The shopper will choose exact change if available (a dollar bill and two quarters), or a combination of smaller denominations (six quarters), or if necessary, a larger unit such as a five dollar bank note. If they hand too much money, say $5, to the shop owner they will expect $3.50 change, which they will return to their wallet and have available for future transactions. Similarly, a bitcoin transaction must be created from a user's UTXO in whatever denominations that user has available. They cannot cut a UTXO in half any more than they can cut a dollar bill in half and use it as currency. The user's wallet application will typically select from the user's available UTXO various units to compose an amount greater than or equal to the desired transaction amount. As with real life, the bitcoin application can use several strategies to satisfy the purchase amount: combining several smaller units, finding exact change, or using a single unit larger than the transaction value and making change.
Imagine a shopper buying a $1.50 beverage, reaching into their wallet and trying to find a combination of coins and bank notes to cover the $1.50 cost. The shopper will choose exact change if available (a dollar bill and two quarters), or a combination of smaller denominations (six quarters), or if necessary, a larger unit such as a five dollar bank note. If they hand too much money, say $5, to the shop owner they will expect $3.50 change, which they will return to their wallet and have available for future transactions.
Similarly, a bitcoin transaction must be created from a user's UTXO in whatever denominations that user has available. They cannot cut a UTXO in half any more than they can cut a dollar bill in half and use it as currency. The user's wallet application will typically select from the user's available UTXO various units to compose an amount greater than or equal to the desired transaction amount.
As with real life, the bitcoin application can use several strategies to satisfy the purchase amount: combining several smaller units, finding exact change, or using a single unit larger than the transaction value and making change. All of this complex assembly of spendable UTXO is done by the user's wallet automatically and is invisible to users. It is only relevant if you are programmatically constructing raw transactions from UTXO.
The UTXO consumed by a transaction are called transaction inputs, while the UTXO created by a transaction are called transaction outputs. This way, chunks of bitcoin value move forward from owner to owner in a chain of transactions consuming and creating UTXO. Transactions consume UTXO unlocking it with the signature of the current owner and create UTXO locking it to the bitcoin address of the new owner.
The exception to the output and input chain is a special type of transaction called the _coinbase_ transaction, which is the first transaction in each block. This transaction is placed there by the "winning" miner and creates brand-new bitcoin payable to that miner as a reward for mining. This is how bitcoin's money supply is created during the mining process as we will see in <<mining>>.
[TIP]
====
What comes first? Inputs or outputs, the chicken or the egg? Strictly speaking, outputs come first because coinbase transactions, which generate new bitcoin, have no inputs and create outputs from nothing.
@ -103,11 +112,37 @@ The transaction scripting language, used in the locking script mentioned above,
[options="header"]
|=======
|Size| Field | Description
| 8 bytes | Amount | Bitcoin Value in Satoshis (10^^-8 bitcoin)
| 8 bytes | Amount | Bitcoin Value in Satoshis (10^-8^ bitcoin)
| 1-9 bytes (VarInt) | Locking-Script Size | Locking-Script length in bytes, to follow
| Variable | Locking-Script | A script defining the conditions needed to spend the output
|=======
In the example below, we use the blockchain.info API to find the unspent outputs (UTXO) of a specific address:
[[get_utxo]]
.A script that calls the blockchain.info API to find the UTXO related to an address
====
[source, python]
----
include::code/get-utxo.py[]
----
====
Running the script, we see a list of transaction IDs, a colon, the index number of the specific unspent transaction output (UTXO), and the value of that UTXO in Satoshis. The locking script is not shown in this output:
[[get_utxo_run]]
.Running the get-utxo.py script
====
[source,bash]
----
$ python get-utxo.py
ebadfaa92f1fd29e2fe296eda702c48bd11ffd52313e986e99ddad9084062167:1 - 8000000 Satoshis
6596fd070679de96e405d52b51b8e1d644029108ec4cbfe451454486796a1ecf:0 - 16050000 Satoshis
74d788804e2aae10891d72753d1520da1206e6f4f20481cc1555b7f2cb44aca0:0 - 5000000 Satoshis
b2affea89ff82557c60d635a2a3137b8f88f12ecec85082f7d0a1f82ee203ac4:0 - 10000000 Satoshis
...
----
====
===== Spending Conditions (Encumbrances)
Transaction outputs associate a specific amount (in satoshis) to a specific _encumbrance_ or locking-script that defines the condition that must be met to spend that amount. In most cases the locking script will lock the output to a specific bitcoin address, thereby transferring ownership of that amount to the new owner. When Alice paid Bob's Cafe for a cup of coffee, her transaction created a 0.015 bitcoin output _encumbered_ or locked to the Cafe's bitcoin address. That 0.015 bitcoin output was recorded on the blockchain and became part of the Unspent Transaction Output set, meaning it showed in Bob's wallet as part of the available balance. When Bob chooses to spend that amount, his transaction will release the encumbrance, unlocking the output by providing an unlocking script containing a signature from Bob's private key.
@ -117,7 +152,36 @@ Transaction outputs associate a specific amount (in satoshis) to a specific _enc
In simple terms, transaction inputs are pointers to UTXO. They point to a specific UTXO by reference to the transaction hash and sequence number where the UTXO is recorded in the blockchain. To spend UTXO, a transaction input also includes unlocking-scripts that satisfy the spending conditions set by the UTXO. The unlocking script is usually a signature proving ownership of the bitcoin address that is in the locking script.
When a user makes a payment, their wallet constructs a transaction by selecting from the available UTXO. For example, to make a 0.015 bitcoin payment, the wallet app may select a 0.01 UTXO and a 0.005 UTXO, using them both to add up to the desired payment amount. The wallet then produces unlocking scripts containing signatures for each of the UTXO, thereby making them spendable by satisfying their locking script conditions. The wallet adds these UTXO references and unlocking scripts as inputs to the transaction.
When a user makes a payment, their wallet constructs a transaction by selecting from the available UTXO. For example, to make a 0.015 bitcoin payment, the wallet app may select a 0.01 UTXO and a 0.005 UTXO, using them both to add up to the desired payment amount.
In the example below, we show the use of a "greedy" algorithm to select from available UTXO in order to make a specific payment amount. In the example, the available UTXO are provided as a constant array, but in reality, the available UTXO would be retrieved with an RPC call to Bitcoin Core, or to a third-party API as shown in <<get_utxo>>.
[[select_utxo]]
.A script for calculating how much total bitcoin will be issued
====
[source, python]
----
include::code/select-utxo.py[]
----
====
If we run the select-utxo.py script without a parameter it will attempt to construct a set of UTXO (and change) for a payment of 55000000 Satoshis (0.55 bitcoin). If you provide a target payment amount as a parameter, the script will select UTXO to make that target payment amount. Below, we run the script trying to make a payment of 0.5 bitcoin or 50000000 Satoshis:
[[select_utxo_run]]
.Running the select-utxo.py script
====
[source,bash]
----
$ python select-utxo.py 50000000
For transaction amount 50000000 Satoshis (0.500000 bitcoin) use:
([<7dbc497969c7475e45d952c4a872e213fb15d45e5cd3473c386a71a1b0c136a1:0 with 25000000 Satoshis>, \
<7f42eda67921ee92eae5f79bd37c68c9cb859b899ce70dba68c48338857b7818:0 with 16100000 Satoshis>, \
<6596fd070679de96e405d52b51b8e1d644029108ec4cbfe451454486796a1ecf:0 with 16050000 Satoshis>],\
'Change: 7150000 Satoshis')
----
====
Once the UTXO is selected, the wallet then produces unlocking scripts containing signatures for each of the UTXO, thereby making them spendable by satisfying their locking script conditions. The wallet adds these UTXO references and unlocking scripts as inputs to the transaction.
[[tx_in_structure]]
.The structure of a transaction input
@ -131,7 +195,7 @@ When a user makes a payment, their wallet constructs a transaction by selecting
| 4 bytes | Sequence Number | Currently-disabled Tx-replacement feature, set to 0xFFFFFFFF
|=======
Note: The sequence number is used to override a transaction prior to the expiration of the transaction locktime, which is a feature that is currently disabled in bitcoin. Most transactions set this value to the maximum integer value (0xFFFFFFFF) and it is ignored by the bitcoin network. If the transaction has a non-zero locktime, at least one of its inputs must have a sequence number below 0xFFFFFFFF in order to enable locktime.
Note: The sequence number is used to override a transaction prior to the expiration of the transaction locktime, which is a feature that is currently disabled in bitcoin. Most transactions set this value to the maximum integer value (0xFFFFFFFF) and it is ignored by the bitcoin network. If the transaction has a non-zero locktime, at least one of its inputs must have a sequence number below 0xFFFFFFFF in order to enable locktime.
[[tx_fees]]
==== Transaction Fees
@ -168,7 +232,11 @@ If you forget to add a change output in a manually constructed transaction you w
Let's see how this works in practice, by looking at Alice's coffee purchase again. Alice wants to spend 0.015 bitcoin to pay for coffee. To ensure this transaction is processed promptly, she will want to include a transaction fee, say 0.001. That will mean that the total cost of the transaction will be 0.016. Her wallet must therefore source a set of UTXO that adds up to 0.016 bitcoin or more and if necessary create change. Let's say her wallet has a 0.2 bitcoin UTXO available. It will therefore need to consume this UTXO, create one output to Bob's Cafe for 0.015, and a second output with 0.184 bitcoin in change back to her own wallet, leaving 0.001 bitcoin unallocated, as an implicit fee for the transaction.
Now let's look at a different scenario. Eugenia, our children's charity director in the Philippines has completed a fundraiser to purchase school books for the children. She received several thousand small donations from people all around the world, totaling 50 bitcoin. Now she wants to purchase hundreds of school books from a local publisher, paying in bitcoin. The charity received thousands of small donations from all around the world. As Eugenia's wallet application tries to construct a single larger payment transaction, it must source from the available UTXO set which is composed of many smaller amounts. That means that the resulting transaction will source from more than a hundred small-value UTXO as inputs and only one output, paying the book publisher. A transaction with that many inputs will be larger than one kilobyte, perhaps 2-3 kilobytes in size. As a result, it will require a higher fee than the minimal network fee of 0.0001 bitcoin. Eugenia's wallet application will calculate the appropriate fee by measuring the size of the transaction and multiplying that by the per-kilobyte fee. Many wallets will overpay fees for larger transactions to ensure the transaction is processed promptly. The higher fee is not because Eugenia is spending more money, but because her transaction is more complex and larger in size - the fee is independent of the transaction's bitcoin value.
Now let's look at a different scenario. Eugenia, our children's charity director in the Philippines has completed a fundraiser to purchase school books for the children. She received several thousand small donations from people all around the world, totaling 50 bitcoin. Now she wants to purchase hundreds of school books from a local publisher, paying in bitcoin. The charity received thousands of small donations from all around the world, so her wallet is full of very small payments (UTXO).
As Eugenia's wallet application tries to construct a single larger payment transaction, it must source from the available UTXO set which is composed of many smaller amounts. That means that the resulting transaction will source from more than a hundred small-value UTXO as inputs and only one output, paying the book publisher. A transaction with that many inputs will be larger than one kilobyte, perhaps 2-3 kilobytes in size. As a result, it will require a higher fee than the minimal network fee of 0.0001 bitcoin.
Eugenia's wallet application will calculate the appropriate fee by measuring the size of the transaction and multiplying that by the per-kilobyte fee. Many wallets will overpay fees for larger transactions to ensure the transaction is processed promptly. The higher fee is not because Eugenia is spending more money, but because her transaction is more complex and larger in size - the fee is independent of the transaction's bitcoin value.
[[tx_chains]]
=== Transaction Chaining and Orphan Transactions
@ -201,7 +269,11 @@ A locking script is an encumbrance placed on an output, and it specifies the con
An unlocking script is a script that "solves", or satisfies, the conditions placed on an output by a locking script and allows the output to be spent. Unlocking scripts are part of every transaction input and most of the time they contain a digital signature produced by the user's wallet from their private key. Historically, the unlocking script is called _scriptSig_, because it usually contained a digital signature. In this book we refer to it as an "unlocking script" to acknowledge the much broader range of locking script requirements, as not all unlocking scripts must contain signatures. As mentioned above, in most bitcoin applications the source code will refer to the unlocking script as "scriptSig".
Every bitcoin client will validate transactions by executing the locking and unlocking scripts together. For each input in the transaction, the validation software will first retrieve the UTXO referenced by the input. That UTXO contains a locking script defining the conditions required to spend it. The validation software will then take the unlocking script contained in the input that is attempting to spend this UTXO and concatenate them. The locking script is added to the end of the unlocking script and then the entire combined script is executed using the script execution engine. If the result of executing the combined script is "TRUE", the unlocking script has succeeded in resolving the conditions imposed by the locking script and therefore the input is a valid authorization to spend the UTXO. If any result other than "TRUE" remains after execution of the combined script, the input is invalid as it has failed to satisfy the spending conditions placed on the UTXO. Note that the UTXO is permanently recorded in the blockchain, and therefore is invariable and is unaffected by failed attempts to spend it by reference in a new transaction. Only a valid transaction that correctly satisfies the conditions of the UTXO results in the UTXO being marked as "spent" and removed from the set of available (unspent) UTXO.
Every bitcoin client will validate transactions by executing the locking and unlocking scripts together. For each input in the transaction, the validation software will first retrieve the UTXO referenced by the input. That UTXO contains a locking script defining the conditions required to spend it. The validation software will then take the unlocking script contained in the input that is attempting to spend this UTXO and execute the two scripts.
In the original bitcoin client, the unlocking and locking scripts were concatenated and executed in sequence. For security reasons, this was changed in 2010, because of a vulnerability that allowed a malformed unlocking script to push data onto the stack and corrupt the locking script. In the current implementation the scripts are executed separately with the stack transferred between the two executions, as described below.
First, the unlocking script is executed, using the stack execution engine. If the unlocking script executed without errors (e.g it has no "dangling" operators leftover), the main stack (not the alternate stack) is copied and the locking script is executed. If the result of executing the locking script with the stack data copied from the unlocking script is "TRUE", the unlocking script has succeeded in resolving the conditions imposed by the locking script and therefore the input is a valid authorization to spend the UTXO. If any result other than "TRUE" remains after execution of the combined script, the input is invalid as it has failed to satisfy the spending conditions placed on the UTXO. Note that the UTXO is permanently recorded in the blockchain, and therefore is invariable and is unaffected by failed attempts to spend it by reference in a new transaction. Only a valid transaction that correctly satisfies the conditions of the UTXO results in the UTXO being marked as "spent" and removed from the set of available (unspent) UTXO.
Below is an example of the unlocking and locking scripts for the most common type of bitcoin transaction (a payment to a public key hash), showing the combined script resulting from the concatenation of the unlocking and locking scripts prior to script validation:
@ -255,7 +327,7 @@ As we saw in the step-by-step example above, when this script is executed the re
[TIP]
====
Transactions are valid if the top result on the stack is TRUE (1), any other non-zero value or if the stack is empty after script execution. Transactions are invalid if the top value on the stack is FALSE (0) or if script execution is halted explicitly by an operator, such as OP_VERIFY, OP_RETURN or a conditional terminator such as OP_ENDIF. See <<tx_script_ops>> for details.
Transactions are valid if the top result on the stack is TRUE (noted as +{0x01}+), any other non-zero value or if the stack is empty after script execution. Transactions are invalid if the top value on the stack is FALSE (a zero-length empty value, noted as +{}+) or if script execution is halted explicitly by an operator, such as OP_VERIFY, OP_RETURN or a conditional terminator such as OP_ENDIF. See <<tx_script_ops>> for details.
====
==== Turing Incompleteness
@ -297,7 +369,8 @@ The locking script above can be satisfied with an unlocking script of the form:
The two scripts together would form the combined validation script below:
----
<Cafe Signature> <Cafe Public Key> OP_DUP OP_HASH160 <Cafe Public Key Hash> OP_EQUAL OP_CHECKSIG
<Cafe Signature> <Cafe Public Key> OP_DUP OP_HASH160 \
<Cafe Public Key Hash> OP_EQUAL OP_CHECKSIG
----
When executed, this combined script will evaluate to TRUE if, and only if, the unlocking script matches the conditions set by the locking script. In other words, the result will be TRUE if the unlocking script has a valid signature from the Cafe's private key which corresponds to the public key hash set as an encumbrance.
@ -315,7 +388,7 @@ image::images/Tx_Script_P2PubKeyHash_2.png["Tx_Script_P2PubKeyHash_2"]
[[p2pk]]
==== Pay-to-Public-Key
Pay-to-Public-Key is a simpler form of a bitcoin payment than Pay-to-Public-Key-Hash. With this script form, the public key itself is stored in the locking script, rather than a public-key-hash as with P2PKH above, which is much shorter. The disadvantage of this form of locking script is that it consumes more space in the blockchain to store these types of payments, because a public key is 264 or 520 bits long (depending on whether it is compressed), whereas a public key hash is only 160 bits long. For legacy compatibility, Pay-to-Public-Key is used in all coinbase generation transactions, the transactions that pay the reward to the miners.
Pay-to-Public-Key is a simpler form of a bitcoin payment than Pay-to-Public-Key-Hash. With this script form, the public key itself is stored in the locking script, rather than a public-key-hash as with P2PKH above, which is much shorter. Pay-to-Public-Key-Hash was invented by Satoshi to make bitcoin addresses shorter, for ease of use. Pay-to-Public-Key is now most often seen in coinbase transactions, generated by older mining software that has not been updated to use P2PKH.
A Pay-to-Public-Key locking script looks like this:
----
@ -335,7 +408,7 @@ The combined script, which is validated by the transaction validation software i
The script above is a simple invocation of the CHECKSIG operator which validates the signature as belonging to the correct key and returns TRUE on the stack.
[[multisig]]
==== Mutli-Signature
==== Multi-Signature
Multi-signature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to release the encumbrance. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multi-signature is one where 3 public keys are listed as potential signers and at least 2 of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multi-signature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multi-signature or any combination within that range. The limitation to 15 listed keys may be lifted by the time of publication of this book, so check the +isStandard()+ function to see what is currently accepted by the network.
@ -361,7 +434,8 @@ _Note: The prefix OP_0 is required because of a bug in the original implementati
The two scripts together would form the combined validation script below:
----
OP_0 <Signature B> <Signature C> 2 <Public Key A> <Public Key B> <Public Key C> 3 OP_CHECKMULTISIG
OP_0 <Signature B> <Signature C>\
2 <Public Key A> <Public Key B> <Public Key C> 3 OP_CHECKMULTISIG
----
When executed, this combined script will evaluate to TRUE if, and only if, the unlocking script matches the conditions set by the locking script. In this case, the condition is whether the unlocking script has a valid signature from the two private keys that correspond to two of the three public keys set as an encumbrance.
@ -385,7 +459,7 @@ where the data portion is limited to 40 bytes and most often represents a hash,
Keep in mind that there is no "unlocking script" that corresponds to OP_RETURN that could possibly be used to "spend" an OP_RETURN output. The whole point of OP_RETURN is that you can't spend the money locked in that output and therefore it does not need to be held in the UTXO set as potentially spendable - OP_RETURN is _provably un-spendable_. OP_RETURN is usually an output with a zero bitcoin amount, since any bitcoin assigned to such an output is effectively lost forever. If an OP_RETURN is encountered by the script validation software, it results immediately in halting the execution of the validation script and marking the transaction as invalid. Thus, if you accidentally reference an OP_RETURN output as an input in a transaction, that transaction is invalid.
A valid transaction can have only one OP_RETURN output. However, a single OP_RETURN output can be combined in a transaction with outputs of any other type.
A standard transaction (one that conforms to the +isStandard()+ checks) can have only one OP_RETURN output. However, a single OP_RETURN output can be combined in a transaction with outputs of any other type.
[[p2sh]]
==== Pay to Script Hash (P2SH)
@ -397,14 +471,15 @@ In chapter 1 we introduced Mohammed, an electronics importer based in Dubai. Moh
The resulting script is quite long and looks like this:
----
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> <Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> \
<Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
----
While multi-signature scripts are a powerful feature, they are cumbersome to use. Given the script above, Mohammed would have to communicate this script to every customer prior to payment. Each customer would have to use special bitcoin wallet software with the ability to create custom transaction scripts and each customer would have to understand how to create a transaction using custom scripts. Furthermore, the resulting transaction would be about five times larger than a simple payment transaction, as this script contains very long public keys. The burden of that extra-large transaction would be borne by the customer in the form of fees. Finally, a large transaction script like this would be carried in the UTXO set in RAM in every full node, until it was spent. All of these issues make using complex output scripts difficult in practice.
Pay-to-Script-Hash (P2SH) was developed to resolve these practical difficulties and to make the use of complex scripts as easy as a payment to a bitcoin address. With P2SH payments, the complex locking script is replaced with its digital fingerprint, a cryptographic hash. When a transaction attempting to spend the UTXO is presented later, it must contain the script that matches the hash, in addition to the unlocking script. In simple terms, P2SH means "pay to a script matching this hash, a script which will be presented later when this output is spent".
In P2SH transactions, the locking script that is replaced by a hash is referred to as the _redeemScript_ because it is presented to the system at redemption time rather than as a locking script.
In P2SH transactions, the locking script that is replaced by a hash is referred to as the _redeem script_ because it is presented to the system at redemption time rather than as a locking script.
[[without_p2sh]]
.Complex Script without P2SH
@ -417,28 +492,35 @@ In P2SH transactions, the locking script that is replaced by a hash is referred
.Complex Script as P2SH
|=======
| Redeem Script | 2 PubKey1 PubKey2 PubKey3 PubKey4 PubKey5 5 OP_CHECKMULTISIG
| Locking Script | OP_HASH160 <20-byte hash of redeemScript> OP_EQUAL
| Unlocking Script | Sig1 Sig2 redeemScript
| Locking Script | OP_HASH160 <20-byte hash of redeem script> OP_EQUAL
| Unlocking Script | Sig1 Sig2 redeem script
|=======
As you can see from the tables above, with P2SH the complex script that details the conditions for spending the output (redeemScript) is not presented in the locking script. Instead, only a hash of it is in the locking script and the redeemScript itself is presented later, as part of the unlocking script when the output is spent. This shifts the burden in fees and complexity from the sender to the recipient (spender) of the transaction.
As you can see from the tables above, with P2SH the complex script that details the conditions for spending the output (redeem script) is not presented in the locking script. Instead, only a hash of it is in the locking script and the redeem script itself is presented later, as part of the unlocking script when the output is spent. This shifts the burden in fees and complexity from the sender to the recipient (spender) of the transaction.
Let's look at Mohammed's company, their complex multi-signature script and the resulting P2SH scripts.
First, the multi-signature script that Mohammed's company uses for all incoming payments from customers:
----
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> <Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> \
<Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
----
If the placeholders above are replaced by actual public keys (shown below as 520 bit numbers starting with 04) you can see that this script becomes very long:
----
2
04C16B8698A9ABF84250A7C3EA7EEDEF9897D1C8C6ADF47F06CF73370D74DCCA01CDCA79DCC5C395D7EEC6984D83F1F50C900A24DD47F569FD4193AF5DE762C587
04A2192968D8655D6A935BEAF2CA23E3FB87A3495E7AF308EDF08DAC3C1FCBFC2C75B4B0F4D0B1B70CD2423657738C0C2B1D5CE65C97D78D0E34224858008E8B49
047E63248B75DB7379BE9CDA8CE5751D16485F431E46117B9D0C1837C9D5737812F393DA7D4420D7E1A9162F0279CFC10F1E8E8F3020DECDBC3C0DD389D9977965
0421D65CBD7149B255382ED7F78E946580657EE6FDA162A187543A9D85BAAA93A4AB3A8F044DADA618D087227440645ABE8A35DA8C5B73997AD343BE5C2AFD94A5
043752580AFA1ECED3C68D446BCAB69AC0BA7DF50D56231BE0AABF1FDEEC78A6A45E394BA29A1EDF518C022DD618DA774D207D137AAB59E0B000EB7ED238F4D800
5 OP_CHECKMULTISIG
04C16B8698A9ABF84250A7C3EA7EEDEF9897D1C8C6ADF47F06CF73370\
D74DCCA01CDCA79DCC5C395D7EEC6984D83F1F50C900A24DD47F569FD\
4193AF5DE762C58704A2192968D8655D6A935BEAF2CA23E3FB87A3495\
E7AF308EDF08DAC3C1FCBFC2C75B4B0F4D0B1B70CD2423657738C0C2B\
1D5CE65C97D78D0E34224858008E8B49047E63248B75DB7379BE9CDA8\
CE5751D16485F431E46117B9D0C1837C9D5737812F393DA7D4420D7E1\
A9162F0279CFC10F1E8E8F3020DECDBC3C0DD389D99779650421D65CB\
D7149B255382ED7F78E946580657EE6FDA162A187543A9D85BAAA93A4\
AB3A8F044DADA618D087227440645ABE8A35DA8C5B73997AD343BE5C2\
AFD94A5043752580AFA1ECED3C68D446BCAB69AC0BA7DF50D56231BE0\
AABF1FDEEC78A6A45E394BA29A1EDF518C022DD618DA774D207D137AA\
B59E0B000EB7ED238F4D800 5 OP_CHECKMULTISIG
----
The entire script above can instead be represented by a 20-byte cryptographic hash, by first applying the SHA256 hashing algorithm and then applying the RIPEMD160 algorithm on the result. The 20-byte hash of the above script is:
@ -451,17 +533,17 @@ A P2SH transaction locks the output to this hash instead of the longer script, u
----
OP_HASH160 54c557e07dde5bb6cb791c7a540e0a4796f5e97e OP_EQUAL
----
which, as you can see is much shorter. Instead of "pay to this 5-key multi-signature script", the P2SH equivalent transaction is "pay to a script with this hash". A customer making a payment to Mohammed's company need only include this much shorter locking script in their payment. When Mohammed wants to spend this UTXO, they must present the original redeemScript (the one whose hash locked the UTXO) and the signatures necessary to unlock it, like this:
which, as you can see is much shorter. Instead of "pay to this 5-key multi-signature script", the P2SH equivalent transaction is "pay to a script with this hash". A customer making a payment to Mohammed's company need only include this much shorter locking script in their payment. When Mohammed wants to spend this UTXO, they must present the original redeem script (the one whose hash locked the UTXO) and the signatures necessary to unlock it, like this:
----
<Sig1> <Sig2> <2 PK1 PK2 PK3 PK4 PK5 5 OP_CHECKMULTISIG>
----
The two scripts are combined in two stages. First, the redeemScript is checked against the locking script to make sure the hash matches:
The two scripts are combined in two stages. First, the redeem script is checked against the locking script to make sure the hash matches:
----
<2 PK1 PK2 PK3 PK4 PK5 5 OP_CHECKMULTISIG> OP_HASH160 <redeemScriptHash> OP_EQUAL
<2 PK1 PK2 PK3 PK4 PK5 5 OP_CHECKMULTISIG> OP_HASH160 <redeem scriptHash> OP_EQUAL
----
If the redeemScript hash matches, then the unlocking script is executed on its own, to unlock the redeemScript:
If the redeem script hash matches, then the unlocking script is executed on its own, to unlock the redeem script:
----
<Sig1> <Sig2> 2 PK1 PK2 PK3 PK4 PK5 5 OP_CHECKMULTISIG
----
@ -481,180 +563,19 @@ The Pay-to-Script-Hash feature offers the following benefits compared to the dir
* P2SH shifts the burden of constructing the script to the recipient not the sender
* P2SH shifts the burden in data storage for the long script from the output (which is in the UTXO set and therefore impacts memory) to the input (only stored on the blockchain)
* P2SH shifts the burden in data storage for the long script from the present time (payment) to a future time (when it is spent)
* P2SH shifts the transaction fee cost of a long script from the sender to the recipient who has to include the long redeemScript to spend it
* P2SH shifts the transaction fee cost of a long script from the sender to the recipient who has to include the long redeem script to spend it
===== Redeem Script and isStandard Validation
Pay-to-Script-Hash is currently limited to the standard types of bitcoin transaction scripts, by the +isStandard()+ function. That means that the redeemScript presented in the spending transaction must be one of the standard types: P2PK, P2PKH or Multi-Sig, excluding OP_RETURN and P2SH itself. You cannot reference a P2SH script inside a redeemScript and you can't use an OP_RETURN inside a P2SH redeemScript.
Prior to version 0.9.2 of the Bitcoin Core client, Pay-to-Script-Hash was limited to the standard types of bitcoin transaction scripts, by the +isStandard()+ function. That means that the redeem script presented in the spending transaction could only be one of the standard types: P2PK, P2PKH or Multi-Sig, excluding OP_RETURN and P2SH itself.
This limitation of redeemScript to only standard transaction scripts is temporary and will likely be removed in future versions of the bitcoin reference implementation, allowing the use of any valid script inside a P2SH redeemScript. You will still not be able to put a P2SH inside a P2SH redeemScript, because the P2SH specification is not recursive. You will still not be able to use OP_RETURN in a redeemScript because OP_RETURN cannot be redeemed by definition. But you will be able someday to use all the other operators to create a vast range of complex and novel scripts that can be used as redeemScripts and referenced as P2SH payment to their hash.
As of version 0.9.2 of the Bitcoin Core client, P2SH transactions can contain any valid script, making the P2SH standard much more flexible and allowing for experimentation with many novel and complex types of transactions.
Note that since the redeemScript is not presented to the network until you attempt to spend a P2SH output, if you lock an output with the hash of a non-standard transaction it will be processed as valid. However, you will not be able to spend it as the spending transaction which includes the redeemScript will not be accepted, as it is non-standard. This creates a risk, as you can lock bitcoin in a P2SH which cannot be later spent. The network will accept the P2SH encumbrance even if it corresponds to a non-standard or invalid redeemScript, because the script hash gives no indication of the script it represents.
Note that you are not able to put a P2SH inside a P2SH redeem script, because the P2SH specification is not recursive. You are also still not be able to use OP_RETURN in a redeem script because OP_RETURN cannot be redeemed by definition.
Note that since the redeem script is not presented to the network until you attempt to spend a P2SH output, if you lock an output with the hash of an invalid transaction it will be processed regardless. However, you will not be able to spend it as the spending transaction which includes the redeem script will not be accepted, as it is an invalid script. This creates a risk, because you can lock bitcoin in a P2SH which cannot be later spent. The network will accept the P2SH encumbrance even if it corresponds to an invalid redeem script, because the script hash gives no indication of the script it represents.
[WARNING]
====
P2SH locking scripts contain the hash of a redeemScript which gives no clues as to the content of the redeemScript itself. The P2SH transaction will be considered valid and accepted even if the redeemScript is invalid or non-standard. You may accidentally lock bitcoin in such a way that it cannot later be spent.
P2SH locking scripts contain the hash of a redeem script which gives no clues as to the content of the redeem script itself. The P2SH transaction will be considered valid and accepted even if the redeem script is invalid. You may accidentally lock bitcoin in such a way that it cannot later be spent.
====
[[tx_script_ops]]
=== Transaction Script Language Operators, Constants and Symbols
[[tx_script_ops_table_pushdata]]
.Push Value onto Stack
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_0 or OP_FALSE | 0x00 | An empty array is pushed on to the stack
| 1-75 | 0x01-0x4b | Push the next N bytes onto the stack, where N is 1 to 75 bytes
| OP_PUSHDATA1 | 0x4c | The next script byte contains N, push the following N bytes onto the stack
| OP_PUSHDATA2 | 0x4d | The next two script bytes contain N, push the following N bytes onto the stack
| OP_PUSHDATA4 | 0x4e | The next four script bytes contain N, push the following N bytes onto the stack
| OP_1NEGATE | 0x4f | Push the value "-1" onto the stack
| OP_RESERVED | 0x50 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_1 or OP_TRUE| 0x51 | Push the value "1" onto the stack
| OP_2 to OP_16 | 0x52 to 0x60 | For OP_N, push the value "N" onto the stack. E.g., OP_2 pushes "2"
|=======
[[tx_script_ops_table_control]]
.Conditional Flow Control
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP | 0x61 | Do nothing
| OP_VER | 0x62 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_IF | 0x63 | Execute the statements following if top of stack is not 0
| OP_NOTIF | 0x64 | Execute the statements following if top of stack is 0
| OP_VERIF | 0x65 | Halt - Invalid transaction
| OP_VERNOTIF | 0x66 | Halt - Invalid transaction
| OP_ELSE | 0x67 | Execute only if the previous statements were not executed
| OP_ENDIF | 0x68 | Ends the OP_IF, OP_NOTIF, OP_ELSE block
| OP_VERIFY | 0x69 | Check the top of the stack, Halt and Invalidate transaction if not TRUE
| OP_RETURN | 0x6a | Halt and invalidate transaction
|=======
[[tx_script_ops_table_stack]]
.Stack Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_TOALTSTACK | 0x6b | Pop top item from stack and push to alternative stack
| OP_FROMALTSTACK | 0x6c | Pop top item from alternative stack and push to stack
| OP_2DROP | 0x6d | Pop top two stack items
| OP_2DUP | 0x6e | Duplicate top two stack items
| OP_3DUP | 0x6f | Duplicate top three stack items
| OP_2OVER | 0x70 | Copies the third and fourth items in the stack to the top
| OP_2ROT | 0x71 | Moves the fifth and sixth items in the stack to the top
| OP_2SWAP | 0x72 | Swap the two top pairs of items in the stack
| OP_IFDUP | 0x73 | Duplicate the top item in the stack if it is not 0
| OP_DEPTH | 0x74 | Count the items on the stack and push the resulting count
| OP_DROP | 0x75 | Pop the top item in the stack
| OP_DUP | 0x76 | Duplicate the top item in the stack
| OP_NIP | 0x77 | Pop the second item in the stack
| OP_OVER | 0x78 | Copy the second item in the stack and push it on to the top
| OP_PICK | 0x79 | Pop value N from top, then copy the Nth item to the top of the stack
| OP_ROLL | 0x7a | Pop value N from top, then move the Nth item to the top of the stack
| OP_ROT | 0x7b | Rotate the top three items in the stack
| OP_SWAP | 0x7c | Swap the top three items in the stack
| OP_TUCK | 0x7d | Copy the top item and insert it between the top and second item.
|=======
[[tx_script_ops_table_splice]]
.String Splice Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_CAT_ | 0x7e | Disabled (Concatenates top two items)
| _OP_SUBSTR_ | 0x7f | Disabled (Returns substring)
| _OP_LEFT_ | 0x80 | Disabled (Returns left substring)
| _OP_RIGHT_ | 0x81 | Disabled (Returns right substring)
| OP_SIZE | 0x82 | Calculate string length of top item and push the result
|=======
[[tx_script_ops_table_binmath]]
.Binary Arithmetic and Conditionals
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_INVERT_ | 0x83 | Disabled (Flip the bits of the top item)
| _OP_AND_ | 0x84 | Disabled (Boolean AND of two top items)
| _OP_OR_ | 0x85 | Disabled (Boolean OR of two top items)
| _OP_XOR_ | 0x86 | Disabled (Boolean XOR of two top items)
| OP_EQUAL | 0x87 | Push TRUE (1) if top two items are exactly equal, push FALSE (0) otherwise
| OP_EQUALVERIFY | 0x88 | Same as OP_EQUAL, but run OP_VERIFY after to halt if not TRUE
| OP_RESERVED1 | 0x89 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_RESERVED2 | 0x8a | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
|=======
[[tx_script_ops_table_numbers]]
.Numeric Operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_1ADD | 0x8b | Add 1 to the top item
| OP_1SUB | 0x8c | Subtract 1 from the top item
| _OP_2MUL_ | 0x8d | Disabled (Multiply top item by 2)
| _OP_2DIV_ | 0x8e | Disabled (Divide top item by 2)
| OP_NEGATE | 0x8f | Flip the sign of top item
| OP_ABS | 0x90 | Change the sign of the top item to positive
| OP_NOT | 0x91 | If top item is 0 or 1 boolean flip it, otherwise return 0
| OP_0NOTEQUAL | 0x92 | If top item is 0 return 0, otherwise return 1
| OP_ADD | 0x93 | Pop top two items, add them and push result
| OP_SUB | 0x94 | Pop top two items, subtract first form second, push result
| OP_MUL | 0x95 | Disabled (Multiply top two items)
| OP_DIV | 0x96 | Disabled (Divide second item by first item)
| OP_MOD | 0x97 | Disabled (Remainder divide second item by first item)
| OP_LSHIFT | 0x98 | Disabled (Shift second item left by first item number of bits)
| OP_RSHIFT | 0x99 | Disabled (Shift second item right by first item number of bits)
| OP_BOOLAND | 0x9a | Boolean AND of top two items
| OP_BOOLOR | 0x9b | Boolean OR of top two items
| OP_NUMEQUAL | 0x9c | Return TRUE if top two items are equal numbers
| OP_NUMEQUALVERIFY | 0x9d | Same as NUMEQUAL, then OP_VERIFY to halt if not TRUE
| OP_NUMNOTEQUAL | 0x9e | Return TRUE if top two items are not equal numbers
| OP_LESSTHAN | 0x9f | Return TRUE if second item is less than top item
| OP_GREATERTHAN | 0xa0 | Return TRUE if second item is greater than top item
| OP_LESSTHANOREQUAL | 0xa1 | Return TRUE if second item is less than or equal to top item
| OP_GREATERTHANOREQUAL | 0xa2 | Return TRUE if second item is great than or equal to top item
| OP_MIN | 0xa3 | Return the smaller of the two top items
| OP_MAX | 0xa4 | Return the larger of the two top items
| OP_WITHIN | 0xa5 | Return TRUE if the third item is between the second item (or equal) and first item
|=======
[[tx_script_ops_table_crypto]]
.Cryptographic and Hashing Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_RIPEMD160 | 0xa6 | Return RIPEMD160 hash of top item
| OP_SHA1 | 0xa7 | Return SHA1 hash of top item
| OP_SHA256 | 0xa8 | Return SHA256 hash of top item
| OP_HASH160 | 0xa9 | Return RIPEMD160(SHA256(x)) hash of top item
| OP_HASH256 | 0xaa | Return SHA256(SHA256(x)) hash of top item
| OP_CODESEPARATOR | 0xab | Mark the beginning of signature-checked data
| OP_CHECKSIG | 0xac | Pop a public key and signature and validate the signature for the transaction's hashed data, return TRUE if matching
| OP_CHECKSIGVERIFY | 0xad | Same as CHECKSIG, then OP_VEIRFY to halt if not TRUE
| OP_CHECKMULTISIG | 0xae | Run CHECKSIG for each pair of signature and public key provided. All must match. Bug in implementation pops an extra value, prefix with OP_NOP as workaround
| OP_CHECKMULTISIGVERIFY | 0xaf | Same as CHECKMULTISIG, then OP_VERIFY to halt if not TRUE
|=======
[[tx_script_ops_table_nop]]
.Non-Operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP1-OP_NOP10 | 0xb0-0xb9 | Does nothing, ignored.
|=======
[[tx_script_ops_table_internal]]
.Reserved OP codes for internal use by the parser
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_SMALLDATA | 0xf9 | Represents small data field
| OP_SMALLINTEGER | 0xfa | Represents small integer data field
| OP_PUBKEYS | 0xfb | Represents public key fields
| OP_PUBKEYHASH | 0xfd | Represents a public key hash field
| OP_PUBKEY | 0xfe | Represents a public key field
| OP_INVALIDOPCODE | 0xff | Represents any OP code not currently assigned
|=======

19
ch06.asciidoc
View File

@ -1,8 +1,7 @@
[[ch6]]
[[bitcoin_network]]
== The Bitcoin Network
=== Introduction
=== Peer-to-Peer Network Architecture
Bitcoin is structured as a peer-to-peer network architecture on top of the Internet. The term peer-to-peer or P2P means that the computers that participate in the network are peers to each other, that they are all equal, that there are no "special" nodes and that all nodes share the burden of providing network services. The network nodes interconnect in a mesh network with a "flat" topology. There is no "server", no centralized service, and no hierarchy within the network. Nodes in a peer-to-peer network both provide and consume services at the same time with reciprocity acting as the incentive for participation. Peer-to-peer networks are inherently resilient, de-centralized, and open. The pre-eminent example of a P2P network architecture was the early Internet itself, where nodes on the IP network were equal. Today's Internet architecture is more hierarchical, but the Internet Protocol still retains its flat-topology essence. Beyond bitcoin, the largest and most successful application of P2P technologies is file sharing with Napster as the pioneer and bittorrent as the most recent evolution of the architecture.
@ -21,7 +20,7 @@ image::images/FullNodeReferenceClient_Small.png["FullNodeReferenceClient_Small"]
All nodes include the routing function to participate in the network and may include other functionality. All nodes validate and propagate transactions and blocks, and discover and maintain connections to peers. In the full node example above, the routing function is indicated by an orange circle named "Network Routing Node".
Some nodes, called full nodes, also maintain a complete and up-to-date copy of the blockchain. Full nodes can autonomously and authoritatively verify any transaction without external reference. Some nodes maintain only a subset of the blockchain and verify transactions using a method called _Simple Payment Verification_ or SPV. These nodes are known as SPV or Lightweight nodes. In the full node example above, the full node blockchain database function is indicated by a blue circle named "Full Blockchain". SPV nodes are drawn without the blue circle, showing that they do not have a full copy of the blockchain.
Some nodes, called full nodes, also maintain a complete and up-to-date copy of the blockchain. Full nodes can autonomously and authoritatively verify any transaction without external reference. Some nodes maintain only a subset of the blockchain and verify transactions using a method called _Simplified Payment Verification_ or SPV. These nodes are known as SPV or Lightweight nodes. In the full node example above, the full node blockchain database function is indicated by a blue circle named "Full Blockchain". SPV nodes are drawn without the blue circle, showing that they do not have a full copy of the blockchain.
Mining nodes compete to create new blocks by running specialized hardware to solve the proof-of-work algorithm. Some mining nodes are also full nodes, maintaining a full copy of the blockchain while others are lightweight nodes participating in pool mining and depending on a pool server to maintain a full node. The mining function is shown in the full node above as a black circle named "Miner".
@ -81,8 +80,13 @@ image::images/AddressPropagation.png["AddressPropagation"]
A node must connect to a few different peers in order to establish diverse paths into the bitcoin network. Paths are not reliable, nodes come and go, and so the node must continue to discover new nodes as it loses old connections as well as assist other nodes when they bootstrap. Only one connection is needed to bootstrap, as the first node can offer introductions to its peer nodes and those peers can offer further introductions. It's also unnecessary and wasteful of network resources to connect to more than a handful of nodes. After bootstrapping, a node will remember its most recent successful peer connections, so that if it is rebooted it can quickly reestablish connections with its former peer network. If none of the former peers respond to its connection request, the node can use the seed nodes to bootstrap again.
On a node running the Bitcoin Core client, you can list the peer connections with the command +getpeerinfo+:
[source,bash]
----
$ bitcoin-cli getpeerinfo
----
[source,json]
----
[
{
"addr" : "85.213.199.39:8333",
@ -129,7 +133,7 @@ Full nodes are nodes that maintain a full blockchain with all transactions. More
Full blockchain nodes maintain a complete and up-to-date copy of the bitcoin blockchain with all the transactions, which they independently build and verify, starting with the very first block (genesis block) and building up to the latest known block in the network. A full blockchain node can independently and authoritatively verify any transaction without recourse or reliance on any other node or source of information. The full blockchain node relies on the network to receive updates about new blocks of transactions, which it then verifies and incorporates into its local copy of the blockchain.
Running a full blockchain node gives you the pure bitcoin experience: independent verification of all transactions without the need to rely on, or trust, any other systems. It's easy to tell if you're running a full node because it requires several gigabytes of persistent storage (disk space) to store the full blockchain. If you need a lot of disk and it takes 2-3 days to "sync" to the network, you are running a full node. That is the price of complete independence and freedom from central authority.
Running a full blockchain node gives you the pure bitcoin experience: independent verification of all transactions without the need to rely on, or trust, any other systems. It's easy to tell if you're running a full node because it requires 20 plus gigabytes of persistent storage (disk space) to store the full blockchain. If you need a lot of disk and it takes 2-3 days to "sync" to the network, you are running a full node. That is the price of complete independence and freedom from central authority.
There are a few alternative implementations of full-blockchain bitcoin clients, built using different programming languages and software architectures. However, the most common implementation is the reference client Bitcoin Core, also known as the Satoshi Client. More than 90% of the nodes on the bitcoin network run various versions of Bitcoin Core. It is identified as "Satoshi" in the sub-version string sent in the +version+ message and shown by the command +getpeerinfo+ as we saw above, for example +/Satoshi:0.8.6/+.
@ -149,15 +153,16 @@ This process of comparing the local blockchain with the peers and retrieving any
.Node synchronizing the blockchain by retrieving blocks from a peer
image::images/InventorySynchronization.png["InventorySynchronization"]
=== Simple Payment Verification (SPV) Nodes
[[spv_nodes]]
=== Simplified Payment Verification (SPV) Nodes
Not all nodes have the ability to store the full blockchain. Many bitcoin clients are designed to run on space- and power-constrained devices, such as smartphones, tablets or embedded systems. For such devices, a _simple payment verification_ (SPV) method is used to allow them to operate without storing the full blockchain. These types of clients are called SPV clients or lightweight clients. As bitcoin adoption surges, the SPV node is becoming the most common form of bitcoin node, especially for bitcoin wallets.
Not all nodes have the ability to store the full blockchain. Many bitcoin clients are designed to run on space- and power-constrained devices, such as smartphones, tablets or embedded systems. For such devices, a _simplified payment verification_ (SPV) method is used to allow them to operate without storing the full blockchain. These types of clients are called SPV clients or lightweight clients. As bitcoin adoption surges, the SPV node is becoming the most common form of bitcoin node, especially for bitcoin wallets.
SPV nodes download only the block headers and do not download the transactions included in each block. The resulting chain of blocks, without transactions, is 1,000 times smaller than the full blockchain. SPV nodes cannot construct a full picture of all the UTXOs that are available for spending, as they do not know about all the transactions on the network. SPV nodes verify transactions using a slightly different methodology that relies on peers to provide partial views of relevant parts of the blockchain on-demand.
As an analogy, a full node is like a tourist in a strange city, equipped with a detailed map of every street and every address. By comparison, an SPV node is like a tourist in a strange city asking random strangers for turn-by-turn directions while knowing only one main avenue. While both tourists can verify the existence of a street by visiting it, the tourist without a map doesn't know what lies down any of the side streets and doesn't know what other streets exist. Positioned in front of 23 Church Street, the tourist without a map cannot know if there are a dozen other "23 Church Street" addresses in the city and whether this is the right one. The map-less tourist's best chance is to ask enough people and hope some of them are not trying to mug the tourist.
Simple Payment Verification verifies transactions by reference to their _depth_ in the blockchain instead of their _height_. Whereas a full-blockchain node will construct a fully verified chain of thousands of blocks and transactions reaching down the blockchain (back in time) all the way to the genesis block, an SPV node will verify the chain of all blocks and link that chain to the transaction of interest.
Simplified Payment Verification verifies transactions by reference to their _depth_ in the blockchain instead of their _height_. Whereas a full-blockchain node will construct a fully verified chain of thousands of blocks and transactions reaching down the blockchain (back in time) all the way to the genesis block, an SPV node will verify the chain of all blocks and link that chain to the transaction of interest.
For example, when examining a transaction in block 300,000, a full node links all 300,000 blocks down to the genesis block and builds a full database of UTXO, establishing the validity of the transaction by confirming that the UTXO remains unspent. An SPV node cannot validate whether the UTXO is unspent. Instead, the SPV node will establish a link between the transaction and the block that contains it, using a Merkle Path (see <<merkle_trees>>). Then, the SPV node waits until it sees the six blocks 300,001 through 300,006 piled on top of the block containing the transaction and verifies it by establishing its depth under blocks 300,006 to 300,001. The fact that other nodes on the network accepted block 300,000 and then did the necessary work to produce 6 more blocks on top of it is proof, by proxy, that the transaction was not a double-spend.

59
ch07.asciidoc
View File

@ -1,8 +1,9 @@
[[ch7]]
[[blockchain]]
== The Blockchain
[[blockchain]]
=== The Blockchain
=== Introduction
The blockchain data structure is an ordered back-linked list of blocks of transactions. The blockchain can be stored as a flat file, or in a simple database. The bitcoin core client stores the blockchain metadata using Google's LevelDB database. Blocks are linked "back", each referring to the previous block in the chain. The blockchain is often visualized as a vertical stack, with blocks layered on top of each other and the first block ever serving as the foundation of the stack. The visualization of blocks stacked on top of each other results in the use of terms like "height" to refer to the distance from the first block, and "top" or "tip" to refer to the most recently added block.
@ -72,6 +73,7 @@ The first block in the blockchain is called the _genesis block_ and was created
Every node always starts with a blockchain of at least one block because the genesis block is statically encoded within the bitcoin client software, such that it cannot be altered. Every node always "knows" the genesis block's hash and structure, the fixed time it was created and even the single transaction within. Thus, every node has the starting point for the blockchain, a secure "root" from which to build a trusted blockchain.
See the statically encoded genesis block inside the Bitcoin Core client, in chainparams.cpp:
https://github.com/bitcoin/bitcoin/blob/3955c3940eff83518c186facfec6f50545b5aab5/src/chainparams.cpp#L123
The genesis block has the identifier hash +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+. You can search for that block hash in any block explorer website, such as blockchain.info, and you will find a page describing the contents of this block, with a URL containing that hash:
@ -82,8 +84,12 @@ https://blockexplorer.com/block/000000000019d6689c085ae165831e934ff763ae46a2a6c1
Using the Bitcoin Core reference client on the command-line:
[source,bash]
----
$ bitcoind getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
----
[source,json]
----
{
"hash" : "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
"confirmations" : 308321,
@ -111,6 +117,8 @@ Bitcoin nodes maintain a local copy of the blockchain, starting at the genesis b
Let's assume, for example, that a node has 277,314 blocks in the local copy of the blockchain. The last block the node knows about is block 277,314, with a block header hash of +00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249+.
The bitcoin node then receives a new block from the network, which it parses as follows:
[source,json]
----
{
"size" : 43560,
@ -125,7 +133,7 @@ The bitcoin node then receives a new block from the network, which it parses as
"tx" : [
"257e7497fb8bc68421eb2c7b699dbab234831600e7352f0d9e6522c7cf3f6c77",
[... many more transactions omitted ...]
#[... many more transactions omitted ...]
"05cfd38f6ae6aa83674cc99e4d75a1458c165b7ab84725eda41d018a09176634"
]
@ -145,7 +153,7 @@ Each block in the bitcoin blockchain contains a summary of all the transactions
A _Merkle Tree_, also known as a _Binary Hash Tree_ is a data structure used for efficiently summarizing and verifying the integrity of large sets of data. Merkle Trees are binary trees containing cryptographic hashes. The term "tree" is used in computer science to describe a branching data structure, but these trees are usually displayed upside down with the "root" at the top and the "leaves" at the bottom of a diagram, as you will see in the examples that follow.
Merkle trees are used in bitcoin to summarize all the transactions in a block, producing an overall digital fingerprint of the entire set of transactions, which can be used to prove that a transaction is included in the set. A merkle tree is constructed by recursively hashing pairs of nodes until there is only one hash, called the _root_, or _merkle root_. The cryptographic hash algorithm used in bitcoin's merkle trees is SHA256 applied twice, also known as double-SHA256.
Merkle trees are used in bitcoin to summarize all the transactions in a block, producing an overall digital fingerprint of the entire set of transactions, providing a very efficient process to verify if a transaction is included in a block. A merkle tree is constructed by recursively hashing pairs of nodes until there is only one hash, called the _root_, or _merkle root_. The cryptographic hash algorithm used in bitcoin's merkle trees is SHA256 applied twice, also known as double-SHA256.
When N data elements are hashed and summarized in a Merkle Tree, you can check to see if any one data element is included in the tree with at most +2*log~2~(N)+ calculations, making this a very efficient data structure.
@ -175,12 +183,47 @@ The same method for constructing a tree from four transactions can be generalize
.A Merkle Tree summarizing many data elements
image::images/MerkleTreeLarge.png["merkle_tree_large"]
To prove that a specific transaction is included in a block, a node only needs to produce +log~2~(N)+ 32-byte hashes, constituting an _authentication path_ or _merkle path_ connecting the specific transaction to the root of the tree. This is especially important as the number of transactions increases, because the base-2 logarithm of the number of transactions increases much more slowly. This allows bitcoin nodes to efficiently produce paths of ten or twelve hashes (320-384 bytes) which can provide proof of a single transaction out of more than a thousand transactions in a megabyte sized block. In the example below, a node can prove that a transaction K is included in the block by producing a merkle path that is only four 32-byte hashes long (128 bytes total). The path consists of the four hashes H~L~, H~IJ~, H~MNOP~ and H~ABCDEFGH~. With those four hashes provided as an authentication path, any node can prove that H~K~ is included in the merkle root by computing four additional pair-wise hashes H~KL~, H~IJKL~ and H~IJKLMNOP~ that lead to the merkle root.
To prove that a specific transaction is included in a block, a node only needs to produce +log~2~(N)+ 32-byte hashes, constituting an _authentication path_ or _merkle path_ connecting the specific transaction to the root of the tree. This is especially important as the number of transactions increases, because the base-2 logarithm of the number of transactions increases much more slowly. This allows bitcoin nodes to efficiently produce paths of ten or twelve hashes (320-384 bytes) which can provide proof of a single transaction out of more than a thousand transactions in a megabyte sized block. In the example below, a node can prove that a transaction K is included in the block by producing a merkle path that is only four 32-byte hashes long (128 bytes total). The path consists of the four hashes (noted in blue in the diagram below) H~L~, H~IJ~, H~MNOP~ and H~ABCDEFGH~. With those four hashes provided as an authentication path, any node can prove that H~K~ (noted in green in the diagram below) is included in the merkle root by computing four additional pair-wise hashes H~KL~, H~IJKL~ and H~IJKLMNOP~ (outlined in a dotted line in the diagram below) that lead to the merkle root.
[[merkle_tree_path]]
.A Merkle Path used to prove inclusion of a data element
image::images/MerkleTreePathToK.png["merkle_tree_path"]
The code in <<merkle_example>> demonstrates the process of creating a merkle tree from the leaf-node hashes up to the root, using the libbitcoin library for some helper functions:
[[merkle_example]]
.Building a merkle tree
====
[source, cpp]
----
include::code/merkle.cpp[]
----
====
Compiling and running the merkle code:
[[merkle_example_run]]
.Compiling and running the merkle example code
====
[source,bash]
----
$ # Compile the merkle.cpp code
$ g++ -o merkle merkle.cpp $(pkg-config --cflags --libs libbitcoin)
$ # Run the merkle executable
$ ./merkle
Current merkle hash list:
32650049a0418e4380db0af81788635d8b65424d397170b8499cdc28c4d27006
30861db96905c8dc8b99398ca1cd5bd5b84ac3264a4e1b3e65afa1bcee7540c4
Current merkle hash list:
d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
Result: d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
----
====
The efficiency of merkle trees becomes obvious as the scale increases. For example, proving that a transaction is part of a block requires:
[[block_structure]]
@ -194,11 +237,11 @@ The efficiency of merkle trees becomes obvious as the scale increases. For examp
| 65,535 transactions | 16 megabytes | 16 hashes | 512 bytes
|=======
As you can see from the table above, while the block size increases rapidly, from 4KB with 16 transactions to a block size of 16 MB to fit 65,535 transactions, the merkle path required to prove the inclusion of a transaction increases much more slowly, from 128 bytes to only 512 bytes. With merkle trees, a node can download just the block headers (80 bytes per block) and still be able to identify a transaction's inclusion in a block by retrieving a small merkle path from a full node, without storing or transmitting the vast majority of the blockchain which may be several gigabytes in size. Nodes which do not maintain a full blockchain, called Simple Payment Verification or SPV nodes use merkle paths to verify transactions without downloading full blocks.
As you can see from the table above, while the block size increases rapidly, from 4KB with 16 transactions to a block size of 16 MB to fit 65,535 transactions, the merkle path required to prove the inclusion of a transaction increases much more slowly, from 128 bytes to only 512 bytes. With merkle trees, a node can download just the block headers (80 bytes per block) and still be able to identify a transaction's inclusion in a block by retrieving a small merkle path from a full node, without storing or transmitting the vast majority of the blockchain which may be several gigabytes in size. Nodes which do not maintain a full blockchain, called Simplified Payment Verification or SPV nodes use merkle paths to verify transactions without downloading full blocks.
=== Merkle Trees and Simple Payment Verification (SPV)
=== Merkle Trees and Simplified Payment Verification (SPV)
Merkle trees are used extensively by Simple Payment Verification nodes. SPV nodes don't have all transactions and do not download full blocks, just block headers. In order to verify that a transaction is included in a block, without having to download all the transactions in the block, they use an _authentication path_, or merkle path.
Merkle trees are used extensively by Simplified Payment Verification nodes. SPV nodes don't have all transactions and do not download full blocks, just block headers. In order to verify that a transaction is included in a block, without having to download all the transactions in the block, they use an _authentication path_, or merkle path.
Consider for example an SPV node that is interested in incoming payments to an address contained in its wallet. The SPV node will establish a bloom filter on its connections to peers to limit the transactions received to only those containing addresses of interest. When a peer sees a transaction that matches the bloom filter, it will send that block using a +merkleblock+ message. The +merkleblock+ message contains the block header as well as a merkle path that links the transaction of interest to the merkle root in the block. The SPV node can use this merkle path to connect the transaction to the block and verify that the transaction is included in the block. The SPV node also uses the block header to link the block to the rest of the blockchain. The combination of these two links, between the transaction and block, and between the block and blockchain, proves that the transaction is recorded in the blockchain. All in all, the SPV node will have received less than a kilobyte of data for the block header and merkle path, an amount of data that is more than a thousand times less than a full block (about 1 megabyte currently).

188
ch08.asciidoc
View File

@ -2,21 +2,52 @@
== Mining and Consensus
[[mining]]
=== Introduction - Mining and Consensus
=== Introduction
Mining is the process by which new bitcoin is added to the money supply. Mining also serves to secure the bitcoin system against fraudulent transactions or transactions spending the same amount of bitcoin more than once, known as a double-spend. Miners act as a decentralized clearinghouse, validating new transactions and recording them on the global ledger. A new block, containing transactions that occurred since the last block, is "mined" every 10 minutes, thereby adding those transactions to the blockchain. Transactions that become part of a block and added to the blockchain are considered "confirmed", which allows the new owners of bitcoin to spend the bitcoin they received in those transactions. Miners receive two types of reward for mining: new coins created with each new block and transaction fees from all the transactions included in the block. To earn this reward, the miners compete to solve a difficult mathematical problem based on a cryptographic hash algorithm. The solution to the problem, called the Proof-of-Work, is included in the new block and acts as proof that the miner expended significant computing effort. The competition to solve the Proof-of-Work algorithm to earn reward and the right to record transactions on the blockchain is the basis for bitcoin's security model.
Mining is the process by which new bitcoin is added to the money supply. Mining also serves to secure the bitcoin system against fraudulent transactions or transactions spending the same amount of bitcoin more than once, known as a double-spend. Miners provide processing power to the bitcoin network in exchange for the opportunity to be rewarded bitcoin.
The process of new coin generation is called mining, because the reward is designed to simulate diminishing returns, just like mining for precious metals. Bitcoin's money supply is created through mining, similar to how a central bank issues new money by printing bank notes. The amount of newly created bitcoin a miner can add to a block decreases approximately every four years (or precisely every 210,000 blocks). It started at 50 bitcoin per block in January of 2009 and halved to 25 bitcoin per block in November of 2012. It will halve again to 12.5 bitcoin per block sometime in 2016. Based on this formula, bitcoin mining rewards decrease exponentially until approximately the year 2140 when all bitcoin (20.99999998 million) will have been issued. After 2140, no new bitcoins are issued.
Miners validate new transactions and record them on the global ledger. A new block, containing transactions that occurred since the last block, is "mined" every 10 minutes, thereby adding those transactions to the blockchain. Transactions that become part of a block and added to the blockchain are considered "confirmed", which allows the new owners of bitcoin to spend the bitcoin they received in those transactions.
Bitcoin miners also earn fees from transactions. Every transaction may include a transaction fee, in the form of a surplus of bitcoin between the transaction's inputs and outputs. The winning bitcoin miner gets to "keep the change" on the transactions included in the winning block. Today the fees represent 0.5% or less of a bitcoin miner's income, the vast majority coming from the newly minted bitcoins. However, as the reward decreases over time and the number of transactions per block increases, a greater proportion of bitcoin mining earnings will come from fees. After 2140 all bitcoin miner earnings will be in the form of transaction fees.
Miners receive two types of reward for mining: new coins created with each new block and transaction fees from all the transactions included in the block. To earn this reward, the miners compete to solve a difficult mathematical problem based on a cryptographic hash algorithm. The solution to the problem, called the Proof-of-Work, is included in the new block and acts as proof that the miner expended significant computing effort. The competition to solve the Proof-of-Work algorithm to earn reward and the right to record transactions on the blockchain is the basis for bitcoin's security model.
The word "mining" is somewhat misleading. By evoking the extraction of precious metals, it focuses our attention on the reward for mining, the new bitcoins in each block. While mining is incentivized by this reward, the primary purpose of mining is not the reward or the generation of new coins. If you view mining only as the process by which coins are created you are mistaking the means (incentives) as a goal of the process. Mining is the main process of the de-centralized clearinghouse, by which transactions are validated and cleared. Mining secures the bitcoin system and enables the emergence of network-wide consensus without a central authority. Mining is the invention that makes bitcoin special, a de-centralized security mechanism that is the basis for peer-to-peer digital cash. The reward of newly minted coins and transaction fees is an incentive scheme that aligns the actions of miners with the security of the network, while simultaneously implementing the monetary supply.
The process of new coin generation is called mining, because the reward is designed to simulate diminishing returns, just like mining for precious metals. Bitcoin's money supply is created through mining, similar to how a central bank issues new money by printing bank notes. The amount of newly created bitcoin a miner can add to a block decreases approximately every four years (or precisely every 210,000 blocks). It started at 50 bitcoin per block in January of 2009 and halved to 25 bitcoin per block in November of 2012. It will halve again to 12.5 bitcoin per block sometime in 2016. Based on this formula, bitcoin mining rewards decrease exponentially until approximately the year 2140 when all bitcoin (20.99999998 million) will have been issued. After 2140, no new bitcoins will be issued.
Bitcoin miners also earn fees from transactions. Every transaction may include a transaction fee, in the form of a surplus of bitcoin between the transaction's inputs and outputs. The winning bitcoin miner gets to "keep the change" on the transactions included in the winning block. Today, the fees represent 0.5% or less of a bitcoin miner's income, the vast majority coming from the newly minted bitcoins. However, as the reward decreases over time and the number of transactions per block increases, a greater proportion of bitcoin mining earnings will come from fees. After 2140, all bitcoin miner earnings will be in the form of transaction fees.
The word "mining" is somewhat misleading. By evoking the extraction of precious metals, it focuses our attention on the reward for mining, the new bitcoins in each block. While mining is incentivized by this reward, the primary purpose of mining is not the reward or the generation of new coins. If you view mining only as the process by which coins are created you are mistaking the means (incentives) as a goal of the process. Mining is the main process of the de-centralized clearinghouse, by which transactions are validated and cleared. Mining secures the bitcoin system and enables the emergence of network-wide consensus without a central authority.
Mining is the invention that makes bitcoin special, a de-centralized security mechanism that is the basis for peer-to-peer digital cash. The reward of newly minted coins and transaction fees is an incentive scheme that aligns the actions of miners with the security of the network, while simultaneously implementing the monetary supply.
In this chapter, we will first examine mining as a monetary supply mechanism and then look at the most important function of mining, the de-centralized emergent consensus mechanism that underpins bitcoin's security.
==== Bitcoin Economics and Currency Creation
Bitcoins are "minted" during the creation of each block at a fixed and diminishing rate. Each block, generated on average every 10 minutes, contains entirely new bitcoins, created ex nihilo (from nothing). Every 210,000 blocks or approximately every four years the currency issuance rate is decreased by 50%. For the first four years of operation of the network, each block contained 50 new bitcoin. In November of 2012, the new bitcoin issuance rate was decreased to 25 bitcoin per block and it will decrease again to 12.5 bitcoin at block 420,000, which will be mined sometime in 2016. The rate of new coins decreases like this exponentially over 64 "halvings", until block 13,230,000 (mined approximately in year 2137) when it reaches the minimum currency unit of 1 satoshi. Finally, after 13.44 million blocks, in approximately 2140, all 2,099,999,997,690,000 satoshis, or almost 21 million bitcoin will be issued. Thereafter, blocks will contain no new bitcoin, and miners will be rewarded solely through the transaction fees.
Bitcoins are "minted" during the creation of each block at a fixed and diminishing rate. Each block, generated on average every 10 minutes, contains entirely new bitcoins, created from nothing. Every 210,000 blocks or approximately every four years the currency issuance rate is decreased by 50%. For the first four years of operation of the network, each block contained 50 new bitcoin.
In November of 2012, the new bitcoin issuance rate was decreased to 25 bitcoin per block and it will decrease again to 12.5 bitcoin at block 420,000, which will be mined sometime in 2016. The rate of new coins decreases like this exponentially over 64 "halvings", until block 13,230,000 (mined approximately in year 2137) when it reaches the minimum currency unit of 1 satoshi. Finally, after 13.44 million blocks, in approximately 2140, all 2,099,999,997,690,000 satoshis, or almost 21 million bitcoin will be issued. Thereafter, blocks will contain no new bitcoin, and miners will be rewarded solely through the transaction fees.
In the example code in <<max_money_>>, we calculate the total amount of bitcoin that will be issued:
[[max_money]]
.A script for calculating how much total bitcoin will be issued
====
[source, python]
----
include::code/max_money.py[]
----
====
Running the script:
[[max_money_run]]
.Running the max_money.py script
====
[source,bash]
----
$ python max_money.py
Total BTC to ever be created: 2099999997690000 Satoshis
----
====
[[bitcoin_money_supply]]
.Supply of bitcoin currency over time based on a geometrically decreasing issuance rate
@ -24,8 +55,8 @@ image::images/BitcoinMoneySupply.png["BitcoinMoneySupply"]
The finite and diminishing issuance creates a fixed monetary supply that resists inflation. Unlike a fiat currency, which can be printed in infinite numbers by a central bank, bitcoin can never be inflated by printing.
===== Deflationary Money
.Deflationary Money
****
The most important and debated consequence of a fixed and diminishing monetary issuance is that the currency will tend to be inherently _deflationary_. Deflation is the phenomenon of appreciation of value due to a mismatch in supply and demand that drives up the value (and exchange rate) of a currency. The opposite of inflation, price deflation means that the money has more purchasing power over time.
Many economists argue that a deflationary economy is a disaster that should be avoided at all costs. That is because in a period of rapid deflation people will tend to hoard money instead of spending it, hoping that prices will fall. Such a phenomenon unfolded during Japan's "Lost Decade", when a complete collapse of demand pushed the currency into a deflationary spiral.
@ -33,6 +64,7 @@ Many economists argue that a deflationary economy is a disaster that should be a
Bitcoin experts argue that deflation is not bad *per se*. Rather, deflation is associated with a collapse in demand because that is the only example of deflation we have to study. In a fiat currency with the possibility of unlimited printing, it is very difficult to enter a deflationary spiral unless there is a complete collapse in demand and an unwillingness to print money. Deflation in bitcoin is not caused by a collapse in demand, but by a predictably constrained supply.
In practice, it has become evident that the hoarding instinct caused by a deflationary currency can be overcome by discounting from vendors, until the discount overcomes the hoarding instinct of the buyer. Since the seller is also motivated to hoard, the discount becomes the equilibrium price at which the two hoarding instincts are matched. With discounts of 30% on the bitcoin price, most bitcoin retailers are not experiencing difficulty overcoming the hoarding instinct and generating revenue. It remains to be seen whether the deflationary aspect of the currency is really a problem when it is not driven by rapid economic retraction.
****
=== De-centralized Consensus
@ -60,24 +92,24 @@ However, before forwarding transactions to its neighbors, every bitcoin node tha
Each node verifies every transaction against a long checklist of criteria:
* Check the syntactic correctness of the transaction's data structure
* Make sure neither lists of inputs or outputs are empty
* The transaction's syntax and data structure must be correct
* Neither lists of inputs or outputs are empty
* The transaction size in bytes is less than MAX_BLOCK_SIZE
* Each output value, as well as the total, must be within the allowed range of values (less than 21m coins, more than 0)
* Check that none of the inputs have hash=0, N=-1 (coinbase transactions should not be relayed)
* Check that nLockTime is less than or equal to INT_MAX
* Check that the transaction size in bytes is greater than or equal to 100
* Check that the number of signature operations contained in the transaction is less than the signature operation limit
* Reject "nonstandard" transactions: unlocking script (scriptSig) doing anything other than pushing numbers on the stack, or the locking script (scriptPubkey) not matching isStandard forms
* Check for a matching transaction in the pool, or in a block in the main branch, if so reject this transaction
* None of the inputs have hash=0, N=-1 (coinbase transactions should not be relayed)
* nLockTime is less than or equal to INT_MAX
* The transaction size in bytes is greater than or equal to 100
* The number of signature operations contained in the transaction is less than the signature operation limit
* The unlocking script (scriptSig) can only push numbers on the stack, and the locking script (scriptPubkey) must match isStandard forms (this rejects "nonstandard" transactions)
* A matching transaction in the pool, or in a block in the main branch, must exist
* For each input, if the referenced output exists in any other transaction in the pool, reject this transaction
* For each input, look in the main branch and the transaction pool to find the referenced output transaction. If the output transaction is missing for any input, this will be an orphan transaction. Add to the orphan transactions, if a matching transaction is not already in the pool
* For each input, if the referenced output transaction is a coinbase output, it must have at least COINBASE_MATURITY (100) confirmations; else reject this transaction
* For each input, if the referenced output does not exist (e.g. never existed or has already been spent), reject this transaction
* For each input, look in the main branch and the transaction pool to find the referenced output transaction. If the output transaction is missing for any input, this will be an orphan transaction. Add to the orphan transactions pool, if a matching transaction is not already in the pool
* For each input, if the referenced output transaction is a coinbase output, it must have at least COINBASE_MATURITY (100) confirmations
* For each input, the referenced output must exist and cannot already be spent
* Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in the allowed range of values (less than 21m coins, more than 0)
* Reject if the sum of input values < sum of output values
* Reject if transaction fee would be too low to get into an empty block
* Verify the unlocking scripts for each input against the corresponding output locking scripts
* The unlocking scripts for each input must validate against the corresponding output locking scripts
These conditions can be seen in detail in the functions +AcceptToMemoryPool+, +CheckTransaction+, and +CheckInputs+ in the bitcoin reference client. Note that the conditions change over time, to address new types of Denial-of-Service attacks or sometimes to relax the rules so as to include more types of transactions.
@ -95,7 +127,7 @@ After validating transactions, a bitcoin node will add them to the _memory pool_
Let's follow the blocks that were created during the time Alice bought a cup of coffee from Bob's Cafe (see <<cup_of_coffee>>). Alice's transaction was included in block 277,316. For the purpose of demonstrating the concepts in this chapter let's assume that block was mined by Jing's mining system and follow Alice's transaction as it becomes part of this new block.
Jing's mining node maintains a local copy of the blockchain, the list of all blocks created since the beginning of the bitcoin system in 2009. By the time Alice buys the cup of coffee, Jing's node has assembled a chain of 277,314 blocks. Jing's node is listening for transactions, trying to mine a new block and also listening for blocks discovered by other nodes. As Jing's node is mining, it receives block 277,315 through the bitcoin network. The arrival of this block signifies the end of the competition for block 277,315 and the beginning of the competition to create block 277,316.
Jing's mining node maintains a local copy of the blockchain, the list of all blocks created since the beginning of the bitcoin system in 2009. By the time Alice buys the cup of coffee, Jing's node has assembled a chain up to block 277,314. Jing's node is listening for transactions, trying to mine a new block and also listening for blocks discovered by other nodes. As Jing's node is mining, it receives block 277,315 through the bitcoin network. The arrival of this block signifies the end of the competition for block 277,315 and the beginning of the competition to create block 277,316.
During the previous 10 minutes, while Jing's node was searching for a solution to block 277,315, it was also collecting transactions in preparation for the next block. By now it has collected a few hundred transactions in the memory pool. Upon receiving block 277,315 and validating it, Jing's node will also check all the transactions in the memory pool and remove any that were included in block 277,315. Whatever transaction remain in the memory pool are unconfirmed and are waiting to be recorded in a new block.
@ -176,7 +208,8 @@ The first transaction added to the block is a special transaction, called a _gen
====
[source, bash]
----
$ bitcoin-cli getrawtransaction d5ada064c6417ca25c4308bd158c34b77e1c0eca2a73cda16c737e7424afba2f 1
$ bitcoin-cli getrawtransaction\
d5ada064c6417ca25c4308bd158c34b77e1c0eca2a73cda16c737e7424afba2f 1
----
====
@ -186,7 +219,9 @@ $ bitcoin-cli getrawtransaction d5ada064c6417ca25c4308bd158c34b77e1c0eca2a73cda1
[source,json]
----
{
"hex" : "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff0f03443b0403858402062f503253482fffffffff0110c08d9500000000232102aa970c592640d19de03ff6f329d6fd2eecb023263b9ba5d1b81c29b523da8b21ac00000000",
"hex" : "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff\
0f03443b0403858402062f503253482fffffffff0110c08d9500000000232102aa970c592640d19de03ff6f329d\
6fd2eecb023263b9ba5d1b81c29b523da8b21ac00000000",
"txid" : "d5ada064c6417ca25c4308bd158c34b77e1c0eca2a73cda16c737e7424afba2f",
"version" : 1,
"locktime" : 0,
@ -201,7 +236,8 @@ $ bitcoin-cli getrawtransaction d5ada064c6417ca25c4308bd158c34b77e1c0eca2a73cda1
"value" : 25.09094928,
"n" : 0,
"scriptPubKey" : {
"asm" : "02aa970c592640d19de03ff6f329d6fd2eecb023263b9ba5d1b81c29b523da8b21 OP_CHECKSIG",
"asm" : "02aa970c592640d19de03ff6f329d6fd2eecb023263b9ba5d1b81c29b523da8b21\
OP_CHECKSIG",
"hex" : "2102aa970c592640d19de03ff6f329d6fd2eecb023263b9ba5d1b81c29b523da8b21ac",
"reqSigs" : 1,
"type" : "pubkey",
@ -305,7 +341,7 @@ In a generation transaction, the first two fields are set to values that do not
Generation transactions do not have an unlocking script (a.k.a. scriptSig) field. Instead, this field is replaced by coinbase data, which must be between 2 and 100 bytes. Except for the first few bytes (see below) the rest of the coinbase data can be used by miners in any way they want; it is arbitrary data.
In the genesis block, for example, Satoshi Nakamoto added the text "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks" in the coinbase data, using it as a proof of the date and to convey a message. Currently, miners use the coinbase data to include extra nonce values (see <<mining>>) and strings identifying the mining pool, as we will see in the following sections.
In the genesis block, for example, Satoshi Nakamoto added the text "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks" in the coinbase data, using it as a proof of the date and to convey a message. Currently, miners use the coinbase data to include extra nonce values and strings identifying the mining pool, as we will see in the following sections.
The first few bytes of the coinbase used to be arbitrary, but that is no longer the case. As per Bitcoin Improvement Proposal 34 (BIP0034), version-2 blocks (blocks with the version field set to 2) must contain the block height index as a script "push" operation in the beginning of the coinbase field.
@ -313,10 +349,36 @@ In block 277,316 we see that the coinbase (see <<generation_tx_example>>), which
The first byte, +03+ instructs the script execution engine to push the next 3 bytes onto the script stack (see <<tx_script_ops_table_pushdata>>). The next 3 bytes, +0x443b04+, are the block height encoded in little-endian format (backwards, least significant byte first). Reverse the order of the bytes and the result is +0x043b44+ which is 277,316 in decimal.
The next few hexadecimal digits (+03858402062+) are used to encode an extra _nonce_, or random value, used to find a suitable Proof-of-Work solution. This is discussed in more detail in the next section on <<mining>>
The next few hexadecimal digits (+03858402062+) are used to encode an extra _nonce_ (See <<extra_nonce_>>), or random value, used to find a suitable Proof-of-Work solution.
The final part of the coinbase data (+2f503253482f+) is the ASCII-encoded string "/P2SH/", which indicates that the mining node that mined this block supports the Pay-to-Script-Hash (P2SH) improvement defined in BIP0016. The introduction of the P2SH capability required a "vote" by miners to endorse either BIP0016 or BIP0017. Those endorsing the BIP0016 implementation were to include "/P2SH/" in their coinbase data. Those endorsing the BIP0017 implementation of P2SH were to include the string "p2sh/CHV" in their coinbase data. The BIP0016 was elected as the winner, and many miners continued including the string "/P2SH/" in their coinbase to indicate support for this feature.
The code example <<satoshi_words>> uses the libbitcoin library introduced in <<alt_libraries>> to extract the coinbase data from the genesis block, displaying Satoshi's message. Note that the libbitcoin library contains a static copy of the genesis block, so the example code below can use retrieve the genesis block directly from the library.
[[satoshi_words]]
.Extract the coinbase data from the genesis block
====
[source, cpp]
----
include::code/satoshi-words.cpp[]
----
====
We compile the code with the GNU C++ compiler and run the resulting executable:
[[satoshi_words_run]]
.Compiling and running the satoshi-words example code
====
[source,bash]
----
$ # Compile the code
$ g++ -o satoshi-words satoshi-words.cpp $(pkg-config --cflags --libs libbitcoin)
$ # Run the executable
$ ./satoshi-words
^D<><44><GS>^A^DEThe Times 03/Jan/2009 Chancellor on brink of second bailout for banks
----
====
=== Constructing the Block Header
To construct the block header, the mining node needs to fill in six fields:
@ -356,7 +418,7 @@ In the simplest terms, mining is the process of hashing the block header repeate
==== Proof-of-Work Algorithm
A hash algorithm takes an arbitrary-length data input and produces a fixed-length deterministic result, a digital fingerprint of the input. For any specific input, the resulting hash will always be the same and can be easily calculated and verified by anyone implementing the same hash algorithm. The key characteristic of a cryptographic hash algorithm is that it is impossible to find two different inputs that produce the same fingerprint. As a corollary, it is also impossible to select an input in such a way as to produce a desired fingerprint, other than trying random inputs.
A hash algorithm takes an arbitrary-length data input and produces a fixed-length deterministic result, a digital fingerprint of the input. For any specific input, the resulting hash will always be the same and can be easily calculated and verified by anyone implementing the same hash algorithm. The key characteristic of a cryptographic hash algorithm is that it is virtually impossible to find two different inputs that produce the same fingerprint. As a corollary, it is also virtually impossible to select an input in such a way as to produce a desired fingerprint, other than trying random inputs.
With SHA-256, the output is always 256 bits long, regardless of the size of the input. In the example below, we will use the Python interpreter to calculate the SHA256 hash of the phrase "I am Satoshi Nakamoto".
@ -432,7 +494,7 @@ To make a challenge out of this algorithm, let's set an arbitrary target: find a
To give a simple analogy, imagine a game where players throw a pair of dice repeatedly, trying to throw less than a specified target. In the first round, the target is 12. Unless you throw double-six, you win. In the next round the target is 11. Players must throw 10 or less to win, again an easy task. Let's say a few rounds later the target is down to 5. Now, more than half the dice throws will add up to more than 5 and therefore be invalid. It takes exponentially more dice throws to win, the lower the target gets. Eventually, when the target is 2 (the minimum possible), only one throw out of every 36, or 2% of them will produce a winning result.
In the example above, the winning "nonce" is 13 and this result can be confirmed by anyone independently. Anyone can add the number 13 as a suffix to the phrase "I am Satoshi Nakamoto" and compute the hash, verifying that it is less than the target. The successful result is also proof-of-work, as it proves we did the work to find that nonce. While it only takes one hash computation to verify, it took us 13 hash computations to find a nonce that worked. If we had a lower target (higher difficulty) it would take many more hash computations to find a suitable nonce, but only one hash computation for anyone to verify. Furthermore, by knowing the target, anyone can estimate the difficulty using statistics and therefore know how much work was needed to find such a nonce.
In the hashing example above, the winning "nonce" is 13 and this result can be confirmed by anyone independently. Anyone can add the number 13 as a suffix to the phrase "I am Satoshi Nakamoto" and compute the hash, verifying that it is less than the target. The successful result is also proof-of-work, as it proves we did the work to find that nonce. While it only takes one hash computation to verify, it took us 13 hash computations to find a nonce that worked. If we had a lower target (higher difficulty) it would take many more hash computations to find a suitable nonce, but only one hash computation for anyone to verify. Furthermore, by knowing the target, anyone can estimate the difficulty using statistics and therefore know how much work was needed to find such a nonce.
Bitcoin's Proof-of-Work is very similar to the problem above. The miner constructs a candidate block filled with transactions. Next, the miner calculates the hash of this block's header and see if it is smaller than the current _target_. If the hash is not less than the target, the miner will modify the nonce (usually just incrementing it by one) and try again. At the current difficulty in the bitcoin network, miners have to try quadrillions of times before finding a nonce that results in a low enough block header hash.
@ -517,6 +579,7 @@ As you can see, increasing the difficulty by 1 bit causes an exponential increas
At the time of writing this, the network is attempting to find a block whose header hash is less than +000000000000004c296e6376db3a241271f43fd3f5de7ba18986e517a243baa7+. As you can see, there are a lot of zeroes at the beginning of that hash, meaning that the acceptable range of hashes is much smaller, hence more difficult to find a valid hash. It will take on average more than 150 quadrillion hash calculations per second for the network to discover the next block. That seems like an impossible task, but fortunately the network is bringing 100 Peta Hashes per second of processing power to bear, which will be able to find a block in about 10 minutes on average.
[[difficulty_bits]]
==== Difficulty Representation
In <<block277316>> we saw that the block contains the difficulty target, in a notation called "difficulty bits" or just "bits", which in block 277,316 has the value of +0x1903a30c+. This notation expresses the difficulty target as a coefficient/exponent format, with the first two hexadecimal digits for the exponent and the next six hex digits as the coefficient. In this block, therefore, the exponent is +0x19+ and the coefficient is +0x03a30c+.
@ -545,6 +608,7 @@ switching back to hexadecimal:
This means that a valid block for height 277,316 is one that has a block header hash that is less than the target. In binary that number would have more than the first 60 bits set to zero. With this level of difficulty, a single miner processing 1 trillion hashes per second (1 tera-hash per second or 1 TH/sec) would only find a solution once every 8,496 blocks or once every 59 days, on average.
[[difficulty_target]]
==== Difficulty Target and Re-Targeting
As we saw above the target determines the difficulty and therefore affects how long it takes to find a solution to the Proof-of-Work algorithm. This leads to the obvious questions: Why is the difficulty adjustable, who adjusts it and how?
@ -565,30 +629,30 @@ Here's the code used in the Bitcoin Core client
[source,cpp]
----
// Go back by what we want to be 14 days worth of blocks
const CBlockIndex* pindexFirst = pindexLast;
for (int i = 0; pindexFirst && i < Params().Interval()-1; i++)
pindexFirst = pindexFirst->pprev;
assert(pindexFirst);
// Go back by what we want to be 14 days worth of blocks
const CBlockIndex* pindexFirst = pindexLast;
for (int i = 0; pindexFirst && i < Params().Interval()-1; i++)
pindexFirst = pindexFirst->pprev;
assert(pindexFirst);
// Limit adjustment step
int64_t nActualTimespan = pindexLast->GetBlockTime() - pindexFirst->GetBlockTime();
LogPrintf(" nActualTimespan = %d before bounds\n", nActualTimespan);
if (nActualTimespan < Params().TargetTimespan()/4)
nActualTimespan = Params().TargetTimespan()/4;
if (nActualTimespan > Params().TargetTimespan()*4)
nActualTimespan = Params().TargetTimespan()*4;
// Limit adjustment step
int64_t nActualTimespan = pindexLast->GetBlockTime() - pindexFirst->GetBlockTime();
LogPrintf(" nActualTimespan = %d before bounds\n", nActualTimespan);
if (nActualTimespan < Params().TargetTimespan()/4)
nActualTimespan = Params().TargetTimespan()/4;
if (nActualTimespan > Params().TargetTimespan()*4)
nActualTimespan = Params().TargetTimespan()*4;
// Retarget
uint256 bnNew;
uint256 bnOld;
bnNew.SetCompact(pindexLast->nBits);
bnOld = bnNew;
bnNew *= nActualTimespan;
bnNew /= Params().TargetTimespan();
// Retarget
uint256 bnNew;
uint256 bnOld;
bnNew.SetCompact(pindexLast->nBits);
bnOld = bnNew;
bnNew *= nActualTimespan;
bnNew /= Params().TargetTimespan();
if (bnNew > Params().ProofOfWorkLimit())
bnNew = Params().ProofOfWorkLimit();
if (bnNew > Params().ProofOfWorkLimit())
bnNew = Params().ProofOfWorkLimit();
----
====
@ -620,14 +684,14 @@ In the next section we'll look at the process each node uses to validate a block
The third step in bitcoin's consensus mechanism is independent validation of each new block by every node on the network. As the newly solved block moves across the network, each node performs a series of tests to validate it before propagating it to its peers. This ensures that only valid blocks are propagated on the network. The independent validation also ensures that miners who act honestly get their blocks incorporated in the blockchain, thus earning the reward. Those miners who act dishonestly have their blocks rejected and not only lose the reward but also waste the effort expended to find a Proof-of-Work solution, thus incurring the cost of electricity without compensation.
When a node receives a new block, it will validate the block by checking it against a long list of criteria. These criteria can be seen in the Bitcoin Core client in the functions +CheckBlock+ and +CheckBlockHeader+. These criteria include:
When a node receives a new block, it will validate the block by checking it against a long list of criteria that must all be met; otherwise the block is rejected. These criteria can be seen in the Bitcoin Core client in the functions +CheckBlock+ and +CheckBlockHeader+ and include:
* Check the syntactic validity of the block data structure
* Check the Proof-of-Work, by checking the block header hash is less than the target difficulty
* Check the block timestamp is less than two hours in the future (allowing for time errors)
* Check the block size is within acceptable limits
* Check the first transaction (and only the first) is a coinbase generation transaction
* Validate all transactions within the block, using the transaction checklist discussed in <<tx_verification>>
* The block data structure is syntactically valid
* The block header hash is less than the target difficulty (enforces the Proof-of-Work)
* The block timestamp is less than two hours in the future (allowing for time errors)
* The block size is within acceptable limits
* The first transaction (and only the first) is a coinbase generation transaction
* All transactions within the block are valid using the transaction checklist discussed in <<tx_verification>>
The independent validation of each new block by every node on the network ensures that the miners can't cheat. In previous sections we saw how the miners get to write a transaction that awards them the new bitcoins created within the block and claim the transaction fees. Why doesn't the miner write themselves a transaction for a thousand bitcoin instead of the correct reward? Because every node validates blocks according to the same rules. An invalid coinbase transaction would make the entire block invalid, which would result in the block being rejected and therefore that transaction would never become part of the ledger. The miner has to construct a perfect block, based on the shared rules that all nodes follow and mine it with a correct solution to the Proof-of-Work. To do so they expend a lot of electricity in mining and if they cheat all the electricity and effort is wasted. This is why independent validation is a key component of decentralized consensus.
@ -696,8 +760,6 @@ It is theoretically possible for a fork to extend to two blocks, if two blocks a
Bitcoin's block interval of 10 minutes is a design compromise between fast confirmation times (settlement of transactions) and the probability of a fork. A faster block time would make transactions clear faster but lead to more frequent blockchain forks, whereas a slower block time would decrease the number of forks but make settlement slower.
=== Mining and the Hashing Race
Bitcoin mining is an extremely competitive industry. The hashing power has increased exponentially, every year of bitcoin's existence. Some years the growth has reflected a complete change of technology, such as in 2010 and 2011 when many miners switched from using CPU mining to Graphical Processing Unit (GPU) mining and Field Programmable Gate Array (FPGA) mining. In 2013 the introduction of Application Specific Integrated Circuit (ASIC) mining lead to another giant leap in mining power, by placing the SHA-256 function directly on silicon chips specialized for the purpose of mining. The first such chips could deliver more mining power in a single box than the entire bitcoin network in 2010.
@ -723,10 +785,12 @@ image::images/BitcoinDifficulty.png["BitcoinDifficulty"]
In the last two years, the ASIC mining chips have become denser and denser, approaching the cutting edge of silicon fabrication with a feature size (resolution) of 22 nanometers (nm). Currently, ASIC manufacturers are aiming to overtake general purpose CPU chip manufacturers, designing chips with a feature size of 16nm, because the profitability of mining is driving this industry even faster than general computing. There are no more giant leaps left in bitcoin mining, because the industry has reached the forefront of "Moore's Law". Still, the mining power of the network continues to advance at an exponential pace as the race for higher density chips is matched with a race for higher density data centers where thousands of these chips can be deployed. It's no longer about how much mining can be done with one chip but how many chips can be squeezed into a building, while still dissipating the heat and providing adequate power.
[[extra_nonce]]
==== The Extra Nonce Solution
Since 2012 bitcoin mining has evolved to resolve a fundamental limitation in the structure of the block header. In the early days of bitcoin, a miner could find a block by iterating through the nonce until the resulting hash was below the target. As difficulty increased, miners often cycled through all 4 billion values of the nonce without finding a block. However, this was easily resolved by updating the block timestamp to account for the elapsed time. Since the timestamp is part of the header, the change would allow miners to iterate through the values of the nonce again with different results. Once mining hardware exceeded 4 GH/sec however, this approach became increasingly difficult as the nonce values were exhausted in less than a second. As ASIC mining equipment started pushing and then exceeding the TH/sec hash rate, the mining software needed more space for nonce values in order to find valid blocks. The timestamp could be stretched a bit, but moving it too far into the future would cause the block to become invalid. A new source of "change" was needed in the block header. The solution was to use the coinbase transaction as a source of extra nonce values. Since the coinbase script can store between 2 and 100 bytes of data, miners started using that space as extra nonce space, allowing them to explore a much larger range of block header values to find valid blocks. The coinbase transaction is included in the merkle tree, which means that any change in the coinbase script causes the merkle root to change. Eight bytes of extra nonce, plus the 4 bytes of "standard" nonce allow miners to explore a total 2^96^ (8 followed by 28 zeroes) possibilities *per second* without having to modify the timestamp. If, in the future, a miner could run through all these possibilities, they could then modify the timestamp. There is also more space in the coinbase script for future expansion of the extra nonce space.
[[mining_pools]]
==== Mining Pools
In this highly competitive environment, individual miners working alone (also known as solo miners) don't stand a chance. The likelihood of them finding a block to offset their electricity and hardware costs is so low that it represents a gamble, like playing the lottery. Even the fastest consumer ASIC mining system cannot keep up with commercial systems that stack tens of thousands of these chips in giant warehouses near hydro-electric power stations. Miners now collaborate to form mining pools, pooling their hashing power and sharing the reward among thousands of participants. By participating in a pool, miners get a smaller share of the overall reward, but typically get rewarded every day, reducing uncertainty.
@ -773,6 +837,12 @@ It is important to note that consensus attacks can only affect future consensus,
One attack scenario against the consensus mechanism is called the "51% attack". In this scenario a group of miners, controlling a majority (51%) of the total network's hashing power, collude to attack bitcoin. With the ability to mine the majority of the blocks, the attacking miners can cause deliberate "forks" in the blockchain and double-spend transactions or execute denial-of-service attacks against specific transactions or addresses. A fork/double-spend attack is one where the attacker causes previously confirmed blocks to be invalidated by forking below them and re-converging on an alternate chain. With sufficient power, an attacker can invalidate six or more blocks in a row, causing transactions that were considered immutable (6 confirmations) to be invalidated. Note that a double-spend can only be done on the attacker's own transactions, for which the attacker can produce a valid signature. Double-spending one's own transactions is profitable if by invalidating a transaction the attacker can get a non-reversible exchange payment or product without paying for it.
Let's examine a practical example of a 51% attack. In the first chapter we looked at a transaction between Alice and Bob for a cup of coffee. Bob, the cafe owner, is willing to accept payment for cups of coffee without waiting for confirmation (mining in a block), because the risk of a double-spend on a cup of coffee is low in comparison to the convenience of rapid customer service. This is similar to the practice of coffee shops that accept credit card payments without a signature for amounts below $25, as the risk of a credit-card chargeback is low while the cost of delaying the transaction to obtain a signature is comparatively larger. In contrast, selling a more expensive item for bitcoin runs the risk of a double-spend attack, where the buyer broadcasts a competing transaction that spends the same inputs (UTXO) and cancels the payment to the merchant. A double-spend attack can happen in two ways: either before a transaction is confirmed, or if the attacker takes advantage of a blockchain fork to undo several blocks. A 51% attack allows an attacker to double-spend their own transactions in the new chain, thus undoing the corresponding transaction in the old chain.
In our example, malicious attacker Mallory goes to Carol's gallery and purchases a beautiful triptych painting depicting Satoshi Nakamoto as Prometheus. Carol sells "The Great Fire" paintings for $250,000 in bitcoin, to Mallory. Instead of waiting for six or more confirmations on the transaction, Carol wraps and hands the paintings to Mallory after only one confirmation. Mallory works with an accomplice, Paul, who operates a large mining pool and the accomplice launches a 51% attack as soon as Mallory's transaction is included in a block. Paul directs the mining pool to re-mine the same block height as the block containing Mallory's transaction replacing Mallory's payment to Carol with a transaction that double-spends the same input as Mallory's payment. The double-spend transaction consumes the same UTXO and pays it back to Mallory's wallet, instead of paying it to Carol, essentially allowing Mallory to keep the bitcoin. Paul then directs the mining pool to mine an additional block, so as to make the chain containing the double-spend transaction longer than the original chain (causing a fork below the block containing Mallory's transaction). When the blockchain fork resolves in favor of the new (longer) chain, the double-spent transaction replaces the original payment to Carol. Carol is now missing the three paintings and also has no bitcoin payment. Throughout all this activity, Paul's mining pool participants may remain blissfully unaware of the double-spend attempt, as they mine with automated miners and cannot monitor every transaction or block.
To protect against this kind of attack, a merchant selling large-value items must wait at least six confirmations before giving the product to the buyer. Alternatively, the merchant should use an escrow multi-signature account, again waiting for several confirmations after the escrow account is funded. The more confirmations elapse, the harder it becomes to invalidate a transaction with a 51% attack. For large-value items, payment by bitcoin will still be convenient and efficient even if the buyer has to wait 24 hours for delivery, which would ensure 144 confirmations.
In addition to a double-spend attack, the other scenario for a consensus attack is to deny service to specific bitcoin participants (specific bitcoin addresses). An attacker with a majority of the mining power can simply ignore specific transactions. If they are included in a block mined by another miner the attacker can deliberately fork and re-mine that block, again excluding the specific transactions. This type of attack can result in a sustained denial of service against a specific address or set of addresses for as long as the attacker controls the majority of the mining power.
Despite its name, the 51% attack scenario doesn't actually require 51% of the hashing power. In fact, such an attack can be attempted with a smaller percentage of the hashing power. The 51% threshold is simply the level at which such an attack is almost guaranteed to succeed. A consensus attack is essentially a tug-of-war for the next block and the "stronger" group is more likely to win. With less hashing power, the probability of success is reduced, as other miners control the generation of some blocks with their "honest" mining power. One way to look at it is that the more hashing power an attacker has, the longer the fork they can deliberately create, the more blocks in the recent past they can invalidate, or the more blocks in the future they can control. Security research groups have used statistical modeling to claim that various types of consensus attacks are possible with as little as 30% of the hashing power.
@ -781,5 +851,5 @@ The massive increase of total hashing power has arguably made bitcoin impervious
Not all attackers will be motivated by profit, however. One potential attack scenario is where an attacker intends to disrupt the bitcoin network without the possibility of profiting from such disruption. A malicious attack aimed at crippling bitcoin would require enormous investment and covert planning, but could conceivably be launched by a well funded, most likely state-sponsored attacker. Alternatively, a well-funded attacker could attack bitcoin's consensus by simultaneously amassing mining hardware, compromising pool operators and attacking other pools with denial-of-service. All of these scenarios are theoretically possible, but increasingly impractical as the bitcoin network's overall hashing power continues to grow exponentially. Recent advancements in bitcoin, such as P2Pool mining, aim to further de-centralize mining control, making bitcoin consensus even harder to attack.
Undoubtedly, a serious consensus attack would erode confidence in bitcoin in the short term, possibly causing a significant price decline. However, the bitcoin network and software is constantly evolving, so consensus attacks would be met with immediate counter-measures by the bitcoin community, making bitcoin hardier, stealthier and more robust.
Undoubtedly, a serious consensus attack would erode confidence in bitcoin in the short term, possibly causing a significant price decline. However, the bitcoin network and software is constantly evolving, so consensus attacks would be met with immediate counter-measures by the bitcoin community, making bitcoin hardier, stealthier, and more robust.

107
ch09.asciidoc
View File

@ -1,9 +1,9 @@
[[ch9]]
== Alternative chains, currencies and applications
== Alternative Chains, Currencies, and Applications
Bitcoin was neither the beginning nor the end of the digital currency evolution. It came from twenty years of research in distributed systems and currencies and brought a revolutionary new technology into the space: the de-centralized consensus mechanism based on Proof-of-Work. The invention at the heart of bitcoin has ushered a wave of innovation in currencies, financial services, economics, distributed systems, voting systems, corporate governance and contracts.
Bitcoin was the result of twenty years of research in distributed systems and currencies and brought a revolutionary new technology into the space: the de-centralized consensus mechanism based on Proof-of-Work. This invention at the heart of bitcoin has ushered a wave of innovation in currencies, financial services, economics, distributed systems, voting systems, corporate governance, and contracts.
In this chapter we'll examine the many offshoots of the bitcoin and blockchain inventions, the alternative chains, currencies and applications built since the introduction of this technology in 2009.
In this chapter we'll examine the many offshoots of the bitcoin and blockchain inventions, the alternative chains, currencies, and applications built since the introduction of this technology in 2009. Mostly, we will look at _alt-coins_, which are digital currencies implemented using the same design pattern as bitcoin, but with a completely separate blockchain and network.
For every alt-coin mentioned in this chapter, 50 or more will go unmentioned, eliciting howls of anger from their creators and fans. The purpose of this chapter is not to evaluate or qualify alt-coins, or to mention the most "significant" ones based on some subjective assessment. Instead, we will highlight a few examples that show the breadth and variety of the ecosystem, noting the first-of-a-kind for each innovation or significant differentiation. Some of the most interesting examples of alt-coins are in fact complete failures from a monetary perspective. That perhaps makes them even more interesting for study and highlights the fact that this chapter is not to be used as an investment guide.
@ -11,13 +11,13 @@ With new coins introduced every day, it would be impossible not to miss some imp
=== A taxonomy of alternative currencies and chains
Bitcoin is an open source project and its code has been used as the basis for many other software projects. The most common form of software spawned from bitcoin's source code are alternative de-centralized currencies, or _alt-coins_, which use the same basic building blocks to implement digital currencies.
Bitcoin is an open source project, and its code has been used as the basis for many other software projects. The most common form of software spawned from bitcoin's source code are alternative de-centralized currencies, or _alt-coins_, which use the same basic building blocks to implement digital currencies.
There are a number of protocol layers implemented on top of bitcoin's blockchain. These _meta-coins_, _meta-chains_, or _blockchain apps_ use the blockchain as an application platform or extend the bitcoin protocol by adding protocol layers. Examples include Colored Coins, Mastercoin and Counterparty.
There are a number of protocol layers implemented on top of bitcoin's blockchain. These _meta-coins_, _meta-chains_, or _blockchain apps_ use the blockchain as an application platform or extend the bitcoin protocol by adding protocol layers. Examples include Colored Coins, Mastercoin, and Counterparty.
In the next section we will examine a few notable alt-coins, such as Litecoin, Dogecoin, Freicoin, Primecoin, Peercoin, Darkcoin and Zerocoin. These alt-coins are notable for historical reasons or because they are good examples a specific type of alt-coin innovation, not because they are the most valuable or "best" alt-coins.
In the next section we will examine a few notable alt-coins, such as Litecoin, Dogecoin, Freicoin, Primecoin, Peercoin, Darkcoin, and Zerocoin. These alt-coins are notable for historical reasons or because they are good examples for a specific type of alt-coin innovation, not because they are the most valuable or "best" alt-coins.
In addition to the alt-coins, there are also a number of alternative blockchain implementations that are not really "coins", which I call _alt-chains_. These alt-chains implement a consensus algorithm and distributed ledger as a platform for contracts, name registration or other applications. Alt-chains use the same basic building blocks and sometimes also use a currency or token as a payment mechanism, but their primary purpose is not currency. We will look at Namecoin, Ethereum and NXT as examples of alt-chains.
In addition to the alt-coins, there are also a number of alternative blockchain implementations that are not really "coins", which I call _alt-chains_. These alt-chains implement a consensus algorithm and distributed ledger as a platform for contracts, name registration, or other applications. Alt-chains use the same basic building blocks and sometimes also use a currency or token as a payment mechanism, but their primary purpose is not currency. We will look at Namecoin, Ethereum, and NXT as examples of alt-chains.
In addition to the Proof-of-Work consensus mechanism used in bitcoin, alternatives include experimental protocols based on Proof-of-Resource and Proof-of-Publishing. We will examine Maidsafe and Twister as examples of these consensus mechanisms.
@ -25,15 +25,15 @@ Finally, there are a number of bitcoin contenders that offer digital currency or
=== Meta-Coin Platforms
Meta-coins and meta-chains are software layers implemented on top of bitcoin, either implementing a currency-inside-a-currency, or a platform/protocol overlay inside the bitcoin system. These function layers extend the core bitcoin protocol and add features and capabilities by encoding additional data inside bitcoin transactions and bitcoin addresses. The first implementations of meta-coins used various "hacks" to add meta-data to the bitcoin blockchain, such as using bitcoin addresses to encode data or using unused transaction fields (eg. the transaction sequence field) to encode meta-data about the added protocol layer. Since the introduction of the OP_RETURN transaction scripting opcode, the meta-coins have been able to record meta-data more directly in the blockchain and most are migrating to using that instead.
Meta-coins and meta-chains are software layers implemented on top of bitcoin, either implementing a currency-inside-a-currency, or a platform/protocol overlay inside the bitcoin system. These function layers extend the core bitcoin protocol and add features and capabilities by encoding additional data inside bitcoin transactions and bitcoin addresses. The first implementations of meta-coins used various "hacks" to add meta-data to the bitcoin blockchain, such as using bitcoin addresses to encode data or using unused transaction fields (e.g. the transaction sequence field) to encode meta-data about the added protocol layer. Since the introduction of the OP_RETURN transaction scripting opcode, the meta-coins have been able to record meta-data more directly in the blockchain, and most are migrating to using that instead.
==== Colored Coins
Colored Coins is a meta-protocol that overlays information on small amounts of bitcoin. A "colored" coin is an amount of bitcoin repurposed to express another asset. Imagine for example taking a $1 USD note and putting a stamp on it that said "This is a 1 share certificate of Acme Inc.". Now the $1 serves two purposes: it is a currency note and also a share certificate. Because it is more valuable as a share, you would not want to use it to buy candy, so effectively it is no longer useful as currency. Colored coins work in a same way by converting a specific, very small, amount of bitcoin into a traded certificate that represents another asset. The term "color" refers to the idea of giving special meaning through the addition of an attribute such as a color - it is a metaphor not an actual color association. There are no colors in colored coins.
Colored Coins is a meta-protocol that overlays information on small amounts of bitcoin. A "colored" coin is an amount of bitcoin repurposed to express another asset. Imagine for example taking a $1 USD note and putting a stamp on it that said "This is a 1 share certificate of Acme Inc.". Now the $1 serves two purposes: it is a currency note and also a share certificate. Because it is more valuable as a share, you would not want to use it to buy candy, so effectively it is no longer useful as currency. Colored coins work in the same way by converting a specific, very small, amount of bitcoin into a traded certificate that represents another asset. The term "color" refers to the idea of giving special meaning through the addition of an attribute such as a color - it is a metaphor not an actual color association. There are no colors in colored coins.
Colored coins are managed by specialized "wallets" that record and interpret the metadata attached to the "colored" bitcoins. Using such a wallet, the user will convert an amount of bitcoins from uncolored currency, into colored coins, by adding a label that has a special meaning. For example, a label could represent stock certificates, coupons, real property, commodities, collectible tokens etc. It is entirely up to the users of colored coins to assign and interpret the meaning of the "color" associated with specific coins. To color the coins, the user defines the associated metadata, such as the type of issuance, whether it can be subdivided into smaller units, a symbol and description and other related information. Once colored, these coins can be bought and sold, subdivided, aggregated and receive dividend payments. The colored coins can also be "uncolored" to remove the special association and redeem them for their face-value in bitcoin.
Colored coins are managed by specialized "wallets" that record and interpret the metadata attached to the "colored" bitcoins. Using such a wallet, the user will convert an amount of bitcoins from uncolored currency, into colored coins, by adding a label that has a special meaning. For example, a label could represent stock certificates, coupons, real property, commodities, collectible tokens, etc. It is entirely up to the users of colored coins to assign and interpret the meaning of the "color" associated with specific coins. To color the coins, the user defines the associated metadata, such as the type of issuance, whether it can be subdivided into smaller units, a symbol and description, and other related information. Once colored, these coins can be bought and sold, subdivided, aggregated and receive dividend payments. The colored coins can also be "uncolored" by removing the special association and redeem them for their face-value in bitcoin.
To demonstrate the use of colored coins, we have created a set of 20 colored coins with symbol "MasterBTC" that represent coupons for a free copy of this book. Each unit of MasterBTC is represented by these colored coins can now be sold or given to any bitcoin user with a colored-coin-capable wallet, who can then transfer them to others or redeem them with the issuer for a free copy of the book. This example of colored coins can be seen here: https://cpr.sm/FoykwrH6UY
To demonstrate the use of colored coins, we have created a set of 20 colored coins with symbol "MasterBTC" that represent coupons for a free copy of this book. Each unit of MasterBTC, represented by these colored coins, can now be sold or given to any bitcoin user with a colored-coin-capable wallet, who can then transfer them to others or redeem them with the issuer for a free copy of the book. This example of colored coins can be seen here: https://cpr.sm/FoykwrH6UY
.The metadata profile of the colored coins recorded as a coupon for a free copy of the book
====
@ -61,17 +61,17 @@ To demonstrate the use of colored coins, we have created a set of 20 colored coi
==== Mastercoin
Mastercoin is a protocol layer on top of bitcoin that supports a platform for various applications extending the bitcoin system. Mastercoin uses the currency MST as a token for conducting Mastercoin transactions but it not primarily a currency. Rather it is a platform for building other things, such as user currencies, smart property tokens, de-centralized asset exchanges, contracts etc. Think of Mastercoin as an application-layer protocol on top of bitcoin's financial transaction transport-layer, just like HTTP runs on top of TCP.
Mastercoin is a protocol layer on top of bitcoin that supports a platform for various applications extending the bitcoin system. Mastercoin uses the currency MST as a token for conducting Mastercoin transactions but it is not primarily a currency. Rather it is a platform for building other things, such as user currencies, smart property tokens, de-centralized asset exchanges, contracts, etc. Think of Mastercoin as an application-layer protocol on top of bitcoin's financial transaction transport-layer, just like HTTP runs on top of TCP.
Mastercoin operates primarily through transactions sent to and from a special bitcoin address called the "exodus" address (+1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P+), just like HTTP uses a specific TCP port (port 80) to differentiate its traffic from the rest of the TCP traffic. The Mastercoin protocol is gradually transitioning from using the specialized exodus address and multi-signatures to using the OP_RETURN bitcoin operator to encode transaction metadata.
==== Counterparty
Counterparty is another protocol layer implemented on top of bitcoin. Counterparty enables user currencies, tradable tokens, financial instruments, de-centralized asset exchanges and other features. Counterparty is implemented primarily using the OP_RETURN operator in bitcoin's scripting language to record metadata enhancing bitcoin transactions with additional meaning. Counterparty uses the currency XCP as a token for conducting Counterparty transactions.
Counterparty is another protocol layer implemented on top of bitcoin. Counterparty enables user currencies, tradable tokens, financial instruments, de-centralized asset exchanges, and other features. Counterparty is implemented primarily using the OP_RETURN operator in bitcoin's scripting language to record metadata enhancing bitcoin transactions with additional meaning. Counterparty uses the currency XCP as a token for conducting Counterparty transactions.
=== Alt-coins
Alt-coins are digital currencies implemented using the same design pattern as bitcoin, with a completely separate blockchain and network. The vast majority of alt-coins are derived from bitcoin's source code, also known as "forks". Some are implemented "from scratch" based on the blockchain model but without using any of bitcoin's source code. Alt-coins and alt-chains (in the next section) are both separate implementations of blockchain technology and both forms use their own blockchain. The difference in the terms is to indicate that alt-coins are primarily used as currency, whereas alt-chains are used for other purposes, not primarily currency.
The vast majority of alt-coins are derived from bitcoin's source code, also known as "forks". Some are implemented "from scratch" based on the blockchain model but without using any of bitcoin's source code. Alt-coins and alt-chains (in the next section) are both separate implementations of blockchain technology and both forms use their own blockchain. The difference in the terms is to indicate that alt-coins are primarily used as currency, whereas alt-chains are used for other purposes, not primarily currency.
The first alt-coins appeared in August of 2011 as forks of the bitcoin source code. Strictly speaking, the first major fork of bitcoin's code was not an alt-coin but the alt-chain _Namecoin_, which will be discussed in the next section.
@ -79,9 +79,9 @@ Based on the date of announcement, the first alt-coin appears to be _IXCoin_, la
In September of 2011, _Tenebrix_ was launched. Tenebrix was the first crypto-currency to implement an alternative Proof-of-Work algorithm, namely _scrypt_, an algorithm originally designed for password stretching (brute-force resistance). The stated goal of Tenebrix was to make a coin that was resistant to mining with GPUs and ASICs, by using a memory-intensive algorithm. Tenebrix did not succeed as a currency, but it was the basis for Litecoin, which has enjoyed great success and has spawned hundreds of clones.
_Litecoin_, in addition to using scrypt as the Proof-of-Work algorithm, also implemented a faster block generation time, targeted at 2.5 minutes instead of bitcoin's 10 minutes. The resulting currency is touted as "silver to bitcoin's gold" and intended as a light-weight alternative currency. Due to the faster confirmation time and 84 million total currency limit, many adherents of Litecoin believe it is better suited for retail transactions than bitcoin.
_Litecoin_, in addition to using scrypt as the Proof-of-Work algorithm, also implemented a faster block generation time, targeted at 2.5 minutes instead of bitcoin's 10 minutes. The resulting currency is touted as "silver to bitcoin's gold" and is intended as a light-weight alternative currency. Due to the faster confirmation time and the 84 million total currency limit, many adherents of Litecoin believe it is better suited for retail transactions than bitcoin.
Alt-coins continued to proliferate in 2011 and 2012, either based on bitcoin, or on Litecoin. In the beginning of 2013 there were 20 alt-coins vying for position in the market. By the end of 2013 however, this number had exploded to 200, with 2013 quickly becoming the "year of the alt-coins". The growth of alt-coins continued in 2014 with more than 500 alt-coins now in existence. More than half the alt-coins today are clones of Litecoin.
Alt-coins continued to proliferate in 2011 and 2012, either based on bitcoin or on Litecoin. In the beginning of 2013 there were 20 alt-coins vying for position in the market. By the end of 2013 however, this number had exploded to 200, with 2013 quickly becoming the "year of the alt-coins". The growth of alt-coins continued in 2014 with more than 500 alt-coins now in existence. More than half the alt-coins today are clones of Litecoin.
Creating an alt-coin is easy, which is why there are now more than 500 of them. Most of the alt-coins differ very slightly from bitcoin and do not offer anything worth studying. Many are in fact just attempts to enrich their creators. Among the copycats and pump-and-dump schemes, there are however some notable exceptions and very important innovations. These alt-coins take radically different approaches or add significant innovation to bitcoin's design pattern. There are three primary areas where alt-coins differentiate from bitcoin:
@ -89,7 +89,7 @@ Creating an alt-coin is easy, which is why there are now more than 500 of them.
* Different Proof-of-Work or consensus mechanism
* Specific features, such as strong anonymity
A graphical timeline of alt-coins and alt-chains can be found at http://mapofcoins.com
A graphical timeline of alt-coins and alt-chains can be found at http://mapofcoins.com.
==== Evaluating an alt-coin
@ -101,7 +101,7 @@ Here are some questions to ask about how well an alt-coin differentiates from bi
* Does the alt-coin differentiate sufficiently from bitcoin?
* Is the difference compelling enough to attract users away from bitcoin?
* Does the alt-coin address an interesting niche market or application?
* Can the alt-coin attract enough miners to be secure against consensus attacks?
* Can the alt-coin attract enough miners to be secured against consensus attacks?
Here are some of the key financial and market metrics to examine:
@ -119,34 +119,34 @@ Bitcoin has a few monetary parameters that give it distinctive characteristics o
*Litecoin*
One of the first alt-coins, released in 2011, Litecoin is the second most successful digital currency after bitcoin. It's primary innovations were the use of _scrypt_ as the Proof-of-Work algorithm (inherited from Tenebrix) and the faster/lighter currency parameters.
One of the first alt-coins, released in 2011, Litecoin is the second most successful digital currency after bitcoin. Its primary innovations were the use of _scrypt_ as the Proof-of-Work algorithm (inherited from Tenebrix) and the faster/lighter currency parameters.
* Block generation time: 2.5 minutes
* Total currency: 84 million coins by 2140
* Consensus Algorithm: scrypt Proof-of-Work
* Market capitalization: $160 million USD in Summer 2014
* Market capitalization: $160 million USD in mid-2014
*Dogecoin*
Dogecoin was released in December of 2013, based on a fork of Litecoin. Dogecoin is notable because it has a monetary policy of rapid issuance and a very high currency cap, to encourage spending and tipping. Dogecoin is also notable because it was started as a joke but became quite popular, with a large and active community, before declining rapidly in 2014.
* Block generation time: 60 seconds
* Total currency: 100,000,000,000 (100 trillion) Doge by 2015
* Total currency: 100,000,000,000 (100 billion) Doge by 2015
* Consensus algorithm: scrypt Proof-of-Work
* Market capitalization: $12 million USD in Summer 2014
* Market capitalization: $12 million USD in mid-2014
*Freicoin*
Freicoin was introduced in July 2012. It is a _demurrage currency_, meaning that is has a negative interest rate for stored value. Value stored in Freicoin is assessed a 4.5% APR fee, to encourage consumption and discourage hoarding of money. Freicoin is notable in that it implements a monetary policy that is the exact opposite of Bitcoin's deflationary policy. Freicoin has not seen success as a currency, but is an interesting example of the variety of monetary policies that can be expressed by alt-coins.
Freicoin was introduced in July 2012. It is a _demurrage currency_, meaning that it has a negative interest rate for stored value. Value stored in Freicoin is assessed a 4.5% APR fee, to encourage consumption and discourage hoarding of money. Freicoin is notable in that it implements a monetary policy that is the exact opposite of Bitcoin's deflationary policy. Freicoin has not seen success as a currency, but is an interesting example of the variety of monetary policies that can be expressed by alt-coins.
* Block generation: 10 minutes
* Total currency: 100 million coins by 2140
* Consensus algorithm: SHA256 Proof-of-Work
* Market capitalization: $130,000 USD in Summer 2014
* Market capitalization: $130,000 USD in mid-2014
==== Consensus Innovation: Peercoin, Myriad, Blackcoin, Vericoin, NXT
Bitcoin's consensus mechanism is based on Proof-of-Work using the SHA256 algorithm. The first alt-coins introduced scrypt as an alternative Proof-of-Work algorithms, as a way to make mining more CPU-friendly and less susceptible to centralization with ASICs. Since then, innovation in the consensus mechanism has continued at a frenetic pace. Several alt-coins adopted a variety of algorithms such as scrypt, scrypt-N, Skein, Groestl, SHA3, X11, Blake and others. Some alt-coins combined multiple algorithms for Proof-of-Work. In 2013 we saw the invention of an alternative to Proof-of-Work, called _Proof-of-Stake_, which forms the basis of many modern alt-coins.
Bitcoin's consensus mechanism is based on Proof-of-Work using the SHA256 algorithm. The first alt-coins introduced scrypt as an alternative Proof-of-Work algorithms, as a way to make mining more CPU-friendly and less susceptible to centralization with ASICs. Since then, innovation in the consensus mechanism has continued at a frenetic pace. Several alt-coins adopted a variety of algorithms such as scrypt, scrypt-N, Skein, Groestl, SHA3, X11, Blake, and others. Some alt-coins combined multiple algorithms for Proof-of-Work. In 2013 we saw the invention of an alternative to Proof-of-Work, called _Proof-of-Stake_, which forms the basis of many modern alt-coins.
Proof-of-Stake is a system by which existing owners of a currency can "stake" currency as interest-bearing collateral. Somewhat like a Certificate of Deposit (CD), participants can reserve a portion of their currency holdings, while earning an investment return in the form of new currency (issued as interest payments) and transaction fees.
@ -157,7 +157,7 @@ Peercoin was introduced in August of 2012 and is the first alt-coin to use a hyb
* Block generation: 10 minutes
* Total currency: No limit
* Consensus algorithm: (Hybrid) Proof-of-Stake with initial Proof-of-Work
* Market capitalization: $14 million USD in Summer 2014
* Market capitalization: $14 million USD in mid-2014
*Myriad*
@ -166,7 +166,7 @@ Myriad was introduced in February 2014 and is notable because it uses five diffe
* Block generation: 30 second average (2.5 minutes target per mining algorithm)
* Total currency: 2 billion by 2024
* Consensus algorithm: Multi-Algorithm Proof-of-Work
* Market capitalization: $120,000 USD in Summer 2014
* Market capitalization: $120,000 USD in mid-2014
*Blackcoin*
@ -175,16 +175,16 @@ Blackcoin was introduced in February 2014 and uses a Proof-of-Stake consensus al
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Stake
* Market capitalization: $3.7 million USD in Summer 2014
* Market capitalization: $3.7 million USD in mid-2014
*Vericoin*
*VeriCoin*
Vericoin was launched in May 2014. It uses a Proof-of-Stake consensus algorithm with a variable interest rate that dynamically adjusts based on market forces of supply and demand.
VeriCoin was launched in May 2014. It uses a Proof-of-Stake consensus algorithm with a variable interest rate that dynamically adjusts based on market forces of supply and demand. It also is the first alt-coin featuring auto-exchange to Bitcoin for payment in Bitcoin from the wallet.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Stake
* Market capitalization: $1.1 million USD in Summer 2014
* Market capitalization: $1.1 million USD in mid-2014
*NXT*
@ -193,21 +193,21 @@ NXT (pronounced "Next") is a "pure" Proof-of-Stake alt-coin, in that it does not
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Stake
* Market capitalization: $30 million USD in Summer 2014
* Market capitalization: $30 million USD in mid-2014
==== Dual-Purpose Mining Innovation: Primecoin, Curecoin, Gridcoin
Bitcoin's Proof-of-Work algorithm has only one purpose: securing the bitcoin network. Compared to traditional payment system security, the cost of mining is not very high. However, it has been criticized by many as "wasteful". The next set of alt-coins attempt to address this concern. Dual-purpose Proof-of-Work algorithms solve a specific "useful" problem, while producing Proof-of-Work to secure the network. The risk of adding an external use to the currency's security is that it also adds external influence to the supply/demand curve.
Bitcoin's Proof-of-Work algorithm has only one purpose: securing the bitcoin network. Compared to traditional payment system security, the cost of mining is not very high. However, it has been criticized by many as being “wasteful". The next set of alt-coins attempt to address this concern. Dual-purpose Proof-of-Work algorithms solve a specific "useful" problem, while producing Proof-of-Work to secure the network. The risk of adding an external use to the currency's security is that it also adds external influence to the supply/demand curve.
*Primecoin*
Primecoin was announced in July 2013. It's Proof-of-Work algorithm searches for prime numbers, computing Cunningham and bi-twin prime chains. Prime numbers are useful in a variety of scientific disciplines. The Primecoin blockchain contains the discovered prime numbers, thereby producing a public record of scientific discovery in parallel to the public ledger of transactions.
Primecoin was announced in July 2013. Its Proof-of-Work algorithm searches for prime numbers, computing Cunningham and bi-twin prime chains. Prime numbers are useful in a variety of scientific disciplines. The Primecoin blockchain contains the discovered prime numbers, thereby producing a public record of scientific discovery in parallel to the public ledger of transactions.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Work with prime number chain discovery
* Market capitalization: $1.3 million USD in Summer 2014
* Market capitalization: $1.3 million USD in mid-2014
*Curecoin*
@ -216,7 +216,7 @@ Curecoin was announced in May 2013. It combines a SHA256 Proof-of-Work algorithm
* Block generation: 10 minutes
* Total currency: No limit
* Consensus algorithm: Proof-of-Work with protein folding research
* Market capitalization: $58,000 USD in Summer 2014
* Market capitalization: $58,000 USD in mid-2014
*Gridcoin*
@ -225,15 +225,15 @@ Gridcoin was introduced in October 2013. It supplements scrypt-based Proof-of-Wo
* Block generation: 150 seconds
* Total currency: No limit
* Consensus algorithm: Proof-of-Work with BOINC grid-computing subsidy
* Market capitalization: $122,000 USD in Summer 2014
* Market capitalization: $122,000 USD in mid-2014
==== Anonymity-Focused Alt-Coins: CryptoNote, Bytecoin, Monero, Zerocash/Zerocoin, Darkcoin
Bitcoin is often mistakenly characterized as "anonymous" currency. In fact, it is relatively easy to connect identities to bitcoin addresses and, using big-data analytics, connect addresses to each other to form a comprehensive picture of someone's bitcoin spending habits. Several alt-coins aim to address this directly by focusing on strong anonymity. The first such attempt is most likely _Zerocoin_, a meta-coin protocol for preserving anonymity on top of bitcoin, introduced with a paper in the 2013 IEEE Symposium on Security and Privacy. Zerocoin will be implemented as a completely separate alt-coin called Zerocash, currently in development. An alternative approach to anonymity was launched with _CryptoNote_ in a paper published in October 2013. CryptoNote is a foundational technology that is implemented by a number of alt-coin forks discussed below. In addition to Zerocash and Cryptonotes, there are several other independent anonymous coins, such as Darkcoin that use stealth addresses or transaction re-mixing to deliver anonymity.
Bitcoin is often mistakenly characterized as "anonymous" currency. In fact, it is relatively easy to connect identities to bitcoin addresses and, using big-data analytics, connect addresses to each other to form a comprehensive picture of someone's bitcoin spending habits. Several alt-coins aim to address this issue directly by focusing on strong anonymity. The first such attempt is most likely _Zerocoin_, a meta-coin protocol for preserving anonymity on top of bitcoin, introduced with a paper in the 2013 IEEE Symposium on Security and Privacy. Zerocoin will be implemented as a completely separate alt-coin called Zerocash, currently in development. An alternative approach to anonymity was launched with _CryptoNote_ in a paper published in October 2013. CryptoNote is a foundational technology that is implemented by a number of alt-coin forks discussed below. In addition to Zerocash and Cryptonotes, there are several other independent anonymous coins, such as Darkcoin that use stealth addresses or transaction re-mixing to deliver anonymity.
*Zerocoin/Zerocash*
Zerocoin is a theoretical approach to digital currency anonymity introduced in 2013 by researchers at Johns Hopkins. Zerocash is an alt-coin implementation of Zercoin that is in development and not yet released.
Zerocoin is a theoretical approach to digital currency anonymity introduced in 2013 by researchers at Johns Hopkins. Zerocash is an alt-coin implementation of Zerocoin that is in development and not yet released.
*CryptoNote*
@ -241,30 +241,30 @@ CryptoNote is a reference implementation alt-coin that provides the basis for an
*Bytecoin*
Bytecoin was the first implementation spawned from CryptoNote, offering a viable anonymous currency based on the CryptoNote technology. Bytecoin was launched in March of 2014. Note that there was a previous alt-coin named Bytecoin with currency symbol BTE, whereas the CryptoNote-derived Bytecoin has currency symbol BCN. Bytecoin uses the Cryptonight Proof-of-Work algorithm which requires access to at least 2 MB of RAM per instance, making it unsuitable for GPU or ASIC mining. Bytecoin inherits ring-signatures, unlinkable transactions and blockchain-analysis resistant anonymity from CryptoNote.
Bytecoin was the first implementation spawned from CryptoNote, offering a viable anonymous currency based on the CryptoNote technology. Bytecoin was launched in July of 2012. Note that there was a previous alt-coin named Bytecoin with currency symbol BTE, whereas the CryptoNote-derived Bytecoin has currency symbol BCN. Bytecoin uses the Cryptonight Proof-of-Work algorithm which requires access to at least 2 MB of RAM per instance, making it unsuitable for GPU or ASIC mining. Bytecoin inherits ring-signatures, unlinkable transactions and blockchain-analysis resistant anonymity from CryptoNote.
* Block generation: 2 minutes
* Total currency: 184 billion BCN
* Consensus algorithm: Cryptonight Proof-of-Work
* Market capitalization: $3 million USD in Summer 2014
* Market capitalization: $3 million USD in mid-2014
*Monero*
Monero is another implementation of CryptoNote. It has a slightly flatter issuance curve than Bytecoin, issuing 80% of the currency in the first 4 years. It offers the same anonymity features inherited from CryptoNote.
* Block generation: 1 minute
* Total currency: 184 billion XMR
* Total currency: 18.4 million XMR
* Consensus algorithm: Cryptonight Proof-of-Work
* Market capitalization: $5 million USD in Summer 2014
* Market capitalization: $5 million USD in mid-2014
*Darkcoin*
Darkcoin was launched in March of 2014. Darkcoin implements anonymous currency using a re-mixing protocol for all transactions called DarkSend. Darkcoin is also notable for using 11 rounds of different hash functions (blake, bmw, groestl, jh, keccak, skein, luffa, cubehash, shavite, simd, echo) for the Proof-of-Work algorithm.
Darkcoin was launched in January of 2014. Darkcoin implements anonymous currency using a re-mixing protocol for all transactions called DarkSend. Darkcoin is also notable for using 11 rounds of different hash functions (blake, bmw, groestl, jh, keccak, skein, luffa, cubehash, shavite, simd, echo) for the Proof-of-Work algorithm.
* Block generation: 2.5 minutes
* Total currency: 22 million DRK
* Total currency: maximum 22 million DRK
* Consensus algorithm: Multi-algorithm Multi-round Proof-of-Work
* Market capitalization: $19 million USD in Summer 2014
* Market capitalization: $19 million USD in mid-2014
=== Non-currency alt-chains
@ -272,7 +272,7 @@ Alt-chains are alternative implementations of the blockchain design pattern, whi
==== Namecoin
Namecoin was the first "fork" of the bitcoin code. Namecoin is a de-centralized key-value registration and transfer platform using a blockchain. It supports a global domain name registry similar to the domain-name registration system on the Internet. Namecoin is currently used as an alternative Domain Name Service (DNS) for the root-level domain +.bit+. Namecoin can also be used to register names and key-value pairs in other namespaces, for storing things like email addresses, encryption keys, SSL certificates, file signatures, voting systems, stock certificates and a myriad other applications.
Namecoin was the first "fork" of the bitcoin code. Namecoin is a de-centralized key-value registration and transfer platform using a blockchain. It supports a global domain name registry similar to the domain-name registration system on the Internet. Namecoin is currently used as an alternative Domain Name Service (DNS) for the root-level domain +.bit+. Namecoin can also be used to register names and key-value pairs in other namespaces, for storing things like email addresses, encryption keys, SSL certificates, file signatures, voting systems, stock certificates and a myriad of other applications.
The Namecoin system includes the namecoin currency (symbol NMC), which is used to pay transaction fees for registration and transfer of names. At current prices, the fee to register a name is 0.01 NMC or approximately 1 US cent. As in bitcoin, the fees are collected by Namecoin miners.
@ -281,13 +281,13 @@ Namecoin's basic parameters are the same as bitcoin's:
* Block generation: 10 minutes
* Total currency: 21 million NMC by 2140
* Consensus algorithm: SHA256 Proof-of-Work
* Market capitalization: $10 million USD in Summer 2014
* Market capitalization: $10 million USD in mid-2014
Namecoin's namespaces are not restricted, and anyone can use any namespace in any way. However, certain namespaces have an agreed upon specification so that when it is read from the blockchain, software knows how to read and proceed from there. If it is malformed, then whatever software you used to read from the specific namespace will throw an error. Some of the popular namespaces are:
* +d/+ is the domain-name namespace for +.bit+ domains
* +id/+ is the namespace for storing person identifiers such as email addresses, PGP keys etc.
* +u/+ is an additional, more structured specfication to store identities (based on openspecs).
* +u/+ is an additional, more structured specification to store identities (based on openspecs).
The Namecoin client is very similar to Bitcoin Core, as it is derived from the same source code. Upon installation, the client will download a full copy of the namecoin blockchain and then will be ready to query and register names. There are three main commands:
@ -297,8 +297,12 @@ The Namecoin client is very similar to Bitcoin Core, as it is derived from the s
For example, to register the domain +mastering-bitcoin.bit+, we use the command +name_new+ as follows:
[source,bash]
----
$ namecoind name_new d/mastering-bitcoin
----
[source,json]
----
[
"21cbab5b1241c6d1a6ad70a2416b3124eb883ac38e423e5ff591d1968eb6664a",
"a05555e0fc56c023"
@ -335,7 +339,7 @@ Namecoin registrations need to be updated every 36,000 blocks (approximately 200
==== Bitmessage
Bitmessage is a bitcoin alt-chain that implements a de-centralized secure messaging service, essentially a server-less encrypted email system. Bitmessage allows users to compose and send message to each other, using a bitmessage address. The messages operate in much the same way as a bitcoin transaction, but they are transient - they do not persist beyond 2 days and if not delivered to the destination node in that time, they are lost. Senders and recipients are pseudonymous, they have no identifiers other than a bitmessage address, but are strongly authenticated, meaning that messages cannot be "spoofed". Bitmessages are encrypted to the recipient and therefore the bitmessage network is resistant to holistic surveillance - an eavesdropper has to compromise the recipient's device in order to intercept messages.
Bitmessage is a bitcoin alt-chain that implements a de-centralized secure messaging service, essentially a server-less encrypted email system. Bitmessage allows users to compose and send messages to each other, using a bitmessage address. The messages operate in much the same way as a bitcoin transaction, but they are transient - they do not persist beyond 2 days and if not delivered to the destination node in that time, they are lost. Senders and recipients are pseudonymous, they have no identifiers other than a bitmessage address, but are strongly authenticated, meaning that messages cannot be "spoofed". Bitmessages are encrypted to the recipient and therefore the bitmessage network is resistant to holistic surveillance - an eavesdropper has to compromise the recipient's device in order to intercept messages.
==== Ethereum
@ -343,6 +347,7 @@ Ethereum is a Turing-complete contract processing and execution platform based o
Ethereum can implement quite complex systems that are otherwise implemented as alt-chains themselves. For example, below is a Namecoin-like name registration contract written in Ethereum (or more accurately, written in a high-level language that can be compiled to Ethereum code):
[source,python]
----
if !contract.storage[msg.data[0]]: # Is the key not yet taken?
# Then take it!
@ -355,7 +360,7 @@ else:
=== Future of Currencies
The future of cryptographic currencies overall is even brighter than the future of bitcoin. Bitcoin introduced a completely new form of de-centralized organization and consensus that has spawned hundreds of incredible innovations. These inventions will likely affect broad sectors of the economy, from distributed systems science, to finance, economics, currencies, central banking and corporate governance. Many human activities that previously required centralized institutions or organizations to function as authoritative or trusted points of control can now be de-centralized. The invention of the blockchain and consensus system will significantly reduce the cost of organization and coordination on large scale systems, while removing opportunities for concentration of power, corruption and regulatory capture.
The future of cryptographic currencies overall is even brighter than the future of bitcoin. Bitcoin introduced a completely new form of de-centralized organization and consensus that has spawned hundreds of incredible innovations. These inventions will likely affect broad sectors of the economy, from distributed systems science, to finance, economics, currencies, central banking, and corporate governance. Many human activities that previously required centralized institutions or organizations to function as authoritative or trusted points of control can now be de-centralized. The invention of the blockchain and consensus system will significantly reduce the cost of organization and coordination on large scale systems, while removing opportunities for concentration of power, corruption and regulatory capture.

40
code/addr.cpp Normal file
View File

@ -0,0 +1,40 @@
#include <bitcoin/bitcoin.hpp>
int main()
{
// Private secret key.
bc::ec_secret secret = bc::decode_hash(
"038109007313a5807b2eccc082c8c3fbb988a973cacf1a7df9ce725c31b14776");
// Get public key.
bc::ec_point public_key = bc::secret_to_public_key(secret);
std::cout << "Public key: " << bc::encode_hex(public_key) << std::endl;
// Create Bitcoin address.
// Normally you can use:
// bc::payment_address payaddr;
// bc::set_public_key(payaddr, public_key);
// const std::string address = payaddr.encoded();
// Compute hash of public key for P2PKH address.
const bc::short_hash hash = bc::bitcoin_short_hash(public_key);
bc::data_chunk unencoded_address;
// Reserve 25 bytes
// [ version:1 ]
// [ hash:20 ]
// [ checksum:4 ]
unencoded_address.reserve(25);
// Version byte, 0 is normal BTC address (P2PKH).
unencoded_address.push_back(0);
// Hash data
bc::extend_data(unencoded_address, hash);
// Checksum is computed by hashing data, and adding 4 bytes from hash.
bc::append_checksum(unencoded_address);
// Finally we must encode the result in Bitcoin's base58 encoding
assert(unencoded_address.size() == 25);
const std::string address = bc::encode_base58(unencoded_address);
std::cout << "Address: " << address << std::endl;
return 0;
}

53
code/ec-math.py Normal file
View File

@ -0,0 +1,53 @@
import ecdsa
import random
from ecdsa.util import string_to_number, number_to_string
# secp256k1, http://www.oid-info.com/get/1.3.132.0.10
_p = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2FL
_r = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141L
_b = 0x0000000000000000000000000000000000000000000000000000000000000007L
_a = 0x0000000000000000000000000000000000000000000000000000000000000000L
_Gx = 0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798L
_Gy = 0x483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8L
curve_secp256k1 = ecdsa.ellipticcurve.CurveFp(_p, _a, _b)
generator_secp256k1 = ecdsa.ellipticcurve.Point(curve_secp256k1, _Gx, _Gy, _r)
oid_secp256k1 = (1, 3, 132, 0, 10)
SECP256k1 = ecdsa.curves.Curve("SECP256k1", curve_secp256k1, generator_secp256k1, oid_secp256k1)
ec_order = _r
curve = curve_secp256k1
generator = generator_secp256k1
def random_secret():
random_char = lambda: chr(random.randint(0, 255))
convert_to_int = lambda array: int("".join(array).encode("hex"), 16)
byte_array = [random_char() for i in range(32)]
return convert_to_int(byte_array)
def get_point_pubkey(point):
if point.y() & 1:
key = '03' + '%064x' % point.x()
else:
key = '02' + '%064x' % point.x()
return key.decode('hex')
def get_point_pubkey_uncompressed(point):
key = '04' + \
'%064x' % point.x() + \
'%064x' % point.y()
return key.decode('hex')
# Generate a new private key.
secret = random_secret()
print "Secret: ", secret
# Get the public key point.
point = secret * generator
print "EC point:", point
print "BTC public key:", get_point_pubkey(point).encode("hex")
# Given the point (x, y) we can create the object using:
point1 = ecdsa.ellipticcurve.Point(curve, point.x(), point.y(), ec_order)
assert point1 == point

28
code/get-utxo.py Normal file
View File

@ -0,0 +1,28 @@
# get unspent outputs from blockchain API
import json
import requests
# example address
address = '1Dorian4RoXcnBv9hnQ4Y2C1an6NJ4UrjX'
# The API URL is https://blockchain.info/unspent?active=<address>
# It returns a JSON object with a list "unspent_outputs", containing UTXO, like this:
#{ "unspent_outputs":[
# {
# "tx_hash":"ebadfaa92f1fd29e2fe296eda702c48bd11ffd52313e986e99ddad9084062167",
# "tx_index":51919767,
# "tx_output_n": 1,
# "script":"76a9148c7e252f8d64b0b6e313985915110fcfefcf4a2d88ac",
# "value": 8000000,
# "value_hex": "7a1200",
# "confirmations":28691
# },
# ...
#]}
resp = requests.get('https://blockchain.info/unspent?active=%s' % address)
utxo_set = json.loads(resp.text)["unspent_outputs"]
for utxo in utxo_set:
print "%s:%d - %ld Satoshis" % (utxo['tx_hash'], utxo['tx_output_n'], utxo['value'])

15
code/hash_example.py
View File

@ -5,7 +5,14 @@ import hashlib
text = "I am Satoshi Nakamoto"
for nonce in xrange(20): # iterate nonce from 0 to 19
input = text + str(nonce) # add the nonce to the end of the text
hash = hashlib.sha256(input).hexdigest() # calculate the SHA-256 hash of the input (text+nonce)
print input, '=>', hash # show the input and hash result
# iterate nonce from 0 to 19
for nonce in xrange(20):
# add the nonce to the end of the text
input = text + str(nonce)
# calculate the SHA-256 hash of the input (text+nonce)
hash = hashlib.sha256(input).hexdigest()
# show the input and hash result
print input, '=>', hash

15
code/key-to-address-ecc-example.py
View File

@ -5,7 +5,8 @@ valid_private_key = False
while not valid_private_key:
private_key = bitcoin.random_key()
decoded_private_key = bitcoin.decode_privkey(private_key, 'hex')
valid_private_key = 0 < decoded_private_key < bitcoin.N
valid_private_key = 0 < decoded_private_key < bitcoin.N
print "Private Key (hex) is: ", private_key
print "Private Key (decimal) is: ", decoded_private_key
@ -18,7 +19,8 @@ compressed_private_key = private_key + '01'
print "Private Key Compressed (hex) is: ", compressed_private_key
# Generate a WIF format from the compressed private key (WIF-compressed)
wif_compressed_private_key = bitcoin.encode_privkey(bitcoin.decode_privkey(compressed_private_key, 'hex'), 'wif')
wif_compressed_private_key = bitcoin.encode_privkey(
bitcoin.decode_privkey(compressed_private_key, 'hex'), 'wif')
print "Private Key (WIF-Compressed) is: ", wif_compressed_private_key
# Multiply the EC generator point G with the private key to get a public key point
@ -42,11 +44,6 @@ print "Compressed Public Key (hex) is:", hex_compressed_public_key
print "Bitcoin Address (b58check) is:", bitcoin.pubkey_to_address(public_key)
# Generate compressed bitcoin address from compressed public key
print "Compressed Bitcoin Address (b58check) is:", bitcoin.pubkey_to_address(hex_compressed_public_key)
print "Compressed Bitcoin Address (b58check) is:", \
bitcoin.pubkey_to_address(hex_compressed_public_key)

16
code/max_money.py Normal file
View File

@ -0,0 +1,16 @@
# Original block reward for miners was 50 BTC
start_block_reward = 50
# 210000 is around every 4 years with a 10 minute block interval
reward_interval = 210000
def max_money():
# 50 BTC = 50 0000 0000 Satoshis
current_reward = 50 * 10**8
total = 0
while current_reward > 0:
total += reward_interval * current_reward
current_reward /= 2
return total
print "Total BTC to ever be created:", max_money(), "Satoshis"

62
code/merkle.cpp Normal file
View File

@ -0,0 +1,62 @@
#include <bitcoin/bitcoin.hpp>
bc::hash_digest create_merkle(bc::hash_digest_list& merkle)
{
// Stop if hash list is empty.
if (merkle.empty())
return bc::null_hash;
else if (merkle.size() == 1)
return merkle[0];
// While there is more than 1 hash in the list, keep looping...
while (merkle.size() > 1)
{
// If number of hashes is odd, duplicate last hash in the list.
if (merkle.size() % 2 != 0)
merkle.push_back(merkle.back());
// List size is now even.
assert(merkle.size() % 2 == 0);
// New hash list.
bc::hash_digest_list new_merkle;
// Loop through hashes 2 at a time.
for (auto it = merkle.begin(); it != merkle.end(); it += 2)
{
// Join both current hashes together (concatenate).
bc::data_chunk concat_data(bc::hash_size * 2);
auto concat = bc::make_serializer(concat_data.begin());
concat.write_hash(*it);
concat.write_hash(*(it + 1));
assert(concat.iterator() == concat_data.end());
// Hash both of the hashes.
bc::hash_digest new_root = bc::bitcoin_hash(concat_data);
// Add this to the new list.
new_merkle.push_back(new_root);
}
// This is the new list.
merkle = new_merkle;
// DEBUG output -------------------------------------
std::cout << "Current merkle hash list:" << std::endl;
for (const auto& hash: merkle)
std::cout << " " << bc::encode_hex(hash) << std::endl;
std::cout << std::endl;
// --------------------------------------------------
}
// Finally we end up with a single item.
return merkle[0];
}
int main()
{
// Replace these hashes with ones from a block to reproduce the same merkle root.
bc::hash_digest_list tx_hashes{{
bc::decode_hash("0000000000000000000000000000000000000000000000000000000000000000"),
bc::decode_hash("0000000000000000000000000000000000000000000000000000000000000011"),
bc::decode_hash("0000000000000000000000000000000000000000000000000000000000000022"),
}};
const bc::hash_digest merkle_root = create_merkle(tx_hashes);
std::cout << "Result: " << bc::encode_hex(merkle_root) << std::endl;
return 0;
}

21
code/proof-of-work-example.py Normal file → Executable file
View File

@ -8,11 +8,13 @@ max_nonce = 2 ** 32 # 4 billion
def proof_of_work(header, difficulty_bits):
# calculate the difficulty target
target = 2 ** (256-difficulty_bits)
for nonce in xrange(max_nonce):
hash_result = hashlib.sha256(str(header)+str(nonce)).hexdigest()
# check if this is a valid result, below the target
if long(hash_result, 16) < target:
print "Success with nonce %d" % nonce
print "Hash is %s" % hash_result
@ -26,21 +28,34 @@ if __name__ == '__main__':
nonce = 0
hash_result = ''
# difficulty from 0 to 31 bits
for difficulty_bits in xrange(32):
difficulty = 2 ** difficulty_bits
print "Difficulty: %ld (%d bits)" % (difficulty, difficulty_bits)
print "Starting search..."
# checkpoint the current time
start_time = time.time()
new_block = 'test block with transactions' + hash_result # make a new block which includes the hash from the previous block
(hash_result, nonce) = proof_of_work(new_block, difficulty_bits) # find a nonce for the new block
# make a new block which includes the hash from the previous block
# we fake a block of transactions - just a string
new_block = 'test block with transactions' + hash_result
# find a valid nonce for the new block
(hash_result, nonce) = proof_of_work(new_block, difficulty_bits)
# checkpoint how long it took to find a result
end_time = time.time()
elapsed_time = end_time - start_time
print "Elapsed Time: %.4f seconds" % elapsed_time
if elapsed_time > 0:
# estimate the hashes per second
hash_power = float(long(nonce)/elapsed_time)
print "Hashing Power: %ld hashes per second" % hash_power

28
code/satoshi-words.cpp Normal file
View File

@ -0,0 +1,28 @@
/*
Display the genesis block message by Satoshi.
*/
#include <iostream>
#include <bitcoin/bitcoin.hpp>
int main()
{
// Create genesis block.
bc::block_type block = bc::genesis_block();
// Genesis block contains a single coinbase transaction.
assert(block.transactions.size() == 1);
// Get first transaction in block (coinbase).
const bc::transaction_type& coinbase_tx = block.transactions[0];
// Coinbase tx has a single input.
assert(coinbase_tx.inputs.size() == 1);
const bc::transaction_input_type& coinbase_input = coinbase_tx.inputs[0];
// Convert the input script to its raw format.
const bc::data_chunk& raw_message = save_script(coinbase_input.script);
// Convert this to an std::string.
std::string message;
message.resize(raw_message.size());
std::copy(raw_message.begin(), raw_message.end(), message.begin());
// Display the genesis block message.
std::cout << message << std::endl;
return 0;
}

68
code/select-utxo.py Normal file
View File

@ -0,0 +1,68 @@
# Selects outputs from a UTXO list using a greedy algorithm.
from sys import argv
class OutputInfo:
def __init__(self, tx_hash, tx_index, value):
self.tx_hash = tx_hash
self.tx_index = tx_index
self.value = value
def __repr__(self):
return "<%s:%s with %s Satoshis>" % (self.tx_hash, self.tx_index,
self.value)
# Select optimal outputs for a send from unspent outputs list.
# Returns output list and remaining change to be sent to
# a change address.
def select_outputs_greedy(unspent, min_value):
# Fail if empty.
if not unspent:
return None
# Partition into 2 lists.
lessers = [utxo for utxo in unspent if utxo.value < min_value]
greaters = [utxo for utxo in unspent if utxo.value >= min_value]
key_func = lambda utxo: utxo.value
if greaters:
# Not-empty. Find the smallest greater.
min_greater = min(greaters)
change = min_greater.value - min_value
return [min_greater], change
# Not found in greaters. Try several lessers instead.
# Rearrange them from biggest to smallest. We want to use the least
# amount of inputs as possible.
lessers.sort(key=key_func, reverse=True)
result = []
accum = 0
for utxo in lessers:
result.append(utxo)
accum += utxo.value
if accum >= min_value:
change = accum - min_value
return result, "Change: %d Satoshis" % change
# No results found.
return None, 0
def main():
unspent = [
OutputInfo("ebadfaa92f1fd29e2fe296eda702c48bd11ffd52313e986e99ddad9084062167", 1, 8000000),
OutputInfo("6596fd070679de96e405d52b51b8e1d644029108ec4cbfe451454486796a1ecf", 0, 16050000),
OutputInfo("b2affea89ff82557c60d635a2a3137b8f88f12ecec85082f7d0a1f82ee203ac4", 0, 10000000),
OutputInfo("7dbc497969c7475e45d952c4a872e213fb15d45e5cd3473c386a71a1b0c136a1", 0, 25000000),
OutputInfo("55ea01bd7e9afd3d3ab9790199e777d62a0709cf0725e80a7350fdb22d7b8ec6", 17, 5470541),
OutputInfo("12b6a7934c1df821945ee9ee3b3326d07ca7a65fd6416ea44ce8c3db0c078c64", 0, 10000000),
OutputInfo("7f42eda67921ee92eae5f79bd37c68c9cb859b899ce70dba68c48338857b7818", 0, 16100000),
]
if len(argv) > 1:
target = long(argv[1])
else:
target = 55000000
print "For transaction amount %d Satoshis (%f bitcoin) use: " % (target, target/10.0**8)
print select_outputs_greedy(unspent, target)
if __name__ == "__main__":
main()

70
code/vanity-miner.cpp Normal file
View File

@ -0,0 +1,70 @@
#include <bitcoin/bitcoin.hpp>
// The string we are searching for
const std::string search = "1kid";
// Generate a random secret key. A random 32 bytes.
bc::ec_secret random_secret(std::default_random_engine& engine);
// Extract the Bitcoin address from an EC secret.
std::string bitcoin_address(const bc::ec_secret& secret);
// Case insensitive comparison with the search string.
bool match_found(const std::string& address);
int main()
{
std::random_device random;
std::default_random_engine engine(random());
// Loop continuously...
while (true)
{
// Generate a random secret.
bc::ec_secret secret = random_secret(engine);
// Get the address.
std::string address = bitcoin_address(secret);
// Does it match our search string? (1kid)
if (match_found(address))
{
// Success!
std::cout << "Found vanity address! " << address << std::endl;
std::cout << "Secret: " << bc::encode_hex(secret) << std::endl;
return 0;
}
}
// Should never reach here!
return 0;
}
bc::ec_secret random_secret(std::default_random_engine& engine)
{
// Create new secret...
bc::ec_secret secret;
// Iterate through every byte setting a random value...
for (uint8_t& byte: secret)
byte = engine() % std::numeric_limits<uint8_t>::max();
// Return result.
return secret;
}
std::string bitcoin_address(const bc::ec_secret& secret)
{
// Convert secret to pubkey...
bc::ec_point pubkey = bc::secret_to_public_key(secret);
// Finally create address.
bc::payment_address payaddr;
bc::set_public_key(payaddr, pubkey);
// Return encoded form.
return payaddr.encoded();
}
bool match_found(const std::string& address)
{
auto addr_it = address.begin();
// Loop through the search string comparing it to the lower case
// character of the supplied address.
for (auto it = search.begin(); it != search.end(); ++it, ++addr_it)
if (*it != std::tolower(*addr_it))
return false;
// Reached end of search string, so address matches.
return true;
}

BIN
images/Bitcoin_Overview.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 144 KiB

After

Width:  |  Height:  |  Size: 129 KiB

BIN
images/ChildHardPrivateDerivation.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
images/ChildPrivateDerivation.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 37 KiB

BIN
images/ChildPublicDerivation.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 28 KiB

BIN
images/HDWalletFromRootSeed.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 31 KiB

BIN
images/HD_wallet.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 28 KiB

After

Width:  |  Height:  |  Size: 30 KiB

BIN
images/MultibitReceive.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

BIN
images/MultibitWallet.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 45 KiB

BIN
images/MultibitWelcome.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 60 KiB

BIN
images/Transaction_Chain.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 86 KiB

After

Width:  |  Height:  |  Size: 67 KiB

BIN
images/paper_wallet_bpw.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 284 KiB

After

Width:  |  Height:  |  Size: 270 KiB

139
preface.asciidoc
View File

@ -5,23 +5,11 @@
I first stumbled upon bitcoin in mid-2011. My immediate reaction was more or less "Pfft! Nerd money!" and I ignored it for another 6 months, failing to grasp its importance. This is a reaction which I have seen repeated among many of the smartest people I know, which gives me some consolation. The second time I came across bitcoin in a mailing list discussion, I decided to read the white paper written by Satoshi Nakamoto, to study the authoritative source and see what it was all about. I still remember the moment I finished reading those 9 pages, when I realized that bitcoin was not simply a digital currency, but a network of trust that could also provide the basis for so much more than just currencies. That realization: "This isn't money, it's a de-centralized trust network," started me on a four month journey to devour every scrap of information about bitcoin I could find. I became obsessed and enthralled, spending twelve or more hours each day glued to a screen, reading, writing, coding and learning as much as I could. I emerged from this state of fugue, more than 20 lbs lighter from lack of consistent meals, determined to dedicate myself to working on bitcoin.
Two years later, after creating a number of small startups to explore various bitcoin-related services and products, I decided that it was time to write my first book. Bitcoin was the topic that had driven me into a frenzy of creativity, consumed my thoughts and is the most exciting technology I have encountered since the Internet. It was now time to share my discovery of this amazing technology and my passion with a broader audience. This is the bitcoin book.
Two years later, after creating a number of small startups to explore various bitcoin-related services and products, I decided that it was time to write my first book. Bitcoin was the topic that had driven me into a frenzy of creativity, consumed my thoughts and was the most exciting technology I had encountered since the Internet. It was now time to share my passion about this amazing technology with a broader audience.
=== Intended Audience
This book is mostly intended for coders. If you can use a programming language, this book will teach you how cryptographic currencies work, how to use them and how to develop software that works with them. The first few chapters are also suitable as an in-depth introduction to bitcoin for non-coders - those trying to understand the inner workings of bitcoin and cryptocurrencies. The examples are illustrated in Python and on the command-line of a Unix-like operating system such as Linux.
=== Early-Release Note
The early release version of the book is a *raw and rough draft* and will change regularly. New chapters will be added as they are drafted and there will be plenty of changes to the content, examples and diagrams. There will be factual and technical errors in the early release and some of the examples may not work or refer to obsolete versions of the code. Nevertheless, I hope you will enjoy the content and find it useful. I also hope that you will take the opportunity to "fork" the source code of the book and provide feedback by creating a pull request or submitting a patch. I present this work in the spirit of Cunningham's Law, named after the inventor of the wiki, Ward Cunningham:
_The best way to get the right answer on the Internet is not to ask a question, it's to post the wrong answer_
I hope you can help me find and publish the "right answer" by the time this book is ready to print.
The early-release source code is published on Github: https://github.com/aantonop/bitcoinbook
File an "Issue", or fork, modify and create a "Pull Request" with your changes.
This book is mostly intended for coders. If you can use a programming language, this book will teach you how cryptographic currencies work, how to use them and how to develop software that works with them. The first few chapters are also suitable as an in-depth introduction to bitcoin for non-coders - those trying to understand the inner workings of bitcoin and crypto-currencies.
=== Why Are There Bugs On The Cover?
@ -29,7 +17,7 @@ The Leafcutter Ant is a species that exhibits highly complex behavior in a colon
While ants form a caste-based society and have a queen for producing offspring, there is no central authority or leader in an ant colony. The highly intelligent and sophisticated behavior exhibited by a multi-million member colony is an emergent property from the interaction of the individuals in a social network.
Nature demonstrates that de-centralized systems can be resilient and can produce emergent complexity and sophistication without the need for a central authority, hierarchy or complex parts.
Nature demonstrates that de-centralized systems can be resilient and can produce emergent complexity and incredible sophistication without the need for a central authority, hierarchy or complex parts.
Bitcoin is a highly sophisticated de-centralized trust network that can support a myriad of financial processes. Yet, each node in the bitcoin network follows a few simple mathematical rules. The interaction between many nodes is what leads to the emergence of the sophisticated behavior, not any inherent complexity or trust in any single node. Like an ant colony, the bitcoin network is a resilient network of simple nodes following simple rules that together can do amazing things without any central coordination.
@ -56,11 +44,31 @@ This icon signifies a tip, suggestion, or general note.
This icon indicates a warning or caution.
====
=== Github Source Code
This book is available on Github, as a repository that contains the text, images and code:
https://github.com/aantonop/bitcoinbook
Fork the book code, try the code examples, or submit corrections via Github.
=== Code Examples
The examples are illustrated in Python, C++ and using the command-line of a Unix-like operating system such as Linux or Mac OSX. All code snippets are available in the Github repository and can be accessed online at https://github.com/aantonop/bitcoinbook in the +code+ subdirectory of the main repository.
All the code snippets can be replicated on most operating systems with a minimal installation of compilers and interpreters for the corresponding languages. Where necessary, we provide basic installation instructions and step-by-step examples of the output of those instructions.
Some of the code snippets and code output have been re-formatted for print. In all such cases, the lines have been split by a backslash "\" character, followed by a newline character. When transcribing the examples, remove those two characters and join the lines again and you should see identical results as shown in the example.
All the code snippets use real values and calculations where possible, so that you can build from example to example and see the same results in any code you write to calculate the same values. For example, the private keys and corresponding public keys and addresses are all real. The sample transactions, blocks and blockchain references have all been introduced in the actual bitcoin blockchain and are part of the public ledger, so you can review them on any bitcoin system.
=== Using Code Examples
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “_Book Title_ by Some Author (O’Reilly). Copyright 2012 Some Copyright Holder, 978-0-596-xxxx-x.”
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “_Mastering Bitcoin_ by Andreas M. Antonopoulos (O’Reilly). Copyright 2014 Andreas M. Antonopoulos, 978-1449374044.”
Some editions of this books are offered under an open source license, such as CC-BY-NC (creativecommons.org) in which case the terms of that licenses apply.
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at pass:[<email>permissions@oreilly.com</email>].
@ -103,6 +111,65 @@ Find us on Facebook: link:$$http://facebook.com/oreilly$$[]
Follow us on Twitter: link:$$http://twitter.com/oreillymedia$$[]
Watch us on YouTube: link:$$http://www.youtube.com/oreillymedia$$[]
++++
<?hard-pagebreak?>
++++
=== Acknowledgments
This book represents the efforts and contributions of many people. I am grateful for all the help I received from friends, colleagues and even complete strangers, who joined me in this effort to write the definitive technical book on crypto-currencies and bitcoin.
It is impossible to make a distinction between the bitcoin technology and the bitcoin community, and this book is as much a product of that community as it is a book on the technology. My work on this book was encouraged, cheered on, supported and rewarded by the entire bitcoin community from the very beginning until the very end. More than anything, this book has allowed me to be part of a wonderful community for two years and I can't thank you enough for accepting me in this community. There are far too many people to mention by name - people I've met at conferences, events, seminars, meetups, pizza gatherings and small private gatherings, as well as many who communicated with me by twitter, on reddit, on bitcointalk.org and on github who have had an impact on this book. Every idea, analogy, question, answer, and explanation you find in this book was at some point inspired, tested or improved through my interactions with the community. Thank you all for your support, without you this book would not have happened. I am forever grateful.
The journey to becoming an author starts long before the first book, of course. My first language (and schooling) was Greek, so I had to take a remedial English Writing course in my first year of university. I owe thanks to Diana Kordas, my English Writing teacher, who helped me build confidence and skills that year. Later, as a professional, I developed my technical writing skills on the topic of data centers, writing for Network World magazine. I owe thanks to John Dix and John Gallant who gave me my first writing job as a columnist at Network World and to my editor Michael Cooney and my colleague Johna Till Johnson who edited my columns and made them fit for publication. Writing 500 words a week for four years gave me enough experience to eventually consider becoming an author. Thanks to Jean for her early encouragement to become an author and for always believing and insisting that I had a book in me.
Thanks also to those who supported me when I submitted my book proposal to O'Reilly, by providing references and reviewing the proposal. Specifically, thanks to John Gallant, Gregory Ness, Richard Stiennon, Joel Snyder, Adam B. Levine, Sandra Gittlen, John Dix, Johna Till Johnson, Roger Ver and Jon Matonis. Special thanks to Richard Kagan and Tymon Mattoszko who reviewed early versions of the proposal and Matthew Owain Taylor who copy-edited the proposal.
Thanks to Cricket Liu, author of O'Reilly title "DNS and BIND" who introduced me to O'Reilly. Thanks also to Michael Loukides and Allyson MacDonald at O'Reilly who worked for months to help make this book happen. Allyson was especially patient when deadlines were missed and deliverables delayed as life intervened in our planned schedule.
The first few drafts of the first few chapters were the hardest, because bitcoin is a difficult subject to unravel. Every time I pulled on one thread of the bitcoin technology, I had to pull in the whole thing. I repeatedly got stuck and a bit despondent as I struggled to make the topic easy to understand and create a narrative around such a dense technical subject. Eventually, I decided to tell the story of bitcoin through the stories of the people using bitcoin and the whole book became a lot easier to write. I owe thanks to my friend and mentor, Richard Kagan, who helped me unravel the story and get past the moments of writer's block, and Pamela Morgan who reviewed early drafts of each chapter and asked the hard questions to make them better. Also, thanks to the developers of the San Francisco Bitcoin Developers Meetup group and Taariq Lewis, the group's co-founder, for helping to test the early material.
During the development of the book, I made early drafts available on Github and invited public comments. More than a hundred comments, suggestions, corrections and contributions were submitted in response. Those contributions are explicitly acknowledged, with my thanks, in <<github_contrib>>. Special thanks to Minh T. Nguyen who volunteered to manage the Github contributions and added many significant contributions himself. Thanks also to Andrew Naugler for infographic design.
Once the book was drafted, it went through several rounds of technical review. Thanks to Cricket Liu and Lorne Lantz for their thorough review, comments and support.
Several bitcoin developers contributed code samples, reviews, comments and encouragement. Thanks to Amir Taaki for example code snippets and many great comments, Vitalik Buterin and Richard Kiss for help with elliptic curve math and code contributions, Gavin Andresen for corrections, comments and encouragement, Michalis Kargakis for comments, contributions and btcd writeup.
I owe my love of words and books to my mother, Theresa, who raised me in a house with books lining every wall. My mother also bought me my first computer in 1982, despite being a self-described technophobe. My father, Menelaos, a civil engineer who just published his first book at 80 years old, was the one who taught me logical and analytical thinking and a love of science and engineering.
Thank you all for supporting me throughout this journey.
[[github_contrib]]
==== Early Release Draft (Github Contributions)
Many contributors offered comments, corrections and additions to the early-release draft on Github. Thank you all for your contributions to this book. Notable contributors included the following:
===== (Name - Github ID)
* *Minh T. Nguyen - enderminh: Github contribution editor*
* Ed Eykholt - edeykholt
* Michalis Kargakis - kargakis
* Erik Wahlström - erikwam
* Richard Kiss - richardkiss
* Eric Winchell - winchell
* Sergej Kotliar - ziggamon
* Nagaraj Hubli - nagarajhubli
* ethers
* Alex Waters - alexwaters
* Mihail Russu - MihailRussu
* Ish Ot Jr. - ishotjr
* James Addison - jayaddison
* Nekomata - nekomata-3
* Simon de la Rouviere - simondlr
* Chapman Shoop - belovachap
* Holger Schinzel - schinzelh
* effectsToCause - vericoin
* Stephan Oeste - Emzy
* Joe Bauers - joebauers
* Jason Bisterfeldt - jbisterfeldt
* Ed Leafe - EdLeafe
++++
<?hard-pagebreak?>
++++
@ -115,7 +182,11 @@ address (aka public key)::
((("bitcoin address")))
((("address", see="bitcoin address")))
((("public key", see="bitcoin address")))
A bitcoin address looks like +1DSrfJdB2AnWaFNgSbv3MZC2m74996JafV+, they always start with a one. You can have as many as you like, share them so people can send you coins.
A bitcoin address looks like +1DSrfJdB2AnWaFNgSbv3MZC2m74996JafV+ - they consist of a string of letters and numbers starting with a "1" (number one). Just like you ask others to send an email to your email address, you would ask others to send you bitcoin to your bitcoin address.
bip::
((("bip")))
Bitcoin Improvement Proposals. A set of proposals that members of the bitcoin community have submitted to improve bitcoin. For example BIP0021 is a proposal to improve the bitcoin URI scheme.
bitcoin::
((("bitcoin")))
@ -131,7 +202,7 @@ blockchain::
confirmations::
((("confirmations")))
Once a transaction is included in a block, it has "one confirmation". As soon as _another_ block is mined on the same blockchain, the transaction has two confirmations etc. Six or more confirmations is considered final.
Once a transaction is included in a block, it has "one confirmation". As soon as _another_ block is mined on the same blockchain, the transaction has two confirmations etc. Six or more confirmations is considered sufficient proof that a transaction cannot be reversed.
difficulty::
((("difficulty")))
@ -147,7 +218,7 @@ difficulty re-targeting::
fees::
((("fees")))
An excess amount included in each transaction as a network fee or additional reward to the miner who finds the proof-of-work for the new block. Currently 0.5 mBTC minimum.
The sender of a transaction often includes a fee to the network for processing their requested transaction. Most transactions require a minimum fee of 0.5mBTC.
hash::
((("hash")))
@ -176,7 +247,7 @@ reward::
secret key (aka private key)::
((("secret key")))
((("private key", see="secret key")))
The secret number that unlocks bitcoins sent to the corresponding address.
The secret number that unlocks bitcoins sent to the corresponding address. A secret key looks like +5J76sF8L5jTtzE96r66Sf8cka9y44wdpJjMwCxR3tzLh3ibVPxh+
transaction::
((("transaction")))
@ -184,30 +255,4 @@ transaction::
wallet::
((("wallet")))
Software that holds all your addresses. Use it to send bitcoin and manage your keys.
++++
<?hard-pagebreak?>
++++
=== Acknowledgments
==== Early Release Draft (Github Contributions)
Many contributors offered comments, corrections and additions to the early-release draft on Github. Thank you all for your contributions to this book. Notable contributors included the following:
===== (Name - Github ID)
* *Minh T. Nguyen - enderminh: Github contribution editor*
* Erik Wahlström - erikwam
* Eric Winchell - winchell
* Richard Kiss - richardkiss
* Sergej Kotliar - ziggamon
* Nagaraj Hubli - nagarajhubli
* Michalis Kargakis - kargakis
* Ish Ot Jr. - ishotjr
* Alex Waters - alexwaters
* Mihail Russu - MihailRussu
* James Addison - jayaddison
* Joe Bauers - joebauers
* Stephan Oeste - Emzy
Software that holds all your bitcoin addresses and secret keys. Use it to send, receive and store your bitcoin.