arno
/
bitcoinbook
mirror of https://github.com/bitcoinbook/bitcoinbook
1
0
Fork 0
Code Issues Releases Wiki Activity

editorial changes for final print

Browse Source
pull/161/head Edition1Print1
Andreas M. Antonopoulos 10 years ago
parent f64f12d61f 7332e8d0da
commit 665f0b69ba
179 changed files with 6072 additions and 5459 deletions
Show Stats Download Patch File Download Diff File

1
.gitignore vendored
Unescape View File

@ -1,5 +1,4 @@
*.html
*.txt
dump.asciidoc
code/python-env
*.csv

270
appdx-bips.asciidoc
Unescape View File

@ -1,133 +1,137 @@
== Appendix - Bitcoin Improvement Proposals
Bitcoin Improvement Proposals are design documents providing information to the Bitcoin community, or describing a new feature for Bitcoin or its processes or environment.
As per BIP0001 _BIP Purpose and Guidelines_, there are three kinds of BIP:
* A _Standards_ Track BIP describes any change that affects most or all Bitcoin implementations, such as a change to the network protocol, a change in block or transaction validity rules, or any change or addition that affects the interoperability of applications using Bitcoin.
* An _Informational_ BIP describes a Bitcoin design issue, or provides general guidelines or information to the Bitcoin community, but does not propose a new feature. Informational BIPs do not necessarily represent a Bitcoin community consensus or recommendation, so users and implementors are free to ignore Informational BIPs or follow their advice.
* A _Process_ BIP describes a process surrounding Bitcoin, or proposes a change to (or an event in) a process. Process BIPs are like Standards Track BIPs but apply to areas other than the Bitcoin protocol itself. They may propose an implementation, but not to Bitcoin's codebase; they often require community consensus; unlike Informational BIPs, they are more than recommendations, and users are typically not free to ignore them. Examples include procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Bitcoin development. Any meta-BIP is also considered a Process BIP.
Bitcoin Improvement Proposals are recorded in a versioned repository on Github at https://github.com/bitcoin/bips. The list below is a snapshot of BIPs in the Fall of 2014. Consult the authoritative repository for up-to-date information on existing BIPs and their contents.
[options="header"]
|=======================================================================
|BIP# | Link | Title |Owner |Type |Status
|[[bip0001]]1|link:https://github.com/bitcoin/bips/blob/master/bip-0001.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0001.mediawiki]|BIP Purpose and Guidelines |Amir Taaki
|Standard |Active
|[[bip0010]]10|link:https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki]|Multi-Sig Transaction Distribution |Alan
Reiner |Informational |Draft
|[[bip0011]]11|link:https://github.com/bitcoin/bips/blob/master/bip-0011.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0011.mediawiki]|M-of-N Standard Transactions |Gavin
Andresen |Standard |Accepted
|[[bip0012]]12|link:https://github.com/bitcoin/bips/blob/master/bip-0012.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0012.mediawiki]|OP_EVAL |Gavin Andresen |Standard
|Withdrawn
|[[bip0013]]13|link:https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki]|Address Format for pay-to-script-hash
|Gavin Andresen |Standard |Final
|[[bip0014]]14|link:https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki]|Protocol Version and User Agent |Amir
Taaki, Patrick Strateman |Standard |Accepted
|[[bip0015]]15|link:https://github.com/bitcoin/bips/blob/master/bip-0015.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0015.mediawiki]|Aliases |Amir Taaki |Standard |Withdrawn
|[[bip0016]]16|link:https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki]|Pay To Script Hash |Gavin Andresen
|Standard |Accepted
|[[bip0017]]17|link:https://github.com/bitcoin/bips/blob/master/bip-0017.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0017.mediawiki]|OP_CHECKHASHVERIFY (CHV) |Luke Dashjr
|Withdrawn |Draft
|[[bip0018]]18|link:https://github.com/bitcoin/bips/blob/master/bip-0018.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0018.mediawiki]|hashScriptCheck |Luke Dashjr |Standard
|Draft
|[[bip0019]]19|link:https://github.com/bitcoin/bips/blob/master/bip-0019.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0019.mediawiki]|M-of-N Standard Transactions (Low SigOp)
|Luke Dashjr |Standard |Draft
|[[bip0020]]20|link:https://github.com/bitcoin/bips/blob/master/bip-0020.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0020.mediawiki]|URI Scheme |Luke Dashjr |Standard
|Replaced
|[[bip0021]]21|link:https://github.com/bitcoin/bips/blob/master/bip-0021.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0021.mediawiki]|URI Scheme |Nils Schneider, Matt Corallo
|Standard |Accepted
|[[bip0022]]22|link:https://github.com/bitcoin/bips/blob/master/bip-0022.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0022.mediawiki]|getblocktemplate - Fundamentals |Luke
Dashjr |Standard |Accepted
|[[bip0023]]23|link:https://github.com/bitcoin/bips/blob/master/bip-0023.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0023.mediawiki]|getblocktemplate - Pooled Mining |Luke
Dashjr |Standard |Accepted
|[[bip0030]]30|link:https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki]|Duplicate transactions |Pieter Wuille
|Standard |Accepted
|[[bip0031]]31|link:https://github.com/bitcoin/bips/blob/master/bip-0031.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0031.mediawiki]|Pong message |Mike Hearn |Standard
|Accepted
|[[bip0032]]32|link:https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki]|Hierarchical Deterministic Wallets |Pieter
Wuille |Informational |Accepted
|[[bip0033]]33|link:https://github.com/bitcoin/bips/blob/master/bip-0033.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0033.mediawiki]|Stratized Nodes |Amir Taaki |Standard
|Draft
|[[bip0034]]34|link:https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki]|Block v2, Height in coinbase |Gavin
Andresen |Standard |Accepted
|[[bip0035]]35|link:https://github.com/bitcoin/bips/blob/master/bip-0035.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0035.mediawiki]|mempool message |Jeff Garzik |Standard
|Accepted
|[[bip0036]]36|link:https://github.com/bitcoin/bips/blob/master/bip-0036.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0036.mediawiki]|Custom Services |Stefan Thomas |Standard
|Draft
|[[bip0037]]37|link:https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki]|Bloom filtering |Mike Hearn and Matt
Corallo |Standard |Accepted
|[[bip0038]]38|link:https://github.com/bitcoin/bips/blob/master/bip-0038.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0038.mediawiki]|Passphrase-protected private key |Mike
Caldwell |Standard |Draft
|[[bip0039]]39|link:https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki]|Mnemonic code for generating deterministic
keys |Slush |Standard |Draft
|[[bip0040]]40|link:https://github.com/bitcoin/bips/blob/master/bip-0040.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0040.mediawiki]|Stratum wire protocol |Slush |Standard |BIP number allocated
|[[bip0041]]41|link:https://github.com/bitcoin/bips/blob/master/bip-0041.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0041.mediawiki]|Stratum mining protocol |Slush |Standard |BIP number allocated
|[[bip0042]]42|link:https://github.com/bitcoin/bips/blob/master/bip-0042.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0042.mediawiki]|A finite monetary supply for Bitcoin
|Pieter Wuille |Standard |Draft
|[[bip0043]]43|link:https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki]|Purpose Field for Deterministic Wallets
|Slush |Standard |Draft
|[[bip0044]]44|link:https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki]|Multi-Account Hierarchy for Deterministic
Wallets |Slush |Standard |Draft
|[[bip0050]]50|link:https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki]|March 2013 Chain Fork Post-Mortem |Gavin
Andresen |Informational |Draft
|[[bip0060]]60|link:https://github.com/bitcoin/bips/blob/master/bip-0060.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0060.mediawiki]|Fixed Length "version" Message
(Relay-Transactions Field) |Amir Taaki |Standard |Draft
|[[bip0061]]61|link:https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki]|"reject" P2P message |Gavin Andresen
|Standard |Draft
|[[bip0062]]62|link:https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki]|Dealing with malleability |Pieter Wuille
|Standard |Draft
|[[bip0063]]63|link:https://github.com/bitcoin/bips/blob/master/bip-0063.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0063.mediawiki]|Stealth Addresses |Peter Todd |Standard |BIP number allocated
|[[bip0064]]64|link:https://github.com/bitcoin/bips/blob/master/bip-0064.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0064.mediawiki]|getutxos message |Mike Hearn |Standard
|Draft
|[[bip0070]]70|link:https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki]|Payment protocol |Gavin Andresen |Standard
|Draft
|[[bip0071]]71|link:https://github.com/bitcoin/bips/blob/master/bip-0071.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0071.mediawiki]|Payment protocol MIME types |Gavin
Andresen |Standard |Draft
|[[bip0072]]72|link:https://github.com/bitcoin/bips/blob/master/bip-0072.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0072.mediawiki]|Payment protocol URIs |Gavin Andresen
|Standard |Draft
|[[bip0073]]73|link:https://github.com/bitcoin/bips/blob/master/bip-0073.mediawiki[https://github.com/bitcoin/bips/blob/master/bip-0073.mediawiki]|Use "Accept" header with Payment Request
URLs |Stephen Pair |Standard |Draft
|=======================================================================
[[appdxbitcoinimpproposals]]
[appendix]
== Bitcoin Improvement Proposals
((("Bitcoin improvement proposals", id="ix_appdx-bips-asciidoc0", range="startofrange")))Bitcoin improvement proposals are design documents providing information to the bitcoin community, or describing a new feature for bitcoin or its processes or environment.
As per BIP0001 _BIP Purpose and Guidelines_, there are three kinds of BIP:
_Standard_ BIP:: Describes any change that affects most or all bitcoin implementations, such as a change to the network protocol, a change in block or transaction validity rules, or any change or addition that affects the interoperability of applications using bitcoin.
_Informational_ BIP:: Describes a bitcoin design issue, or provides general guidelines or information to the bitcoin community, but does not propose a new feature. Informational BIPs do not necessarily represent a bitcoin community consensus or recommendation, so users and implementors may ignore informational BIPs or follow their advice.
_Process_ BIP:: Describes a bitcoin process, or proposes a change to (or an event in) a process. Process BIPs are like standard BIPs but apply to areas other than the bitcoin protocol itself. They might propose an implementation, but not to bitcoin's codebase; they often require community consensus; and unlike informational BIPs, they are more than recommendations, and users are typically not free to ignore them. Examples include procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Bitcoin development. Any meta-BIP is also considered a process BIP.
Bitcoin improvement proposals are recorded in a versioned repository on https://github.com/bitcoin/bips[GitHub]. <<table_d-1>> shows a snapshot of BIPs in the Fall of 2014. Consult the authoritative repository for up-to-date information on existing BIPs and their contents.
[[table_d-1]]
.Snapshot of BIPs
[options="header"]
|=======================================================================
|BIP# | Link | Title |Owner |Type |Status
|[[bip0001]]1|https://github.com/bitcoin/bips/blob/master/bip-0001.mediawiki|BIP Purpose and Guidelines |Amir Taaki
|Standard |Active
|[[bip0010]]10|https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki|Multi-Sig Transaction Distribution |Alan
Reiner |Informational |Draft
|[[bip0011]]11|https://github.com/bitcoin/bips/blob/master/bip-0011.mediawiki|M-of-N Standard Transactions |Gavin
Andresen |Standard |Accepted
|[[bip0012]]12|https://github.com/bitcoin/bips/blob/master/bip-0012.mediawiki|OP_EVAL |Gavin Andresen |Standard
|Withdrawn
|[[bip0013]]13|https://github.com/bitcoin/bips/blob/master/bip-0013.mediawiki|Address Format for pay-to-script-hash
|Gavin Andresen |Standard |Final
|[[bip0014]]14|https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki|Protocol Version and User Agent |Amir
Taaki, Patrick Strateman |Standard |Accepted
|[[bip0015]]15|https://github.com/bitcoin/bips/blob/master/bip-0015.mediawiki|Aliases |Amir Taaki |Standard |Withdrawn
|[[bip0016]]16|https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki|Pay To Script Hash |Gavin Andresen
|Standard |Accepted
|[[bip0017]]17|https://github.com/bitcoin/bips/blob/master/bip-0017.mediawiki|OP_CHECKHASHVERIFY (CHV) |Luke Dashjr
|Withdrawn |Draft
|[[bip0018]]18|https://github.com/bitcoin/bips/blob/master/bip-0018.mediawikilink:|hashScriptCheck |Luke Dashjr |Standard
|Draft
|[[bip0019]]19|https://github.com/bitcoin/bips/blob/master/bip-0019.mediawiki|M-of-N Standard Transactions (Low SigOp)
|Luke Dashjr |Standard |Draft
|[[bip0020]]20|https://github.com/bitcoin/bips/blob/master/bip-0020.mediawiki|URI Scheme |Luke Dashjr |Standard
|Replaced
|[[bip0021]]21|https://github.com/bitcoin/bips/blob/master/bip-0021.mediawiki|URI Scheme |Nils Schneider, Matt Corallo
|Standard |Accepted
|[[bip0022]]22|https://github.com/bitcoin/bips/blob/master/bip-0022.mediawiki|getblocktemplate - Fundamentals |Luke
Dashjr |Standard |Accepted
|[[bip0023]]23|https://github.com/bitcoin/bips/blob/master/bip-0023.mediawiki|getblocktemplate - Pooled Mining |Luke
Dashjr |Standard |Accepted
|[[bip0030]]30|https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki|Duplicate transactions |Pieter Wuille
|Standard |Accepted
|[[bip0031]]31|https://github.com/bitcoin/bips/blob/master/bip-0031.mediawiki|Pong message |Mike Hearn |Standard
|Accepted
|[[bip0032]]32|https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki|Hierarchical Deterministic Wallets |Pieter
Wuille |Informational |Accepted
|[[bip0033]]33|https://github.com/bitcoin/bips/blob/master/bip-0033.mediawiki|Stratized Nodes |Amir Taaki |Standard
|Draft
|[[bip0034]]34|https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki|Block v2, Height in coinbase |Gavin
Andresen |Standard |Accepted
|[[bip0035]]35|https://github.com/bitcoin/bips/blob/master/bip-0035.mediawiki|mempool message |Jeff Garzik |Standard
|Accepted
|[[bip0036]]36|https://github.com/bitcoin/bips/blob/master/bip-0036.mediawiki|Custom Services |Stefan Thomas |Standard
|Draft
|[[bip0037]]37|https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki|Bloom filtering |Mike Hearn and Matt
Corallo |Standard |Accepted
|[[bip0038]]38|https://github.com/bitcoin/bips/blob/master/bip-0038.mediawiki|Passphrase-protected private key |Mike
Caldwell |Standard |Draft
|[[bip0039]]39|https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki|Mnemonic code for generating deterministic
keys |Slush |Standard |Draft
|[[bip0040]]40||Stratum wire protocol |Slush |Standard |BIP number allocated
|[[bip0041]]41||Stratum mining protocol |Slush |Standard |BIP number allocated
|[[bip0042]]42|https://github.com/bitcoin/bips/blob/master/bip-0042.mediawiki|A finite monetary supply for bitcoin
|Pieter Wuille |Standard |Draft
|[[bip0043]]43|https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki|Purpose Field for Deterministic Wallets
|Slush |Standard |Draft
|[[bip0044]]44|https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki|Multi-Account Hierarchy for Deterministic
Wallets |Slush |Standard |Draft
|[[bip0050]]50|https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki|March 2013 Chain Fork Post-Mortem |Gavin
Andresen |Informational |Draft
|[[bip0060]]60|https://github.com/bitcoin/bips/blob/master/bip-0060.mediawiki|Fixed Length "version" Message
(Relay-Transactions Field) |Amir Taaki |Standard |Draft
|[[bip0061]]61|https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki|"reject" P2P message |Gavin Andresen
|Standard |Draft
|[[bip0062]]62|https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki|Dealing with malleability |Pieter Wuille
|Standard |Draft
|[[bip0063]]63||Stealth Addresses |Peter Todd |Standard |BIP number allocated
|[[bip0064]]64|https://github.com/bitcoin/bips/blob/master/bip-0064.mediawiki|getutxos message |Mike Hearn |Standard
|Draft
|[[bip0070]]70|https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki|Payment protocol |Gavin Andresen |Standard
|Draft
|[[bip0071]]71|https://github.com/bitcoin/bips/blob/master/bip-0071.mediawiki|Payment protocol MIME types |Gavin
Andresen |Standard |Draft
|[[bip0072]]72|https://github.com/bitcoin/bips/blob/master/bip-0072.mediawiki|Payment protocol URIs |Gavin Andresen
|Standard |Draft
|[[bip0073]]73|https://github.com/bitcoin/bips/blob/master/bip-0073.mediawiki|Use "Accept" header with Payment Request
URLs |Stephen Pair |Standard |Draft(((range="endofrange", startref="ix_appdx-bips-asciidoc0")))
|=======================================================================

440
appdx-pycoin.asciidoc
Unescape View File

@ -0,0 +1,440 @@
[[appdx-pycoin]]
[appendix]
== pycoin, ku, and tx
The Python library http://github.com/richardkiss/pycoin[+pycoin+], originally written and maintained by Richard Kiss, is a Python-based library that supports manipulation of bitcoin keys and transactions, even supporting the scripting language enough to properly deal with nonstandard transactions.
The pycoin library supports both Python 2 (2.7.x) and Python 3 (after 3.3), and comes with some handy command-line utilities, ku and tx.
=== Key Utility (KU)
((("key utility (ku)", id="ix_appdx-pycoin-asciidoc0", range="startofrange")))The command-line utility +ku+ ("key utility") is a Swiss Army knife for manipulating keys. It supports BIP32 keys, WIF, and addresses (bitcoin and alt coins). Following are some examples.
Create a BIP32 key using the default entropy sources of GPG and _/dev/random_:
====
----
$ ku create
input : create
network : Bitcoin
wallet key : xprv9s21ZrQH143K3LU5ctPZTBnb9kTjA5Su9DcWHvXJemiJBsY7VqXUG7hipgdWaU
m2nhnzdvxJf5KJo9vjP2nABX65c5sFsWsV8oXcbpehtJi
public version : xpub661MyMwAqRbcFpYYiuvZpKjKhnJDZYAkWSY76JvvD7FH4fsG3Nqiov2CfxzxY8
DGcpfT56AMFeo8M8KPkFMfLUtvwjwb6WPv8rY65L2q8Hz
tree depth : 0
fingerprint : 9d9c6092
parent f'print : 00000000
child index : 0
chain code : 80574fb260edaa4905bc86c9a47d30c697c50047ed466c0d4a5167f6821e8f3c
private key : yes
secret exponent : 112471538590155650688604752840386134637231974546906847202389294096567806844862
hex : f8a8a28b28a916e1043cc0aca52033a18a13cab1638d544006469bc171fddfbe
wif : L5Z54xi6qJusQT42JHA44mfPVZGjyb4XBRWfxAzUWwRiGx1kV4sP
uncompressed : 5KhoEavGNNH4GHKoy2Ptu4KfdNp4r56L5B5un8FP6RZnbsz5Nmb
public pair x : 76460638240546478364843397478278468101877117767873462127021560368290114016034
public pair y : 59807879657469774102040120298272207730921291736633247737077406753676825777701
x as hex : a90b3008792432060fa04365941e09a8e4adf928bdbdb9dad41131274e379322
y as hex : 843a0f6ed9c0eb1962c74533795406914fe3f1957c5238951f4fe245a4fcd625
y parity : odd
key pair as sec : 03a90b3008792432060fa04365941e09a8e4adf928bdbdb9dad41131274e379322
uncompressed : 04a90b3008792432060fa04365941e09a8e4adf928bdbdb9dad41131274e379322
843a0f6ed9c0eb1962c74533795406914fe3f1957c5238951f4fe245a4fcd625
hash160 : 9d9c609247174ae323acfc96c852753fe3c8819d
uncompressed : 8870d869800c9b91ce1eb460f4c60540f87c15d7
Bitcoin address : 1FNNRQ5fSv1wBi5gyfVBs2rkNheMGt86sp
uncompressed : 1DSS5isnH4FsVaLVjeVXewVSpfqktdiQAM
----
====
Create a BIP32 key from a passphrase:
[WARNING]
====
The passphrase in this example is way too easy to guess.
====
----
$ ku P:foo
input : P:foo
network : Bitcoin
wallet key : xprv9s21ZrQH143K31AgNK5pyVvW23gHnkBq2wh5aEk6g1s496M8ZMjxncCKZKgb5j
ZoY5eSJMJ2Vbyvi2hbmQnCuHBujZ2WXGTux1X2k9Krdtq
public version : xpub661MyMwAqRbcFVF9ULcqLdsEa5WnCCugQAcgNd9iEMQ31tgH6u4DLQWoQayvtS
VYFvXz2vPPpbXE1qpjoUFidhjFj82pVShWu9curWmb2zy
tree depth : 0
fingerprint : 5d353a2e
parent f'print : 00000000
child index : 0
chain code : 5eeb1023fd6dd1ae52a005ce0e73420821e1d90e08be980a85e9111fd7646bbc
private key : yes
secret exponent : 65825730547097305716057160437970790220123864299761908948746835886007793998275
hex : 91880b0e3017ba586b735fe7d04f1790f3c46b818a2151fb2def5f14dd2fd9c3
wif : L26c3H6jEPVSqAr1usXUp9qtQJw6NHgApq6Ls4ncyqtsvcq2MwKH
uncompressed : 5JvNzA5vXDoKYJdw8SwwLHxUxaWvn9mDea6k1vRPCX7KLUVWa7W
public pair x : 81821982719381104061777349269130419024493616650993589394553404347774393168191
public pair y : 58994218069605424278320703250689780154785099509277691723126325051200459038290
x as hex : b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f
y as hex : 826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52
y parity : even
key pair as sec : 02b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f
uncompressed : 04b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f
826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52
hash160 : 5d353a2ecdb262477172852d57a3f11de0c19286
uncompressed : e5bd3a7e6cb62b4c820e51200fb1c148d79e67da
Bitcoin address : 19Vqc8uLTfUonmxUEZac7fz1M5c5ZZbAii
uncompressed : 1MwkRkogzBRMehBntgcq2aJhXCXStJTXHT
----
====
Get info as JSON:
====
----
$ ku P:foo -P -j
----
[source,json]
----
{
"y_parity": "even",
"public_pair_y_hex": "826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52",
"private_key": "no",
"parent_fingerprint": "00000000",
"tree_depth": "0",
"network": "Bitcoin",
"btc_address_uncompressed": "1MwkRkogzBRMehBntgcq2aJhXCXStJTXHT",
"key_pair_as_sec_uncompressed": "04b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f826d8b4d3010aea16ff4c1c1d3ae68541d9a04df54a2c48cc241c2983544de52",
"public_pair_x_hex": "b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f",
"wallet_key": "xpub661MyMwAqRbcFVF9ULcqLdsEa5WnCCugQAcgNd9iEMQ31tgH6u4DLQWoQayvtSVYFvXz2vPPpbXE1qpjoUFidhjFj82pVShWu9curWmb2zy",
"chain_code": "5eeb1023fd6dd1ae52a005ce0e73420821e1d90e08be980a85e9111fd7646bbc",
"child_index": "0",
"hash160_uncompressed": "e5bd3a7e6cb62b4c820e51200fb1c148d79e67da",
"btc_address": "19Vqc8uLTfUonmxUEZac7fz1M5c5ZZbAii",
"fingerprint": "5d353a2e",
"hash160": "5d353a2ecdb262477172852d57a3f11de0c19286",
"input": "P:foo",
"public_pair_x": "81821982719381104061777349269130419024493616650993589394553404347774393168191",
"public_pair_y": "58994218069605424278320703250689780154785099509277691723126325051200459038290",
"key_pair_as_sec": "02b4e599dfa44555a4ed38bcfff0071d5af676a86abf123c5b4b4e8e67a0b0b13f"
}
----
====
Public BIP32 key:
====
----
$ ku -w -P P:foo
xpub661MyMwAqRbcFVF9ULcqLdsEa5WnCCugQAcgNd9iEMQ31tgH6u4DLQWoQayvtSVYFvXz2vPPpbXE1qpjoUFidhjFj82pVShWu9curWmb2zy
----
====
Generate a subkey:
====
----
$ ku -w -s3/2 P:foo
xprv9wTErTSkjVyJa1v4cUTFMFkWMe5eu8ErbQcs9xajnsUzCBT7ykHAwdrxvG3g3f6BFk7ms5hHBvmbdutNmyg6iogWKxx6mefEw4M8EroLgKj
----
====
Hardened subkey:
====
----
$ ku -w -s3/2H P:foo
xprv9wTErTSu5AWGkDeUPmqBcbZWX1xq85ZNX9iQRQW9DXwygFp7iRGJo79dsVctcsCHsnZ3XU3DhsuaGZbDh8iDkBN45k67UKsJUXM1JfRCdn1
----
====
WIF:
====
----
$ ku -W P:foo
L26c3H6jEPVSqAr1usXUp9qtQJw6NHgApq6Ls4ncyqtsvcq2MwKH
----
====
Address:
====
----
$ ku -a P:foo
19Vqc8uLTfUonmxUEZac7fz1M5c5ZZbAii
----
====
Generate a bunch of subkeys:
====
----
$ ku P:foo -s 0/0-5 -w
xprv9xWkBDfyBXmZjBG9EiXBpy67KK72fphUp9utJokEBFtjsjiuKUUDF5V3TU8U8cDzytqYnSekc8bYuJS8G3bhXxKWB89Ggn2dzLcoJsuEdRK
xprv9xWkBDfyBXmZnzKf3bAGifK593gT7WJZPnYAmvc77gUQVej5QHckc5Adtwxa28ACmANi9XhCrRvtFqQcUxt8rUgFz3souMiDdWxJDZnQxzx
xprv9xWkBDfyBXmZqdXA8y4SWqfBdy71gSW9sjx9JpCiJEiBwSMQyRxan6srXUPBtj3PTxQFkZJAiwoUpmvtrxKZu4zfsnr3pqyy2vthpkwuoVq
xprv9xWkBDfyBXmZsA85GyWj9uYPyoQv826YAadKWMaaEosNrFBKgj2TqWuiWY3zuqxYGpHfv9cnGj5P7e8EskpzKL1Y8Gk9aX6QbryA5raK73p
xprv9xWkBDfyBXmZv2q3N66hhZ8DAcEnQDnXML1J62krJAcf7Xb1HJwuW2VMJQrCofY2jtFXdiEY8UsRNJfqK6DAdyZXoMvtaLHyWQx3FS4A9zw
xprv9xWkBDfyBXmZw4jEYXUHYc9fT25k9irP87n2RqfJ5bqbjKdT84Mm7Wtc2xmzFuKg7iYf7XFHKkSsaYKWKJbR54bnyAD9GzjUYbAYTtN4ruo
----
====
Generate the corresponding addresses:
====
----
$ ku P:foo -s 0/0-5 -a
1MrjE78H1R1rqdFrmkjdHnPUdLCJALbv3x
1AnYyVEcuqeoVzH96zj1eYKwoWfwte2pxu
1GXr1kZfxE1FcK6ZRD5sqqqs5YfvuzA1Lb
116AXZc4bDVQrqmcinzu4aaPdrYqvuiBEK
1Cz2rTLjRM6pMnxPNrRKp9ZSvRtj5dDUML
1WstdwPnU6HEUPme1DQayN9nm6j7nDVEM
----
====
Generate the corresponding WIFs:
====
----
$ ku P:foo -s 0/0-5 -W
L5a4iE5k9gcJKGqX3FWmxzBYQc29PvZ6pgBaePLVqT5YByEnBomx
Kyjgne6GZwPGB6G6kJEhoPbmyjMP7D5d3zRbHVjwcq4iQXD9QqKQ
L4B3ygQxK6zH2NQGxLDee2H9v4Lvwg14cLJW7QwWPzCtKHdWMaQz
L2L2PZdorybUqkPjrmhem4Ax5EJvP7ijmxbNoQKnmTDMrqemY8UF
L2oD6vA4TUyqPF8QG4vhUFSgwCyuuvFZ3v8SKHYFDwkbM765Nrfd
KzChTbc3kZFxUSJ3Kt54cxsogeFAD9CCM4zGB22si8nfKcThQn8C
----
====
Check that it works by choosing a BIP32 string (the one corresponding to subkey 0/3):
====
----
$ ku -W xprv9xWkBDfyBXmZsA85GyWj9uYPyoQv826YAadKWMaaEosNrFBKgj2TqWuiWY3zuqxYGpHfv9cnGj5P7e8EskpzKL1Y8Gk9aX6QbryA5raK73p
L2L2PZdorybUqkPjrmhem4Ax5EJvP7ijmxbNoQKnmTDMrqemY8UF
$ ku -a xprv9xWkBDfyBXmZsA85GyWj9uYPyoQv826YAadKWMaaEosNrFBKgj2TqWuiWY3zuqxYGpHfv9cnGj5P7e8EskpzKL1Y8Gk9aX6QbryA5raK73p
116AXZc4bDVQrqmcinzu4aaPdrYqvuiBEK
----
====
Yep, looks familiar.
From secret exponent:
====
----
$ ku 1
input : 1
network : Bitcoin
secret exponent : 1
hex : 1
wif : KwDiBf89QgGbjEhKnhXJuH7LrciVrZi3qYjgd9M7rFU73sVHnoWn
uncompressed : 5HpHagT65TZzG1PH3CSu63k8DbpvD8s5ip4nEB3kEsreAnchuDf
public pair x : 55066263022277343669578718895168534326250603453777594175500187360389116729240
public pair y : 32670510020758816978083085130507043184471273380659243275938904335757337482424
x as hex : 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
y as hex : 483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
y parity : even
key pair as sec : 0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
uncompressed : 0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
hash160 : 751e76e8199196d454941c45d1b3a323f1433bd6
uncompressed : 91b24bf9f5288532960ac687abb035127b1d28a5
Bitcoin address : 1BgGZ9tcN4rm9KBzDn7KprQz87SZ26SAMH
uncompressed : 1EHNa6Q4Jz2uvNExL497mE43ikXhwF6kZm
----
====
Litecoin version:
====
----
$ ku -nL 1
input : 1
network : Litecoin
secret exponent : 1
hex : 1
wif : T33ydQRKp4FCW5LCLLUB7deioUMoveiwekdwUwyfRDeGZm76aUjV
uncompressed : 6u823ozcyt2rjPH8Z2ErsSXJB5PPQwK7VVTwwN4mxLBFrao69XQ
public pair x : 55066263022277343669578718895168534326250603453777594175500187360389116729240
public pair y : 32670510020758816978083085130507043184471273380659243275938904335757337482424
x as hex : 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
y as hex : 483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
y parity : even
key pair as sec : 0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
uncompressed : 0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
hash160 : 751e76e8199196d454941c45d1b3a323f1433bd6
uncompressed : 91b24bf9f5288532960ac687abb035127b1d28a5
Litecoin address : LVuDpNCSSj6pQ7t9Pv6d6sUkLKoqDEVUnJ
uncompressed : LYWKqJhtPeGyBAw7WC8R3F7ovxtzAiubdM
----
====
Dogecoin((("Dogecoin"))) WIF:
====
----
$ ku -nD -W 1
QNcdLVw8fHkixm6NNyN6nVwxKek4u7qrioRbQmjxac5TVoTtZuot
----
====
From public pair (on Testnet):
====
----
$ ku -nT 55066263022277343669578718895168534326250603453777594175500187360389116729240,even
input : 550662630222773436695787188951685343262506034537775941755001873603
89116729240,even
network : Bitcoin testnet
public pair x : 55066263022277343669578718895168534326250603453777594175500187360389116729240
public pair y : 32670510020758816978083085130507043184471273380659243275938904335757337482424
x as hex : 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
y as hex : 483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
y parity : even
key pair as sec : 0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
uncompressed : 0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
hash160 : 751e76e8199196d454941c45d1b3a323f1433bd6
uncompressed : 91b24bf9f5288532960ac687abb035127b1d28a5
Bitcoin testnet address : mrCDrCybB6J1vRfbwM5hemdJz73FwDBC8r
uncompressed : mtoKs9V381UAhUia3d7Vb9GNak8Qvmcsme
----
====
From hash160:
====
----
$ ku 751e76e8199196d454941c45d1b3a323f1433bd6
input : 751e76e8199196d454941c45d1b3a323f1433bd6
network : Bitcoin
hash160 : 751e76e8199196d454941c45d1b3a323f1433bd6
Bitcoin address : 1BgGZ9tcN4rm9KBzDn7KprQz87SZ26SAMH
----
====
As a Dogecoin address:(((range="endofrange", startref="ix_appdx-pycoin-asciidoc0")))
====
----
$ ku -nD 751e76e8199196d454941c45d1b3a323f1433bd6
input : 751e76e8199196d454941c45d1b3a323f1433bd6
network : Dogecoin
hash160 : 751e76e8199196d454941c45d1b3a323f1433bd6
Dogecoin address : DFpN6QqFfUm3gKNaxN6tNcab1FArL9cZLE
----
==== Transaction Utility (TX)
((("transaction utility (tx)")))The command-line utility +tx+ will display transactions in human-readable form, fetch base transactions from pycoin's transaction cache or from web services (blockchain.info, blockr.io, and biteasy.com are currently supported), merge transactions, add or delete inputs or outputs, and sign transactions.
Following are some examples.
View the famous "pizza" transaction [PIZZA]:
====
----
$ tx 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
warning: consider setting environment variable PYCOIN_CACHE_DIR=~/.pycoin_cache to cache transactions fetched via web services
warning: no service providers found for get_tx; consider setting environment variable PYCOIN_SERVICE_PROVIDERS=BLOCKR_IO:BLOCKCHAIN_INFO:BITEASY:BLOCKEXPLORER
usage: tx [-h] [-t TRANSACTION_VERSION] [-l LOCK_TIME] [-n NETWORK] [-a]
[-i address] [-f path-to-private-keys] [-g GPG_ARGUMENT]
[--remove-tx-in tx_in_index_to_delete]
[--remove-tx-out tx_out_index_to_delete] [-F transaction-fee] [-u]
[-b BITCOIND_URL] [-o path-to-output-file]
argument [argument ...]
tx: error: can't find Tx with id 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
----
====
Oops! We don't have web services set up. Let's do that now:
====
[source,bash]
----
$ PYCOIN_CACHE_DIR=~/.pycoin_cache
$ PYCOIN_SERVICE_PROVIDERS=BLOCKR_IO:BLOCKCHAIN_INFO:BITEASY:BLOCKEXPLORER
$ export PYCOIN_CACHE_DIR PYCOIN_SERVICE_PROVIDERS
----
====
It's not done automatically so a command-line tool won't leak potentially private information about what transactions you're interested in to a third-party website. If you don't care, you could put these lines into your _.profile_.
Let's try again:
====
----
$ tx 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
Version: 1 tx hash 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a 159 bytes
TxIn count: 1; TxOut count: 1
Lock time: 0 (valid anytime)
Input:
0: (unknown) from 1e133f7de73ac7d074e2746a3d6717dfc99ecaa8e9f9fade2cb8b0b20a5e0441:0
Output:
0: 1CZDM6oTttND6WPdt3D6bydo7DYKzd9Qik receives 10000000.00000 mBTC
Total output 10000000.00000 mBTC
including unspents in hex dump since transaction not fully signed
010000000141045e0ab2b0b82cdefaf9e9a8ca9ec9df17673d6a74e274d0c73ae77d3f131e000000004a493046022100a7f26eda874931999c90f87f01ff1ffc76bcd058fe16137e0e63fdb6a35c2d78022100a61e9199238eb73f07c8f209504c84b80f03e30ed8169edd44f80ed17ddf451901ffffffff010010a5d4e80000001976a9147ec1003336542cae8bded8909cdd6b5e48ba0ab688ac00000000
** can't validate transaction as source transactions missing
----
====
The final line appears because to validate the transactions' signatures, you technically need the source transactions. So let's add +-a+ to augment the transactions with source information:
====
----
$ tx -a 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a
warning: transaction fees recommendations casually calculated and estimates may be incorrect
warning: transaction fee lower than (casually calculated) expected value of 0.1 mBTC, transaction might not propogate
Version: 1 tx hash 49d2adb6e476fa46d8357babf78b1b501fd39e177ac7833124b3f67b17c40c2a 159 bytes
TxIn count: 1; TxOut count: 1
Lock time: 0 (valid anytime)
Input:
0: 17WFx2GQZUmh6Up2NDNCEDk3deYomdNCfk from 1e133f7de73ac7d074e2746a3d6717dfc99ecaa8e9f9fade2cb8b0b20a5e0441:0 10000000.00000 mBTC sig ok
Output:
0: 1CZDM6oTttND6WPdt3D6bydo7DYKzd9Qik receives 10000000.00000 mBTC
Total input 10000000.00000 mBTC
Total output 10000000.00000 mBTC
Total fees 0.00000 mBTC
010000000141045e0ab2b0b82cdefaf9e9a8ca9ec9df17673d6a74e274d0c73ae77d3f131e000000004a493046022100a7f26eda874931999c90f87f01ff1ffc76bcd058fe16137e0e63fdb6a35c2d78022100a61e9199238eb73f07c8f209504c84b80f03e30ed8169edd44f80ed17ddf451901ffffffff010010a5d4e80000001976a9147ec1003336542cae8bded8909cdd6b5e48ba0ab688ac00000000
all incoming transaction values validated
----
====
Now, let's look at unspent outputs for a specific address (UTXO). In block #1, we see a coinbase transaction to +12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX+. Let's use +fetch_unspent+ to find all coins in this address:
====
----
$ fetch_unspent 12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX
a3a6f902a51a2cbebede144e48a88c05e608c2cce28024041a5b9874013a1e2a/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/333000
cea36d008badf5c7866894b191d3239de9582d89b6b452b596f1f1b76347f8cb/31/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/10000
065ef6b1463f552f675622a5d1fd2c08d6324b4402049f68e767a719e2049e8d/86/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/10000
a66dddd42f9f2491d3c336ce5527d45cc5c2163aaed3158f81dc054447f447a2/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/10000
ffd901679de65d4398de90cefe68d2c3ef073c41f7e8dbec2fb5cd75fe71dfe7/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/100
d658ab87cc053b8dbcfd4aa2717fd23cc3edfe90ec75351fadd6a0f7993b461d/5/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/911
36ebe0ca3237002acb12e1474a3859bde0ac84b419ec4ae373e63363ebef731c/1/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/100000
fd87f9adebb17f4ebb1673da76ff48ad29e64b7afa02fda0f2c14e43d220fe24/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/1
dfdf0b375a987f17056e5e919ee6eadd87dad36c09c4016d4a03cea15e5c05e3/1/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/1337
cb2679bfd0a557b2dc0d8a6116822f3fcbe281ca3f3e18d3855aa7ea378fa373/0/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/1337
d6be34ccf6edddc3cf69842dce99fe503bf632ba2c2adb0f95c63f6706ae0c52/1/76a914119b098e2e980a229e139a9ed01a469e518e6f2688ac/2000000
0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098/0/410496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858eeac/5000000000
----
====

348
appdx-scriptops.asciidoc
Unescape View File

@ -1,163 +1,185 @@
[[tx_script_ops]]
== Appendix: Transaction Script Language Operators, Constants and Symbols
[[tx_script_ops_table_pushdata]]
.Push Value onto Stack
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_0 or OP_FALSE | 0x00 | An empty array is pushed on to the stack
| 1-75 | 0x01-0x4b | Push the next N bytes onto the stack, where N is 1 to 75 bytes
| OP_PUSHDATA1 | 0x4c | The next script byte contains N, push the following N bytes onto the stack
| OP_PUSHDATA2 | 0x4d | The next two script bytes contain N, push the following N bytes onto the stack
| OP_PUSHDATA4 | 0x4e | The next four script bytes contain N, push the following N bytes onto the stack
| OP_1NEGATE | 0x4f | Push the value "-1" onto the stack
| OP_RESERVED | 0x50 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_1 or OP_TRUE| 0x51 | Push the value "1" onto the stack
| OP_2 to OP_16 | 0x52 to 0x60 | For OP_N, push the value "N" onto the stack. E.g., OP_2 pushes "2"
|=======
[[tx_script_ops_table_control]]
.Conditional Flow Control
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP | 0x61 | Do nothing
| OP_VER | 0x62 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_IF | 0x63 | Execute the statements following if top of stack is not 0
| OP_NOTIF | 0x64 | Execute the statements following if top of stack is 0
| OP_VERIF | 0x65 | Halt - Invalid transaction
| OP_VERNOTIF | 0x66 | Halt - Invalid transaction
| OP_ELSE | 0x67 | Execute only if the previous statements were not executed
| OP_ENDIF | 0x68 | Ends the OP_IF, OP_NOTIF, OP_ELSE block
| OP_VERIFY | 0x69 | Check the top of the stack, Halt and Invalidate transaction if not TRUE
| OP_RETURN | 0x6a | Halt and invalidate transaction
|=======
[[tx_script_ops_table_stack]]
.Stack Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_TOALTSTACK | 0x6b | Pop top item from stack and push to alternative stack
| OP_FROMALTSTACK | 0x6c | Pop top item from alternative stack and push to stack
| OP_2DROP | 0x6d | Pop top two stack items
| OP_2DUP | 0x6e | Duplicate top two stack items
| OP_3DUP | 0x6f | Duplicate top three stack items
| OP_2OVER | 0x70 | Copies the third and fourth items in the stack to the top
| OP_2ROT | 0x71 | Moves the fifth and sixth items in the stack to the top
| OP_2SWAP | 0x72 | Swap the two top pairs of items in the stack
| OP_IFDUP | 0x73 | Duplicate the top item in the stack if it is not 0
| OP_DEPTH | 0x74 | Count the items on the stack and push the resulting count
| OP_DROP | 0x75 | Pop the top item in the stack
| OP_DUP | 0x76 | Duplicate the top item in the stack
| OP_NIP | 0x77 | Pop the second item in the stack
| OP_OVER | 0x78 | Copy the second item in the stack and push it on to the top
| OP_PICK | 0x79 | Pop value N from top, then copy the Nth item to the top of the stack
| OP_ROLL | 0x7a | Pop value N from top, then move the Nth item to the top of the stack
| OP_ROT | 0x7b | Rotate the top three items in the stack
| OP_SWAP | 0x7c | Swap the top three items in the stack
| OP_TUCK | 0x7d | Copy the top item and insert it between the top and second item.
|=======
[[tx_script_ops_table_splice]]
.String Splice Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_CAT_ | 0x7e | Disabled (Concatenates top two items)
| _OP_SUBSTR_ | 0x7f | Disabled (Returns substring)
| _OP_LEFT_ | 0x80 | Disabled (Returns left substring)
| _OP_RIGHT_ | 0x81 | Disabled (Returns right substring)
| OP_SIZE | 0x82 | Calculate string length of top item and push the result
|=======
[[tx_script_ops_table_binmath]]
.Binary Arithmetic and Conditionals
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_INVERT_ | 0x83 | Disabled (Flip the bits of the top item)
| _OP_AND_ | 0x84 | Disabled (Boolean AND of two top items)
| _OP_OR_ | 0x85 | Disabled (Boolean OR of two top items)
| _OP_XOR_ | 0x86 | Disabled (Boolean XOR of two top items)
| OP_EQUAL | 0x87 | Push TRUE (1) if top two items are exactly equal, push FALSE (0) otherwise
| OP_EQUALVERIFY | 0x88 | Same as OP_EQUAL, but run OP_VERIFY after to halt if not TRUE
| OP_RESERVED1 | 0x89 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_RESERVED2 | 0x8a | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
|=======
[[tx_script_ops_table_numbers]]
.Numeric Operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_1ADD | 0x8b | Add 1 to the top item
| OP_1SUB | 0x8c | Subtract 1 from the top item
| _OP_2MUL_ | 0x8d | Disabled (Multiply top item by 2)
| _OP_2DIV_ | 0x8e | Disabled (Divide top item by 2)
| OP_NEGATE | 0x8f | Flip the sign of top item
| OP_ABS | 0x90 | Change the sign of the top item to positive
| OP_NOT | 0x91 | If top item is 0 or 1 boolean flip it, otherwise return 0
| OP_0NOTEQUAL | 0x92 | If top item is 0 return 0, otherwise return 1
| OP_ADD | 0x93 | Pop top two items, add them and push result
| OP_SUB | 0x94 | Pop top two items, subtract first form second, push result
| OP_MUL | 0x95 | Disabled (Multiply top two items)
| OP_DIV | 0x96 | Disabled (Divide second item by first item)
| OP_MOD | 0x97 | Disabled (Remainder divide second item by first item)
| OP_LSHIFT | 0x98 | Disabled (Shift second item left by first item number of bits)
| OP_RSHIFT | 0x99 | Disabled (Shift second item right by first item number of bits)
| OP_BOOLAND | 0x9a | Boolean AND of top two items
| OP_BOOLOR | 0x9b | Boolean OR of top two items
| OP_NUMEQUAL | 0x9c | Return TRUE if top two items are equal numbers
| OP_NUMEQUALVERIFY | 0x9d | Same as NUMEQUAL, then OP_VERIFY to halt if not TRUE
| OP_NUMNOTEQUAL | 0x9e | Return TRUE if top two items are not equal numbers
| OP_LESSTHAN | 0x9f | Return TRUE if second item is less than top item
| OP_GREATERTHAN | 0xa0 | Return TRUE if second item is greater than top item
| OP_LESSTHANOREQUAL | 0xa1 | Return TRUE if second item is less than or equal to top item
| OP_GREATERTHANOREQUAL | 0xa2 | Return TRUE if second item is great than or equal to top item
| OP_MIN | 0xa3 | Return the smaller of the two top items
| OP_MAX | 0xa4 | Return the larger of the two top items
| OP_WITHIN | 0xa5 | Return TRUE if the third item is between the second item (or equal) and first item
|=======
[[tx_script_ops_table_crypto]]
.Cryptographic and Hashing Operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_RIPEMD160 | 0xa6 | Return RIPEMD160 hash of top item
| OP_SHA1 | 0xa7 | Return SHA1 hash of top item
| OP_SHA256 | 0xa8 | Return SHA256 hash of top item
| OP_HASH160 | 0xa9 | Return RIPEMD160(SHA256(x)) hash of top item
| OP_HASH256 | 0xaa | Return SHA256(SHA256(x)) hash of top item
| OP_CODESEPARATOR | 0xab | Mark the beginning of signature-checked data
| OP_CHECKSIG | 0xac | Pop a public key and signature and validate the signature for the transaction's hashed data, return TRUE if matching
| OP_CHECKSIGVERIFY | 0xad | Same as CHECKSIG, then OP_VEIRFY to halt if not TRUE
| OP_CHECKMULTISIG | 0xae | Run CHECKSIG for each pair of signature and public key provided. All must match. Bug in implementation pops an extra value, prefix with OP_NOP as workaround
| OP_CHECKMULTISIGVERIFY | 0xaf | Same as CHECKMULTISIG, then OP_VERIFY to halt if not TRUE
|=======
[[tx_script_ops_table_nop]]
.Non-Operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP1-OP_NOP10 | 0xb0-0xb9 | Does nothing, ignored.
|=======
[[tx_script_ops_table_internal]]
.Reserved OP codes for internal use by the parser
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_SMALLDATA | 0xf9 | Represents small data field
| OP_SMALLINTEGER | 0xfa | Represents small integer data field
| OP_PUBKEYS | 0xfb | Represents public key fields
| OP_PUBKEYHASH | 0xfd | Represents a public key hash field
| OP_PUBKEY | 0xfe | Represents a public key field
| OP_INVALIDOPCODE | 0xff | Represents any OP code not currently assigned
|=======
[[tx_script_ops]]
[appendix]
== Transaction Script Language Operators, Constants, and Symbols
((("Script language", id="ix_appdx-scriptops-asciidoc0", range="startofrange")))((("Script language","reserved operator codes", id="ix_appdx-scriptops-asciidoc1", range="startofrange")))<<tx_script_ops_table_pushdata>> shows operators for pushing values onto the stack.((("Script language","push operators")))
[[tx_script_ops_table_pushdata]]
.Push value onto stack
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_0 or OP_FALSE | 0x00 | An empty array is pushed onto the stack
| 1-75 | 0x01-0x4b | Push the next N bytes onto the stack, where N is 1 to 75 bytes
| OP_PUSHDATA1 | 0x4c | The next script byte contains N, push the following N bytes onto the stack
| OP_PUSHDATA2 | 0x4d | The next two script bytes contain N, push the following N bytes onto the stack
| OP_PUSHDATA4 | 0x4e | The next four script bytes contain N, push the following N bytes onto the stack
| OP_1NEGATE | 0x4f | Push the value "–1" onto the stack
| OP_RESERVED | 0x50 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_1 or OP_TRUE| 0x51 | Push the value "1" onto the stack
| OP_2 to OP_16 | 0x52 to 0x60 | For OP_N, push the value "N" onto the stack. E.g., OP_2 pushes "2"
|=======
<<tx_script_ops_table_control>> shows conditional flow control operators.((("Script language","conditional flow operators")))
[[tx_script_ops_table_control]]
.Conditional flow control
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP | 0x61 | Do nothing
| OP_VER | 0x62 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_IF | 0x63 | Execute the statements following if top of stack is not 0
| OP_NOTIF | 0x64 | Execute the statements following if top of stack is 0
| OP_VERIF | 0x65 | Halt - Invalid transaction
| OP_VERNOTIF | 0x66 | Halt - Invalid transaction
| OP_ELSE | 0x67 | Execute only if the previous statements were not executed
| OP_ENDIF | 0x68 | End the OP_IF, OP_NOTIF, OP_ELSE block
| OP_VERIFY | 0x69 | Check the top of the stack, halt and invalidate transaction if not TRUE
| OP_RETURN | 0x6a | Halt and invalidate transaction
|=======
<<tx_script_ops_table_stack>> shows operators used to manipulate the stack.((("Script language","stack manipulation operators")))
[[tx_script_ops_table_stack]]
.Stack operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_TOALTSTACK | 0x6b | Pop top item from stack and push to alternative stack
| OP_FROMALTSTACK | 0x6c | Pop top item from alternative stack and push to stack
| OP_2DROP | 0x6d | Pop top two stack items
| OP_2DUP | 0x6e | Duplicate top two stack items
| OP_3DUP | 0x6f | Duplicate top three stack items
| OP_2OVER | 0x70 | Copy the third and fourth items in the stack to the top
| OP_2ROT | 0x71 | Move the fifth and sixth items in the stack to the top
| OP_2SWAP | 0x72 | Swap the two top pairs of items in the stack
| OP_IFDUP | 0x73 | Duplicate the top item in the stack if it is not 0
| OP_DEPTH | 0x74 | Count the items on the stack and push the resulting count
| OP_DROP | 0x75 | Pop the top item in the stack
| OP_DUP | 0x76 | Duplicate the top item in the stack
| OP_NIP | 0x77 | Pop the second item in the stack
| OP_OVER | 0x78 | Copy the second item in the stack and push it onto the top
| OP_PICK | 0x79 | Pop value N from top, then copy the Nth item to the top of the stack
| OP_ROLL | 0x7a | Pop value N from top, then move the Nth item to the top of the stack
| OP_ROT | 0x7b | Rotate the top three items in the stack
| OP_SWAP | 0x7c | Swap the top three items in the stack
| OP_TUCK | 0x7d | Copy the top item and insert it between the top and second item.
|=======
<<tx_script_ops_table_splice>> shows string operators.((("Script language","string operators")))
[[tx_script_ops_table_splice]]
.String splice operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_CAT_ | 0x7e | Disabled (concatenates top two items)
| _OP_SUBSTR_ | 0x7f | Disabled (returns substring)
| _OP_LEFT_ | 0x80 | Disabled (returns left substring)
| _OP_RIGHT_ | 0x81 | Disabled (returns right substring)
| OP_SIZE | 0x82 | Calculate string length of top item and push the result
|=======
<<tx_script_ops_table_binmath>> shows binary arithmetic and boolean logic operators.((("Script language","binary arithmetic operators")))((("Script language","boolean logic operators")))
[[tx_script_ops_table_binmath]]
.Binary arithmetic and conditionals
[options="header"]
|=======
| Symbol | Value (hex) | Description
| _OP_INVERT_ | 0x83 | Disabled (Flip the bits of the top item)
| _OP_AND_ | 0x84 | Disabled (Boolean AND of two top items)
| _OP_OR_ | 0x85 | Disabled (Boolean OR of two top items)
| _OP_XOR_ | 0x86 | Disabled (Boolean XOR of two top items)
| OP_EQUAL | 0x87 | Push TRUE (1) if top two items are exactly equal, push FALSE (0) otherwise
| OP_EQUALVERIFY | 0x88 | Same as OP_EQUAL, but run OP_VERIFY after to halt if not TRUE
| OP_RESERVED1 | 0x89 | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
| OP_RESERVED2 | 0x8a | Halt - Invalid transaction unless found in an unexecuted OP_IF clause
|=======
<<tx_script_ops_table_numbers>> shows numeric (arithmetic) operators.((("Script language","numeric operators")))
[[tx_script_ops_table_numbers]]
.Numeric operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_1ADD | 0x8b | Add 1 to the top item
| OP_1SUB | 0x8c | Subtract 1 from the top item
| _OP_2MUL_ | 0x8d | Disabled (multiply top item by 2)
| _OP_2DIV_ | 0x8e | Disabled (divide top item by 2)
| OP_NEGATE | 0x8f | Flip the sign of top item
| OP_ABS | 0x90 | Change the sign of the top item to positive
| OP_NOT | 0x91 | If top item is 0 or 1 Boolean flip it, otherwise return 0
| OP_0NOTEQUAL | 0x92 | If top item is 0 return 0, otherwise return 1
| OP_ADD | 0x93 | Pop top two items, add them and push result
| OP_SUB | 0x94 | Pop top two items, subtract first from second, push result
| OP_MUL | 0x95 | Disabled (multiply top two items)
| OP_DIV | 0x96 | Disabled (divide second item by first item)
| OP_MOD | 0x97 | Disabled (remainder divide second item by first item)
| OP_LSHIFT | 0x98 | Disabled (shift second item left by first item number of bits)
| OP_RSHIFT | 0x99 | Disabled (shift second item right by first item number of bits)
| OP_BOOLAND | 0x9a | Boolean AND of top two items
| OP_BOOLOR | 0x9b | Boolean OR of top two items
| OP_NUMEQUAL | 0x9c | Return TRUE if top two items are equal numbers
| OP_NUMEQUALVERIFY | 0x9d | Same as NUMEQUAL, then OP_VERIFY to halt if not TRUE
| OP_NUMNOTEQUAL | 0x9e | Return TRUE if top two items are not equal numbers
| OP_LESSTHAN | 0x9f | Return TRUE if second item is less than top item
| OP_GREATERTHAN | 0xa0 | Return TRUE if second item is greater than top item
| OP_LESSTHANOREQUAL | 0xa1 | Return TRUE if second item is less than or equal to top item
| OP_GREATERTHANOREQUAL | 0xa2 | Return TRUE if second item is great than or equal to top item
| OP_MIN | 0xa3 | Return the smaller of the two top items
| OP_MAX | 0xa4 | Return the larger of the two top items
| OP_WITHIN | 0xa5 | Return TRUE if the third item is between the second item (or equal) and first item
|=======
<<tx_script_ops_table_crypto>> shows cryptographic function operators.((("Script language","cryptographic function operators")))
[[tx_script_ops_table_crypto]]
.Cryptographic and hashing operations
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_RIPEMD160 | 0xa6 | Return RIPEMD160 hash of top item
| OP_SHA1 | 0xa7 | Return SHA1 hash of top item
| OP_SHA256 | 0xa8 | Return SHA256 hash of top item
| OP_HASH160 | 0xa9 | Return RIPEMD160(SHA256(x)) hash of top item
| OP_HASH256 | 0xaa | Return SHA256(SHA256(x)) hash of top item
| OP_CODESEPARATOR | 0xab | Mark the beginning of signature-checked data
| OP_CHECKSIG | 0xac | Pop a public key and signature and validate the signature for the transaction's hashed data, return TRUE if matching
| OP_CHECKSIGVERIFY | 0xad | Same as CHECKSIG, then OP_VERIFY to halt if not TRUE
| OP_CHECKMULTISIG | 0xae | Run CHECKSIG for each pair of signature and public key provided. All must match. Bug in implementation pops an extra value, prefix with OP_NOP as workaround
| OP_CHECKMULTISIGVERIFY | 0xaf | Same as CHECKMULTISIG, then OP_VERIFY to halt if not TRUE
|=======
<<tx_script_ops_table_nop>> shows nonoperator symbols((("Script language","symbols")))
[[tx_script_ops_table_nop]]
.Non-operators
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_NOP1-OP_NOP10 | 0xb0-0xb9 | Does nothing, ignored
|=======
++++
<?hard-pagebreak?>
++++
<<tx_script_ops_table_internal>> shows operator codes reserved for use by the internal script parser.(((range="endofrange", startref="ix_appdx-scriptops-asciidoc1")))(((range="endofrange", startref="ix_appdx-scriptops-asciidoc0")))
[[tx_script_ops_table_internal]]
.Reserved OP codes for internal use by the parser
[options="header"]
|=======
| Symbol | Value (hex) | Description
| OP_SMALLDATA | 0xf9 | Represents small data field
| OP_SMALLINTEGER | 0xfa | Represents small integer data field
| OP_PUBKEYS | 0xfb | Represents public key fields
| OP_PUBKEYHASH | 0xfd | Represents a public key hash field
| OP_PUBKEY | 0xfe | Represents a public key field
| OP_INVALIDOPCODE | 0xff | Represents any OP code not currently assigned
|=======

344
appdx-sx.asciidoc
Unescape View File

@ -1,133 +1,211 @@
[[sx_cmds]]
== Appendix: Available commands with sx tools
----
The sx commands are:
DEPRECATED
ELECTRUM STYLE DETERMINISTIC KEYS AND ADDRESSES
genaddr Generate a Bitcoin address deterministically from a wallet
seed or master public key.
genpriv Generate a private key deterministically from a seed.
genpub Generate a public key deterministically from a wallet
seed or master public key.
mpk Extract a master public key from a deterministic wallet seed.
newseed Create a new deterministic wallet seed.
EXPERIMENTAL
APPS
wallet Experimental command line wallet.
OFFLINE BLOCKCHAIN
HEADERS
showblkhead Show the details of a block header.
OFFLINE KEYS AND ADDRESSES
BASIC
addr See Bitcoin address of a public or private key.
embed-addr Generate an address used for embedding record of data into the blockchain
get-pubkey Get the pubkey of an address if available
newkey Create a new private key.
pubkey See the public part of a private key.
validaddr Validate an address.
BRAIN STORAGE
brainwallet Make 256 bit bitcoin private key from an arbitrary passphrase.
mnemonic Make 12 word mnemonic out of 128 bit electrum or bip32 seed.
HD / BIP32
hd-priv Create an private HD key from another HD private key.
hd-pub Create an HD public key from another HD private or public key.
hd-seed Create a random new HD key.
hd-to-address Convert an HD public or private key to a Bitcoin address.
hd-to-wif Convert an HD private key to a WIF private key.
MULTISIG ADDRESSES
scripthash Create BIP 16 script hash address from raw script hex.
STEALTH
stealth-addr See a stealth address from given input.
stealth-initiate Initiate a new stealth payment.
stealth-newkey Generate new stealth keys and an address.
stealth-show-addr Show details for a stealth address.
stealth-uncover Uncover a stealth address.
stealth-uncover-secret Uncover a stealth secret.
OFFLINE TRANSACTIONS
SCRIPTING
mktx Create an unsigned tx.
rawscript Create the raw hex representation from a script.
set-input Set a transaction input.
showscript Show the details of a raw script.
showtx Show the details of a transaction.
sign-input Sign a transaction input.
unwrap Validates checksum and recovers version byte and original data from hexstring.
validsig Validate a transaction input's signature.
wrap Adds version byte and checksum to hexstring.
ONLINE (BITCOIN P2P)
BLOCKCHAIN UPDATES
sendtx-node Send transaction to a single node.
sendtx-p2p Send tx to bitcoin network.
ONLINE (BLOCKCHAIN.INFO)
BLOCKCHAIN QUERIES (blockchain.info)
bci-fetch-last-height Fetch the last block height using blockchain.info.
bci-history Get list of output points, values, and their spends
from blockchain.info
BLOCKCHAIN UPDATES
sendtx-bci Send tx to blockchain.info/pushtx.
ONLINE (BLOCKEXPLORER.COM)
BLOCKCHAIN QUERIES (blockexplorer.com)
blke-fetch-transaction Fetches a transaction from blockexplorer.com
ONLINE (OBELISK)
BLOCKCHAIN QUERIES
balance Show balance of a Bitcoin address in satoshis.
fetch-block-header Fetch raw block header.
fetch-last-height Fetch the last block height.
fetch-stealth Fetch a stealth information using a network connection to
make requests against the obelisk load balancer backend.
fetch-transaction Fetch a raw transaction using a network connection to
make requests against the obelisk load balancer backend.
fetch-transaction-index
Fetch block height and index in block of transaction.
get-utxo Get enough unspent transaction outputs from a given set of
addresses to pay a given number of satoshis
history Get list of output points, values, and their spends for an
address. grep can filter for just unspent outputs which can
be fed into mktx.
validtx Validate a transaction.
BLOCKCHAIN UPDATES
sendtx-obelisk Send tx to obelisk server.
BLOCKCHAIN WATCHING
monitor Monitor an address.
watchtx Watch transactions from the network searching for a certain hash.
OBELISK ADMIN
initchain Initialize a new blockchain.
UTILITY
EC MATH
ec-add-modp Calculate the result of INTEGER + INTEGER.
ec-multiply Multiply an integer and a point together.
ec-tweak-add Calculate the result of POINT + INTEGER * G.
FORMAT (BASE 58)
base58-decode Convert from base58 to hex
base58-encode Convert from hex to base58
FORMAT (BASE58CHECK)
base58check-decode Convert from base58check to hex
base58check-encode Convert from hex to base58check
decode-addr Decode a address from base58check form to internal RIPEMD representation
encode-addr Encode an address from internal RIPEMD representation to base58check form
FORMAT (WIF)
secret-to-wif Convert a secret exponent value to Wallet Import Format
wif-to-secret Convert a Wallet Import Format to secret exponent value.
HASHES
ripemd-hash RIPEMD hash data from STDIN.
sha256 Perform SHA256 hash of data.
MISC
qrcode Generate Bitcoin QR codes offline.
SATOSHI MATH
btc Convert Satoshis into Bitcoins.
satoshi Convert Bitcoins into Satoshis.
See 'sx help COMMAND' for more information on a specific command.
----
[[appdx_sx]]
[appendix]
== Available Commands with sx Tools
((("sx tools","commands in", id="ix_appdx-sx-asciidoc0", range="startofrange")))
----
The sx commands are:
DEPRECATED
ELECTRUM STYLE DETERMINISTIC KEYS AND ADDRESSES
genaddr Generate a Bitcoin address deterministically from a wallet
seed or master public key.
genpriv Generate a private key deterministically from a seed.
genpub Generate a public key deterministically from a wallet
seed or master public key.
mpk Extract a master public key from a deterministic wallet seed.
newseed Create a new deterministic wallet seed.
EXPERIMENTAL
APPS
wallet Experimental command-line wallet.
OFFLINE BLOCKCHAIN
HEADERS
showblkhead Show the details of a block header.
OFFLINE KEYS AND ADDRESSES
BASIC
addr See Bitcoin address of a public or private key.
embed-addr Generate an address used for embedding record of data into the
blockchain
get-pubkey Get the pubkey of an address if available.
newkey Create a new private key.
pubkey See the public part of a private key.
validaddr Validate an address.
BRAIN STORAGE
brainwallet Make 256 bit bitcoin private key from an arbitrary passphrase.
mnemonic Make 12 word mnemonic out of 128 bit electrum or bip32 seed.
HD / BIP32
hd-priv Create a private HD key from another HD private key.
hd-pub Create an HD public key from another HD private or public key.
hd-seed Create a random new HD key.
hd-to-address Convert an HD public or private key to a Bitcoin address.
hd-to-wif Convert an HD private key to a WIF private key.
MULTISIG ADDRESSES
scripthash Create BIP 16 script hash address from raw script hex.
STEALTH
stealth-addr See a stealth address from given input.
stealth-initiate Initiate a new stealth payment.
stealth-newkey Generate new stealth keys and an address.
stealth-show-addr Show details for a stealth address.
stealth-uncover Uncover a stealth address.
stealth-uncover-secret Uncover a stealth secret.
OFFLINE TRANSACTIONS
SCRIPTING
mktx Create an unsigned tx.
rawscript Create the raw hex representation from a script.
set-input Set a transaction input.
showscript Show the details of a raw script.
showtx Show the details of a transaction.
sign-input Sign a transaction input.
unwrap Validates checksum and recovers version byte and original data
from hexstring.
validsig Validate a transaction input's signature.
wrap Adds version byte and checksum to hexstring.
ONLINE (BITCOIN P2P)
BLOCKCHAIN UPDATES
sendtx-node Send transaction to a single node.
sendtx-p2p Send tx to bitcoin network.
ONLINE (BLOCKCHAIN.INFO)
BLOCKCHAIN QUERIES (blockchain.info)
bci-fetch-last-height Fetch the last block height using blockchain.info.
bci-history Get list of output points, values, and their spends
from blockchain.info
BLOCKCHAIN UPDATES
sendtx-bci Send tx to blockchain.info/pushtx.
ONLINE (BLOCKEXPLORER.COM)
BLOCKCHAIN QUERIES (blockexplorer.com)
blke-fetch-transaction Fetches a transaction from blockexplorer.com
ONLINE (OBELISK)
BLOCKCHAIN QUERIES
balance Show balance of a Bitcoin address in satoshis.
fetch-block-header Fetch raw block header.
fetch-last-height Fetch the last block height.
fetch-stealth Fetch a stealth information using a network connection
to make requests against the obelisk load balancer backend.
fetch-transaction Fetch a raw transaction using a network connection to
make requests against the obelisk load balancer
backend.
fetch-transaction-index
Fetch block height and index in block of transaction.
get-utxo Get enough unspent transaction outputs from a given set
of addresses to pay a given number of satoshis.
history Get list of output points, values, and their spends for
an address. grep can filter for just unspent outputs which can
be fed into mktx.
validtx Validate a transaction.
BLOCKCHAIN UPDATES
sendtx-obelisk Send tx to obelisk server.
BLOCKCHAIN WATCHING
monitor Monitor an address.
watchtx Watch transactions from the network searching for a certain
hash.
OBELISK ADMIN
initchain Initialize a new blockchain.
UTILITY
EC MATH
ec-add-modp Calculate the result of INTEGER + INTEGER.
ec-multiply Multiply an integer and a point together.
ec-tweak-add Calculate the result of POINT + INTEGER * G.
FORMAT (BASE 58)
base58-decode Convert from base58 to hex.
base58-encode Convert from hex to base58.
FORMAT (BASE58CHECK)
base58check-decode Convert from base58check to hex.
base58check-encode Convert from hex to base58check.
decode-addr Decode a address from base58check form to internal RIPEMD
representation.
encode-addr Encode an address from internal RIPEMD representation to
base58check form.
FORMAT (WIF)
secret-to-wif Convert a secret exponent value to Wallet Import Format.
wif-to-secret Convert a Wallet Import Format to secret exponent value.
HASHES
ripemd-hash RIPEMD hash data from STDIN.
sha256 Perform SHA256 hash of data.
MISC
qrcode Generate Bitcoin QR codes offline.
SATOSHI MATH
btc Convert Satoshis into Bitcoins.
satoshi Convert Bitcoins into Satoshis.
See 'sx help COMMAND' for more information on a specific command.
----
Next, we look at some examples of using sx tools to experiment with keys and addresses.
Generate a new private key with the operating system's random number generator by using the +newkey+ command. We save the standard output into the file _private_key_:
----
$ sx newkey > private_key
$ cat private_key
5Jgx3UAaXw8AcCQCi1j7uaTaqpz2fqNR9K3r4apxdYn6rTzR1PL
----
Now, generate the public key from that private key using the +pubkey+ command. Pass the _private_key_ file into the standard input and save the standard output of the command into a new file _public_key_:
----
$ sx pubkey < private_key > public_key
$ cat public_key
02fca46a6006a62dfdd2dbb2149359d0d97a04f430f12a7626dd409256c12be500
----
We can reformat the +public_key+ as an address using the +addr+ command. We pass the +public_key+ into standard input:
----
$ sx addr < public_key
17re1S4Q8ZHyCP8Kw7xQad1Lr6XUzWUnkG
----
The keys generated are so called type-0 nondeterministic keys. That means that each one is generated from a random number generator. The sx tools also support type-2 deterministic keys, where a "master" key is created and then extended to produce a chain or tree of subkeys.
First, we generate a "seed" that will be used as the basis to derive a chain of keys, compatible with the Electrum wallet and other similar implementations. We use the +newseed+ command to produce a seed value:
----
$ sx newseed > seed
$ cat seed
eb68ee9f3df6bd4441a9feadec179ff1
----
The seed value can also be exported as a word mnemonic that is human readable and easier to store and type than a hexadecimal string
using the +mnemonic+ command:
----
$ sx mnemonic < seed > words
$ cat words
adore repeat vision worst especially veil inch woman cast recall dwell appreciate
----
The mnemonic words can be used to reproduce the seed using the +mnemonic+ command again:
----
$ sx mnemonic < words
eb68ee9f3df6bd4441a9feadec179ff1
----
With the seed, we can now generate a sequence of private and public keys, a key chain. We use the +genpriv+ command to generate a sequence of private keys from a seed and the +addr+ command to generate the corresponding public key:
[source,bash]
----
$ sx genpriv 0 < seed
5JzY2cPZGViPGgXZ4Syb9Y4eUGjJpVt6sR8noxrpEcqgyj7LK7i
$ sx genpriv 0 < seed | sx addr
1esVQV2vR9JZPhFeRaeWkAhzmWq7Fi7t7
$ sx genpriv 1 < seed
5JdtL7ckAn3iFBFyVG1Bs3A5TqziFTaB9f8NeyNo8crnE2Sw5Mz
$ sx genpriv 1 < seed | sx addr
1G1oTeXitk76c2fvQWny4pryTdH1RTqSPW
----
With deterministic keys we can generate and regenerate thousands of keys, all derived from a single seed in a deterministic chain. This technique is used in many wallet applications to generate keys that can be backed up and restored with a simple multiword mnemonic. This is easier than having to back up the wallet with all its randomly generated keys every time a new key is created.(((range="endofrange", startref="ix_appdx-sx-asciidoc0")))

161
appdx01.asciidoc
Unescape View File

@ -1,80 +1,81 @@
[appendix]
== Appendix: Bitcoin financial services
[[appdx01]]
This appendix describes the main financial services offered in the bitcoin economy, comparing them to traditional financial services that are already familiar to consumers. It's not a list of sites or companies, as that would go stale immediately. Instead it is a list of service primitives with examples of existing implementations. For example, an escrow
service would be described as an archetype, by analogy to a real-estate escrow, showing the unique characteristics, use case and need for escrow in the bitcoin world. The escrow service archetype would be followed by two or three examples of well implemented actual escrow services, each demonstrating a capability unique to bitcoin.
=== Currency Exchanges
=== Bitcoin market data services
=== Bitcoin ticker, order book, chart and analysis services
=== Peer-to-peer exchange
=== OTC and Web-of-Trust (WoT)
=== Escrow services
=== Monitoring services
=== Alert and notification services
=== Lending
=== P2P Lending
=== Securities
=== Mutual Funds
=== Angel investing
== Appendix: Bitcoin markets and applications
As above, this appendix describes services offered in the bitcoin economy. Each service is described a a service archetype which is compared to a real-world example familiar to anyone. The description of such a service is followed by real-world examples that express
these bitcoin features.
=== Currency transfer
=== US domestic
=== Other in-country
=== International
=== Retail commerce
=== Physical (tangible) goods
=== Intangible products
=== Services
=== Technology services
=== Re-selling and Cross-selling
=== Wholesale commerce
== Appendix: Bitcoin Protocol Structure and Conventions
Reference index of main protocol primitives, packet structure, opcodes, state enumerations, protocol mechanics, time diagrams and protocol validation mechanisms.
== Appendix: Bitcoin Transaction Script Operands and Tokens
More reference material as above
== Appendix: Bitcoin Cryptography Algorithms, Conventions and Conversions.
More reference material as above
== Appendix: Bitcoin Meta-protocols: Mining Pool Protocols, Lightweight Client Protocols
More reference material as above
[[appdx01]]
[appendix]
== Bitcoin financial services
This appendix describes the main financial services offered in the bitcoin economy, comparing them to traditional financial services that are already familiar to consumers. It's not a list of sites or companies, as that would go stale immediately. Instead it is a list of service primitives with examples of existing implementations. For example, an escrow
service would be described as an archetype, by analogy to a real-estate escrow, showing the unique characteristics, use case and need for escrow in the bitcoin world. The escrow service archetype would be followed by two or three examples of well implemented actual escrow services, each demonstrating a capability unique to bitcoin.
=== Currency Exchanges
=== Bitcoin market data services
=== Bitcoin ticker, order book, chart and analysis services
=== Peer-to-peer exchange
=== OTC and Web-of-Trust (WoT)
=== Escrow services
=== Monitoring services
=== Alert and notification services
=== Lending
=== P2P Lending
=== Securities
=== Mutual Funds
=== Angel investing
== Appendix: Bitcoin markets and applications
As above, this appendix describes services offered in the bitcoin economy. Each service is described a a service archetype which is compared to a real-world example familiar to anyone. The description of such a service is followed by real-world examples that express
these bitcoin features.
=== Currency transfer
=== US domestic
=== Other in-country
=== International
=== Retail commerce
=== Physical (tangible) goods
=== Intangible products
=== Services
=== Technology services
=== Re-selling and Cross-selling
=== Wholesale commerce
== Appendix: Bitcoin Protocol Structure and Conventions
Reference index of main protocol primitives, packet structure, opcodes, state enumerations, protocol mechanics, time diagrams and protocol validation mechanisms.
== Appendix: Bitcoin Transaction Script Operands and Tokens
More reference material as above
== Appendix: Bitcoin Cryptography Algorithms, Conventions and Conversions.
More reference material as above
== Appendix: Bitcoin Meta-protocols: Mining Pool Protocols, Lightweight Client Protocols
More reference material as above

16
book.asciidoc
Unescape View File

@ -2,8 +2,12 @@
= Mastering Bitcoin
include::praise.asciidoc[]
include::preface.asciidoc[]
include::glossary.asciidoc[]
include::ch01.asciidoc[]
include::ch02.asciidoc[]
@ -22,6 +26,16 @@ include::ch08.asciidoc[]
include::ch09.asciidoc[]
include::ch10.asciidoc[]
include::appdx-scriptops.asciidoc[]
include::appdx-sx.asciidoc[]
include::appdx-bips.asciidoc[]
include::appdx-pycoin.asciidoc[]
include::appdx-sx.asciidoc[]
include::index.asciidoc[]
include::colo.asciidoc[]

405
ch01.asciidoc
Unescape View File

@ -1,198 +1,207 @@
[[ch01_intro_what_is_bitcoin]]
== Introduction
=== What is Bitcoin?
Bitcoin is a collection of concepts and technologies that form the basis of a digital money ecosystem. Units of currency called bitcoins are used to store and transmit value among participants in the bitcoin network. Bitcoin users communicate with each other using the bitcoin protocol primarily via the Internet, although other transport networks can also be used. The bitcoin protocol stack, available as open source software, can be run on a wide range of computing devices, including laptops and smartphones, making the technology easily accessible.
Users can transfer bitcoin over the network to do just about anything that can be done with conventional currencies, such as buy and sell goods, send money to people or organizations, or extend credit. Bitcoin technology includes features that are based on encryption and digital signatures to ensure the security of the bitcoin network. Bitcoins can be purchased, sold and exchanged for other currencies at specialized currency exchanges. Bitcoin in a sense is the perfect form of money for the Internet because it is fast, secure, and borderless.
Unlike traditional currencies, bitcoins are entirely virtual. There are no physical coins or even digital coins per se. The coins are implied in transactions which transfer value from sender to recipient. Users of bitcoin own keys which allow them to prove ownership of transactions in the bitcoin network, unlocking the value to spend it and transfer it to a new recipient. Those keys are often stored in a digital wallet on each user’s computer. Possession of the key that unlocks a transaction is the only prerequisite to spending bitcoins, putting the control entirely in the hands of each user.
Bitcoin is a fully-distributed, peer-to-peer system. As such there is no "central" server or point of control. Bitcoins are created through a process called "mining", which involves looking for a solution to a difficult problem. Any participant in the bitcoin network (i.e., any device running the full bitcoin protocol stack) may operate as a miner, using their computer's processing power to attempt to find solutions to this problem. Every 10 minutes on average, a new solution is found by someone who then is able to validate the transactions of the past 10 minutes and is rewarded with brand new bitcoins. Essentially, bitcoin mining de-centralizes the currency-issuance and clearing functions of a central bank and replaces the need for any central bank with this global competition.
The bitcoin protocol includes built-in algorithms that regulate the mining function across the network. The difficulty of the problem that miners must solve is adjusted dynamically so that, on average, someone finds a correct answer every 10 minutes regardless of how many miners (and CPUs) are working on the problem at any moment. The protocol also halves the rate at which new bitcoins are created every 4 years, and limits the total number of bitcoins that will be created to a fixed total of 21 million coins. The result is that the number of bitcoins in circulation closely follows an easily predictable curve that reaches 21 million by the year 2140. Due to bitcoin's diminishing rate of issuance, over the long term, the bitcoin currency is deflationary. Furthermore, bitcoin cannot be inflated by "printing" new money above and beyond the expected issuance rate.
Behind the scenes, bitcoin is also the name of the protocol, a network and a distributed computing innovation. The bitcoin currency is really only the first application of this invention. As a developer, I see bitcoin as akin to the Internet of money, a network for propagating value and securing the ownership of digital assets via distributed computation. There's a lot more to bitcoin than first meets the eye.
In this chapter we'll get started by explaining some of the main concepts and terms, getting the necessary software and using bitcoin for simple transactions. In following chapters we'll start unwrapping the layers of technology that make bitcoin possible and examine the inner workings of the bitcoin network and protocol.
.Digital Currencies Before Bitcoin
****
The emergence of viable digital money is closely linked to developments in cryptography. This is not surprising when one considers the fundamental challenges involved with using bits to represent value that can be exchanged for goods and services. Two fundamental questions for anyone accepting digital money are:
1. Can I trust the money is authentic and not counterfeit?
2. Can I be sure that no one else can claim that this money belongs to them and not me? (aka the “double-spend” problem)
Issuers of paper money are constantly battling the counterfeiting problem by using increasingly sophisticated papers and printing technology. Physical money addresses the double-spend issue easily because the same paper note cannot be in two places at once. Of course, conventional money is also often stored and transmitted digitally. In this case the counterfeiting and double-spend issues are handled by clearing all electronic transactions through central authorities that have a global view of the currency in circulation. For digital money, which cannot take advantage of esoteric inks or holographic strips, cryptography provides the basis for trusting the legitimacy of a user’s claim to value. Specifically, cryptographic digital signatures enable a user to sign a digital asset or transaction proving the ownership of that asset. With the appropriate architecture, digital signatures also can be used to address the double-spend issue.
When cryptography started becoming more broadly available and understood in the late 1980s, many researchers began trying to use cryptography to build digital currencies. These early digital currency projects issued digital money, usually backed by a national currency or precious metal such as gold.
While these earlier digital currencies worked, they were centralized and as a result they were easy to attack by governments and hackers. Early digital currencies used a central clearinghouse to settle all transactions at regular intervals, just like a traditional banking system. Unfortunately, in most cases these nascent digital currencies were targeted by worried governments and eventually litigated out of existence. Some failed in spectacular crashes when the parent company liquidated abruptly. To be robust against intervention by antagonists, whether legitimate governments or criminal elements, a de-centralized digital currency was needed to avoid a single point of attack. Bitcoin is such a system, completely de-centralized by design, and free of any central authority or point of control that can be attacked or corrupted.
Bitcoin represents the culmination of decades of research in cryptography and distributed systems and includes four key innovations brought together in a unique and powerful combination. Bitcoin consists of:
* A de-centralized peer-to-peer network (the bitcoin protocol);
* A public transaction ledger (the blockchain);
* A de-centralized mathematical and deterministic currency issuance (distributed mining), and;
* A de-centralized transaction verification system (transaction script).
****
=== History of Bitcoin
Bitcoin was invented in 2008 by Satoshi Nakamoto with the publication of a paper titled "Bitcoin: A Peer-to-Peer Electronic Cash System". Satoshi Nakamoto combined several prior inventions such as b-money and HashCash to create a completely de-centralized electronic cash system that does not rely on a central authority for currency issuance or settlement and validation of transactions. The key innovation was to use a distributed computation system (called a "Proof-Of-Work" algorithm) to conduct a global "election" every 10 minutes, allowing the de-centralized network to arrive at _consensus_ about the state of transactions. This elegantly solves the issue of double-spend where a single currency unit can be spent twice. Previously, the double-spend problem was a weakness of digital currency and was addressed by clearing all transactions through a central clearinghouse.
The bitcoin network started in 2009, based on a reference implementation published by Nakamoto and since revised by many other programmers. The distributed computation that provides security and resilience for bitcoin has increased exponentially and now exceeds that combined processing capacity of the world's top super-computers. Bitcoin's total market value is estimated at between 5 and 10 billion US dollars, depending on the dollar/bitcoin exchange rate. The largest transaction processed so far by the network was $150 million US dollars, transmitted instantly and processed without any fees.
Satoshi Nakamoto withdrew from the public in April of 2011, leaving the responsibility of developing the code and network to a thriving group of volunteers. The name Satoshi Nakamoto is an alias and the identity of the person or people behind this invention is currently unknown. However, neither Satoshi Nakamoto nor anyone else exerts control over the bitcoin system, which operates based on fully transparent mathematical principles. The invention itself is groundbreaking and has already spawned new science in the fields of distributed computing, economics and econometrics.
.A Solution To a Distributed Computing Problem
****
Satoshi Nakamoto's invention is also a practical solution to a previously unsolved problem in distributed computing, known as the Byzantine Generals' Problem. Briefly, the problem consists of trying to agree on a course of action by exchanging information over an unreliable and potentially compromised network. Satoshi Nakamoto's solution, which uses the concept of Proof-of-Work to achieve consensus without a central trusted authority represents a breakthrough in distributed computing science and has wide applicability beyond currency. It can be used to achieve consensus on decentralized networks for provably-fair elections, lotteries, asset registries, digital notarization and more.
****
[[user-stories]]
=== Bitcoin Uses, Users and Their Stories
Bitcoin is a technology, but it expresses money which is fundamentally a language for exchanging value between people. Let's look at the people who are using bitcoin and some of the most common uses of the currency and protocol through their stories. We will re-use these stories throughout the book to illustrate the real-life uses of digital money and how they are made possible by the various technologies that are part of bitcoin.
North American Low Value Retail::
Alice lives in Northern California's Bay Area. She has heard about bitcoin from her techie friends and wants to start using it. We will follow her story as she learns about bitcoin, acquires some and then spends some of her bitcoin to buy a cup of coffee at Bob's Cafe in Palo Alto. This story will introduce us to the software, the exchanges and basic transactions from the perspective of a retail consumer.
North American High Value Retail::
Carol is an art gallery owner in San Francisco. She sells expensive paintings for bitcoin. This story will introduce the risks of a "51%" consensus attack for retailers of high-value items.
Offshore Contract Services::
Bob, the cafe owner in Palo Alto is building a new website. He has contracted with an Indian web developer, Gopesh, who lives in Bangalore, India. Gopesh has agreed to be paid in bitcoin. This story will examine the use of bitcoin for outsourcing, contract services and international wire transfers.
Charitable Donations::
Eugenia is the director of a children's charity in the Philippines. Recently she has discovered bitcoin and wants to use it to reach a whole new group of foreign and domestic donors to fundraise for her charity. She's also investigating ways to use bitcoin to distribute funds quickly to areas of need. This story will show the use of bitcoin for global fundraising across currencies and borders and the use of an open ledger for transparency in charitable organizations.
Import/Export::
Mohammed is an electronics importer in Dubai. He's trying to use bitcoin to buy electronics from the USA and China for import into the U.A.E. to accelerate the process of payments for imports. This story will show how bitcoin can be used for large business-to-business international payments tied to physical goods.
Mining for Bitcoin::
Jing is a computer engineering student in Shanghai. He has built a "mining" rig to mine for bitcoins, using his engineering skills to supplement his income. This story will examine the "industrial" base of bitcoin, the specialized equipment used to secure the bitcoin network and issue new currency.
Each of the stories above is based on real people and real industries that are currently using bitcoin to create new markets, new industries and innovative solutions to global economic issues.
=== Getting Started
To join the bitcoin network and start using the currency, all a user has to do is download an application or use a web application. Since bitcoin is a standard, there are many implementations of the bitcoin client software. There is also a "reference implementation", also known as the Satoshi Client, which is managed as an open source project by a team of developers and is derived from the original implementation written by Satoshi Nakamoto.
The three primary forms of bitcoin clients are:
Full Client:: A full client, or "full node" is a client that stores the entire history of bitcoin transactions (every transaction by every user, ever), manages the user's wallets and can initiate transactions directly on the bitcoin network. This is similar to a standalone email server, in that it handles all aspects of the protocol without relying on any other servers or third party services.
Light Client:: A lightweight client stores the user's wallet but relies on third-party owned servers for access to the bitcoin transactions and network. The light client does not store a full copy of all transactions and therefore must trust the third party servers for transaction validation. This is similar to a standalone email client that connects to a mail server for access to a mailbox, in that it relies on a third party for interactions with the network.
Web Client:: Web-clients are accessed through a web browser and store the user's wallet on a server owned by a third party. This is similar to webmail in that it relies entirely on a third party server.
Mobile Client:: Mobile clients for smartphones, such as those based on the Android system, can either operate as full clients, light clients or web clients. Some mobile clients are synchronized with a web or desktop client, providing a multi-platform wallet across multiple devices but with a common source of funds.
The choice of bitcoin client depends on how much control the user wants over funds. A full client will offer the highest level of control and independence for the user, but in turn puts the burden of backups and security on the user. On the other end of the range of choices, a web client is the easiest to set up and use, but the tradeoff with a web client is that counterparty risk is introduced because security and control is shared by the user and the owner of the web service. If a web-wallet service is compromised, as many have been, the users can lose all their funds. Conversely, if a user has a full client without adequate backups, they may lose their funds through a computer mishap.
For the purposes of this book, we will be demonstrating the use of a variety of bitcoin clients, from the reference implementation (the Satoshi client) to web-wallets. Some of the examples will require the use of the reference client which exposes APIs to the wallet, network and transaction services. If you are planning to explore the programmatic interfaces into the bitcoin system, you will need the reference client.
==== Quick Start
Alice, who we introduced in <<user-stories>>, is not a technical user and only recently heard about bitcoin from a friend. She starts her journey by visiting the official website bitcoin.org, where she finds a broad selection of bitcoin clients. Following the advice on the bitcoin.org site, she chooses the lightweight bitcoin client _Multibit_.
Alice follows a link from the bitcoin.org site to download and install Multibit on her desktop. Multibit is available for Windows, Mac OS and Linux desktops.
[WARNING]
====
A bitcoin wallet must be protected by a password or passphrase. There are many bad actors attempting to break weak passwords, so take care to select one that cannot be easily broken. Use a combination of upper and lower-case characters, numbers and symbols. Avoid personal information such as birth-dates or names of sports teams. Avoid any words commonly found in dictionaries, in any language. If you can, use a password generator to create a completely random password that is at least 12 characters in length. Remember: bitcoin is money and can be instantly moved anywhere in the world. If it is not well protected, it can be easily stolen.
====
Once Alice has downloaded and installed the Multibit application, she runs it and is greeted by a "welcome" screen:
[[multibit-welcome]]
.The Multibit Bitcoin Client - Welcome Screen
image::images/MultibitWelcome.png["MultibitWelcome"]
Multibit automatically creates a wallet and a new bitcoin address for Alice, which Alice can see by clicking on the "Request" tab:
[[multibit-request]]
.Alice's new bitcoin address, in the "Request" tab of the Multibit client
image::images/MultibitReceive.png["MultibitReceive"]
The most important part of this screen is Alice's _bitcoin address_. Like an email address, Alice can share this address and anyone can use it to send money directly to her new wallet. On the screen it appears as a long string of letters and numbers: +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. Next to the wallet's bitcoin address, there is a QR code, a form of barcode that contains the same information in a format that can be easily scanned by a smartphone's camera. The QR code is the black and white square on the right side of the window. Alice can copy the bitcoin address or the QR code onto her clipboard by clicking on the copy button adjacent to each of them. Clicking on the QR code itself will magnify it, so that it can be easily scanned by a smartphone camera.
Alice can also print the QR code as a way to easily give her address to others without them having to type the long string of letters and numbers.
[TIP]
====
Bitcoin addresses start with the digit "1" or "3". Like email addresses, they can be shared with other bitcoin users who can use them to send bitcoin directly to your wallet. Unlike email addresses, you can create new addresses as often as you like, all of which will direct funds to your wallet. A wallet is simply a collection of addresses and the keys that unlock the funds within. There is practically no limit to the number of addresses a user can create.
====
Alice is now ready to start using her new bitcoin wallet.
[[getting_first_bitcoin]]
==== Getting your first bitcoins
It is not possible to buy bitcoins at a bank or foreign exchange kiosks at this time. As of 2014, it is still quite difficult to acquire bitcoins in most countries. There are a number of specialized currency exchanges where you can buy and sell bitcoin in exchange for a local currency. These operate as web-based currency markets and include:
* Bitstamp (bitstamp.net), a European currency market that supports several currencies including euros (EUR) and US dollars (USD) via wire transfer
* Coinbase (coinbase.com), a US-based bitcoin wallet and platform where merchants and consumers can transact in bitcoin. Coinbase makes it easy to buy and sell bitcoin, allowing users to connect to US checking accounts via the ACH system.
Crypto-currency exchanges such as these operate at the intersection of national currencies and crypto-currencies. As such, they are subject to national and international regulations and are often specific to a single country or economic area and specialize in the national currencies of that area. Your choice of currency exchange will be specific to the national currency you use and limited to the exchanges that operate within the legal jurisdiction of your country. Similar to opening a bank account, it takes several days or weeks to set up the necessary accounts with the above services because they require various forms of identification to comply with KYC (Know Your Customer) and AML (Anti-Money Laundering) banking regulations. Once you have an account on a bitcoin exchange, you can then buy or sell bitcoins quickly just as you could with foreign currency with a brokerage account.
A more complete list can be found at http://bitcoincharts.com/markets/, a site that offers price quotes and other market data across many dozens of currency exchanges.
There are three other methods for getting bitcoins as a new user:
* Find a friend who has bitcoins and buy some from them directly. Many bitcoin users started this way.
* Use a classified service like localbitcoins.com to find a seller in your area to buy bitcoins for cash in an in-person transaction.
* Sell a product or service for bitcoin. If you're a programmer, sell your programming skills. If you have an online store, see <<bitcoin-commerce>> to sell in bitcoin.
* Use a bitcoin ATM in your city. A map of bitcoin ATMs can be found at http://www.coindesk.com/bitcoin-atm-map/
Alice was introduced to bitcoin by a friend and so she has an easy way of getting her first bitcoin while she waits for her account on a California currency market to be verified and activated.
[[sending_receiving]]
==== Sending and receiving bitcoins
Alice has created her bitcoin wallet and she is now ready to receive funds. Her wallet application randomly generated a private key (described in more detail in <<private_keys>>) together with its corresponding bitcoin address. At this point, her bitcoin address is not known to the bitcoin network or "registered" with any part of the bitcoin system. Her bitcoin address is simply a number that corresponds to a key that she can use to control access to the funds. There is no account or association between that address and an account. Until the moment this address is referenced as the recipient of value in a transaction posted on the bitcoin ledger (the blockchain), it is simply part of the vast number of possible addresses that are "valid" in bitcoin. Once it has been associated with a transaction, it becomes part of the known addresses in the network and Alice can check its balance on the public ledger.
Alice meets her friend Joe who introduced her to bitcoin at a local restaurant so they can exchange some US dollars and put some bitcoins into her account. She has brought a printout of her address and the QR code as displayed in her bitcoin wallet. There is nothing sensitive, from a security perspective, about the bitcoin address. It can be posted anywhere without risking the security of her account.
Alice wants to convert just $10 US dollars into bitcoin, so as not to risk too much money on this new technology. She gives Joe a $10 bill and the printout of her address so that Joe can send her the equivalent amount of bitcoin.
Next, Joe has to figure out the exchange rate so that he can give the correct amount of bitcoin to Alice. There are hundreds of applications and web sites that can provide the current market rate, here are some of the most popular:
* bitcoincharts.com, a market data listing service that shows the market rate of bitcoin across many exchanges around the globe, denominated in different local currencies
* bitcoinaverage.com, a site that provides a simple view of the volume-weighted-average for each currency
* ZeroBlock, a free Android and iOS application that can display a bitcoin price from different exchanges
* bitcoinwisdom.com, another market data listing service
[[zeroblock-android]]
.ZeroBlock - A bitcoin market-rate application for Android and iOS
image::images/zeroblock.png["zeroblock screenshot"]
Using one of the applications or websites above, Joe determines the price of bitcoin to be approximately $100 US dollars per bitcoin. At that rate he should give Alice 0.10 bitcoin, also known as 100 milliBits, in return for the $10 US dollars she gave him.
Once Joe has established a fair exchange price, he opens his mobile wallet application and selects to "send" bitcoin. He is presented with a screen requesting two inputs:
* The destination bitcoin address for the transaction
* The amount of bitcoin to send
[[blockchain-mobile-send]]
.Bitcoin mobile wallet - Send bitcoin screen
image::images/blockchain-mobile-send.png["blockchain mobile send screen"]
In the input field for the bitcoin address, there is a small icon that looks like a QR code. This allows Joe to scan the barcode with his smartphone camera so that he doesn't have to type in Alice's bitcoin address (+1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+), which is quite long and difficult to type. Joe taps on the QR code icon and activates the smartphone camera, scanning the QR code from Alice's printed wallet that she brought with her. The mobile wallet application fills in the bitcoin address and Joe can check that it scanned correctly by comparing a few digits from the address with the address printed by Alice.
Joe then enters the bitcoin value for the transaction, 0.10 bitcoin. He carefully checks to make sure he has entered the correct amount, as he is about to transmit money and any mistake could be costly. Finally, he presses "Send" to transmit the transaction. Joe's mobile bitcoin wallet constructs a transaction that assigns 0.10 bitcoin to the address provided by Alice, sourcing the funds from Joe's wallet and signing the transaction with Joe's private keys. This tells the bitcoin network that Joe has authorized a transfer of value from one of his addresses to Alice's new address. As the transaction is transmitted via the peer-to-peer protocol, it quickly propagates across the bitcoin network. In less than a second, most of the well-connected nodes in the network receive the transaction and see Alice's address for the first time.
If Alice has a smartphone or laptop with her, she will also be able to see the transaction. The bitcoin ledger - a constantly growing file that records every bitcoin transaction that has ever occurred - is public, meaning that all she has to do is look up her own address and see if any funds have been sent to it. She can do this quite easily at the blockchain.info website by entering her address in the search box. The website will show her a page (https://blockchain.info/address/1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK) listing all the transactions to and from that address. If Alice is watching that page, it will update to show a new transaction transferring 0.10 bitcoin to her balance soon after Joe hits "Send".
.Confirmations
****
At first, Alice's address will show the transaction from Joe as "Unconfirmed". This means that the transaction has been propagated to the network but has not yet been included in the bitcoin transaction ledger, known as the blockchain. To be included, the transaction must be "picked up" by a miner and included in a block of transactions. Once a new block is created, in approximately 10 minutes, the transactions within the block will be accepted as "confirmed" by the network and can be spent. The transaction is seen by all instantly, but it is only "trusted" by all when it is included in a newly mined block.
****
Alice is now the proud owner of 0.10 bitcoin which she can spend. In the next chapter we will look at her first purchase with bitcoin and examine the underlying transaction and propagation technologies in more detail.
[[ch01_intro_what_is_bitcoin]]
== Introduction
=== What Is Bitcoin?
((("bitcoin", id="ix_ch01-asciidoc0", range="startofrange")))((("bitcoin","defined")))Bitcoin is a collection of concepts and technologies that form the basis of a digital money ecosystem. Units of currency called bitcoins are used to store and transmit value among participants in the bitcoin network. Bitcoin users communicate with each other using the bitcoin protocol primarily via the Internet, although other transport networks can also be used. The bitcoin protocol stack, available as open source software, can be run on a wide range of computing devices, including laptops and smartphones, making the technology easily accessible.
Users can transfer bitcoins over the network to do just about anything that can be done with conventional currencies, including buy and sell goods, send money to people or organizations, or extend credit. Bitcoins can be purchased, sold, and exchanged for other currencies at specialized currency exchanges. Bitcoin in a sense is the perfect form of money for the Internet because it is fast, secure, and borderless.
Unlike traditional currencies, bitcoins are entirely virtual. There are no physical coins or even digital coins per se. The coins are implied in transactions that transfer value from sender to recipient. Users of bitcoin own keys that allow them to prove ownership of transactions in the bitcoin network, unlocking the value to spend it and transfer it to a new recipient. Those keys are often stored in a digital wallet on each user’s computer. Possession of the key that unlocks a transaction is the only prerequisite to spending bitcoins, putting the control entirely in the hands of each user.
Bitcoin is a distributed, peer-to-peer system. As such there is no "central" server or point of control. Bitcoins are created through a process called "mining," which involves competing to find solutions to a mathematical problem while processing bitcoin transactions. Any participant in the bitcoin network (i.e., anyone using a device running the full bitcoin protocol stack) may operate as a miner, using their computer's processing power to verify and record transactions. Every 10 minutes on average, someone is able to validate the transactions of the past 10 minutes and is rewarded with brand new bitcoins. Essentially, bitcoin mining decentralizes the currency-issuance and clearing functions of a central bank and replaces the need for any central bank with this global competition.
((("mining","algorithms regulating")))The bitcoin protocol includes built-in algorithms that regulate the mining function across the network. The difficulty of the processing task that miners must perform—to successfully record a block of transactions for the bitcoin network—is adjusted dynamically so that, on average, someone succeeds every 10 minutes regardless of how many miners (and CPUs) are working on the task at any moment. ((("bitcoin","rate of issuance")))The protocol also halves the rate at which new bitcoins are created every four years, and limits the total number of bitcoins that will be created to a fixed total of 21 million coins. The result is that the number of bitcoins in circulation closely follows an easily predictable curve that reaches 21 million by the year 2140. Due to bitcoin's diminishing rate of issuance, over the long term, the bitcoin currency is deflationary. Furthermore, bitcoin cannot be inflated by "printing" new money above and beyond the expected issuance rate.
Behind the scenes, bitcoin is also the name of the protocol, a network, and a distributed computing innovation. The bitcoin currency is really only the first application of this invention. As a developer, I see bitcoin as akin to the Internet of money, a network for propagating value and securing the ownership of digital assets via distributed computation. There's a lot more to bitcoin than first meets the eye.
In this chapter we'll get started by explaining some of the main concepts and terms, getting the necessary software, and using bitcoin for simple transactions. In following chapters we'll start unwrapping the layers of technology that make bitcoin possible and examine the inner workings of the bitcoin network and protocol.
.Digital Currencies Before Bitcoin
****
((("bitcoin","precursors to")))The emergence of viable digital money is closely linked to developments in cryptography. This is not surprising when one considers the fundamental challenges involved with using bits to represent value that can be exchanged for goods and services. Two basic questions for anyone accepting digital money are:
1. Can I trust the money is authentic and not counterfeit?
2. Can I be sure that no one else can claim that this money belongs to them and not me? (Aka the((("double-spend problem"))) “double-spend” problem.)
((("counterfeiting")))((("crypto-currency","counterfeiting")))Issuers of paper money are constantly battling the counterfeiting problem by using increasingly sophisticated papers and printing technology. Physical money addresses the double-spend issue easily because the same paper note cannot be in two places at once. Of course, conventional money is also often stored and transmitted digitally. In these cases, the counterfeiting and double-spend issues are handled by clearing all electronic transactions through central authorities that have a global view of the currency in circulation. For digital money, which cannot take advantage of esoteric inks or holographic strips,((("cryptography"))) cryptography provides the basis for trusting the legitimacy of a user’s claim to value. Specifically, cryptographic digital signatures enable a user to sign a digital asset or transaction proving the ownership of that asset. With the appropriate architecture, digital signatures also can be used to address the double-spend issue.
When cryptography started becoming more broadly available and understood in the late 1980s, many researchers began trying to use cryptography to build digital currencies. These early digital currency projects issued digital money, usually backed by a national currency or precious metal such as gold.
Although these earlier digital currencies worked, they were centralized and, as a result, they were easy to attack by governments and hackers. Early digital currencies used a central clearinghouse to settle all transactions at regular intervals, just like a traditional banking system. Unfortunately, in most cases these nascent digital currencies were targeted by worried governments and eventually litigated out of existence. Some failed in spectacular crashes when the parent company liquidated abruptly. To be robust against intervention by antagonists, whether legitimate governments or criminal elements, a decentralized digital currency was needed to avoid a single point of attack. Bitcoin is such a system, completely decentralized by design, and free of any central authority or point of control that can be attacked or corrupted.
Bitcoin represents the culmination of decades of research in cryptography and distributed systems and includes four key innovations brought together in a unique and powerful combination. Bitcoin consists of:
* A decentralized peer-to-peer network (the bitcoin protocol)
* A public transaction ledger (the blockchain)
* A decentralized mathematical and deterministic currency issuance (distributed mining)
* A decentralized transaction verification system (transaction script)
****
=== History of Bitcoin
((("bitcoin","development of")))((("Nakamoto, Satoshi")))Bitcoin was invented in 2008 with the publication of a paper titled((("Bitcoin: A Peer-to-Peer Electronic Cash System. (Nakamoto)"))) "Bitcoin: A Peer-to-Peer Electronic Cash System," written under the alias of Satoshi Nakamoto. Nakamoto combined several prior inventions such as((("b-money")))((("HashCash"))) b-money and HashCash to create a completely decentralized electronic cash system that does not rely on a central authority for currency issuance or settlement and validation of transactions. The key innovation was to use a distributed computation system (called a((("proof-of-work algorithm"))) "proof-of-work" algorithm) to conduct a global "election" every 10 minutes, allowing the decentralized network to arrive at _consensus_ about the state of transactions. This elegantly solves the issue of double-spend where a single currency unit can be spent twice. Previously, the double-spend problem was a weakness of digital currency and was addressed by clearing all transactions through a central clearinghouse.
((("bitcoin network","origin of")))The bitcoin network started in 2009, based on a reference implementation published by Nakamoto and since revised by many other programmers. The distributed computation that provides security and resilience for bitcoin has increased exponentially, and now exceeds that combined processing capacity of the world's top super-computers. Bitcoin's total market value is estimated at between 5 billion and 10 billion US dollars, depending on the bitcoin-to-dollar exchange rate. The largest transaction processed so far by the network was 150 million US dollars, transmitted instantly and processed without any fees.
Satoshi Nakamoto withdrew from the public in April of 2011, leaving the responsibility of developing the code and network to a thriving group of volunteers. The identity of the person or people behind bitcoin is still unknown. However, neither Satoshi Nakamoto nor anyone else exerts control over the bitcoin system, which operates based on fully transparent mathematical principles. The invention itself is groundbreaking and has already spawned new science in the fields of distributed computing, economics, and econometrics.
.A Solution to a Distributed Computing Problem
****
((("Byzantine Generals Problem")))Satoshi Nakamoto's invention is also a practical solution to a previously unsolved problem in distributed computing, known as the "Byzantine Generals' Problem." Briefly, the problem consists of trying to agree on a course of action by exchanging information over an unreliable and potentially compromised network. Satoshi Nakamoto's solution, which uses the concept of proof-of-work to achieve consensus without a central trusted authority, represents a breakthrough in distributed computing science and has wide applicability beyond currency. It can be used to achieve consensus on decentralized networks to prove the fairness of elections, lotteries, asset registries, digital notarization, and more.
****
[[user-stories]]
=== Bitcoin Uses, Users, and Their Stories
Bitcoin is a technology, but it expresses money that is fundamentally a language for exchanging value between people. Let's look at the people who are using bitcoin and some of the most common uses of the currency and protocol through their stories. We will reuse these stories throughout the book to illustrate the real-life uses of digital money and how they are made possible by the various technologies that are part of bitcoin.
North American low-value retail::
Alice lives in Northern California's Bay Area. She has heard about bitcoin from her techie friends and wants to start using it. We will follow her story as she learns about bitcoin, acquires some, and then spends some of her bitcoin to buy a cup of coffee at Bob's Cafe in Palo Alto. This story will introduce us to the software, the exchanges, and basic transactions from the perspective of a retail consumer.
North American high-value retail::
Carol is an art gallery owner in San Francisco. She sells expensive paintings for bitcoin. This story will introduce the risks of a "51%" consensus attack for retailers of high-value items.
Offshore contract services::
Bob, the cafe owner in Palo Alto, is building a new website. He has contracted with an Indian web developer, Gopesh, who lives in Bangalore, India. Gopesh has agreed to be paid in bitcoin. This story will examine the use of bitcoin for outsourcing, contract services, and international wire transfers.
Charitable donations::
Eugenia is the director of a children's charity in the Philippines. Recently she has discovered bitcoin and wants to use it to reach a whole new group of foreign and domestic donors to fundraise for her charity. She's also investigating ways to use bitcoin to distribute funds quickly to areas of need. This story will show the use of bitcoin for global fundraising across currencies and borders and the use of an open ledger for transparency in charitable organizations.
Import/export::
Mohammed is an electronics importer in Dubai. He's trying to use bitcoin to buy electronics from the US and China for import into the UAE to accelerate the process of payments for imports. This story will show how bitcoin can be used for large business-to-business international payments tied to physical goods.
Mining for bitcoin::
Jing is a computer engineering student in Shanghai. He has built a "mining" rig to mine for bitcoins, using his engineering skills to supplement his income. This story will examine the "industrial" base of bitcoin: the specialized equipment used to secure the bitcoin network and issue new currency.
Each of these stories is based on real people and real industries that are currently using bitcoin to create new markets, new industries, and innovative solutions to global economic issues.
=== Getting Started
((("bitcoin","forms of")))To join the bitcoin network and start using the currency, all a user has to do is download an application or use a web application. Because bitcoin is a standard, there are many implementations of the bitcoin client software. There is also a reference implementation, also known as the Satoshi client, which is managed as an open source project by a team of developers and is derived from the original implementation written by Satoshi Nakamoto.
The three main forms of bitcoin clients are:
Full client:: ((("full nodes")))A full client, or "full node," is a client that stores the entire history of bitcoin transactions (every transaction by every user, ever), manages the users' wallets, and can initiate transactions directly on the bitcoin network. This is similar to a standalone email server, in that it handles all aspects of the protocol without relying on any other servers or third-party services.
Lightweight client:: ((("lightweight client")))A lightweight client stores the user's wallet but relies on third-party–owned servers for access to the bitcoin transactions and network. The light client does not store a full copy of all transactions and therefore must trust the third-party servers for transaction validation. This is similar to a standalone email client that connects to a mail server for access to a mailbox, in that it relies on a third party for interactions with the network.
Web client:: ((("web clients")))Web clients are accessed through a web browser and store the user's wallet on a server owned by a third party. This is similar to webmail in that it relies entirely on a third-party server.
.Mobile Bitcoin
****
((("mobile clients")))((("smartphones, bitcoin clients for")))Mobile clients for smartphones, such as those based on the Android system, can either operate as full clients, lightweight clients, or web clients. Some mobile clients are synchronized with a web or desktop client, providing a multiplatform wallet across multiple devices but with a common source of funds.
****
The choice of bitcoin client depends on how much control the user wants over funds. A full client will offer the highest level of control and independence for the user, but in turn puts the burden of backups and security on the user. On the other end of the range of choices, a web client is the easiest to set up and use, but the trade-off with a web client is that counterparty risk is introduced because security and control is shared with the user and the owner of the web service. If a web-wallet service is compromised, as many have been, the users can lose all their funds. Conversely, if users have a full client without adequate backups, they might lose their funds through a computer mishap.
For the purposes of this book, we will be demonstrating the use of a variety of downloadable bitcoin clients, from the reference implementation (the Satoshi client) to web wallets. Some of the examples will require the use of the reference client, which, in addition to being a full client, also exposes APIs to the wallet, network, and transaction services. If you are planning to explore the programmatic interfaces into the bitcoin system, you will need the reference client.
==== Quick Start
((("bitcoin","wallet setup")))((("wallets","setting up")))Alice, who we introduced in <<user-stories>>, is not a technical user and only recently heard about bitcoin from a friend. She starts her journey by visiting the((("bitcoin.org"))) official website http://www.bitcoin.org[bitcoin.org], where she finds a broad selection of bitcoin clients. Following the advice on the bitcoin.org site, she chooses the lightweight bitcoin client((("Multibit client"))) Multibit.
Alice follows a link from the bitcoin.org site to download and install Multibit on her desktop. Multibit is available for Windows, Mac OS, and Linux desktops.
[WARNING]
====
((("wallets","security of")))A bitcoin wallet must be protected by a password or passphrase. There are many bad actors attempting to break weak passwords, so take care to select one that cannot be easily broken. Use a combination of upper and lowercase characters, numbers, and symbols. Avoid personal information such as birth dates or names of sports teams. Avoid any words commonly found in dictionaries, in any language. If you can, use a password generator to create a completely random password that is at least 12 characters in length. Remember: bitcoin is money and can be instantly moved anywhere in the world. If it is not well protected, it can be easily stolen.
====
Once Alice has downloaded and installed the Multibit application, she runs it and is greeted by a Welcome screen, as shown in <<multibit-welcome>>.
[[multibit-welcome]]
.The Multibit bitcoin client Welcome screen
image::images/msbt_0101.png["MultibitWelcome"]
((("addresses, bitcoin","created by Multibit")))Multibit automatically creates a wallet and a new bitcoin address for Alice, which Alice can see by clicking the Request tab shown in <<multibit-request>>.
[[multibit-request]]
.Alice's new bitcoin address, in the Request tab of the Multibit client
image::images/msbt_0102.png["MultibitReceive"]
The most important part of this screen is Alice's _bitcoin address_. Like an email address, Alice can share this address and anyone can use it to send money directly to her new wallet. On the screen it appears as a long string of letters and numbers: +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. Next to the wallet's bitcoin address is a QR code, a form of barcode that contains the same information in a format that can be scanned by a smartphone camera. The QR code is the black-and-white square on the right side of the window. Alice can copy the bitcoin address or the QR code onto her clipboard by clicking the copy button adjacent to each of them. Clicking the QR code itself will magnify it, so that it can be easily scanned by a smartphone camera.
Alice can also print the QR code as a way to easily give her address to others without them having to type the long string of letters and numbers.
[TIP]
====
((("addresses, bitcoin","sharing")))Bitcoin addresses start with the digit 1 or 3. Like email addresses, they can be shared with other bitcoin users who can use them to send bitcoin directly to your wallet. Unlike email addresses, you can create new addresses as often as you like, all of which will direct funds to your wallet. A wallet is simply a collection of addresses and the keys that unlock the funds within. You can increase your privacy by using a different address for every transaction. There is practically no limit to the number of addresses a user can create.
====
Alice is now ready to start using her new bitcoin wallet.
[[getting_first_bitcoin]]
==== Getting Your First Bitcoins
((("bitcoin","acquiring")))((("currency markets")))It is not possible to buy bitcoins at a bank or foreign exchange kiosks at this time. As of 2014, it is still quite difficult to acquire bitcoins in most countries. There are a number of specialized currency exchanges where you can buy and sell bitcoin in exchange for a local currency. These operate as web-based currency markets and include:
http://bitstamp.net[Bitstamp]:: A European currency market that supports several currencies including euros (EUR) and US dollars (USD) via wire transfer.((("Bitstamp currency market")))
http://www.coinbase.com[Coinbase]:: A US-based bitcoin wallet and platform where merchants and consumers can transact in bitcoin. Coinbase makes it easy to buy and sell bitcoin, allowing users to connect to US checking accounts via the ACH system.((("Coinbase.com")))
Cryptocurrency exchanges such as these operate at the intersection of national currencies and cryptocurrencies. As such, they are subject to national and international regulations, and are often specific to a single country or economic area and specialize in the national currencies of that area. Your choice of currency exchange will be specific to the national currency you use and limited to the exchanges that operate within the legal jurisdiction of your country. Similar to opening a bank account, it takes several days or weeks to set up the necessary accounts with these services because they require various forms of identification to comply with((("AML (Anti-Money Laundering) banking regulations")))((("banking regulations and bitcoin")))((("KYC (Know Your Customer) banking regulations"))) KYC (know your customer) and AML (anti-money laundering) banking regulations. Once you have an account on a bitcoin exchange, you can then buy or sell bitcoins quickly just as you could with foreign currency with a brokerage account.
You can find a more complete list at http://bitcoincharts.com/markets[bitcoin charts], a site that offers price quotes and other market data across many dozens of currency exchanges.
There are four other methods for getting bitcoins as a new user:
* Find((("bitcoins, buying for cash"))) a friend who has bitcoins and buy some from him directly. Many bitcoin users start this way.
* Use a classified service such as localbitcoins.com to find a seller in your area to buy bitcoins for cash in an in-person transaction.
* Sell a product or service for bitcoin. If you're a programmer, sell your programming skills.
* Use((("ATMs, bitcoin")))((("bitcoin ATMs"))) a bitcoin ATM in your city. Find a bitcoin ATM close to you using an online map from http://www.coindesk.com/bitcoin-atm-map/[CoinDesk].
Alice was introduced to bitcoin by a friend and so she has an easy way of getting her first bitcoins while she waits for her account on a California currency market to be verified and activated.
[[sending_receiving]]
==== Sending and Receiving Bitcoins
((("bitcoin","sending/receiving", id="ix_ch01-asciidoc1", range="startofrange")))Alice has created her bitcoin wallet and she is now ready to receive funds. Her wallet application randomly generated a private key (described in more detail in <<private_keys>>) together with its corresponding bitcoin address. At this point, her bitcoin address is not known to the bitcoin network or "registered" with any part of the bitcoin system. Her bitcoin address is simply a number that corresponds to a key that she can use to control access to the funds. There is no account or association between that address and an account. Until the moment this address is referenced as the recipient of value in a transaction posted on the bitcoin ledger (the blockchain), it is simply part of the vast number of possible addresses that are "valid" in bitcoin. Once it has been associated with a transaction, it becomes part of the known addresses in the network and Alice can check its balance on the public ledger.
Alice meets her friend Joe, who introduced her to bitcoin, at a local restaurant so they can exchange some US dollars and put some bitcoins into her account. She has brought a printout of her address and the QR code as displayed in her bitcoin wallet. There is nothing sensitive, from a security perspective, about the bitcoin address. It can be posted anywhere without risking the security of her account.
Alice wants to convert just 10 US dollars into bitcoin, so as not to risk too much money on this new technology. She gives Joe a $10 bill and the printout of her address so that Joe can send her the equivalent amount of bitcoin.
((("exchange rate, finding")))Next, Joe has to figure out the exchange rate so that he can give the correct amount of bitcoin to Alice. There are hundreds of applications and websites that can provide the current market rate. Here are some of the most popular:
http://bitcoincharts.com[Bitcoin Charts]:: ((("bitcoincharts.com")))A market data listing service that shows the market rate of bitcoin across many exchanges around the globe, denominated in different local currencies
http://bitcoinaverage.com/[Bitcoin Average]:: ((("bitcoinaverage.com")))A site that provides a simple view of the volume-weighted-average for each currency
http://www.zeroblock.com/[ZeroBlock]:: ((("ZeroBlock")))A free Android and iOS application that can display a bitcoin price from different exchanges (see <<zeroblock-android>>)
http://www.bitcoinwisdom.com/[Bitcoin Wisdom]:: ((("bitcoinwisdom.com")))Another market data listing service
[[zeroblock-android]]
.ZeroBlock, a bitcoin market-rate application for Android and iOS
image::images/msbt_0103.png["zeroblock screenshot"]
Using one of the applications or websites just listed, Joe determines the price of bitcoin to be approximately 100 US dollars per bitcoin. At that rate he should give Alice 0.10 bitcoin, also known as 100 millibits, in return for the 10 US dollars she gave him.
Once Joe has established a fair exchange price, he opens his mobile wallet application and selects to "send" bitcoin. For example, if using the Blockchain mobile wallet on an Android phone, he would see a screen requesting two inputs, as shown in <<blockchain-mobile-send>>.
* The destination bitcoin address for the transaction
* The amount of bitcoin to send
In the input field for the bitcoin address, there is a small icon that looks like a QR code. This allows Joe to scan the barcode with his smartphone camera so that he doesn't have to type in Alice's bitcoin address (+1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+), which is quite long and difficult to type. Joe taps the QR code icon and activates the smartphone camera, scanning the QR code from Alice's printed wallet that she brought with her. The mobile wallet application fills in the bitcoin address and Joe can check that it scanned correctly by comparing a few digits from the address with the address printed by Alice.
[[blockchain-mobile-send]]
.Blockchain mobile wallet's bitcoin send screen
image::images/msbt_0104.png["blockchain mobile send screen"]
Joe then enters the bitcoin value for the transaction, 0.10 bitcoin. He carefully checks to make sure he has entered the correct amount, because he is about to transmit money and any mistake could be costly. Finally, he presses Send to transmit the transaction. Joe's mobile bitcoin wallet constructs a transaction that assigns 0.10 bitcoin to the address provided by Alice, sourcing the funds from Joe's wallet and signing the transaction with Joe's private keys. This tells the bitcoin network that Joe has authorized a transfer of value from one of his addresses to Alice's new address. As the transaction is transmitted via the peer-to-peer protocol, it quickly propagates across the bitcoin network. In less than a second, most of the well-connected nodes in the network receive the transaction and see Alice's address for the first time.
If Alice has a smartphone or laptop with her, she will also be able to see the transaction. The bitcoin ledger—a constantly growing file that records every bitcoin transaction that has ever occurred—is public, meaning that all she has to do is look up her own address and see if any funds have been sent to it. She can do this quite easily at the((("blockchain.info website"))) blockchain.info website by entering her address in the search box. The website will show her a http://bit.ly/1u0FFKL[page] listing all the transactions to and from that address. If Alice is watching that page, it will update to show a new transaction transferring 0.10 bitcoin to her balance soon after Joe hits Send.
++++
<?hard-pagebreak?>
++++
.Confirmations
****
((("confirmation of transactions")))At first, Alice's address will show the transaction from Joe as "Unconfirmed." This means that the transaction has been propagated to the network but has not yet been included in the bitcoin transaction ledger, known as the blockchain. To be included, the transaction must be "picked up" by a miner and included in a block of transactions. Once a new block is created, in approximately 10 minutes, the transactions within the block will be accepted as "confirmed" by the network and can be spent. The transaction is seen by all instantly, but it is only "trusted" by all when it is included in a newly mined block.
****
Alice is now the proud owner of 0.10 bitcoin that she can spend. In the next chapter we will look at her first purchase with bitcoin, and examine the underlying transaction and propagation technologies in more detail.(((range="endofrange", startref="ix_ch01-asciidoc1")))(((range="endofrange", startref="ix_ch01-asciidoc0")))

523
ch02.asciidoc
Unescape View File

@ -1,263 +1,260 @@
[[ch02_bitcoin_overview]]
== How Bitcoin Works
=== Transactions, Blocks, Mining, and the Blockchain
The bitcoin system, unlike traditional banking and payment systems, is based on de-centralized trust. Instead of a central trusted authority, in bitcoin, trust is achieved as an emergent property from the interactions of different participants in the bitcoin system. In this chapter we will examine bitcoin from a high-level by tracking a single transaction through the bitcoin system and watch as it becomes "trusted" and accepted by the bitcoin mechanism of distributed consensus and is finally recorded on the blockchain, the distributed ledger of all transactions.
Each example below is based upon an actual transaction made on the bitcoin network, simulating the interactions between the users (Joe, Alice and Bob) by sending funds from one wallet to another. While tracking a transaction through the bitcoin network and blockchain, we will use a _blockchain explorer_ site to visualize each step. A blockchain explorer is a web application that operates as a bitcoin search engine, in that it allows you to search for addresses, transactions and blocks and see the relationships and flows between them.
Popular blockchain explorers include:
* blockchain.info
* blockexplorer.com
* insight.bitpay.com
* blockr.io
Each of these has a search function that can take an address, transaction hash or block number and find the equivalent data on the bitcoin network and blockchain. With each example, we will provide a URL that takes you directly to the relevant entry, so you can study it in detail.
==== Bitcoin Overview
In the overview diagram below, we see that the bitcoin system consists of users with wallets containing keys, transactions which are propagated across the network and miners who produce (through competitive computation) the consensus blockchain, the authoritative ledger of all transactions. In this chapter, we will trace a single transaction as it travels across the network and examine the interactions between each part of the bitcoin system, at a high level. Subsequent chapters will delve into the technology behind wallets, mining and merchant systems.
[[bitcoin-overview]]
.Bitcoin Overview
image::images/Bitcoin_Overview.png["Bitcoin Overview"]
[[cup_of_coffee]]
==== Buying a cup of coffee
Alice, introduced in the previous chapter, is a new user who has just acquired her first bitcoin. In <<getting_first_bitcoin>>, Alice met with her friend Joe to exchange some cash for bitcoin. The transaction created by Joe, funded Alice's wallet with 0.10 BTC. Now Alice will make her first retail transaction, buying a cup of coffee at Bob's coffee shop in Palo Alto, California. Bob's coffee shop recently started accepting bitcoin payments, by adding a bitcoin option to his point-of-sale system. The prices at Bob's Cafe are listed in the local currency (US dollars) but at the register, customers have the option of paying in either dollars or bitcoin. Alice places her order for a cup of coffee and Bob enters the transaction at the register. The point-of-sale system will convert the total price from US dollars to bitcoins at the prevailing market rate and display the prices in both currencies, as well as showing a QR code containing a _payment request_ for this transaction:
.Displayed on Bob's cash register
----
Total:
$1.50 USD
0.015 BTC
----
[[payment-request-QR]]
.Payment Request QR Code - Hint: Try to scan this!
image::images/payment-request-qr.png["payment-request"]
[[payment-request-URL]]
.The payment request QR code above encodes the following URL, defined in BIP0021
----
bitcoin:1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA?\
amount=0.015&\
label=Bob%27s%20Cafe&\
message=Purchase%20at%20Bob%27s%20Cafe
Components of the URL
A bitcoin address: "1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA"
The payment amount: "0.015"
A label for the recipient address: "Bob's Cafe"
A description for the payment: "Purchase at Bob's Cafe"
----
[TIP]
====
Unlike a QR code that simply contains a destination bitcoin address, a "payment request" is a QR encoded URL that contains a destination address, a payment amount and a generic description such as "Bob's Cafe". This allows a bitcoin wallet application to pre-fill the information used to send the payment while showing a human-readable description to the user. You can scan the QR code above with a bitcoin wallet application to see what Alice would see.
====
Bob says "That's one-dollar-fifty, or fifteen milliBits".
Alice uses her smartphone to scan the barcode on display. Her smartphone shows a payment of +0.0150 BTC+ to +Bob's Cafe+ and she selects +Send+ to authorize the payment. Within a few seconds (about the same time as a credit card authorization), Bob would see the transaction on the register, completing the transaction.
In the following sections we will examine this transaction in more detail, see how Alice's wallet constructed it, how it was propagated across the network, how it was verified and finally how Bob, the owner of the cafe, can spend that amount in subsequent transactions.
[NOTE]
====
The bitcoin network can transact in fractional values, e.g. from milli-bitcoins (1/1000th of a bitcoin) down to 1/100,000,000th of a bitcoin, which is known as a Satoshi. Throughout this book we’ll use the term “bitcoins” to refer to any quantity of bitcoin currency, from the smallest unit (1 Satoshi) to the total number (21,000,000) of all bitcoins that will ever be mined.
====
=== Bitcoin Transactions
In simple terms, a transaction tells the network that the owner of a number of bitcoins has authorized the transfer of some of those bitcoins to another owner. The new owner can now spend these bitcoins by creating another transaction that authorizes transfer to another owner, and so on, in a chain of ownership.
Transactions are like lines in a double-entry bookkeeping ledger. In simple terms, each transaction contains one or more "inputs", which are debits against a bitcoin account. On the other side of the transaction, there are one or more "outputs", which are credits added to a bitcoin account. The inputs and outputs (debits and credits) do not necessarily add up to the same amount. Instead, outputs add up to slightly less than inputs and the difference represents an implied "transaction fee", a small payment collected by the miner who includes the transaction in the ledger.
[[transaction-double-entry]]
.Transaction as Double-Entry Bookkeeping
image::images/Transaction_Double_Entry.png["Transaction Double-Entry"]
The transaction also contains proof of ownership for each amount of bitcoin (inputs) whose value is transferred, in the form of a digital signature from the owner, which can be independently validated by anyone. In bitcoin terms, "spending" is signing a transaction which transfers value from a previous transaction over to a new owner identified by a bitcoin address.
[TIP]
====
_Transactions_ move value *from* _transaction inputs_ *to* _transaction outputs_. An input is where the coin value is coming from, usually a previous transaction's output. A transaction output assigns a new owner to the value by associating it with a key. The destination key is called an _encumbrance_. It imposes a requirement for a signature for the funds to be redeemed in future transactions. Outputs from one transaction can be used as inputs in a new transaction, thus creating a chain of ownership as the value is moved from address to address.
====
[[blockchain-mnemonic]]
.A chain of transactions, where the output of one transaction is the input of the next transaction
image::images/Transaction_Chain.png["Transaction chain"]
Alice's payment to Bob's Cafe utilizes a previous transaction as its input. In the previous chapter Alice received bitcoin from her friend Joe in return for cash. That transaction has a number of bitcoins locked (encumbered) against Alice's key. Her new transaction to Bob's Cafe references the previous transaction as an input and creates new outputs to pay for the cup of coffee and receive change. The transactions form a chain, where the inputs from the latest transaction correspond to outputs from previous transactions. Alice's key provides the signature which unlocks those previous transaction outputs, thereby proving to the bitcoin network that she owns the funds. She attaches the payment for coffee to Bob's address, thereby "encumbering" that output with the requirement that Bob produces a signature in order to spend that amount. This represents a transfer of value between Alice and Bob.
==== Common Transaction Forms
The most common form of transaction is a simple payment from one address to another, which often includes some "change" returned to the original owner. This type of transaction has one input and two outputs and is shown below:
[[transaction-common]]
.Most Common Transaction
image::images/Bitcoin_Transaction_Structure_Common.png["Common Transaction"]
Another common form of transaction is a transaction that aggregates several inputs into a single output. This represents the real-world equivalent of exchanging a pile of coins and currency notes for a single larger note. Transactions like these are sometimes generated by wallet applications to clean up lots of smaller amounts that were received as change for payments.
[[transaction-aggregating]]
.Transaction Aggregating Funds
image::images/Bitcoin_Transaction_Structure_Aggregating.png["Aggregating Transaction"]
Finally, another transaction form that is seen often on the bitcoin ledger is a transaction that distributes one input to multiple outputs representing multiple recipients. This type of transaction is sometimes used by commercial entities to distribute funds, such as when processing payroll payments to multiple employees.
[[transaction-distributing]]
.Transaction Distributing Funds
image::images/Bitcoin_Transaction_Structure_Distribution.png["Distributing Transaction"]
=== Constructing a Transaction
Alice's wallet application contains all the logic for selecting appropriate inputs and outputs to build a transaction to Alice's specification. Alice only needs to specify a destination and an amount and the rest happens in the wallet application without her seeing the details. Importantly, a wallet application can construct transactions even if it is completely offline. Like writing a cheque at home and later sending it to the bank in an envelope, the transaction does not need to be constructed and signed while connected to the bitcoin network. It only has to be sent to the network eventually for it to be executed.
==== Getting the right inputs
Alice's wallet application will first have to find inputs that can pay for the amount she wants to send to Bob. Most wallet applications keep a small database of "unspent transaction outputs" that are locked (encumbered) with the wallet's own keys. Therefore, Alice's wallet would contain a copy of the transaction output from Joe's transaction which was created in exchange for cash (see <<getting_first_bitcoin>>). A bitcoin wallet application that runs as a full-index client actually contains a copy of every unspent output from every transaction in the blockchain. This allows a wallet to construct transaction inputs as well as to quickly verify incoming transactions as having correct inputs. However, since a full-index client takes up a lot of disk space, most user wallets run "lightweight" clients that track only the user's own unspent outputs.
If the wallet application does not maintain a copy of unspent transaction outputs, it can query the bitcoin network to retrieve this information, using a variety of APIs available by different providers, or by asking a full-index node using the bitcoin JSON RPC API. Below we see an example of a RESTful API request, constructed as an HTTP GET command to a specific URL. This URL will return all the unspent transaction outputs for an address, giving any application the information it needs to construct transaction inputs for spending. We use the simple command-line HTTP client _cURL_ to retrieve the response:
.Look up all the unspent outputs for Alice's bitcoin address
[source,bash]
----
$ curl https://blockchain.info/unspent?active=1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK
----
[source,json]
----
{
"unspent_outputs":[
{
"tx_hash":"186f9f998a5...2836dd734d2804fe65fa35779",
"tx_index":104810202,
"tx_output_n": 0,
"script":"76a9147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a888ac",
"value": 10000000,
"value_hex": "00989680",
"confirmations":0
}
]
}
----
The response above shows that the bitcoin network knows of one unspent output (one that has not been redeemed yet) under the ownership of Alice's address +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. The response includes the reference to the transaction in which this unspent output is contained (the payment from Joe) and its value in Satoshis, at 10 million, equivalent to 0.10 bitcoin. With this information, Alice's wallet application can construct a transaction to transfer that value to new owner addresses.
[TIP]
====
Use the following link to look up the transaction from Joe to Alice:
https://blockchain.info/tx/7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18
====
As you can see, Alice's wallet contains enough bitcoins in a single unspent output to pay for the cup of coffee. Had this not been the case, Alice's wallet application might have to "rummage" through a pile of smaller unspent outputs, like picking coins from a purse until it could find enough to pay for coffee. In both cases, there might be a need to get some change back, which we will see in the next section, as the wallet application creates the transaction outputs (payments).
==== Creating the outputs
A transaction output is created in the form of a script that creates an encumbrance on the value and can only be redeemed by the introduction of a solution to the script. In simpler terms, Alice's transaction output will contain a script that says something like "This output is payable to whoever can present a signature from the key corresponding to Bob's public address". Since only Bob has the wallet with the keys corresponding to that address, only Bob's wallet can present such a signature to redeem this output. Alice will therefore "encumber" the output value with a demand for a signature from Bob.
This transaction will also include a second output, because Alice's funds are in the form of a 0.10 BTC output, too much money for the 0.015 BTC cup of coffee. Alice will need 0.085 BTC in change. Alice's change payment is created _by Alice's wallet_ in the very same transaction as the payment to Bob. Essentially, Alice's wallet breaks her funds into two payments: one to Bob, and one back to herself. She can then use the change output in a subsequent transaction, thus spending it later.
Finally, for the transaction to be processed by the network in a timely fashion, Alice's wallet application will add a small fee. This is not explicit in the transaction, it is implied by the difference between inputs and outputs. If instead of taking 0.085 in change, Alice creates only 0.0845 as the second output, there will be 0.0005 BTC (half a millibitcoin) left over. The input's 0.10 BTC is not fully spent with the two outputs, as they will add up to less than 0.10. The resulting difference is the _transaction fee_ which is collected by the miner as a fee for including the transaction in a block and putting it on the blockchain ledger.
The resulting transaction can be seen using a blockchain explorer web application
[[transaction-alice]]
.Alice's transaction to Bob's Cafe
image::images/AliceCoffeeTransaction.png["Alice Coffee Transaction"]
[[transaction-alice-url]]
[TIP]
====
Use the following link to look up the transaction from Alice to Bob's Cafe:
https://blockchain.info/tx/0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2
====
==== Adding the transaction to the ledger
The transaction created by Alice's wallet application is 258 bytes long and contains everything necessary to confirm ownership of the funds and assign new owners. Now, the transaction must be transmitted to the bitcoin network where it will become part of the distributed ledger, the blockchain. In the next section we will see how a transaction becomes part of a new block and how the block is "mined". Finally, we will see how the new block, once added to the blockchain is increasingly trusted by the network as more blocks are added.
===== Transmitting the transaction
Since the transaction contains all the information necessary to process, it does not matter how or where it is transmitted to the bitcoin network. The bitcoin network is a peer-to-peer network, with each bitcoin client participating by connecting to several other bitcoin clients. The purpose of the bitcoin network is to propagate transactions and blocks to all participants.
===== How it propagates
Alice's wallet application can send the new transaction to any of the other bitcoin clients it is connected to over any Internet connection: wired, WiFi, or mobile. Her bitcoin wallet does not have to be connected to Bob's bitcoin wallet directly and she does not have to use the Internet connection offered by the cafe, though both those options are possible too. Any bitcoin network node (other client) that receives a valid transaction it has not seen before, will immediately forward it to other nodes to which it is connected. Thus, the transaction rapidly propagates out across the peer-to-peer network, reaching a large percentage of the nodes within a few seconds.
===== Bob's view
If Bob's bitcoin wallet application is directly connected to Alice's wallet application, Bob's wallet application may be the first node to receive the transaction. However, even if Alice's wallet sends the transaction through other nodes, it will reach Bob's wallet within a few seconds. Bob's wallet will immediately identify Alice's transaction as an incoming payment because it contains outputs redeemable by Bob's keys. Bob's wallet application can also independently verify that the transaction is well-formed, uses previously-unspent inputs and contains sufficient transaction fees to be included in the next block. At this point Bob can assume, with little risk, that the transaction will shortly be included in a block and confirmed.
[TIP]
====
A common misconception about bitcoin transactions is that they must be "confirmed" by waiting 10 minutes for a new block, or up to sixty minutes for a full six confirmations. While confirmations ensure the transaction has been accepted by the whole network, such a delay is unnecessary for small value items like a cup of coffee. A merchant may accept a valid small-value transaction with no confirmations, with no more risk than a credit card payment made without ID or a signature, like merchants routinely accept today.
====
=== Bitcoin Mining
The transaction is now propagated on the bitcoin network. It does not become part of the shared ledger (the _blockchain_) until it is verified and included in a block by a process called _mining_. See <<mining>> for a detailed explanation.
The bitcoin system of trust is based on computation. Transactions are bundled into _blocks_, which require an enormous amount of computation to prove, but only a small amount of computation to verify as proven. This process is called _mining_ and serves two purposes in bitcoin:
* Mining creates new bitcoins in each block, almost like a central bank printing new money. The amount of bitcoin created per block is fixed and diminishes with time.
* Mining creates trust by ensuring that transactions are only confirmed if enough computational power was devoted to the block that contains them. More blocks mean more computation which means more trust.
A good way to describe mining is like a giant competitive game of sudoku that resets every time someone finds a solution and whose difficulty automatically adjusts so that it takes approximately 10 minutes to find a solution. Imagine a giant sudoku puzzle, several thousand rows and columns in size. If I show you a completed puzzle you can verify it quite quickly. However, if the puzzle has a few squares filled and the rest is empty, it takes a lot of work to solve! The difficulty of the sudoku can be adjusted by changing its size (more or fewer rows and columns), but it can still be verified quite easily even if it is very large. The "puzzle" used in bitcoin is based on a cryptographic hash and exhibits similar characteristics: it is asymmetrically hard to solve but easy to verify, and its difficulty can be adjusted.
In <<user-stories>> we introduced Jing, a computer engineering student in Shanghai. Jing is participating in the bitcoin network as a miner. Every 10 minutes or so, Jing joins thousands of other miners in a global race to find a solution to a block of transactions. Finding such a solution, the so-called "Proof-of-Work", requires quadrillions of hashing operations per second across the entire bitcoin network. The algorithm for "Proof-of-Work" involves repeatedly hashing the header of the block and a random number with the SHA256 cryptographic algorithm until a solution matching a pre-determined pattern emerges. The first miner to find such a solution wins the round of competition and publishes that block into the blockchain.
Jing started mining in 2010 using a very fast desktop computer to find a suitable Proof-of-Work for new blocks. As more miners started joining the bitcoin network, the difficulty of the problem increased rapidly. Soon, Jing and other miners upgraded to more specialized hardware, such as Graphical Processing Units (GPUs), as used in gaming desktops or consoles. As this book is written, by 2014, the difficulty is so high that it is only profitable to mine with Application Specific Integrated Circuits (ASIC), essentially hundreds of mining algorithms printed in hardware, running in parallel on a single silicon chip. Jing also joined a "mining pool", which much like a lottery-pool allows several participants to share their efforts and the rewards. Jing now runs two USB-connected ASIC machines to mine for bitcoin 24 hours a day. He pays his electricity costs by selling the bitcoin he is able to generate from mining, creating some income from the profits. His computer runs a copy of bitcoind, the reference bitcoin client, as a back-end to his specialized mining software.
=== Mining transactions in blocks
A transaction transmitted across the network is not verified until it becomes part of the global distributed ledger, the blockchain. Every ten minutes on average, miners generate a new block that contains all the transactions since the last block. New transactions are constantly flowing into the network from user wallets and other applications. As these are seen by the bitcoin network nodes, they get added to a temporary "pool" of unverified transactions maintained by each node. As miners build a new block, they add unverified transactions from this pool to a new block and then attempt to solve a very hard problem (aka Proof-of-Work) to prove the validity of that new block. The process of mining is explained in detail in <<mining>>.
Transactions are added to the new block, prioritized by the highest-fee transactions first and a few other criteria. Each miner starts the process of mining a new block of transactions as soon as they receive the previous block from the network, knowing they have lost that previous round of competition. They immediately create a new block, fill it with transactions and the fingerprint of the previous block and start calculating the Proof-of-Work for the new block. Each miner includes a special transaction in their block, one that pays their own bitcoin address a reward of newly created bitcoins (currently 25 BTC per block). If they find a solution that makes that block valid, they "win" this reward because their successful block is added to the global blockchain and the reward transaction they included becomes spendable. Jing, who participates in a mining pool, has set up his software to create new blocks that assign the reward to a pool address. From there, a share of the reward is distributed to Jing and other miners in proportion to the amount of work they contributed in the last round.
Alice's transaction was picked up by the network and included in the pool of unverified transactions. Since it had sufficient fees, it was included in a new block generated by Jing's mining pool. Approximately 5 minutes after the transaction was first transmitted by Alice's wallet, Jing's ASIC miner found a solution for the block and published it as block #277316, containing 419 other transactions. Jing's ASIC miner published the new block on the bitcoin network, where other miners validated it and started the race to generate the next block.
You can see the block that includes Alice's transaction here:
https://blockchain.info/block-height/277316
A few minutes later, a new block, #277317 is mined by another miner. As this new block is based on the previous block (#277316) that contained Alice's transaction, it added even more computation on top of that block, thereby strengthening the trust in those transactions. One block mined on top of the one containing the transaction is called "one confirmation" for that transaction. As the blocks pile on top of each other, it becomes exponentially harder to reverse the transaction, thereby making it more and more trusted by the network.
In the diagram below we can see block #277316, which contains Alice's transaction. Below it are 277,316 blocks (including block #0), linked to each other in a chain of blocks (blockchain) all the way back to block #0, the genesis block. Over time, as the "height" in blocks increases, so does the computation difficulty for each block and the chain as a whole. The blocks mined after the one that contains Alice's transaction act as further assurance, as they pile on more computation in a longer and longer chain. The blocks above count as "confirmations". By convention, any block with more than 6 confirmations is considered irrevocable, as it would require an immense amount of computation to invalidate and re-calculate six blocks. We will examine the process of mining and the way it builds trust in more detail in <<mining>>.
[[block-alice]]
.Alice's transaction included in block #277,316
image::images/Blockchain_height_and_depth.png["Alice's transaction included in a block"]
=== Spending the transaction
Now that Alice's transaction has been embedded in the blockchain as part of a block, it is part of the distributed ledger of bitcoin and visible to all bitcoin applications. Each bitcoin client can independently verify the transaction as valid and spendable. Full-index clients can track the source of the funds from the moment the bitcoins were first generated in a block, incrementally from transaction to transaction, until they reach Bob's address. Lightweight clients can do a Simplified Payment Verification (See <<spv_nodes>>) by confirming that the transaction is in the blockchain and has several blocks mined after it, thus providing assurance that the network accepts it as valid.
Bob can now spend the output from this and other transactions, by creating his own transactions that reference these outputs as their inputs and assign them new ownership. For example, Bob can pay a contractor or supplier by transferring value from Alice's coffee cup payment to these new owners. Most likely, Bob's bitcoin software will aggregate many small payments into a larger payment, perhaps concentrating all the day's bitcoin revenue into a single transaction. This would move the various payments into a single address, utilized as the store's general "checking" account. For a diagram of an aggregating transaction, see <<transaction-aggregating>>.
As Bob spends the payments received from Alice and other customers, he extends the chain of transactions which in turn are added to the global blockchain ledger for all to see and trust. Let's assume that Bob pays his web designer Gopesh in Bangalore for a new web site page. Now the chain of transactions will look like this:
[[block-alice]]
.Alice's transaction as part of a transaction chain from Joe to Gopesh
image::images/Alices_Transaction_Chain.png["Alice's transaction as part of a transaction chain"]
[[ch02_bitcoin_overview]]
== How Bitcoin Works
=== Transactions, Blocks, Mining, and the Blockchain
((("bitcoin","implementation of", id="ix_ch02-asciidoc0", range="startofrange")))The bitcoin system, unlike traditional banking and payment systems, is based on de-centralized trust. Instead of a central trusted authority, in bitcoin, trust is achieved as an emergent property from the interactions of different participants in the bitcoin system. In this chapter, we will examine bitcoin from a high level by tracking a single transaction through the bitcoin system and watch as it becomes "trusted" and accepted by the bitcoin mechanism of distributed consensus and is finally recorded on the blockchain, the distributed ledger of all transactions.
Each example is based on an actual transaction made on the bitcoin network, simulating the interactions between the users (Joe, Alice, and Bob) by sending funds from one wallet to another. While tracking a transaction through the bitcoin network and blockchain, we will use a((("blockchain explorer websites"))) _blockchain explorer_ site to visualize each step. A blockchain explorer is a web application that operates as a bitcoin search engine, in that it allows you to search for addresses, transactions, and blocks and see the relationships and flows between them.
Popular blockchain explorers include: ((("blockchain.info website")))((("blockexplorer.com")))((("blockr.io website")))((("insight.bitpay.com")))
* http://blockchain.info[Blockchain info]
* http://blockexplorer.com[Bitcoin Block Explorer]
* http://insight.bitpay.com[insight]
* http://blockr.io[blockr Block Reader]
Each of these has a search function that can take an address, transaction hash, or block number and find the equivalent data on the bitcoin network and blockchain. With each example, we will provide a URL that takes you directly to the relevant entry, so you can study it in detail.
==== Bitcoin Overview
In the overview diagram shown in <<bitcoin-overview>>, we see that the bitcoin system consists of users with wallets containing keys, transactions that are propagated across the network, and miners who produce (through competitive computation) the consensus blockchain, which is the authoritative ledger of all transactions. In this chapter, we will trace a single transaction as it travels across the network and examine the interactions between each part of the bitcoin system, at a high level. Subsequent chapters will delve into the technology behind wallets, mining, and merchant systems.
[[bitcoin-overview]]
.Bitcoin overview
image::images/msbt_0201.png["Bitcoin Overview"]
[[cup_of_coffee]]
==== Buying a Cup of Coffee
((("transactions", id="ix_ch02-asciidoc1", range="startofrange")))((("transactions","simple example of", id="ix_ch02-asciidoc2", range="startofrange")))Alice, introduced in the previous chapter, is a new user who has just acquired her first bitcoin. In <<getting_first_bitcoin>>, Alice met with her friend Joe to exchange some cash for bitcoin. The transaction created by Joe funded Alice's wallet with 0.10 BTC. Now Alice will make her first retail transaction, buying a cup of coffee at Bob's coffee shop in Palo Alto, California. Bob's coffee shop recently started accepting bitcoin payments, by adding a bitcoin option to his point-of-sale system. The prices at Bob's Cafe are listed in the local currency (US dollars), but at the register, customers have the option of paying in either dollars or bitcoin. Alice places her order for a cup of coffee and Bob enters the transaction at the register. The point-of-sale system will convert the total price from US dollars to bitcoins at the prevailing market rate and display the prices in both currencies, as well as show a QR code containing a _payment request_ for this transaction (see <<payment-request-QR>>):
----
Total:
$1.50 USD
0.015 BTC
----
[[payment-request-QR]]
.Payment request QR code (Hint: Try to scan this!)
image::images/msbt_0202.png["payment-request"]
[[payment-request-URL]]
.The payment request QR code encodes the following URL, defined in BIP0021:
----
bitcoin:1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA?
amount=0.015&
label=Bob%27s%20Cafe&
message=Purchase%20at%20Bob%27s%20Cafe
Components of the URL
A bitcoin address: "1GdK9UzpHBzqzX2A9JFP3Di4weBwqgmoQA"
The payment amount: "0.015"
A label for the recipient address: "Bob's Cafe"
A description for the payment: "Purchase at Bob's Cafe"
----
[TIP]
====
((("QR codes","payment requests as")))Unlike a QR code that simply contains a destination bitcoin address, a payment request is a QR-encoded URL that contains a destination address, a payment amount, and a generic description such as "Bob's Cafe." This allows a bitcoin wallet application to prefill the information used to send the payment while showing a human-readable description to the user. You can scan the QR code with a bitcoin wallet application to see what Alice would see.
====
Bob says, "That's one-dollar-fifty, or fifteen millibits."
Alice uses her smartphone to scan the barcode on display. Her smartphone shows a payment of +0.0150 BTC+ to +Bob's Cafe+ and she selects +Send+ to authorize the payment. Within a few seconds (about the same amount of time as a credit card authorization), Bob would see the transaction on the register, completing the transaction.
In the following sections we will examine this transaction in more detail, see how Alice's wallet constructed it, how it was propagated across the network, how it was verified, and finally, how Bob can spend that amount in subsequent transactions.
[NOTE]
====
The bitcoin network can transact in fractional values, e.g., from milli-bitcoins (1/1000th of a bitcoin) down to 1/100,000,000th of a bitcoin, which is known as a((("satoshis","defined"))) satoshi. Throughout this book we’ll use the term “bitcoin” to refer to any quantity of bitcoin currency, from the smallest unit (1 satoshi) to the total number (21,000,000) of all bitcoins that will ever be mined.(((range="endofrange", startref="ix_ch02-asciidoc2")))
====
=== Bitcoin Transactions
((("transactions","defined")))In simple terms, a transaction tells the network that the owner of a number of bitcoins has authorized the transfer of some of those bitcoins to another owner. The new owner can now spend these bitcoins by creating another transaction that authorizes transfer to another owner, and so on, in a chain of ownership.
Transactions are like lines in a double-entry bookkeeping ledger. ((("inputs, defined")))In simple terms, each transaction contains one or more "inputs," which are debits against a bitcoin account. ((("outputs, defined")))On the other side of the transaction, there are one or more "outputs," which are credits added to a bitcoin account. The inputs and outputs (debits and credits) do not necessarily add up to the same amount. Instead, outputs add up to slightly less than inputs and the difference represents an implied "transaction fee," which is a small payment collected by the miner who includes the transaction in the ledger. A bitcoin transaction is shown as a bookkeeping ledger entry in <<transaction-double-entry>>.
The transaction also contains proof of ownership for each amount of bitcoin (inputs) whose value is transferred, in the form of a digital signature from the owner, which can be independently validated by anyone. In bitcoin terms, "spending" is signing a transaction that transfers value from a previous transaction over to a new owner identified by a bitcoin address.
[TIP]
====
_Transactions_ move value from _transaction inputs_ to _transaction outputs_. An input is where the coin value is coming from, usually a previous transaction's output. A transaction output assigns a new owner to the value by associating it with a key. The destination key is called an _encumbrance_. It imposes a requirement for a signature for the funds to be redeemed in future transactions. Outputs from one transaction can be used as inputs in a new transaction, thus creating a chain of ownership as the value is moved from address to address (see <<blockchain-mnemonic>>).
====
[[transaction-double-entry]]
.Transaction as double-entry bookkeeping
image::images/msbt_0203.png["Transaction Double-Entry"]
[[blockchain-mnemonic]]
.A chain of transactions, where the output of one transaction is the input of the next transaction
image::images/msbt_0204.png["Transaction chain"]
Alice's payment to Bob's Cafe uses a previous transaction as its input. In the previous chapter Alice received bitcoin from her friend Joe in return for cash. That transaction has a number of bitcoins locked (encumbered) against Alice's key. Her new transaction to Bob's Cafe references the previous transaction as an input and creates new outputs to pay for the cup of coffee and receive change. The transactions form a chain, where the inputs from the latest transaction correspond to outputs from previous transactions. Alice's key provides the signature that unlocks those previous transaction outputs, thereby proving to the bitcoin network that she owns the funds. She attaches the payment for coffee to Bob's address, thereby "encumbering" that output with the requirement that Bob produces a signature in order to spend that amount. This represents a transfer of value between Alice and Bob. This chain of transactions, from Joe to Alice to Bob, is illustrated in <<blockchain-mnemonic>>.
==== Common Transaction Forms
((("transactions","common forms of", id="ix_ch02-asciidoc3", range="startofrange")))The most common form of transaction is a simple payment from one address to another, which often includes some "change" returned to the original owner. This type of transaction has one input and two outputs and is shown in <<transaction-common>>.
[[transaction-common]]
.Most common transaction
image::images/msbt_0205.png["Common Transaction"]
Another common form of transaction is one that aggregates several inputs into a single output (see <<transaction-aggregating>>). This represents the real-world equivalent of exchanging a pile of coins and currency notes for a single larger note. Transactions like these are sometimes generated by wallet applications to clean up lots of smaller amounts that were received as change for payments.
[[transaction-aggregating]]
.Transaction aggregating funds
image::images/msbt_0206.png["Aggregating Transaction"]
Finally, another transaction form that is seen often on the bitcoin ledger is a transaction that distributes one input to multiple outputs representing multiple recipients (see <<transaction-distributing>>). This type of transaction is sometimes used by commercial entities to distribute funds, such as when processing payroll payments to multiple employees.(((range="endofrange", startref="ix_ch02-asciidoc3")))
[[transaction-distributing]]
.Transaction distributing funds
image::images/msbt_0207.png["Distributing Transaction"]
=== Constructing a Transaction
((("transactions","constructing", id="ix_ch02-asciidoc4", range="startofrange")))Alice's wallet application contains all the logic for selecting appropriate inputs and outputs to build a transaction to Alice's specification. Alice only needs to specify a destination and an amount and the rest happens in the wallet application without her seeing the details. ((("offline transactions")))Importantly, a wallet application can construct transactions even if it is completely offline. Like writing a check at home and later sending it to the bank in an envelope, the transaction does not need to be constructed and signed while connected to the bitcoin network. It only has to be sent to the network eventually for it to be executed.
==== Getting the Right Inputs
((("transactions","inputs, getting", id="ix_ch02-asciidoc5", range="startofrange")))Alice's wallet application will first have to find inputs that can pay for the amount she wants to send to Bob. Most wallet applications keep a small database of "unspent transaction outputs" that are locked (encumbered) with the wallet's own keys. Therefore, Alice's wallet would contain a copy of the transaction output from Joe's transaction, which was created in exchange for cash (see <<getting_first_bitcoin>>). A bitcoin wallet application that runs as a full-index client actually contains a copy of every unspent output from every transaction in the blockchain. This allows a wallet to construct transaction inputs as well as quickly verify incoming transactions as having correct inputs. However, because a full-index client takes up a lot of disk space, most user wallets run "lightweight" clients that track only the user's own unspent outputs.
((("wallets","blockchain storage in")))If the wallet application does not maintain a copy of unspent transaction outputs, it can query the bitcoin network to retrieve this information, using a variety of APIs available by different providers or by asking a full-index node using the bitcoin JSON RPC API. <<example_2-1>> shows a RESTful API request, constructed as an HTTP GET command to a specific URL. This URL will return all the unspent transaction outputs for an address, giving any application the information it needs to construct transaction inputs for spending. We use the simple command-line HTTP client((("cURL HTTP client"))) _cURL_ to retrieve the response.
[[example_2-1]]
.Look up all the unspent outputs for Alice's bitcoin address
====
[source,bash]
----
$ curl https://blockchain.info/unspent?active=1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK
----
====
[[example_2-2]]
.Response to the lookup
====
[source,json]
----
{
"unspent_outputs":[
{
"tx_hash":"186f9f998a5...2836dd734d2804fe65fa35779",
"tx_index":104810202,
"tx_output_n": 0,
"script":"76a9147f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a888ac",
"value": 10000000,
"value_hex": "00989680",
"confirmations":0
}
]
}
----
====
The response in <<example_2-2>> shows one unspent output (one that has not been redeemed yet) under the ownership of Alice's address +1Cdid9KFAaatwczBwBttQcwXYCpvK8h7FK+. The response includes the reference to the transaction in which this unspent output is contained (the payment from Joe) and its value in satoshis, at 10 million, equivalent to 0.10 bitcoin. With this information, Alice's wallet application can construct a transaction to transfer that value to new owner addresses.
[TIP]
====
View the http://bit.ly/1tAeeGr[transaction from Joe to Alice].
====
As you can see, Alice's wallet contains enough bitcoins in a single unspent output to pay for the cup of coffee. Had this not been the case, Alice's wallet application might have to "rummage" through a pile of smaller unspent outputs, like picking coins from a purse until it could find enough to pay for coffee. In both cases, there might be a need to get some change back, which we will see in the next section, as the wallet application creates the transaction outputs (payments).(((range="endofrange", startref="ix_ch02-asciidoc5")))
==== Creating the Outputs
((("transactions","outputs, creating")))A transaction output is created in the form of a script that creates an encumbrance on the value and can only be redeemed by the introduction of a solution to the script. In simpler terms, Alice's transaction output will contain a script that says something like, "This output is payable to whoever can present a signature from the key corresponding to Bob's public address." Because only Bob has the wallet with the keys corresponding to that address, only Bob's wallet can present such a signature to redeem this output. Alice will therefore "encumber" the output value with a demand for a signature from Bob.
This transaction will also include a second output, because Alice's funds are in the form of a 0.10 BTC output, too much money for the 0.015 BTC cup of coffee. Alice will need 0.085 BTC in change. Alice's change payment is created _by Alice's wallet_ in the very same transaction as the payment to Bob. Essentially, Alice's wallet breaks her funds into two payments: one to Bob, and one back to herself. She can then use the change output in a subsequent transaction, thus spending it later.
Finally, for the transaction to be processed by the network in a timely fashion, Alice's wallet application will add a small fee. This is not explicit in the transaction; it is implied by the difference between inputs and outputs. If instead of taking 0.085 in change, Alice creates only 0.0845 as the second output, there will be 0.0005 BTC (half a millibitcoin) left over. The input's 0.10 BTC is not fully spent with the two outputs, because they will add up to less than 0.10. The resulting difference is the _transaction fee_ that is collected by the miner as a fee for including the transaction in a block and putting it on the blockchain ledger.
The resulting transaction can be seen using a blockchain explorer web application, as shown in <<transaction-alice>>.
[[transaction-alice]]
.Alice's transaction to Bob's Cafe
image::images/msbt_0208.png["Alice Coffee Transaction"]
[[transaction-alice-url]]
[TIP]
====
View the http://bit.ly/1u0FIGs[transaction from Alice to Bob's Cafe].
====
==== Adding the Transaction to the Ledger
((("transactions","adding to ledger")))The transaction created by Alice's wallet application is 258 bytes long and contains everything necessary to confirm ownership of the funds and assign new owners. Now, the transaction must be transmitted to the bitcoin network where it will become part of the distributed ledger (the blockchain). In the next section we will see how a transaction becomes part of a new block and how the block is "mined." Finally, we will see how the new block, once added to the blockchain, is increasingly trusted by the network as more blocks are added.
===== Transmitting the transaction
((("transactions","transmitting")))((("transmitting transactions")))Because the transaction contains all the information necessary to process, it does not matter how or where it is transmitted to the bitcoin network. The bitcoin network is a peer-to-peer network, with each bitcoin client participating by connecting to several other bitcoin clients. The purpose of the bitcoin network is to propagate transactions and blocks to all participants.
===== How it propagates
((("transactions","propagating")))Alice's wallet application can send the new transaction to any of the other bitcoin clients it is connected to over any Internet connection: wired, WiFi, or mobile. Her bitcoin wallet does not have to be connected to Bob's bitcoin wallet directly and she does not have to use the Internet connection offered by the cafe, though both those options are possible, too. Any bitcoin network node (other client) that receives a valid transaction it has not seen before will immediately forward it to other nodes to which it is connected. Thus, the transaction rapidly propagates out across the peer-to-peer network, reaching a large percentage of the nodes within a few seconds.
===== Bob's view
If Bob's bitcoin wallet application is directly connected to Alice's wallet application, Bob's wallet application might be the first node to receive the transaction. However, even if Alice's wallet sends the transaction through other nodes, it will reach Bob's wallet within a few seconds. Bob's wallet will immediately identify Alice's transaction as an incoming payment because it contains outputs redeemable by Bob's keys. Bob's wallet application can also independently verify that the transaction is well formed, uses previously unspent inputs, and contains sufficient transaction fees to be included in the next block. At this point Bob can assume, with little risk, that the transaction will shortly be included in a block and confirmed.
[TIP]
====
((("transactions","accepting without confirmations")))A common misconception about bitcoin transactions is that they must be "confirmed" by waiting 10 minutes for a new block, or up to 60 minutes for a full six confirmations. Although confirmations ensure the transaction has been accepted by the whole network, such a delay is unnecessary for small-value items such as a cup of coffee. A merchant may accept a valid small-value transaction with no confirmations, with no more risk than a credit card payment made without an ID or a signature, as merchants routinely accept today.(((range="endofrange", startref="ix_ch02-asciidoc4")))(((range="endofrange", startref="ix_ch02-asciidoc1")))
====
=== Bitcoin Mining
((("mining","blockchains")))The transaction is now propagated on the bitcoin network. It does not become part of the shared ledger (the _blockchain_) until it is verified and included in a block by a process called _mining_. See <<ch8>> for a detailed explanation.
The bitcoin system of trust is based on computation. Transactions are bundled into _blocks_, which require an enormous amount of computation to prove, but only a small amount of computation to verify as proven. The mining process serves two purposes in bitcoin:
* Mining creates new bitcoins in each block, almost like a central bank printing new money. The amount of bitcoin created per block is fixed and diminishes with time.
* Mining creates trust by ensuring that transactions are only confirmed if enough computational power was devoted to the block that contains them. More blocks mean more computation, which means more trust.
A good way to describe mining is like a giant competitive game of sudoku that resets every time someone finds a solution and whose difficulty automatically adjusts so that it takes approximately 10 minutes to find a solution. Imagine a giant sudoku puzzle, several thousand rows and columns in size. If I show you a completed puzzle you can verify it quite quickly. However, if the puzzle has a few squares filled and the rest are empty, it takes a lot of work to solve! The difficulty of the sudoku can be adjusted by changing its size (more or fewer rows and columns), but it can still be verified quite easily even if it is very large. The "puzzle" used in bitcoin is based on a cryptographic hash and exhibits similar characteristics: it is asymmetrically hard to solve but easy to verify, and its difficulty can be adjusted.
In <<user-stories>>, we introduced Jing, a computer engineering student in Shanghai. Jing is participating in the bitcoin network as a miner. Every 10 minutes or so, Jing joins thousands of other miners in a global race to find a solution to a block of transactions. Finding such a solution, the so-called proof of work, requires quadrillions of hashing operations per second across the entire bitcoin network. The algorithm for proof of work involves repeatedly hashing the header of the block and a random number with the SHA256 cryptographic algorithm until a solution matching a predetermined pattern emerges. The first miner to find such a solution wins the round of competition and publishes that block into the blockchain.
((("mining","profitability of")))Jing started mining in 2010 using a very fast desktop computer to find a suitable proof of work for new blocks. As more miners started joining the bitcoin network, the difficulty of the problem increased rapidly. Soon, Jing and other miners upgraded to more specialized hardware, such as high-end dedicated graphical processing units (GPUs) cards such as those used in gaming desktops or consoles. At the time of this writing, the difficulty is so high that it is profitable only to mine with application-specific integrated circuits (ASIC), essentially hundreds of mining algorithms printed in hardware, running in parallel on a single silicon chip. Jing also joined a "mining pool," which much like a lottery pool allows several participants to share their efforts and the rewards. Jing now runs two USB-connected ASIC machines to mine for bitcoin 24 hours a day. He pays his electricity costs by selling the bitcoin he is able to generate from mining, creating some income from the profits. His computer runs a copy of bitcoind, the reference bitcoin client, as a backend to his specialized mining software.
=== Mining Transactions in Blocks
((("mining","transactions in blocks")))((("transactions","mining in blocks")))A transaction transmitted across the network is not verified until it becomes part of the global distributed ledger, the blockchain. Every 10 minutes on average, miners generate a new block that contains all the transactions since the last block. New transactions are constantly flowing into the network from user wallets and other applications. As these are seen by the bitcoin network nodes, they get added to a temporary pool of unverified transactions maintained by each node. As miners build a new block, they add unverified transactions from this pool to a new block and then attempt to solve a very hard problem (a.k.a., proof of work) to prove the validity of that new block. The process of mining is explained in detail in <<mining>>.
Transactions are added to the new block, prioritized by the highest-fee transactions first and a few other criteria. Each miner starts the process of mining a new block of transactions as soon as he receives the previous block from the network, knowing he has lost that previous round of competition. He immediately creates a new block, fills it with transactions and the fingerprint of the previous block, and starts calculating the proof of work for the new block. Each miner includes a special transaction in his block, one that pays his own bitcoin address a reward of newly created bitcoins (currently 25 BTC per block). If he finds a solution that makes that block valid, he "wins" this reward because his successful block is added to the global blockchain and the reward transaction he included becomes spendable. Jing, who participates in a mining pool, has set up his software to create new blocks that assign the reward to a pool address. From there, a share of the reward is distributed to Jing and other miners in proportion to the amount of work they contributed in the last round.
Alice's transaction was picked up by the network and included in the pool of unverified transactions. Because it had sufficient fees, it was included in a new block generated by Jing's mining pool. Approximately five minutes after the transaction was first transmitted by Alice's wallet, Jing's ASIC miner found a solution for the block and published it as block #277316, containing 419 other transactions. Jing's ASIC miner published the new block on the bitcoin network, where other miners validated it and started the race to generate the next block.
You can see the block that includes https://blockchain.info/block-height/277316[Alice's transaction].
A few minutes later, a new block, #277317, is mined by another miner. Because this new block is based on the previous block (#277316) that contained Alice's transaction, it added even more computation on top of that block, thereby strengthening the trust in those transactions. The block containing Alice's transaction is counted as one "confirmation" of that transaction. Each block mined on top of the one containing the transaction is an additional confirmation. As the blocks pile on top of each other, it becomes exponentially harder to reverse the transaction, thereby making it more and more trusted by the network.
In the diagram in <<block-alice1>> we can see block #277316, which contains Alice's transaction. Below it are 277,316 blocks (including block #0), linked to each other in a chain of blocks (blockchain) all the way back to block #0, known as the _genesis block_. Over time, as the "height" in blocks increases, so does the computation difficulty for each block and the chain as a whole. The blocks mined after the one that contains Alice's transaction act as further assurance, as they pile on more computation in a longer and longer chain. By convention, any block with more than six confirmations is considered irrevocable, because it would require an immense amount of computation to invalidate and recalculate six blocks. We will examine the process of mining and the way it builds trust in more detail in <<ch8>>.
[[block-alice1]]
.Alice's transaction included in block #277316
image::images/msbt_0209.png["Alice's transaction included in a block"]
=== Spending the Transaction
((("transactions","spending")))Now that Alice's transaction has been embedded in the blockchain as part of a block, it is part of the distributed ledger of bitcoin and visible to all bitcoin applications. Each bitcoin client can independently verify the transaction as valid and spendable. Full-index clients can track the source of the funds from the moment the bitcoins were first generated in a block, incrementally from transaction to transaction, until they reach Bob's address. Lightweight clients can do what is called a simplified payment verification (see <<spv_nodes>>) by confirming that the transaction is in the blockchain and has several blocks mined after it, thus providing assurance that the network accepts it as valid.
Bob can now spend the output from this and other transactions, by creating his own transactions that reference these outputs as their inputs and assign them new ownership. For example, Bob can pay a contractor or supplier by transferring value from Alice's coffee cup payment to these new owners. Most likely, Bob's bitcoin software will aggregate many small payments into a larger payment, perhaps concentrating all the day's bitcoin revenue into a single transaction. This would move the various payments into a single address, used as the store's general "checking" account. For a diagram of an aggregating transaction, see <<transaction-aggregating>>.
As Bob spends the payments received from Alice and other customers, he extends the chain of transactions, which in turn are added to the global blockchain ledger for all to see and trust. Let's assume that Bob pays his web designer Gopesh in Bangalore for a new website page. Now the chain of transactions will look like <<block-alice2>>.(((range="endofrange", startref="ix_ch02-asciidoc0")))
[[block-alice2]]
.Alice's transaction as part of a transaction chain from Joe to Gopesh
image::images/msbt_0210.png["Alice's transaction as part of a transaction chain"]

2578
ch03.asciidoc
View File

File diff suppressed because it is too large Load Diff

2036
ch04.asciidoc
View File

File diff suppressed because it is too large Load Diff

352
ch05.asciidoc
Unescape View File

@ -5,48 +5,46 @@
[[ch5_intro]]
=== Introduction
Transactions are the most important part of the bitcoin system. Everything else in bitcoin is designed to ensure that transactions can be created, propagated on the network, validated, and finally added to the global ledger of transactions, the blockchain. Transactions are data structures that encode the transfer of value between participants in the bitcoin system. Each transaction is a public entry in bitcoin's global double-entry bookkeeping ledger, the blockchain.
((("transactions", id="ix_ch05-asciidoc0", range="startofrange")))Transactions are the most important part of the bitcoin system. Everything else in bitcoin is designed to ensure that transactions can be created, propagated on the network, validated, and finally added to the global ledger of transactions (the blockchain). Transactions are data structures that encode the transfer of value between participants in the bitcoin system. Each transaction is a public entry in bitcoin's blockchain, the global double-entry bookkeeping ledger.
In this chapter we will examine all the various forms of transactions, what they contain, how to create them, how they are verified, and how they become part of the permanent record of all transactions.
[[tx_lifecycle]]
=== Transaction Lifecycle
A transaction's lifecycle starts with the transaction's creation, also known as origination. The transaction is then signed, with one or more signatures indicating the authorization to spend the funds referenced by the transaction. The transaction is then broadcast on the bitcoin network, where each network node (participant) validates and propagates the transaction until it reaches (almost) every node in the network. Finally, the transaction is verified by a mining node and included in a block of transactions that is recorded on the blockchain.
((("transactions","lifecycle of", id="ix_ch05-asciidoc1", range="startofrange")))A transaction's lifecycle starts with the transaction's creation, also known as((("origination of transactions"))) _origination_. The transaction is then signed with one or more signatures indicating the authorization to spend the funds referenced by the transaction. The transaction is then broadcast on the bitcoin network, where each network node (participant) validates and propagates the transaction until it reaches (almost) every node in the network. Finally, the transaction is verified by a mining node and included in a block of transactions that is recorded on the blockchain.
Once recorded on the blockchain and confirmed by sufficient subsequent blocks (confirmations), the transaction is a permanent part of the bitcoin ledger and is accepted as valid by all participants. The funds allocated to a new owner by the transaction can then be spent in a new transaction, extending the chain of ownership and beginning the lifecycle of a transaction again.
[[tx_origination]]
==== Creating Transactions
In some ways it helps to think of a transaction in the same way as a paper cheque. Like a cheque, a transaction is an instrument that expresses the intent to transfer money and is not visible to the financial system until it is submitted for execution. Like a cheque, the originator of the transaction does not have to be the one signing the transaction.
((("transactions","creating")))In some ways it helps to think of a transaction in the same way as a paper check. Like a check, a transaction is an instrument that expresses the intent to transfer money and is not visible to the financial system until it is submitted for execution. Like a check, the originator of the transaction does not have to be the one signing the transaction.
Transactions can be created online or offline by anyone, even if the person creating the transaction is not an authorized signer on the account. For example an accounts payable clerk might process payable cheques for signature by the CEO. Similarly, an accounts payable clerk can create bitcoin transactions and then have the CEO apply digital signatures to make them valid. While a cheque references a specific account as the source of the funds, a bitcoin transaction references a specific previous transaction as its source, rather than an account.
Transactions can be created online or offline by anyone, even if the person creating the transaction is not an authorized signer on the account. For example, an accounts payable clerk might process payable checks for signature by the CEO. Similarly, an accounts payable clerk can create bitcoin transactions and then have the CEO apply digital signatures to make them valid. Whereas a check references a specific account as the source of the funds, a bitcoin transaction references a specific previous transaction as its source, rather than an account.
Once a transaction has been created, it is signed by the owner (or owners) of the source funds. If it was properly formed and signed, the signed transaction is now valid and contains all the information needed to execute the transfer of funds. Finally, the valid transaction has to reach the bitcoin network so that it can be propagated until it reaches a miner for inclusion in the public ledger, the blockchain.
Once a transaction has been created, it is signed by the owner (or owners) of the source funds. If it is properly formed and signed, the signed transaction is now valid and contains all the information needed to execute the transfer of funds. Finally, the valid transaction has to reach the bitcoin network so that it can be propagated until it reaches a miner for inclusion in the pubic ledger (the blockchain).
[[tx_bcast]]
==== Broadcasting Transactions to the Bitcoin Network
First, a transaction needs to be delivered to the bitcoin network so that it can be propagated and be included in the blockchain. In essence, a bitcoin transaction is just 300-400 bytes of data and has to reach any one of tens of thousands of bitcoin nodes. The sender does not need to trust the nodes they use to broadcast the transaction, as long as they use more than one to ensure that it propagates. The nodes don't need to trust the sender or establish the sender's "identity". Since the transaction is signed and contains no confidential information, private keys or credentials, it can be publicly broadcast using any underlying network transport that is convenient. Unlike credit card transactions, for example, which contain sensitive information and can only be transmitted on encrypted networks, a bitcoin transaction can be sent over any network. As long as the transaction can reach a bitcoin node that will propagate it into the bitcoin network, it doesn't matter how it is transported to the first node.
((("bitcoin network","broadcasting transactions to")))((("transactions","broadcasting to network")))First, a transaction needs to be delivered to the bitcoin network so that it can be propagated and included in the blockchain. In essence, a bitcoin transaction is just 300 to 400 bytes of data and has to reach any one of tens of thousands of bitcoin nodes. The senders do not need to trust the nodes they use to broadcast the transaction, as long as they use more than one to ensure that it propagates. The nodes don't need to trust the sender or establish the sender's "identity." Because the transaction is signed and contains no confidential information, private keys, or credentials, it can be publicly broadcast using any underlying network transport that is convenient. Unlike credit card transactions, for example, which contain sensitive information and can only be transmitted on encrypted networks, a bitcoin transaction can be sent over any network. As long as the transaction can reach a bitcoin node that will propagate it into the bitcoin network, it doesn't matter how it is transported to the first node.
Bitcoin transactions can therefore be transmitted to the bitcoin network over insecure networks such as Wifi, Bluetooth, NFC, Chirp, barcodes or by copying and pasting into a web form. In extreme cases, a bitcoin transaction could be transmitted over packet radio, satellite relay or shortwave using burst transmission, spread spectrum or frequency hopping to evade detection and jamming. A bitcoin transaction could even be encoded as smileys (emoticons) and posted in a public forum or sent as a text message or Skype chat message. Bitcoin has turned money into a data structure, making it virtually impossible to stop anyone from creating and executing a bitcoin transaction.
((("insecure networks, transmitting bitcoin over")))Bitcoin transactions can therefore be transmitted to the bitcoin network over insecure networks such as WiFi, Bluetooth, NFC, Chirp, barcodes, or by copying and pasting into a web form. In extreme cases, a bitcoin transaction could be transmitted over packet radio, satellite relay, or shortwave using burst transmission, spread spectrum, or frequency hopping to evade detection and jamming. A bitcoin transaction could even be encoded as smileys (emoticons) and posted in a public forum or sent as a text message or Skype chat message. Bitcoin has turned money into a data structure, making it virtually impossible to stop anyone from creating and executing a bitcoin transaction.
[[tx_propagation]]
==== Propagating Transactions on the Bitcoin Network
Once a bitcoin transaction is sent to any node connected to the bitcoin network, the transaction will be validated by that node. If valid, that node will propagate it to the other nodes to which it is connected and a success message will be returned synchronously to the originator. If the transaction is invalid, the node will reject it and synchronously return a rejection message to the originator.
((("bitcoin network","propagating transactions on")))((("transactions","propagating")))Once a bitcoin transaction is sent to any node connected to the bitcoin network, the transaction will be validated by that node. If valid, that node will propagate it to the other nodes to which it is connected, and a success message will be returned synchronously to the originator. If the transaction is invalid, the node will reject it and synchronously return a rejection message to the originator.
The bitcoin network is a peer-to-peer network meaning that each bitcoin node is connected to a few other bitcoin nodes that it discovers during startup through the peer-to-peer protocol. The entire network forms a loosely connected mesh without a fixed topology or any structure making all nodes equal peers. Messages, including transactions and blocks, are propagated from each node to the peers to which it is connected. A new validated transaction injected into any node on the network will be sent to 3 to 4 of the neighboring nodes, each of which will send it to 3 to 4 more nodes and so on. In this way, within a few seconds a valid transaction will propagate in an exponentially expanding ripple across the network until all connected nodes have received it.
The bitcoin network is a peer-to-peer network, meaning that each bitcoin node is connected to a few other bitcoin nodes that it discovers during startup through the peer-to-peer protocol. The entire network forms a loosely connected mesh without a fixed topology or any structure, making all nodes equal peers. Messages, including transactions and blocks, are propagated from each node to the peers to which it is connected. A new validated transaction injected into any node on the network will be sent to three to four of the neighboring nodes, each of which will send it to three to four more nodes, and so on. In this way, within a few seconds a valid transaction will propagate in an exponentially expanding ripple across the network until all connected nodes have received it.
The bitcoin network is designed to propagate transactions and blocks to all nodes in an efficient and resilient manner that is resistant to attacks. To prevent spamming, denial of service attacks, or other nuisance attacks against the bitcoin system, every node will independently validate every transaction before propagating it further. A malformed transaction will not get beyond one node. The rules by which transactions are validated are explained in more detail in <<tx_verification>>.
The bitcoin network is designed to propagate transactions and blocks to all nodes in an efficient and resilient manner that is resistant to attacks. To prevent spamming, denial-of-service attacks, or other nuisance attacks against the bitcoin system, every node independently validates every transaction before propagating it further. A malformed transaction will not get beyond one node. The rules by which transactions are validated are explained in more detail in <<tx_verification>>.(((range="endofrange", startref="ix_ch05-asciidoc1")))
[[tx_structure]]
=== Transaction Structure
A transaction is a _data structure_ that encodes a transfer of value from a source of funds, called an _input_, to a destination, called an _output_. Transaction inputs and outputs are not related to accounts or identities. Instead you should think of them as bitcoin amounts, chunks of bitcoin, being locked with a specific secret which only the owner, or person who knows the secret, can unlock.
A transaction contains a number of fields, as follows:
((("transactions","structure of")))A transaction is a((("data structure"))) _data structure_ that encodes a transfer of value from a source of funds, called an((("inputs, defined"))) _input_, to a destination, called an((("outputs, defined"))) _output_. Transaction inputs and outputs are not related to accounts or identities. Instead, you should think of them as bitcoin amounts—chunks of bitcoin—being locked with a specific secret that only the owner, or person who knows the secret, can unlock. A transaction contains a number of fields, as shown in <<tx_data_structure>>.
[[tx_data_structure]]
.The structure of a transaction
@ -54,39 +52,39 @@ A transaction contains a number of fields, as follows:
|=======
|Size| Field | Description
| 4 bytes | Version | Specifies which rules this transaction follows
| 1-9 bytes (VarInt) | Input Counter | How many inputs are included
| Variable | Inputs | One or more Transaction Inputs
| 1-9 bytes (VarInt) | Output Counter | How many outputs are included
| Variable | Outputs | One or more Transaction Outputs
| 4 bytes | Locktime | A unix timestamp or block number
| 1–9 bytes (VarInt) | Input Counter | How many inputs are included
| Variable | Inputs | One or more transaction inputs
| 1–9 bytes (VarInt) | Output Counter | How many outputs are included
| Variable | Outputs | One or more transaction outputs
| 4 bytes | Locktime | A Unix timestamp or block number
|=======
.Transaction Locktime
****
Locktime defines the earliest time that a transaction can be added to the blockchain. It is set to zero in most transactions to indicate immediate execution. If locktime is non-zero and below 500 million, it is interpreted as a block height, meaning the transaction is not included in the blockchain prior to the specified block height. If it is above 500 million, it is interpreted as a Unix Epoch timestamp (seconds since Jan-1-1970) and the transaction is not included in the blockchain prior to the specified time. The use of locktime is equivalent to post-dating a paper cheque.
((("locktime")))((("transactions","locktime")))Locktime defines the earliest time that a transaction can be added to the blockchain. It is set to zero in most transactions to indicate immediate execution. If locktime is nonzero and below 500 million, it is interpreted as a block height, meaning the transaction is not included in the blockchain prior to the specified block height. If it is above 500 million, it is interpreted as a Unix Epoch timestamp (seconds since Jan-1-1970) and the transaction is not included in the blockchain prior to the specified time. The use of locktime is equivalent to postdating a paper check.
****
[[tx_inputs_outputs]]
=== Transaction Outputs and Inputs
The fundamental building block of a bitcoin transaction is an _unspent transaction output_ or UTXO. UTXO are indivisible chunks of bitcoin currency locked to a specific owner, recorded on the blockchain, and recognized as currency units by the entire network. The bitcoin network tracks all available (unspent) UTXO currently numbering in the millions. Whenever a user receives bitcoin, that amount is recorded within the blockchain as a UTXO. Thus, a user's bitcoin may be scattered as UTXO amongst hundreds of transactions and hundreds of blocks. In effect, there is no such thing as a stored balance of a bitcoin address or account; there are only scattered UTXO, locked to specific owners. The concept of a user's bitcoin balance is a derived construct created by the wallet application. The wallet calculates the user's balance by scanning the blockchain and aggregating all UTXO belonging to that user.
((("transactions","unspent transaction output (UTXO)")))((("unspent transaction output (UTXO)")))The fundamental building block of a bitcoin transaction is an _unspent transaction output_, or UTXO. UTXO are indivisible chunks of bitcoin currency locked to a specific owner, recorded on the blockchain, and recognized as currency units by the entire network. The bitcoin network tracks all available (unspent) UTXO currently numbering in the millions. Whenever a user receives bitcoin, that amount is recorded within the blockchain as a UTXO. Thus, a user's bitcoin might be scattered as UTXO amongst hundreds of transactions and hundreds of blocks. In effect, there is no such thing as a stored balance of a bitcoin address or account; there are only scattered UTXO, locked to specific owners. The concept of a user's bitcoin balance is a derived construct created by the wallet application. The wallet calculates the user's balance by scanning the blockchain and aggregating all UTXO belonging to that user.
[TIP]
====
There are no accounts or balances in bitcoin, there are only _unspent transaction outputs_ (UTXO) scattered in the blockchain.
((("accounts")))((("balances")))There are no accounts or balances in bitcoin; there are only _unspent transaction outputs_ (UTXO) scattered in the blockchain.
====
A UTXO can have an arbitrary value denominated as a multiple of satoshis. Just like dollars can be divided down to two decimal places as cents, bitcoins can be divided down to eight decimal places as satoshis. While UTXO can be any arbitrary value, once created it is indivisible just like a coin that cannot be cut in half. If a UTXO is larger than the desired value of a transaction, it must still be consumed in its entirety and change must be generated in the transaction. In other words, if you have a 20 bitcoin UTXO and want to pay 1 bitcoin, your transaction must consume the entire 20 bitcoin UTXO and produce two outputs: one paying 1 bitcoin to your desired recipient and another paying 19 bitcoin in change back to your wallet. As a result, most bitcoin transactions will generate change.
A UTXO can have an arbitrary value denominated as a multiple of((("satoshis"))) satoshis. Just like dollars can be divided down to two decimal places as cents, bitcoins can be divided down to eight decimal places as satoshis. Although UTXO can be any arbitrary value, once created it is indivisible just like a coin that cannot be cut in half. If a UTXO is larger than the desired value of a transaction, it must still be consumed in its entirety and change must be generated in the transaction. ((("change, making")))In other words, if you have a 20 bitcoin UTXO and want to pay 1 bitcoin, your transaction must consume the entire 20 bitcoin UTXO and produce two outputs: one paying 1 bitcoin to your desired recipient and another paying 19 bitcoin in change back to your wallet. As a result, most bitcoin transactions will generate change.
Imagine a shopper buying a $1.50 beverage, reaching into their wallet and trying to find a combination of coins and bank notes to cover the $1.50 cost. The shopper will choose exact change if available (a dollar bill and two quarters), or a combination of smaller denominations (six quarters), or if necessary, a larger unit such as a five dollar bank note. If they hand too much money, say $5, to the shop owner they will expect $3.50 change, which they will return to their wallet and have available for future transactions.
Imagine a shopper buying a $1.50 beverage, reaching into her wallet and trying to find a combination of coins and bank notes to cover the $1.50 cost. The shopper will choose exact change if available (a dollar bill and two quarters), or a combination of smaller denominations (six quarters), or if necessary, a larger unit such as a five dollar bank note. If she hands too much money, say $5, to the shop owner, she will expect $3.50 change, which she will return to her wallet and have available for future transactions.
Similarly, a bitcoin transaction must be created from a user's UTXO in whatever denominations that user has available. They cannot cut a UTXO in half any more than they can cut a dollar bill in half and use it as currency. The user's wallet application will typically select from the user's available UTXO various units to compose an amount greater than or equal to the desired transaction amount.
Similarly, a bitcoin transaction must be created from a user's UTXO in whatever denominations that user has available. Users cannot cut a UTXO in half any more than they can cut a dollar bill in half and use it as currency. The user's wallet application will typically select from the user's available UTXO various units to compose an amount greater than or equal to the desired transaction amount.
As with real life, the bitcoin application can use several strategies to satisfy the purchase amount: combining several smaller units, finding exact change, or using a single unit larger than the transaction value and making change. All of this complex assembly of spendable UTXO is done by the user's wallet automatically and is invisible to users. It is only relevant if you are programmatically constructing raw transactions from UTXO.
The UTXO consumed by a transaction are called transaction inputs, while the UTXO created by a transaction are called transaction outputs. This way, chunks of bitcoin value move forward from owner to owner in a chain of transactions consuming and creating UTXO. Transactions consume UTXO unlocking it with the signature of the current owner and create UTXO locking it to the bitcoin address of the new owner.
The UTXO consumed by a transaction are called transaction inputs, and the UTXO created by a transaction are called transaction outputs. This way, chunks of bitcoin value move forward from owner to owner in a chain of transactions consuming and creating UTXO. Transactions consume UTXO by unlocking it with the signature of the current owner and create UTXO by locking it to the bitcoin address of the new owner.
The exception to the output and input chain is a special type of transaction called the _coinbase_ transaction, which is the first transaction in each block. This transaction is placed there by the "winning" miner and creates brand-new bitcoin payable to that miner as a reward for mining. This is how bitcoin's money supply is created during the mining process as we will see in <<mining>>.
The exception to the output and input chain is a special type of transaction called the _coinbase_ transaction, which is the first transaction in each block. This transaction is placed there by the "winning" miner and creates brand-new bitcoin payable to that miner as a reward for mining. This is how bitcoin's money supply is created during the mining process, as we will see in <<ch8>>.
[TIP]
====
@ -96,28 +94,29 @@ What comes first? Inputs or outputs, the chicken or the egg? Strictly speaking,
[[tx_outs]]
==== Transaction Outputs
Every bitcoin transaction creates outputs, which are recorded on the bitcoin ledger. Almost all of these outputs, with one exception (see <<op_return>>) create spendable chunks of bitcoin called _unspent transaction outputs_ or UTXO, which are then recognized by the whole network and available for the owner to spend in a future transaction. Sending someone bitcoin is creating an unspent transaction output (UTXO) registered to their address and available for them to spend.
((("bitcoin ledger, outputs in", id="ix_ch05-asciidoc2", range="startofrange")))((("transactions","outputs", id="ix_ch05-asciidoc3", range="startofrange")))((("unspent transaction output (UTXO)", id="ix_ch05-asciidoc4", range="startofrange")))Every bitcoin transaction creates outputs, which are recorded on the bitcoin ledger. Almost all of these outputs, with one exception (see <<op_return>>) create spendable chunks of bitcoin called _unspent transaction outputs_ or UTXO, which are then recognized by the whole network and available for the owner to spend in a future transaction. Sending someone bitcoin is creating an unspent transaction output (UTXO) registered to their address and available for them to spend.
UTXO are tracked by every full node bitcoin client in a database held in memory, called the _UTXO set_ or _UTXO pool_. New transactions consume (spend) one or more of these outputs from the UTXO set.
UTXO are tracked by every full-node bitcoin client in a database held in memory, called the((("UTXO pool")))((("UTXO set"))) _UTXO set_ or _UTXO pool_. New transactions consume (spend) one or more of these outputs from the UTXO set.
Transaction outputs consist of two parts:
* An amount of bitcoin, denominated in _satoshis_, the smallest bitcoin unit
* A _locking script_, also known as an "encumbrance" that "locks" this amount by specifying the conditions that must be met to spend the output
* A((("encumbrance")))((("locking scripts"))) _locking script_, also known as an "encumbrance" that "locks" this amount by specifying the conditions that must be met to spend the output
The transaction scripting language, used in the locking script mentioned above, is discussed in detail in <<tx_script>>
The transaction scripting language, used in the locking script mentioned previously, is discussed in detail in <<tx_script>>. <<tx_out_structure>> shows the structure of a transaction output.
[[tx_out_structure]]
.The structure of a transaction output
[options="header"]
|=======
|Size| Field | Description
| 8 bytes | Amount | Bitcoin Value in Satoshis (10^-8^ bitcoin)
| 8 bytes | Amount | Bitcoin value in satoshis (10^-8^ bitcoin)
| 1-9 bytes (VarInt) | Locking-Script Size | Locking-Script length in bytes, to follow
| Variable | Locking-Script | A script defining the conditions needed to spend the output
|=======
In the example below, we use the blockchain.info API to find the unspent outputs (UTXO) of a specific address:
In <<get_utxo>>, we use the blockchain.info API to find the unspent outputs (UTXO) of a specific address.
[[get_utxo]]
.A script that calls the blockchain.info API to find the UTXO related to an address
====
@ -127,7 +126,7 @@ include::code/get-utxo.py[]
----
====
Running the script, we see a list of transaction IDs, a colon, the index number of the specific unspent transaction output (UTXO), and the value of that UTXO in Satoshis. The locking script is not shown in this output:
Running the script, we see a list of transaction IDs, a colon, the index number of the specific unspent transaction output (UTXO), and the value of that UTXO in satoshis. The locking script is not shown in the output in <<get_utxo_run>>.
[[get_utxo_run]]
.Running the get-utxo.py script
@ -143,18 +142,18 @@ b2affea89ff82557c60d635a2a3137b8f88f12ecec85082f7d0a1f82ee203ac4:0 - 10000000 Sa
----
====
===== Spending Conditions (Encumbrances)
===== Spending conditions (encumbrances)
Transaction outputs associate a specific amount (in satoshis) to a specific _encumbrance_ or locking-script that defines the condition that must be met to spend that amount. In most cases the locking script will lock the output to a specific bitcoin address, thereby transferring ownership of that amount to the new owner. When Alice paid Bob's Cafe for a cup of coffee, her transaction created a 0.015 bitcoin output _encumbered_ or locked to the Cafe's bitcoin address. That 0.015 bitcoin output was recorded on the blockchain and became part of the Unspent Transaction Output set, meaning it showed in Bob's wallet as part of the available balance. When Bob chooses to spend that amount, his transaction will release the encumbrance, unlocking the output by providing an unlocking script containing a signature from Bob's private key.
((("encumbrance")))((("locking scripts")))Transaction outputs associate a specific amount (in satoshis) to a specific _encumbrance_ or locking script that defines the condition that must be met to spend that amount. In most cases, the locking script will lock the output to a specific bitcoin address, thereby transferring ownership of that amount to the new owner. When Alice paid Bob's Cafe for a cup of coffee, her transaction created a 0.015 bitcoin output _encumbered_ or locked to the cafe's bitcoin address. That 0.015 bitcoin output was recorded on the blockchain and became part of the Unspent Transaction Output set, meaning it showed in Bob's wallet as part of the available balance. When Bob chooses to spend that amount, his transaction will release the encumbrance, unlocking the output by providing an unlocking script containing a signature from Bob's private key.(((range="endofrange", startref="ix_ch05-asciidoc4")))(((range="endofrange", startref="ix_ch05-asciidoc3")))(((range="endofrange", startref="ix_ch05-asciidoc2")))
[[tx_inputs]]
==== Transaction Inputs
In simple terms, transaction inputs are pointers to UTXO. They point to a specific UTXO by reference to the transaction hash and sequence number where the UTXO is recorded in the blockchain. To spend UTXO, a transaction input also includes unlocking-scripts that satisfy the spending conditions set by the UTXO. The unlocking script is usually a signature proving ownership of the bitcoin address that is in the locking script.
((("transactions","inputs", id="ix_ch05-asciidoc5", range="startofrange")))In simple terms, transaction inputs are pointers to UTXO. They point to a specific UTXO by reference to the transaction hash and sequence number where the UTXO is recorded in the blockchain. To spend UTXO, a transaction input also includes unlocking scripts that satisfy the spending conditions set by the UTXO. The unlocking script is usually a signature proving ownership of the bitcoin address that is in the locking script.
When a user makes a payment, their wallet constructs a transaction by selecting from the available UTXO. For example, to make a 0.015 bitcoin payment, the wallet app may select a 0.01 UTXO and a 0.005 UTXO, using them both to add up to the desired payment amount.
When users make a payment, their wallet constructs a transaction by selecting from the available UTXO. For example, to make a 0.015 bitcoin payment, the wallet app may select a 0.01 UTXO and a 0.005 UTXO, using them both to add up to the desired payment amount.
In the example below, we show the use of a "greedy" algorithm to select from available UTXO in order to make a specific payment amount. In the example, the available UTXO are provided as a constant array, but in reality, the available UTXO would be retrieved with an RPC call to Bitcoin Core, or to a third-party API as shown in <<get_utxo>>.
In <<select_utxo>>, we show the use of a "greedy" algorithm to select from available UTXO in order to make a specific payment amount. In the example, the available UTXO are provided as a constant array, but in reality, the available UTXO would be retrieved with an RPC call to Bitcoin Core, or to a third-party API as shown in <<get_utxo>>.
[[select_utxo]]
.A script for calculating how much total bitcoin will be issued
@ -165,23 +164,19 @@ include::code/select-utxo.py[]
----
====
If we run the select-utxo.py script without a parameter it will attempt to construct a set of UTXO (and change) for a payment of 55000000 Satoshis (0.55 bitcoin). If you provide a target payment amount as a parameter, the script will select UTXO to make that target payment amount. Below, we run the script trying to make a payment of 0.5 bitcoin or 50000000 Satoshis:
If we run the _select-utxo.py_ script without a parameter, it will attempt to construct a set of UTXO (and change) for a payment of 55,000,000 satoshis (0.55 bitcoin). If you provide a target payment amount as a parameter, the script will select UTXO to make that target payment amount. In <<select_utxo_run>>, we run the script trying to make a payment of 0.5 bitcoin or 50,000,000 satoshis.
[[select_utxo_run]]
.Running the select-utxo.py script
====
[source,bash]
----
$ python select-utxo.py 50000000
For transaction amount 50000000 Satoshis (0.500000 bitcoin) use:
([<7dbc497969c7475e45d952c4a872e213fb15d45e5cd3473c386a71a1b0c136a1:0 with 25000000 Satoshis>, \
<7f42eda67921ee92eae5f79bd37c68c9cb859b899ce70dba68c48338857b7818:0 with 16100000 Satoshis>, \
<6596fd070679de96e405d52b51b8e1d644029108ec4cbfe451454486796a1ecf:0 with 16050000 Satoshis>],\
'Change: 7150000 Satoshis')
([<7dbc497969c7475e45d952c4a872e213fb15d45e5cd3473c386a71a1b0c136a1:0 with 25000000 Satoshis>, <7f42eda67921ee92eae5f79bd37c68c9cb859b899ce70dba68c48338857b7818:0 with 16100000 Satoshis>, <6596fd070679de96e405d52b51b8e1d644029108ec4cbfe451454486796a1ecf:0 with 16050000 Satoshis>], 'Change: 7150000 Satoshis')
----
====
Once the UTXO is selected, the wallet then produces unlocking scripts containing signatures for each of the UTXO, thereby making them spendable by satisfying their locking script conditions. The wallet adds these UTXO references and unlocking scripts as inputs to the transaction.
Once the UTXO is selected, the wallet then produces unlocking scripts containing signatures for each of the UTXO, thereby making them spendable by satisfying their locking script conditions. The wallet adds these UTXO references and unlocking scripts as inputs to the transaction. <<tx_in_structure>> shows the structure of a transaction input.
[[tx_in_structure]]
.The structure of a transaction input
@ -189,131 +184,132 @@ Once the UTXO is selected, the wallet then produces unlocking scripts containing
|=======
|Size| Field | Description
| 32 bytes | Transaction Hash | Pointer to the transaction containing the UTXO to be spent
| 4 bytes | Output Index | The index number of the UTXO to be spent, first one is 0
| 4 bytes | Output Index | The index number of the UTXO to be spent; first one is 0
| 1-9 bytes (VarInt) | Unlocking-Script Size | Unlocking-Script length in bytes, to follow
| Variable | Unlocking-Script | A script that fulfills the conditions of the UTXO locking-script.
| 4 bytes | Sequence Number | Currently-disabled Tx-replacement feature, set to 0xFFFFFFFF
| Variable | Unlocking-Script | A script that fulfills the conditions of the UTXO locking script.
| 4 bytes | Sequence Number | Currently disabled Tx-replacement feature, set to 0xFFFFFFFF
|=======
Note: The sequence number is used to override a transaction prior to the expiration of the transaction locktime, which is a feature that is currently disabled in bitcoin. Most transactions set this value to the maximum integer value (0xFFFFFFFF) and it is ignored by the bitcoin network. If the transaction has a non-zero locktime, at least one of its inputs must have a sequence number below 0xFFFFFFFF in order to enable locktime.
[NOTE]
====
The sequence number is used to override a transaction prior to the expiration of the transaction locktime, which is a feature that is currently disabled in bitcoin. Most transactions set this value to the maximum integer value (0xFFFFFFFF) and it is ignored by the bitcoin network. If the transaction has a nonzero locktime, at least one of its inputs must have a sequence number below 0xFFFFFFFF in order to enable locktime.(((range="endofrange", startref="ix_ch05-asciidoc5")))
====
[[tx_fees]]
==== Transaction Fees
Most transactions include transaction fees, which compensate the bitcoin miners for securing the network. Mining and the fees and rewards collected by miners are discussed in more detail in <<mining>>. This section examines how transaction fees are included in a typical transaction. Most wallets calculate and include transaction fees automatically. However, if you are constructing transactions programmatically, or using a command line interface, you must manually account for and include these fees.
((("fees, transaction", id="ix_ch05-asciidoc6", range="startofrange")))Most transactions include transaction fees, which compensate the bitcoin miners for securing the network. Mining and the fees and rewards collected by miners are discussed in more detail in <<ch8>>. This section examines how transaction fees are included in a typical transaction. Most wallets calculate and include transaction fees automatically. However, if you are constructing transactions programmatically, or using a command-line interface, you must manually account for and include these fees.
Transaction fees serve as an incentive to include (mine) a transaction into the next block and also as a disincentive against "spam" transactions or any kind of abuse of the system, by imposing a small cost on every transaction. Transaction fees are collected by the miner who mines the block that records the transaction on the blockchain.
Transaction fees are calculated based on the size of the transaction in kilobytes, not the value of the transaction in bitcoin. Overall, transaction fees are set based on market forces within the bitcoin network. Miners prioritize transactions based on many different criteria, including fees and may even process transactions for free under certain circumstances. Transaction fees affect the processing priority, meaning that a transaction with sufficient fees is likely to be included in the next-most mined block, while a transaction with insufficient or no fees may be delayed, on a best-effort basis and processed after a few blocks or not at all. Transaction fees are not mandatory and transactions without fees may be processed eventually; however, including transaction fees encourages priority processing.
((("fees, transaction","calculating")))Transaction fees are calculated based on the size of the transaction in kilobytes, not the value of the transaction in bitcoin. Overall, transaction fees are set based on market forces within the bitcoin network. Miners prioritize transactions based on many different criteria, including fees, and might even process transactions for free under certain circumstances. Transaction fees affect the processing priority, meaning that a transaction with sufficient fees is likely to be included in the next-most–mined block, whereas a transaction with insufficient or no fees might be delayed, processed on a best-effort basis after a few blocks, or not processed at all. Transaction fees are not mandatory, and transactions without fees might be processed eventually; however, including transaction fees encourages priority processing.
Over time, the way transaction fees are calculated and the effect they have on transaction prioritization has been evolving. At first, transaction fees were fixed and constant across the network. Gradually, the fee structure has been relaxed so that it may be influenced by market forces, based on network capacity and transaction volume. The current minimum transaction fee is fixed at 0.0001 bitcoin or a tenth of a milli-bitcoin per kilobyte, recently decreased from one milli-bitcoin. Most transactions are less than one kilobyte; however, those with multiple inputs or outputs can be larger. In future revisions of the bitcoin protocol it is expected that wallet applications will use statistical analysis to calculate the most appropriate fee to attach to a transaction based on the average fees of recent transactions.
Over time, the way transaction fees are calculated and the effect they have on transaction prioritization has been evolving. At first, transaction fees were fixed and constant across the network. Gradually, the fee structure has been relaxed so that it may be influenced by market forces, based on network capacity and transaction volume. The current minimum transaction fee is fixed at 0.0001 bitcoin or a tenth of a milli-bitcoin per kilobyte, recently decreased from one milli-bitcoin. Most transactions are less than one kilobyte; however, those with multiple inputs or outputs can be larger. In future revisions of the bitcoin protocol, it is expected that wallet applications will use statistical analysis to calculate the most appropriate fee to attach to a transaction based on the average fees of recent transactions.
The current algorithm used by miners to prioritize transactions for inclusion in a block based on their fees will be examined in detail in <<mining>>.
The current algorithm used by miners to prioritize transactions for inclusion in a block based on their fees is examined in detail in <<ch8>>.(((range="endofrange", startref="ix_ch05-asciidoc6")))
==== Adding Fees to Transactions
The data structure of transactions does not have a field for fees. Instead, fees are implied as the difference between the sum of inputs and the sum of outputs. Any excess amount that remains after all outputs have been deducted from all inputs is the fee that is collected by the miners.
((("fees, transaction","adding", id="ix_ch05-asciidoc7", range="startofrange")))((("transactions","fees", id="ix_ch05-asciidoc8", range="startofrange")))The data structure of transactions does not have a field for fees. Instead, fees are implied as the difference between the sum of inputs and the sum of outputs. Any excess amount that remains after all outputs have been deducted from all inputs is the fee that is collected by the miners.
[[tx_fee_equation]]
.Transaction fees are implied, as the excess of inputs minus outputs
.Transaction fees are implied, as the excess of inputs minus outputs:
----
Fees = Sum(Inputs) - Sum(Outputs)
Fees = Sum(Inputs) – Sum(Outputs)
----
This is a somewhat confusing element of transactions and an important point to understand, because if you are constructing your own transactions you must ensure you do not inadvertently include a very large fee by underspending the inputs. That means that you must account for all inputs, if necessary by creating change, or you will end up giving the miners a very big tip!
For example, if you consume a 20 bitcoin UTXO to make a 1 bitcoin payment, you must include a 19 bitcoin change output back to your wallet. Otherwise, the 19 bitcoin "leftover" will be counted as a transaction fee and will be collected by the miner who mines your transaction in a block. While you will receive priority processing and make a miner very happy, this is probably not what you intended.
For example, if you consume a 20-bitcoin UTXO to make a 1-bitcoin payment, you must include a 19-bitcoin change output back to your wallet. Otherwise, the 19-bitcoin "leftover" will be counted as a transaction fee and will be collected by the miner who mines your transaction in a block. Although you will receive priority processing and make a miner very happy, this is probably not what you intended.
[WARNING]
====
If you forget to add a change output in a manually constructed transaction you will be paying the change as a transaction fee. "Keep the change!" may not be what you intended.
If you forget to add a change output in a manually constructed transaction, you will be paying the change as a transaction fee. "Keep the change!" might not be what you intended.
====
Let's see how this works in practice, by looking at Alice's coffee purchase again. Alice wants to spend 0.015 bitcoin to pay for coffee. To ensure this transaction is processed promptly, she will want to include a transaction fee, say 0.001. That will mean that the total cost of the transaction will be 0.016. Her wallet must therefore source a set of UTXO that adds up to 0.016 bitcoin or more and if necessary create change. Let's say her wallet has a 0.2 bitcoin UTXO available. It will therefore need to consume this UTXO, create one output to Bob's Cafe for 0.015, and a second output with 0.184 bitcoin in change back to her own wallet, leaving 0.001 bitcoin unallocated, as an implicit fee for the transaction.
Let's see how this works in practice, by looking at Alice's coffee purchase again. Alice wants to spend 0.015 bitcoin to pay for coffee. To ensure this transaction is processed promptly, she will want to include a transaction fee, say 0.001. That will mean that the total cost of the transaction will be 0.016. Her wallet must therefore source a set of UTXO that adds up to 0.016 bitcoin or more and, if necessary, create change. Let's say her wallet has a 0.2-bitcoin UTXO available. It will therefore need to consume this UTXO, create one output to Bob's Cafe for 0.015, and a second output with 0.184 bitcoin in change back to her own wallet, leaving 0.001 bitcoin unallocated, as an implicit fee for the transaction.
Now let's look at a different scenario. Eugenia, our children's charity director in the Philippines has completed a fundraiser to purchase school books for the children. She received several thousand small donations from people all around the world, totaling 50 bitcoin. Now she wants to purchase hundreds of school books from a local publisher, paying in bitcoin. The charity received thousands of small donations from all around the world, so her wallet is full of very small payments (UTXO).
Now let's look at a different scenario. Eugenia, our children's charity director in the Philippines, has completed a fundraiser to purchase school books for the children. She received several thousand small donations from people all around the world, totaling 50 bitcoin, so her wallet is full of very small payments (UTXO). Now she wants to purchase hundreds of school books from a local publisher, paying in bitcoin.
As Eugenia's wallet application tries to construct a single larger payment transaction, it must source from the available UTXO set which is composed of many smaller amounts. That means that the resulting transaction will source from more than a hundred small-value UTXO as inputs and only one output, paying the book publisher. A transaction with that many inputs will be larger than one kilobyte, perhaps 2-3 kilobytes in size. As a result, it will require a higher fee than the minimal network fee of 0.0001 bitcoin.
As Eugenia's wallet application tries to construct a single larger payment transaction, it must source from the available UTXO set, which is composed of many smaller amounts. That means that the resulting transaction will source from more than a hundred small-value UTXO as inputs and only one output, paying the book publisher. A transaction with that many inputs will be larger than one kilobyte, perhaps 2 to 3 kilobytes in size. As a result, it will require a higher fee than the minimal network fee of 0.0001 bitcoin.
Eugenia's wallet application will calculate the appropriate fee by measuring the size of the transaction and multiplying that by the per-kilobyte fee. Many wallets will overpay fees for larger transactions to ensure the transaction is processed promptly. The higher fee is not because Eugenia is spending more money, but because her transaction is more complex and larger in size - the fee is independent of the transaction's bitcoin value.
Eugenia's wallet application will calculate the appropriate fee by measuring the size of the transaction and multiplying that by the per-kilobyte fee. Many wallets will overpay fees for larger transactions to ensure the transaction is processed promptly. The higher fee is not because Eugenia is spending more money, but because her transaction is more complex and larger in size—the fee is independent of the transaction's bitcoin value.(((range="endofrange", startref="ix_ch05-asciidoc8")))(((range="endofrange", startref="ix_ch05-asciidoc7")))
[[tx_chains]]
=== Transaction Chaining and Orphan Transactions
As we have seen above, transactions form a chain, whereby one transaction spends the outputs of the previous transaction (known as the parent) and creates outputs for a subsequent transaction (known as the child). Sometimes an entire chain of transactions depending on each other, say a parent, child and grandchild transaction are created at the same time, to fulfill a complex transactional workflow that requires valid children be signed before the parent is signed. For example, this is a technique used in a CoinJoin transactions where multiple parties join transactions together to protect their privacy.
((("chaining transactions")))((("orphan transactions")))((("transactions","chaining")))((("transactions","orphan")))As we have seen, transactions form a chain, whereby one transaction spends the outputs of the previous transaction (known as the parent) and creates outputs for a subsequent transaction (known as the child). Sometimes an entire chain of transactions depending on each other—say a parent, child, and grandchild transaction—are created at the same time, to fulfill a complex transactional workflow that requires valid children to be signed before the parent is signed. For example, this is a technique used in((("CoinJoin"))) CoinJoin transactions where multiple parties join transactions together to protect their privacy.
When a chain of transactions is transmitted across the network, they don't always arrive in the same order. Sometimes, the child might arrive before the parent. In that case, the nodes which see a child first can see that it references a parent transaction that is not yet known. Rather than reject the child, they put it in a temporary pool to await the arrival of its parent and propagate it to every other node. The pool of transactions without parents is known as the orphan transaction pool. Once the parent arrives, any orphans that reference the UTXO created by the parent are released from the pool, revalidated recursively and then the entire chain of transactions can be included in the transaction pool, ready to be mined in block. Transaction chains can be arbitrarily long, with any number of generations transmitted simultaneously. The mechanism of holding orphans in the orphan pool ensures that otherwise valid transactions will not be rejected just because their parent has been delayed and that eventually the chain they belong to is reconstructed in the correct order, regardless of the order of arrival.
When a chain of transactions is transmitted across the network, they don't always arrive in the same order. Sometimes, the child might arrive before the parent. In that case, the nodes that see a child first can see that it references a parent transaction that is not yet known. Rather than reject the child, they put it in a temporary pool to await the arrival of its parent and propagate it to every other node. The pool of transactions without parents is known as the((("orphan transaction pool"))) _orphan transaction pool_. Once the parent arrives, any orphans that reference the UTXO created by the parent are released from the pool, revalidated recursively, and then the entire chain of transactions can be included in the transaction pool, ready to be mined in a block. Transaction chains can be arbitrarily long, with any number of generations transmitted simultaneously. The mechanism of holding orphans in the orphan pool ensures that otherwise valid transactions will not be rejected just because their parent has been delayed and that eventually the chain they belong to is reconstructed in the correct order, regardless of the order of arrival.
There is a limit to the number of orphan transactions stored in memory, to prevent a Denial-of-Service attack against bitcoin nodes. The limit is defined as MAX_ORPHAN_TRANSACTIONS in the source code of the bitcoin reference client. If the number of orphan transactions in the pool exceeds MAX_ORPHAN_TRANSACTIONS, one or more randomly selected orphan transactions are evicted from the pool, until the pool size is back within limits.
There is a limit to the number of orphan transactions stored in memory, to prevent a denial-of-service attack against bitcoin nodes. The limit is defined as((("MAX_ORPHAN_TRANSACTIONS constant"))) +MAX_ORPHAN_TRANSACTIONS+ in the source code of the bitcoin reference client. If the number of orphan transactions in the pool exceeds +MAX_ORPHAN_TRANSACTIONS+, one or more randomly selected orphan transactions are evicted from the pool, until the pool size is back within limits.
[[tx_script]]
=== Transaction Scripts and Script Language
Bitcoin clients validate transactions by executing a script, written in a Forth-like scripting language. Both the locking script (encumbrance) placed on a UTXO and the unlocking script that usually contains a signature are written in this scripting language. When a transaction is validated, the unlocking script in each input is executed alongside the corresponding locking script to see if it satisfies the spending condition.
((("scripts", id="ix_ch05-asciidoc9", range="startofrange")))((("transactions","script language for", id="ix_ch05-asciidoc10", range="startofrange")))((("transactions","validation", id="ix_ch05-asciidoc11", range="startofrange")))((("validation (transaction)", id="ix_ch05-asciidoc12", range="startofrange")))Bitcoin clients validate transactions by executing a script, written in a Forth-like scripting language. Both the locking script (encumbrance) placed on a UTXO and the unlocking script that usually contains a signature are written in this scripting language. When a transaction is validated, the unlocking script in each input is executed alongside the corresponding locking script to see if it satisfies the spending condition.
Today most transactions processed through the bitcoin network have the form "Alice pays Bob" and are based on the same script called a Pay-to-Public-Key-Hash script. However, the use of scripts to lock outputs and unlock inputs means that through use of the programming language, transactions can contain an infinite number of conditions. Bitcoin transactions are not limited to the "Alice pays Bob" form and pattern.
Today, most transactions processed through the bitcoin network have the form "Alice pays Bob" and are based on the same script called a Pay-to-Public-Key-Hash script. However, the use of scripts to lock outputs and unlock inputs means that through use of the programming language, transactions can contain an infinite number of conditions. Bitcoin transactions are not limited to the "Alice pays Bob" form and pattern.
This is only the tip of the iceberg of possibilities that can be expressed with this scripting language. In this section we will demonstrate the components of bitcoins transaction scripting language and show how it can be used to express complex conditions for spending and how those conditions can be satisfied by unlocking scripts.
This is only the tip of the iceberg of possibilities that can be expressed with this scripting language. In this section, we will demonstrate the components of the bitcoin transaction scripting language and show how it can be used to express complex conditions for spending and how those conditions can be satisfied by unlocking scripts.
[TIP]
====
Bitcoin transaction validation is not based on a static pattern, but instead is achieved through the execution of a scripting language. This language allows for a nearly infinite variety of conditions to be expressed. This is how bitcoin gets the power of "programmable money".
Bitcoin transaction validation is not based on a static pattern, but instead is achieved through the execution of a scripting language. This language allows for a nearly infinite variety of conditions to be expressed. This is how bitcoin gets the power of "programmable money."
====
==== Script Construction (Lock + Unlock)
Bitcoin's transaction validation engine relies on two types of scripts to validate transactions -- a locking script and an unlocking script.
((("scripts","construction of")))((("validation (transaction)","script construction for")))Bitcoin's transaction validation engine relies on two types of scripts to validate transactions: a locking script and an unlocking script.
A locking script is an encumbrance placed on an output, and it specifies the conditions that must be met to spend the output in the future. Historically, the locking script was called a _scriptPubKey_, because it usually contained a public key or bitcoin address. In this book we refer to it as a "locking script" to acknowledge the much broader range of possibilities of this scripting technology. In most bitcoin applications, what we refer to as a locking script will appear in the source code as "scriptPubKey".
((("locking scripts","transaction validation and")))((("validation (transaction)","locking scripts")))A locking script is an encumbrance placed on an output, and it specifies the conditions that must be met to spend the output in the future. Historically, the locking script was called a _scriptPubKey_, because it usually contained a public key or bitcoin address. In this book we refer to it as a "locking script" to acknowledge the much broader range of possibilities of this scripting technology. In most bitcoin applications, what we refer to as a locking script will appear in the source code as +scriptPubKey+.
An unlocking script is a script that "solves", or satisfies, the conditions placed on an output by a locking script and allows the output to be spent. Unlocking scripts are part of every transaction input and most of the time they contain a digital signature produced by the user's wallet from their private key. Historically, the unlocking script is called _scriptSig_, because it usually contained a digital signature. In this book we refer to it as an "unlocking script" to acknowledge the much broader range of locking script requirements, as not all unlocking scripts must contain signatures. As mentioned above, in most bitcoin applications the source code will refer to the unlocking script as "scriptSig".
((("unlocking scripts","transaction validation and")))An unlocking script is a script that "solves," or satisfies, the conditions placed on an output by a locking script and allows the output to be spent. Unlocking scripts are part of every transaction input, and most of the time they contain a digital signature produced by the user's wallet from his or her private key. Historically, the unlocking script is called _scriptSig_, because it usually contained a digital signature. In most bitcoin applications, the source code refers to the unlocking script as +scriptSig+. In this book, we refer to it as an "unlocking script" to acknowledge the much broader range of locking script requirements, because not all unlocking scripts must contain signatures.
Every bitcoin client will validate transactions by executing the locking and unlocking scripts together. For each input in the transaction, the validation software will first retrieve the UTXO referenced by the input. That UTXO contains a locking script defining the conditions required to spend it. The validation software will then take the unlocking script contained in the input that is attempting to spend this UTXO and execute the two scripts.
In the original bitcoin client, the unlocking and locking scripts were concatenated and executed in sequence. For security reasons, this was changed in 2010, because of a vulnerability that allowed a malformed unlocking script to push data onto the stack and corrupt the locking script. In the current implementation the scripts are executed separately with the stack transferred between the two executions, as described below.
In the original bitcoin client, the unlocking and locking scripts were concatenated and executed in sequence. For security reasons, this was changed in 2010, because of a vulnerability that allowed a malformed unlocking script to push data onto the stack and corrupt the locking script. In the current implementation, the scripts are executed separately with the stack transferred between the two executions, as described next.
First, the unlocking script is executed, using the stack execution engine. If the unlocking script executed without errors (e.g it has no "dangling" operators leftover), the main stack (not the alternate stack) is copied and the locking script is executed. If the result of executing the locking script with the stack data copied from the unlocking script is "TRUE", the unlocking script has succeeded in resolving the conditions imposed by the locking script and therefore the input is a valid authorization to spend the UTXO. If any result other than "TRUE" remains after execution of the combined script, the input is invalid as it has failed to satisfy the spending conditions placed on the UTXO. Note that the UTXO is permanently recorded in the blockchain, and therefore is invariable and is unaffected by failed attempts to spend it by reference in a new transaction. Only a valid transaction that correctly satisfies the conditions of the UTXO results in the UTXO being marked as "spent" and removed from the set of available (unspent) UTXO.
First, the unlocking script is executed, using the stack execution engine. If the unlocking script executed without errors (e.g., it has no "dangling" operators left over), the main stack (not the alternate stack) is copied and the locking script is executed. If the result of executing the locking script with the stack data copied from the unlocking script is "TRUE," the unlocking script has succeeded in resolving the conditions imposed by the locking script and, therefore, the input is a valid authorization to spend the UTXO. If any result other than "TRUE" remains after execution of the combined script, the input is invalid because it has failed to satisfy the spending conditions placed on the UTXO. Note that the UTXO is permanently recorded in the blockchain, and therefore is invariable and is unaffected by failed attempts to spend it by reference in a new transaction. Only a valid transaction that correctly satisfies the conditions of the UTXO results in the UTXO being marked as "spent" and removed from the set of available (unspent) UTXO.
Below is an example of the unlocking and locking scripts for the most common type of bitcoin transaction (a payment to a public key hash), showing the combined script resulting from the concatenation of the unlocking and locking scripts prior to script validation:
<<scriptSig_and_scriptPubKey>> is an example of the unlocking and locking scripts for the most common type of bitcoin transaction (a payment to a public key hash), showing the combined script resulting from the concatenation of the unlocking and locking scripts prior to script validation.
[[scriptSig and scriptPubKey]]
[[scriptSig_and_scriptPubKey]]
.Combining scriptSig and scriptPubKey to evaluate a transaction script
image::images/scriptSig_and_scriptPubKey.png["scriptSig_and_scriptPubKey"]
image::images/msbt_0501.png["scriptSig_and_scriptPubKey"]
[[tx_script_language]]
==== Scripting Language
The bitcoin transaction script language, also named confusingly _Script_, is a Forth-like reverse-polish notation stack-based execution language. If that sounds like gibberish, you probably haven't studied 1960's programming languages. Script is a very simple, lightweight language that was designed to be limited in scope and executable on a range of hardware, perhaps as simple as an embedded device, like a handheld calculator. It requires minimal processing and cannot do many of the fancy things modern programming languages can do. In the case of programmable money, that is a deliberate security feature.
((("Script language", id="ix_ch05-asciidoc13", range="startofrange")))((("scripts","language for", id="ix_ch05-asciidoc14", range="startofrange")))The bitcoin transaction script language, called _Script_, is a Forth-like reverse-polish notation stack-based execution language. If that sounds like gibberish, you probably haven't studied 1960's programming languages. Script is a very simple language that was designed to be limited in scope and executable on a range of hardware, perhaps as simple as an embedded device, such as a handheld calculator. It requires minimal processing and cannot do many of the fancy things modern programming languages can do. In the case of programmable money, that is a deliberate security feature.
Bitcoin's scripting language is called a stack-based language because it uses a data structure called a _stack_. A stack is a very simple data structure, which can be visualized as a stack of cards. A stack allows two operations: push and pop. Push adds an item on top of the stack. Pop removes the top item from the stack.
Bitcoin's scripting language is called a stack-based language because it uses a data structure called a((("stack, defined"))) _stack_. A stack is a very simple data structure, which can be visualized as a stack of cards. A stack allows two operations: push and pop. Push adds an item on top of the stack. Pop removes the top item from the stack.
The scripting language executes the script by processing each item from left to right. Numbers (data constants) are pushed onto the stack. Operators push or pop one or more parameters from the stack, act on them, and may push a result onto the stack. For example, OP_ADD will pop two items from the stack, add them and push the resulting sum onto the stack.
The scripting language executes the script by processing each item from left to right. Numbers (data constants) are pushed onto the stack. Operators push or pop one or more parameters from the stack, act on them, and might push a result onto the stack. For example, +OP_ADD+ will pop two items from the stack, add them, and push the resulting sum onto the stack.
Conditional operators evaluate a condition producing a boolean result of TRUE or FALSE. For example, OP_EQUAL pops two items from the stack and pushes TRUE (TRUE is represented by the number 1) if they are equal or FALSE (represented by zero) if they are not equal. Bitcoin transaction scripts usually contain a conditional operator, so that they can produce the result TRUE that signifies a valid transaction.
Conditional operators evaluate a condition, producing a boolean result of TRUE or FALSE. For example, +OP_EQUAL+ pops two items from the stack and pushes TRUE (TRUE is represented by the number 1) if they are equal or FALSE (represented by zero) if they are not equal. Bitcoin transaction scripts usually contain a conditional operator, so that they can produce the TRUE result that signifies a valid transaction.
In the following example, the script +2 3 OP_ADD 5 OP_EQUAL+ demonstrates the arithmetic addition operator _OP_ADD_, adding two numbers and putting the result on the stack, followed by the conditional operator OP_EQUAL which checks the resulting sum is equal to +5+. For brevity, the OP_ prefix is omitted in the step-by-step example.
In <<simplemath_script>>, the script +2 3 OP_ADD 5 OP_EQUAL+ demonstrates the arithmetic addition operator +OP_ADD+, adding two numbers and putting the result on the stack, followed by the conditional operator +OP_EQUAL+, which checks that the resulting sum is equal to +5+. For brevity, the +OP_+ prefix is omitted in the step-by-step example.
[[simplemath_script]]
.Bitcoin's script validation doing simple math
image::images/TxScriptSimpleMathExample.png["TxScriptSimpleMathExample"]
Below is a slightly more complex script, which calculates +((2 + 3) * 2) + 1+. Notice that when the script contains several operators in a row, the stack allows the results of one operator to be acted upon by the next operator:
The following is a slightly more complex script, which calculates ++2 + 7 – 3 + 1++. Notice that when the script contains several operators in a row, the stack allows the results of one operator to be acted upon by the next operator:
----
2 3 OP_ADD 2 OP_MUL 1 OP_ADD 11 OP_EQUAL
2 7 OP_ADD 3 OP_SUB 1 OP_ADD 7 OP_EQUAL
----
Try validating the script above yourself, using pencil and paper. When the script execution ends, you should be left with the value TRUE on the stack.
Try validating the preceding script yourself using pencil and paper. When the script execution ends, you should be left with the value TRUE on the stack.
While most locking scripts refer to a bitcoin address or public key, thereby requiring proof of ownership to spend the funds, the script does not have to be that complex. Any combination of locking and unlocking scripts that results in a TRUE value is valid. The simple arithmetic we used as an example of the scripting language above is also a valid locking script that can be used to lock a transaction output.
Although most locking scripts refer to a bitcoin address or public key, thereby requiring proof of ownership to spend the funds, the script does not have to be that complex. Any combination of locking and unlocking scripts that results in a TRUE value is valid. The simple arithmetic we used as an example of the scripting language is also a valid locking script that can be used to lock a transaction output.
Use part of the arithmetic example script as the locking script:
----
3 OP_ADD 5 OP_EQUAL
----
which can be satisfied by transaction containing an input with the unlocking script:
which can be satisfied by a transaction containing an input with the unlocking script:
----
2
----
@ -323,119 +319,135 @@ The validation software combines the locking and unlocking scripts and the resul
2 3 OP_ADD 5 OP_EQUAL
----
As we saw in the step-by-step example above, when this script is executed the result is OP_TRUE, making the transaction valid. Not only is this a valid transaction output locking script, but the resulting UTXO could be spent by anyone with the arithmetic skills to know that the number 2 satisfies the script.
As we saw in the step-by-step example in <<simplemath_script>>, when this script is executed, the result is +OP_TRUE+, making the transaction valid. Not only is this a valid transaction output locking script, but the resulting UTXO could be spent by anyone with the arithmetic skills to know that the number 2 satisfies the script. (((range="endofrange", startref="ix_ch05-asciidoc14")))(((range="endofrange", startref="ix_ch05-asciidoc13")))
[[simplemath_script]]
.Bitcoin's script validation doing simple math
image::images/msbt_0502.png["TxScriptSimpleMathExample"]
[TIP]
====
Transactions are valid if the top result on the stack is TRUE (noted as +{0x01}+), any other non-zero value or if the stack is empty after script execution. Transactions are invalid if the top value on the stack is FALSE (a zero-length empty value, noted as +{}+) or if script execution is halted explicitly by an operator, such as OP_VERIFY, OP_RETURN or a conditional terminator such as OP_ENDIF. See <<tx_script_ops>> for details.
Transactions are valid if the top result on the stack is TRUE (noted as ++&#x7b;0x01&#x7d;++), any other non-zero value or if the stack is empty after script execution. Transactions are invalid if the top value on the stack is FALSE (a zero-length empty value, noted as ++&#x7b;&#x7d;++) or if script execution is halted explicitly by an operator, such as OP_VERIFY, OP_RETURN, or a conditional terminator such as OP_ENDIF. See <<tx_script_ops>> for details.
====
==== Turing Incompleteness
The bitcoin transaction script language contains many operators but is deliberately limited in one important way -- there are no loops or complex flow control capabilities other than conditional flow control. This ensures that the language is not Turing Complete, meaning that scripts have limited complexity and predictable execution times. Script is not a general-purpose language. These limitations ensure that the language cannot be used to create an infinite loop or other form of "logic bomb" that could be embedded in a transaction in a way that causes a Denial-of-Service attack against the bitcoin network. Remember, every transaction is validated by every full node on the bitcoin network. A limited language prevents the transaction validation mechanism from being used as a vulnerability.
((("Script language","flow-control/loops in")))((("Script language","statelessness of")))((("Turing Complete")))The bitcoin transaction script language contains many operators, but is deliberately limited in one important way—there are no loops or complex flow control capabilities other than conditional flow control. This ensures that the language is not _Turing Complete_, meaning that scripts have limited complexity and predictable execution times. Script is not a general-purpose language. These limitations ensure that the language cannot be used to create an infinite loop or other form of "logic bomb" that could be embedded in a transaction in a way that causes a((("denial-of-service attack","Script language and"))) denial-of-service attack against the bitcoin network. Remember, every transaction is validated by every full node on the bitcoin network. A limited language prevents the transaction validation mechanism from being used as a vulnerability.
==== Stateless Verification
The bitcoin transaction script language is stateless, in that there is no state prior to execution of the script, or state saved after execution of the script. Therefore, all the information needed to execute a script is contained within the script. A script will predictably execute the same way on any system. If your system verifies a script, you can be sure that every other system in the bitcoin network will also verify the script, meaning that a valid transaction is valid for everyone and everyone knows this. This predictability of outcomes is an essential benefit of the bitcoin system.
((("stateless verification of transactions")))((("transactions","statelessness of")))The bitcoin transaction script language is stateless, in that there is no state prior to execution of the script, or state saved after execution of the script. Therefore, all the information needed to execute a script is contained within the script. A script will predictably execute the same way on any system. If your system verifies a script, you can be sure that every other system in the bitcoin network will also verify the script, meaning that a valid transaction is valid for everyone and everyone knows this. This predictability of outcomes is an essential benefit of the bitcoin system.(((range="endofrange", startref="ix_ch05-asciidoc12")))(((range="endofrange", startref="ix_ch05-asciidoc11")))(((range="endofrange", startref="ix_ch05-asciidoc10")))(((range="endofrange", startref="ix_ch05-asciidoc9")))
[[std_tx]]
=== Standard Transactions
In the first few years of bitcoin's development, the developers introduced some limitations in the types of scripts that could be processed by the reference client. These limitations are encoded in a function called +isStandard()+ which defines five types of "standard" transactions. These limitations are temporary and may be lifted by the time you read this. Until then, the five standard types of transaction scripts are the only ones that will be accepted by the reference client and most miners who run the reference client. While it is possible to create a non-standard transaction containing a script that is not one of the standard types, you must find a miner who does not follow these limitations, to mine that transaction into a block.
In the first few years of bitcoin's development, the developers introduced some limitations in the types of scripts that could be processed by the reference client. These limitations are encoded in a function called +isStandard()+, which defines five types of "standard" transactions. These limitations are temporary and might be lifted by the time you read this. Until then, the five standard types of transaction scripts are the only ones that will be accepted by the reference client and most miners who run the reference client. Although it is possible to create a nonstandard transaction containing a script that is not one of the standard types, you must find a miner who does not follow these limitations to mine that transaction into a block.
Check the source code of the bitcoin core client (the reference implementation) to see what is currently allowed as a valid transaction script.
Check the source code of the Bitcoin Core client (the reference implementation) to see what is currently allowed as a valid transaction script.
The five standard types of transaction scripts are Pay-to-Public-Key-Hash (P2PKH), Public-Key, Multi-Signature (limited to 15 keys), Pay-to-Script-Hash (P2SH) and Data Output (OP_RETURN), which are described in more detail below.
The five standard types of transaction scripts are pay-to-public-key-hash (P2PKH), public-key, multi-signature (limited to 15 keys), pay-to-script-hash (P2SH), and data output (OP_RETURN), which are described in more detail in the following sections.
[[p2pkh]]
==== Pay to Public Key Hash (P2PKH)
==== Pay-to-Public-Key-Hash (P2PKH)
The vast majority of transactions processed on the bitcoin network are Pay-to-Public-Key-Hash, also known as P2PKH transactions. These contain a locking script that encumbers the output with a public key hash, more commonly known as a bitcoin address. Transactions that pay a bitcoin address contain P2PKH scripts. An output locked by a P2PKH script can be unlocked (spent) by presenting a public key and a digital signature created by the corresponding private key.
((("pay-to-public-key-hash (P2PKH)", id="ix_ch05-asciidoc15", range="startofrange")))((("transactions","pay-to-public-key-hash", id="ix_ch05-asciidoc16", range="startofrange")))The vast majority of transactions processed on the bitcoin network are P2PKH transactions. These contain a locking script that encumbers the output with a public key hash, more commonly known as a bitcoin address. Transactions that pay a bitcoin address contain P2PKH scripts. An output locked by a P2PKH script can be unlocked (spent) by presenting a public key and a digital signature created by the corresponding private key.
For example, let's look at Alice's payment to Bob's Cafe again. Alice made a payment of 0.015 bitcoin to the Cafe's bitcoin address. That transaction output would have a locking script of the form:
For example, let's look at Alice's payment to Bob's Cafe again. Alice made a payment of 0.015 bitcoin to the cafe's bitcoin address. That transaction output would have a locking script of the form:
----
OP_DUP OP_HASH160 <Cafe Public Key Hash> OP_EQUAL OP_CHECKSIG
----
The +Cafe Public Key Hash+ is equivalent to the bitcoin address of the Cafe, without the Base58Check encoding. Most applications would show the Public Key Hash in hexadecimal encoding and not the familiar bitcoin address Base58Check format that begins with a "1".
The +Cafe Public Key Hash+ is equivalent to the bitcoin address of the cafe, without the Base58Check encoding. Most applications would show the _public key hash_ in hexadecimal encoding and not the familiar bitcoin address Base58Check format that begins with a "1".
The locking script above can be satisfied with an unlocking script of the form:
The preceding locking script can be satisfied with an unlocking script of the form:
----
<Cafe Signature> <Cafe Public Key>
----
The two scripts together would form the combined validation script below:
The two scripts together would form the following combined validation script:
----
<Cafe Signature> <Cafe Public Key> OP_DUP OP_HASH160 \
<Cafe Signature> <Cafe Public Key> OP_DUP OP_HASH160
<Cafe Public Key Hash> OP_EQUAL OP_CHECKSIG
----
When executed, this combined script will evaluate to TRUE if, and only if, the unlocking script matches the conditions set by the locking script. In other words, the result will be TRUE if the unlocking script has a valid signature from the Cafe's private key which corresponds to the public key hash set as an encumbrance.
When executed, this combined script will evaluate to TRUE if, and only if, the unlocking script matches the conditions set by the locking script. In other words, the result will be TRUE if the unlocking script has a valid signature from the cafe's private key that corresponds to the public key hash set as an encumbrance.
Here's a step-by-step execution of the combined script, which will prove this is a valid transaction:
Figures pass:[<xref linkend="P2PubKHash1" xrefstyle="select: labelnumber"/>] and pass:[<xref linkend="P2PubKHash2" xrefstyle="select: labelnumber"/>] show (in two parts) a step-by-step execution of the combined script, which will prove this is a valid transaction.(((range="endofrange", startref="ix_ch05-asciidoc16")))(((range="endofrange", startref="ix_ch05-asciidoc15")))
[[P2PubKHash1]]
.Evaluating a script for a Pay-to-Public-Key-Hash transaction (Part 1 of 2)
image::images/Tx_Script_P2PubKeyHash_1.png["Tx_Script_P2PubKeyHash_1"]
.Evaluating a script for a P2PKH transaction (Part 1 of 2)
image::images/msbt_0503.png["Tx_Script_P2PubKeyHash_1"]
[[P2PubKHash2]]
.Evaluating a script for a Pay-to-Public-Key-Hash transaction (Part 2 of 2)
image::images/Tx_Script_P2PubKeyHash_2.png["Tx_Script_P2PubKeyHash_2"]
[[p2pk]]
==== Pay-to-Public-Key
Pay-to-Public-Key is a simpler form of a bitcoin payment than Pay-to-Public-Key-Hash. With this script form, the public key itself is stored in the locking script, rather than a public-key-hash as with P2PKH above, which is much shorter. Pay-to-Public-Key-Hash was invented by Satoshi to make bitcoin addresses shorter, for ease of use. Pay-to-Public-Key is now most often seen in coinbase transactions, generated by older mining software that has not been updated to use P2PKH.
((("pay-to-public-key")))Pay-to-public-key is a simpler form of a bitcoin payment than pay-to-public-key-hash. With this script form, the public key itself is stored in the locking script, rather than a public-key-hash as with P2PKH earlier, which is much shorter. Pay-to-public-key-hash was invented by Satoshi to make bitcoin addresses shorter, for ease of use. Pay-to-public-key is now most often seen in coinbase transactions, generated by older mining software that has not been updated to use P2PKH.
A pay-to-public-key locking script looks like this:
A Pay-to-Public-Key locking script looks like this:
----
<Public Key A> OP_CHECKSIG
----
The corresponding unlocking script that must be presented to unlock this type of output is a simple signature, like this:
----
<Signature from Private Key A>
----
The combined script, which is validated by the transaction validation software is:
The combined script, which is validated by the transaction validation software, is:
----
<Signature from Private Key A> <Public Key A> OP_CHECKSIG
----
The script above is a simple invocation of the CHECKSIG operator which validates the signature as belonging to the correct key and returns TRUE on the stack.
This script is a simple invocation of the +CHECKSIG+ operator, which validates the signature as belonging to the correct key and returns TRUE on the stack.
[[P2PubKHash2]]
.Evaluating a script for a P2PKH transaction (Part 2 of 2)
image::images/msbt_0504.png["Tx_Script_P2PubKeyHash_2"]
[[multisig]]
==== Multi-Signature
Multi-signature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to release the encumbrance. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multi-signature is one where 3 public keys are listed as potential signers and at least 2 of those must be used to create signatures for a valid transaction to spend the funds. At this time, standard multi-signature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multi-signature or any combination within that range. The limitation to 15 listed keys may be lifted by the time of publication of this book, so check the +isStandard()+ function to see what is currently accepted by the network.
((("multi-signature scripts")))((("transactions","multi-signature scripts")))Multi-signature scripts set a condition where N public keys are recorded in the script and at least M of those must provide signatures to release the encumbrance. This is also known as an M-of-N scheme, where N is the total number of keys and M is the threshold of signatures required for validation. For example, a 2-of-3 multi-signature is one where three public keys are listed as potential signers and at least two of those must be used to create signatures for a valid transaction to spend the funds. ((("multi-signature scripts","limits on")))At this time, standard multi-signature scripts are limited to at most 15 listed public keys, meaning you can do anything from a 1-of-1 to a 15-of-15 multi-signature or any combination within that range. The limitation to 15 listed keys might be lifted by the time this book is published, so check the((("isStandard() function"))) +isStandard()+ function to see what is currently accepted by the network.
The general form of a locking script setting an M-of-N multi-signature condition is:
----
M <Public Key 1> <Public Key 2> ... <Public Key N> N OP_CHECKMULTISIG
----
where N is the total number of listed public keys and M is the threshold of required signatures to spend the output.
A locking script setting a 2-of-3 multi-signature condition looks like this:
----
2 <Public Key A> <Public Key B> <Public Key C> 3 OP_CHECKMULTISIG
----
The locking script above can be satisfied with an unlocking script containing pairs of signatures and public keys:
The preceding locking script can be satisfied with an unlocking script containing pairs of signatures and public keys:
----
OP_0 <Signature B> <Signature C>
----
or any combination of two signatures from the private keys corresponding to the three listed public keys.
_Note: The prefix OP_0 is required because of a bug in the original implementation of CHECKMULTISIG where one item too many is popped off the stack. It is ignored by CHECKMULTISIG and is simply a placeholder._
[NOTE]
====
((("CHECKMULTISIG implementation")))The prefix +OP_0+ is required because of a bug in the original implementation of +CHECKMULTISIG+ where one item too many is popped off the stack. It is ignored by +CHECKMULTISIG+ and is simply a placeholder.
====
The two scripts together would form the combined validation script:
The two scripts together would form the combined validation script below:
----
OP_0 <Signature B> <Signature C>\
2 <Public Key A> <Public Key B> <Public Key C> 3 OP_CHECKMULTISIG
OP_0 <Signature B> <Signature C> 2 <Public Key A> <Public Key B> <Public Key C> 3 OP_CHECKMULTISIG
----
When executed, this combined script will evaluate to TRUE if, and only if, the unlocking script matches the conditions set by the locking script. In this case, the condition is whether the unlocking script has a valid signature from the two private keys that correspond to two of the three public keys set as an encumbrance.
@ -443,139 +455,133 @@ When executed, this combined script will evaluate to TRUE if, and only if, the u
[[op_return]]
==== Data Output (OP_RETURN)
Bitcoin's distributed and timestamped ledger, the blockchain, has potential uses far beyond payments. Many developers have tried to use the transaction scripting language to take advantage of the security and resilience of the system for applications such as digital notary services, stock certificates, and smart contracts. Early attempts to use bitcoin's script language for these purposes involved creating transaction outputs that recorded data on the blockchain, for example to record a digital fingerprint of a file in such a way that anyone could establish proof-of-existence of that file on a specific date by reference to that transaction.
((("ledger, storing unrelated information in")))((("OP_RETURN operator")))((("transactions","storing unrelated information in")))Bitcoin's distributed and timestamped ledger, the blockchain, has potential uses far beyond payments. Many developers have tried to use the transaction scripting language to take advantage of the security and resilience of the system for applications such as((("digital notary services")))((("smart contracts")))((("stock certificates"))) digital notary services, stock certificates, and smart contracts. Early attempts to use bitcoin's script language for these purposes involved creating transaction outputs that recorded data on the blockchain; for example, to record a digital fingerprint of a file in such a way that anyone could establish proof-of-existence of that file on a specific date by reference to that transaction.
The use of bitcoin's blockchain to store data unrelated to bitcoin payments is a controversial subject. Many developers consider such use abusive and want to discourage it. Others view it as a demonstration of the powerful capabilities of blockchain technology and want to encourage such experimentation. Those who object to the inclusion of non-payment data argue that it causes "blockchain bloat", burdening those running full bitcoin nodes with carrying the cost of disk storage for data that the blockchain was not intended to carry. Moreover, such transactions create UTXO that cannot be spent, using the destination bitcoin address as a free-form 20-byte field. Since the address is used for data, it doesn't correspond to a private key and the resulting UTXO can _never_ be spent, it's a fake payment. This practice causes the size of the in-memory UTXO set to increase and these transactions which can never be spent are therefore never removed, forcing bitcoin nodes to carry these forever in RAM which is far more expensive.
((("blockchains","storing unrelated information in")))The use of bitcoin's blockchain to store data unrelated to bitcoin payments is a controversial subject. Many developers consider such use abusive and want to discourage it. Others view it as a demonstration of the powerful capabilities of blockchain technology and want to encourage such experimentation. Those who object to the inclusion of non-payment data argue that it causes "blockchain bloat," burdening those running full bitcoin nodes with carrying the cost of disk storage for data that the blockchain was not intended to carry. Moreover, such transactions create UTXO that cannot be spent, using the destination bitcoin address as a free-form 20-byte field. Because the address is used for data, it doesn't correspond to a private key and the resulting UTXO can _never_ be spent; it's a fake payment. This practice causes the size of the in-memory UTXO set to increase and these transactions that can never be spent are therefore never removed, forcing bitcoin nodes to carry these forever in RAM, which is far more expensive.
In version 0.9 of the bitcoin core client, a compromise was reached, with the introduction of the OP_RETURN operator. OP_RETURN allows developers to add 40 bytes of non-payment data to a transaction output. However, unlike the use of "fake" UTXO, the OP_RETURN operator creates an explicitly _provably un-spendable_ output, which does not need to be stored in the UTXO set. OP_RETURN outputs are recorded on the blockchain, so they consume disk space and contribute to the increase in the blockchain's size, but they are not stored in the UTXO set and therefore do not bloat the UTXO memory pool and burden full nodes with the cost of more expensive RAM.
In version 0.9 of the Bitcoin Core client, a compromise was reached with the introduction of the +OP_RETURN+ operator. +OP_RETURN+ allows developers to add 40 bytes of nonpayment data to a transaction output. However, unlike the use of "fake" UTXO, the +OP_RETURN+ operator creates an explicitly _provably unspendable_ output, which does not need to be stored in the UTXO set. +OP_RETURN+ outputs are recorded on the blockchain, so they consume disk space and contribute to the increase in the blockchain's size, but they are not stored in the UTXO set and therefore do not bloat the UTXO memory pool and burden full nodes with the cost of more expensive RAM.
OP_RETURN scripts look like this:
+OP_RETURN+ scripts look like this:
----
OP_RETURN <data>
----
where the data portion is limited to 40 bytes and most often represents a hash, such as the output from the SHA256 algorithm (32 bytes). Many applications put a prefix in front of the data to help identify the application. For example, the proofofexistence.com digital notarization service uses the 8-byte prefix "DOCPROOF" which is ASCII encoded as 44f4350524f4f46 in hexadecimal.
The data portion is limited to 40 bytes and most often represents a hash, such as the output from the SHA256 algorithm (32 bytes). Many applications put a prefix in front of the data to help identify the application. For example, the http://proofofexistence.com[Proof of Existence] digital notarization service uses the 8-byte prefix "DOCPROOF," which is ASCII encoded as +44f4350524f4f46+ in hexadecimal.
Keep in mind that there is no "unlocking script" that corresponds to OP_RETURN that could possibly be used to "spend" an OP_RETURN output. The whole point of OP_RETURN is that you can't spend the money locked in that output and therefore it does not need to be held in the UTXO set as potentially spendable - OP_RETURN is _provably un-spendable_. OP_RETURN is usually an output with a zero bitcoin amount, since any bitcoin assigned to such an output is effectively lost forever. If an OP_RETURN is encountered by the script validation software, it results immediately in halting the execution of the validation script and marking the transaction as invalid. Thus, if you accidentally reference an OP_RETURN output as an input in a transaction, that transaction is invalid.
Keep in mind that there is no "unlocking script" that corresponds to +OP_RETURN+ that could possibly be used to "spend" an +OP_RETURN+ output. The whole point of +OP_RETURN+ is that you can't spend the money locked in that output, and therefore it does not need to be held in the UTXO set as potentially spendable—+OP_RETURN+ is _provably un-spendable_. +OP_RETURN+ is usually an output with a zero bitcoin amount, because any bitcoin assigned to such an output is effectively lost forever. If an +OP_RETURN+ is encountered by the script validation software, it results immediately in halting the execution of the validation script and marking the transaction as invalid. Thus, if you accidentally reference an +OP_RETURN+ output as an input in a transaction, that transaction is invalid.
A standard transaction (one that conforms to the +isStandard()+ checks) can have only one OP_RETURN output. However, a single OP_RETURN output can be combined in a transaction with outputs of any other type.
A standard transaction (one that conforms to the +isStandard()+ checks) can have only one +OP_RETURN+ output. However, a single +OP_RETURN+ output can be combined in a transaction with outputs of any other type.
[[p2sh]]
==== Pay to Script Hash (P2SH)
==== Pay-to-Script-Hash (P2SH)
Pay-to-Script-Hash (P2SH) was introduced in the winter of 2012 as a powerful new type of transaction that greatly simplifies the use of complex transaction scripts. To explain the need for P2SH, let's look at a practical example.
((("multi-signature scripts","P2SH and", id="ix_ch05-asciidoc17", range="startofrange")))((("Pay-to-script-hash (P2SH)", id="ix_ch05-asciidoc18", range="startofrange")))((("transactions","Pay-to-script-hash", id="ix_ch05-asciidoc19", range="startofrange")))Pay-to-script-hash (P2SH) was introduced in 2012 as a powerful new type of transaction that greatly simplifies the use of complex transaction scripts. To explain the need for P2SH, let's look at a practical example.
In chapter 1 we introduced Mohammed, an electronics importer based in Dubai. Mohammed's company uses bitcoin's multi-signature feature extensively for its corporate accounts. Multi-signature scripts are one of the most common uses of bitcoin's advanced scripting capabilities and are a very powerful feature. Mohammed's company uses a multi-signature script for all customer payments, known in accounting terms as "accounts receivable" or AR. With the multi-signature scheme, any payments made by customers are locked in such a way that they require at least two signatures to release, from Mohammed and one of his partners or from his attorney who has a backup key. A multi-signature scheme like that offers corporate governance controls and protects against theft, embezzlement or loss.
In <<ch01_intro_what_is_bitcoin>> we introduced Mohammed, an electronics importer based in Dubai. Mohammed's company uses bitcoin's multi-signature feature extensively for its corporate accounts. Multi-signature scripts are one of the most common uses of bitcoin's advanced scripting capabilities and are a very powerful feature. Mohammed's company uses a multi-signature script for all customer payments, known in accounting terms as "accounts receivable," or AR. With the multi-signature scheme, any payments made by customers are locked in such a way that they require at least two signatures to release, from Mohammed and one of his partners or from his attorney who has a backup key. A multi-signature scheme like that offers corporate governance controls and protects against theft, embezzlement, or loss.
The resulting script is quite long and looks like this:
----
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> \
<Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> <Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
----
While multi-signature scripts are a powerful feature, they are cumbersome to use. Given the script above, Mohammed would have to communicate this script to every customer prior to payment. Each customer would have to use special bitcoin wallet software with the ability to create custom transaction scripts and each customer would have to understand how to create a transaction using custom scripts. Furthermore, the resulting transaction would be about five times larger than a simple payment transaction, as this script contains very long public keys. The burden of that extra-large transaction would be borne by the customer in the form of fees. Finally, a large transaction script like this would be carried in the UTXO set in RAM in every full node, until it was spent. All of these issues make using complex output scripts difficult in practice.
Although multi-signature scripts are a powerful feature, they are cumbersome to use. Given the preceding script, Mohammed would have to communicate this script to every customer prior to payment. Each customer would have to use special bitcoin wallet software with the ability to create custom transaction scripts, and each customer would have to understand how to create a transaction using custom scripts. Furthermore, the resulting transaction would be about five times larger than a simple payment transaction, because this script contains very long public keys. The burden of that extra-large transaction would be borne by the customer in the form of fees. Finally, a large transaction script like this would be carried in the UTXO set in RAM in every full node, until it was spent. All of these issues make using complex output scripts difficult in practice.
Pay-to-Script-Hash (P2SH) was developed to resolve these practical difficulties and to make the use of complex scripts as easy as a payment to a bitcoin address. With P2SH payments, the complex locking script is replaced with its digital fingerprint, a cryptographic hash. When a transaction attempting to spend the UTXO is presented later, it must contain the script that matches the hash, in addition to the unlocking script. In simple terms, P2SH means "pay to a script matching this hash, a script which will be presented later when this output is spent".
Pay-to-script-hash (P2SH) was developed to resolve these practical difficulties and to make the use of complex scripts as easy as a payment to a bitcoin address. With P2SH payments, the complex locking script is replaced with its digital fingerprint, a cryptographic hash. When a transaction attempting to spend the UTXO is presented later, it must contain the script that matches the hash, in addition to the unlocking script. In simple terms, P2SH means "pay to a script matching this hash, a script that will be presented later when this output is spent."
In P2SH transactions, the locking script that is replaced by a hash is referred to as the _redeem script_ because it is presented to the system at redemption time rather than as a locking script.
In P2SH transactions, the locking script that is replaced by a hash is referred to as the((("redeem script"))) _redeem script_ because it is presented to the system at redemption time rather than as a locking script. <<without_p2sh>> shows the script without P2SH and <<with_p2sh>> shows the same script encoded with P2SH.
[[without_p2sh]]
.Complex Script without P2SH
.Complex script without P2SH
|=======
| Locking Script | 2 PubKey1 PubKey2 PubKey3 PubKey4 PubKey5 5 OP_CHECKMULTISIG
| Unlocking Script | Sig1 Sig2
|=======
[[with_p2sh]]
.Complex Script as P2SH
.Complex script as P2SH
|=======
| Redeem Script | 2 PubKey1 PubKey2 PubKey3 PubKey4 PubKey5 5 OP_CHECKMULTISIG
| Locking Script | OP_HASH160 <20-byte hash of redeem script> OP_EQUAL
| Unlocking Script | Sig1 Sig2 redeem script
|=======
As you can see from the tables above, with P2SH the complex script that details the conditions for spending the output (redeem script) is not presented in the locking script. Instead, only a hash of it is in the locking script and the redeem script itself is presented later, as part of the unlocking script when the output is spent. This shifts the burden in fees and complexity from the sender to the recipient (spender) of the transaction.
As you can see from the tables, with P2SH the complex script that details the conditions for spending the output (redeem script) is not presented in the locking script. Instead, only a hash of it is in the locking script and the redeem script itself is presented later, as part of the unlocking script when the output is spent. This shifts the burden in fees and complexity from the sender to the recipient (spender) of the transaction.
Let's look at Mohammed's company, their complex multi-signature script and the resulting P2SH scripts.
Let's look at Mohammed's company, the complex multi-signature script, and the resulting P2SH scripts.
First, the multi-signature script that Mohammed's company uses for all incoming payments from customers:
----
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> \
<Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
2 <Mohammed's Public Key> <Partner1 Public Key> <Partner2 Public Key> <Partner3 Public Key> <Attorney Public Key> 5 OP_CHECKMULTISIG
----
If the placeholders above are replaced by actual public keys (shown below as 520 bit numbers starting with 04) you can see that this script becomes very long:
If the placeholders are replaced by actual public keys (shown here as 520-bit numbers starting with 04) you can see that this script becomes very long:
----
2
04C16B8698A9ABF84250A7C3EA7EEDEF9897D1C8C6ADF47F06CF73370\
D74DCCA01CDCA79DCC5C395D7EEC6984D83F1F50C900A24DD47F569FD\
4193AF5DE762C58704A2192968D8655D6A935BEAF2CA23E3FB87A3495\
E7AF308EDF08DAC3C1FCBFC2C75B4B0F4D0B1B70CD2423657738C0C2B\
1D5CE65C97D78D0E34224858008E8B49047E63248B75DB7379BE9CDA8\
CE5751D16485F431E46117B9D0C1837C9D5737812F393DA7D4420D7E1\
A9162F0279CFC10F1E8E8F3020DECDBC3C0DD389D99779650421D65CB\
D7149B255382ED7F78E946580657EE6FDA162A187543A9D85BAAA93A4\
AB3A8F044DADA618D087227440645ABE8A35DA8C5B73997AD343BE5C2\
AFD94A5043752580AFA1ECED3C68D446BCAB69AC0BA7DF50D56231BE0\
AABF1FDEEC78A6A45E394BA29A1EDF518C022DD618DA774D207D137AA\
B59E0B000EB7ED238F4D800 5 OP_CHECKMULTISIG
04C16B8698A9ABF84250A7C3EA7EEDEF9897D1C8C6ADF47F06CF73370D74DCCA01CDCA79DCC5C395D7EEC6984D83F1F50C900A24DD47F569FD4193AF5DE762C58704A2192968D8655D6A935BEAF2CA23E3FB87A3495E7AF308EDF08DAC3C1FCBFC2C75B4B0F4D0B1B70CD2423657738C0C2B1D5CE65C97D78D0E34224858008E8B49047E63248B75DB7379BE9CDA8CE5751D16485F431E46117B9D0C1837C9D5737812F393DA7D4420D7E1A9162F0279CFC10F1E8E8F3020DECDBC3C0DD389D99779650421D65CBD7149B255382ED7F78E946580657EE6FDA162A187543A9D85BAAA93A4AB3A8F044DADA618D087227440645ABE8A35DA8C5B73997AD343BE5C2AFD94A5043752580AFA1ECED3C68D446BCAB69AC0BA7DF50D56231BE0AABF1FDEEC78A6A45E394BA29A1EDF518C022DD618DA774D207D137AAB59E0B000EB7ED238F4D800 5 OP_CHECKMULTISIG
----
The entire script above can instead be represented by a 20-byte cryptographic hash, by first applying the SHA256 hashing algorithm and then applying the RIPEMD160 algorithm on the result. The 20-byte hash of the above script is:
This entire script can instead be represented by a 20-byte cryptographic hash, by first applying the SHA256 hashing algorithm and then applying the RIPEMD160 algorithm on the result. The 20-byte hash of the preceding script is:
----
54c557e07dde5bb6cb791c7a540e0a4796f5e97e
----
A P2SH transaction locks the output to this hash instead of the longer script, using the locking script:
----
OP_HASH160 54c557e07dde5bb6cb791c7a540e0a4796f5e97e OP_EQUAL
----
which, as you can see is much shorter. Instead of "pay to this 5-key multi-signature script", the P2SH equivalent transaction is "pay to a script with this hash". A customer making a payment to Mohammed's company need only include this much shorter locking script in their payment. When Mohammed wants to spend this UTXO, they must present the original redeem script (the one whose hash locked the UTXO) and the signatures necessary to unlock it, like this:
which, as you can see, is much shorter. Instead of "pay to this 5-key multi-signature script," the P2SH equivalent transaction is "pay to a script with this hash." A customer making a payment to Mohammed's company need only include this much shorter locking script in his payment. When Mohammed wants to spend this UTXO, they must present the original redeem script (the one whose hash locked the UTXO) and the signatures necessary to unlock it, like this:
----
<Sig1> <Sig2> <2 PK1 PK2 PK3 PK4 PK5 5 OP_CHECKMULTISIG>
----
The two scripts are combined in two stages. First, the redeem script is checked against the locking script to make sure the hash matches:
----
<2 PK1 PK2 PK3 PK4 PK5 5 OP_CHECKMULTISIG> OP_HASH160 <redeem scriptHash> OP_EQUAL
----
If the redeem script hash matches, then the unlocking script is executed on its own, to unlock the redeem script:
If the redeem script hash matches, the unlocking script is executed on its own, to unlock the redeem script:
----
<Sig1> <Sig2> 2 PK1 PK2 PK3 PK4 PK5 5 OP_CHECKMULTISIG
----
===== Pay-to-Script-Hash Addresses
===== Pay-to-script-hash addresses
Another important part of the P2SH feature is the ability to encode a script hash as an address, as defined in BIP0013. P2SH addresses are Base58Check encodings of the 20-byte hash of a script, just like bitcoin addresses are Base58Check encodings of the 20-byte hash of a public key. P2SH addresses use the version prefix "5", which results in Base58Check encoded addresses that start with a "3". For example, Mohammed's complex script, hashed and Base58Check encoded as P2SH address becomes +39RF6JqABiHdYHkfChV6USGMe6Nsr66Gzw+. Now, Mohammed can give this "address" to his customers and they can use almost any bitcoin wallet to make a simple payment, as if it were a bitcoin address. The 3 prefix gives them a hint that this is a special type of address, one corresponding to a script instead of a public key, but otherwise it works in exactly the same way as a payment to a bitcoin address.
((("addresses, bitcoin","Pay-to-Script-Hash (P2SH)")))((("Pay-to-script-hash (P2SH)","addresses")))Another important part of the P2SH feature is the ability to encode a script hash as an address, as defined in BIP0013. P2SH addresses are Base58Check encodings of the 20-byte hash of a script, just like bitcoin addresses are Base58Check encodings of the 20-byte hash of a public key. P2SH addresses use the version prefix "5", which results in Base58Check-encoded addresses that start with a "3". For example, Mohammed's complex script, hashed and Base58Check-encoded as a P2SH address becomes +39RF6JqABiHdYHkfChV6USGMe6Nsr66Gzw+. Now, Mohammed can give this "address" to his customers and they can use almost any bitcoin wallet to make a simple payment, as if it were a bitcoin address. The 3 prefix gives them a hint that this is a special type of address, one corresponding to a script instead of a public key, but otherwise it works in exactly the same way as a payment to a bitcoin address.
P2SH addresses hide all of the complexity, so that the person making a payment does not see the script.
===== Benefits of Pay-to-Script-Hash
===== Benefits of pay-to-script-hash
The Pay-to-Script-Hash feature offers the following benefits compared to the direct use of complex scripts in locking outputs:
((("Pay-to-script-hash (P2SH)","benefits of")))The pay-to-script-hash feature offers the following benefits compared to the direct use of complex scripts in locking outputs:
* Complex scripts are replaced by shorter fingerprint in the transaction output, making the transaction smaller
* Scripts can be coded as an address, so the sender and the sender's wallet don't need complex engineering to implement P2SH
* P2SH shifts the burden of constructing the script to the recipient not the sender
* P2SH shifts the burden in data storage for the long script from the output (which is in the UTXO set and therefore impacts memory) to the input (only stored on the blockchain)
* P2SH shifts the burden in data storage for the long script from the present time (payment) to a future time (when it is spent)
* P2SH shifts the transaction fee cost of a long script from the sender to the recipient who has to include the long redeem script to spend it
* Complex scripts are replaced by shorter fingerprints in the transaction output, making the transaction smaller.
* Scripts can be coded as an address, so the sender and the sender's wallet don't need complex engineering to implement P2SH.
* P2SH shifts the burden of constructing the script to the recipient, not the sender.
* P2SH shifts the burden in data storage for the long script from the output (which is in the UTXO set and therefore affect memory) to the input (only stored on the blockchain).
* P2SH shifts the burden in data storage for the long script from the present time (payment) to a future time (when it is spent).
* P2SH shifts the transaction fee cost of a long script from the sender to the recipient, who has to include the long redeem script to spend it.
===== Redeem Script and isStandard Validation
===== Redeem script and isStandard validation
Prior to version 0.9.2 of the Bitcoin Core client, Pay-to-Script-Hash was limited to the standard types of bitcoin transaction scripts, by the +isStandard()+ function. That means that the redeem script presented in the spending transaction could only be one of the standard types: P2PK, P2PKH or Multi-Sig, excluding OP_RETURN and P2SH itself.
((("pay-to-script-hash (P2SH)","isStandard validation")))((("pay-to-script-hash (P2SH)","redeem script for")))Prior to version 0.9.2 of the Bitcoin Core client, pay-to-script-hash was limited to the standard types of bitcoin transaction scripts, by the +isStandard()+ function. That means that the redeem script presented in the spending transaction could only be one of the standard types: P2PK, P2PKH, or multi-sig nature, excluding +OP_RETURN+ and P2SH itself.
As of version 0.9.2 of the Bitcoin Core client, P2SH transactions can contain any valid script, making the P2SH standard much more flexible and allowing for experimentation with many novel and complex types of transactions.
Note that you are not able to put a P2SH inside a P2SH redeem script, because the P2SH specification is not recursive. You are also still not be able to use OP_RETURN in a redeem script because OP_RETURN cannot be redeemed by definition.
Note that you are not able to put a P2SH inside a P2SH redeem script, because the P2SH specification is not recursive. You are also still not able to use +OP_RETURN+ in a redeem script because +OP_RETURN+ cannot be redeemed by definition.
Note that since the redeem script is not presented to the network until you attempt to spend a P2SH output, if you lock an output with the hash of an invalid transaction it will be processed regardless. However, you will not be able to spend it as the spending transaction which includes the redeem script will not be accepted, as it is an invalid script. This creates a risk, because you can lock bitcoin in a P2SH which cannot be later spent. The network will accept the P2SH encumbrance even if it corresponds to an invalid redeem script, because the script hash gives no indication of the script it represents.
Note that because the redeem script is not presented to the network until you attempt to spend a P2SH output, if you lock an output with the hash of an invalid transaction it will be processed regardless. However, you will not be able to spend it because the spending transaction, which includes the redeem script, will not be accepted because it is an invalid script. This creates a risk, because you can lock bitcoin in a P2SH that cannot be spent later. The network will accept the P2SH encumbrance even if it corresponds to an invalid redeem script, because the script hash gives no indication of the script it represents.
[WARNING]
====
P2SH locking scripts contain the hash of a redeem script which gives no clues as to the content of the redeem script itself. The P2SH transaction will be considered valid and accepted even if the redeem script is invalid. You may accidentally lock bitcoin in such a way that it cannot later be spent.
((("Pay-to-Script-Hash (P2SH)","locking scripts")))P2SH locking scripts contain the hash of a redeem script, which gives no clues as to the content of the redeem script itself. The P2SH transaction will be considered valid and accepted even if the redeem script is invalid. You might accidentally lock bitcoin in such a way that it cannot later be spent.(((range="endofrange", startref="ix_ch05-asciidoc19")))(((range="endofrange", startref="ix_ch05-asciidoc18")))(((range="endofrange", startref="ix_ch05-asciidoc17")))(((range="endofrange", startref="ix_ch05-asciidoc0")))
====

586
ch06.asciidoc
Unescape View File

@ -1,287 +1,299 @@
[[ch6]]
[[bitcoin_network]]
== The Bitcoin Network
=== Peer-to-Peer Network Architecture
Bitcoin is structured as a peer-to-peer network architecture on top of the Internet. The term peer-to-peer or P2P means that the computers that participate in the network are peers to each other, that they are all equal, that there are no "special" nodes and that all nodes share the burden of providing network services. The network nodes interconnect in a mesh network with a "flat" topology. There is no "server", no centralized service, and no hierarchy within the network. Nodes in a peer-to-peer network both provide and consume services at the same time with reciprocity acting as the incentive for participation. Peer-to-peer networks are inherently resilient, de-centralized, and open. The pre-eminent example of a P2P network architecture was the early Internet itself, where nodes on the IP network were equal. Today's Internet architecture is more hierarchical, but the Internet Protocol still retains its flat-topology essence. Beyond bitcoin, the largest and most successful application of P2P technologies is file sharing with Napster as the pioneer and bittorrent as the most recent evolution of the architecture.
Bitcoin's P2P network architecture is much more than a topology choice. Bitcoin is a peer-to-peer digital cash system by design, and the network architecture is both a reflection and a foundation of that core characteristic. De-centralization of control is a core design principle and that can only be achieved and maintained by a flat, de-centralized P2P consensus network.
The term "bitcoin network" refers to the collection of nodes running the bitcoin P2P protocol. In addition to the bitcoin P2P protocol, there are other protocols such as Stratum, which are used for mining and lightweight or mobile wallets. These additional protocols are provided by gateway routing servers that access the bitcoin network using the bitcoin P2P protocol and then extend that network to nodes running other protocols. For example, Stratum servers connect Stratum mining nodes via the Stratum protocol to the main bitcoin network and bridge the Stratum protocol to the bitcoin P2P protocol. We use the term "extended bitcoin network" to refer to the overall network that includes the bitcoin P2P protocol, pool mining protocols, the Stratum protocol, and any other related protocols connecting the components of the bitcoin system.
=== Nodes Types and Roles
While nodes in the bitcoin P2P network are equal, they may take on different "roles" depending on the functionality they are supporting. A bitcoin node is a collection of functions: routing, the blockchain database, mining, and wallet services. A full node with all four of these functions is shown below:
[[full_node_reference]]
.A bitcoin network node with all four functions: Wallet, Miner, full Blockchain database, and Network routing
image::images/FullNodeReferenceClient_Small.png["FullNodeReferenceClient_Small"]
All nodes include the routing function to participate in the network and may include other functionality. All nodes validate and propagate transactions and blocks, and discover and maintain connections to peers. In the full node example above, the routing function is indicated by an orange circle named "Network Routing Node".
Some nodes, called full nodes, also maintain a complete and up-to-date copy of the blockchain. Full nodes can autonomously and authoritatively verify any transaction without external reference. Some nodes maintain only a subset of the blockchain and verify transactions using a method called _Simplified Payment Verification_ or SPV. These nodes are known as SPV or Lightweight nodes. In the full node example above, the full node blockchain database function is indicated by a blue circle named "Full Blockchain". SPV nodes are drawn without the blue circle, showing that they do not have a full copy of the blockchain.
Mining nodes compete to create new blocks by running specialized hardware to solve the proof-of-work algorithm. Some mining nodes are also full nodes, maintaining a full copy of the blockchain while others are lightweight nodes participating in pool mining and depending on a pool server to maintain a full node. The mining function is shown in the full node above as a black circle named "Miner".
User wallets may be part of a full node, as is usually the case with desktop bitcoin clients. Increasingly many user wallets, especially those running on resource-constrained devices such as smart phones, are SPV nodes. The wallet function is shown above as a green circle named "Wallet".
In addition to the main node types on the bitcoin P2P protocol, there are servers and nodes running other protocols, such as specialized mining pool protocols and lightweight client access protocols.
Here are the most common node types on the extended bitcoin network:
[[node_type_ledgend]]
.Different types of nodes on the extended bitcoin network
image::images/BitcoinNodeTypes.png["BitcoinNodeTypes"]
=== The Extended Bitcoin Network
The main bitcoin network, running the bitcoin P2P protocol, consists of between 7,000 to 10,000 nodes running various versions of the bitcoin reference client (Bitcoin Core) and a few hundred nodes running various other implementations of the bitcoin P2P protocol, such as BitcoinJ, Libbitcoin, and btcd. A small percentage of the nodes on the bitcoin P2P network are also mining nodes, competing in the mining process, validating transactions, and creating new blocks. Various large companies interface with the bitcoin network by running full-node clients based on the Bitcoin Core client, with full copies of the blockchain and a network node, but without mining or wallet functions. These nodes act as network edge routers, allowing various other services (exchanges, wallets, block explorers, merchant payment processing) to be built on top.
The extended bitcoin network includes the network running the bitcoin P2P protocol, described above, as well as nodes running specialized protocols. Attached to the main bitcoin P2P network are a number of pool servers and protocol gateways that connect nodes running other protocols. These other protocol nodes are mostly pool mining nodes (see <<mining>>) and lightweight wallet clients, which do not carry a full copy of the blockchain.
The diagram below shows the extended bitcoin network with the various types of nodes, gateway servers, edge routers, and wallet clients and the various protocols they use to connect to each other.
[[bitcoin_network]]
.The extended bitcoin network showing various node types, gateways and protocols
image::images/BitcoinNetwork.png["BitcoinNetwork"]
=== Network Discovery
When a new node boots up, it must discover other bitcoin nodes on the network in order to participate. To start this process, a new node must discover at least one existing node on the network and connect to it. The geographic location of the other nodes is irrelevant, the bitcoin network topology is not geographically defined. Therefore, any existing bitcoin nodes can be selected at random.
To connect to a known peer, nodes establish a TCP connection, usually to port 8333 (the bitcoin "well known" port), or an alternative port if one is provided. Upon establishing a connection, the node will start a "handshake" by transmitting a +version+ message, which contains basic identifying information, including:
* PROTOCOL_VERSION, a constant that defines the bitcoin P2P protocol version the client "speaks" (e.g. 70002)
* nLocalServices, a list of local services supported by the node, currently just NODE_NETWORK
* nTime, the current time
* addrYou, the IP address of the remote node as seen from this node
* addrMe, the IP address of the local node, as discovered by the local node
* subver, a sub-version showing the type of software running on this node (e.g. "/Satoshi:0.9.2.1/“)
* BestHeight, the block height of this node's blockchain
(See https://github.com/bitcoin/bitcoin/blob/d3cb2b8acfce36d359262b4afd7e7235eff106b0/src/net.cpp#L562 for an example of the +version+ network message)
The peer node responds with +verack+ to acknowledge and establish a connection, and optionally sends its own +version+ message if it wishes to reciprocate the connection and connect back as a peer.
[[network_handshake]]
.The initial handshake between peers
image::images/NetworkHandshake.png["NetworkHandshake"]
How does a new node find peers? While there are no special nodes in bitcoin, there are some long running stable nodes that are listed in the client as _seed nodes_. While a new node does not have to connect with the seed nodes, it can use them to quickly discover other nodes in the network. In the Bitcoin Core client, the option to use the seed nodes is controlled by the option switch +-dnsseed+, which is set to 1, to use the seed nodes, by default. Alternatively, a bootstrapping node that knows nothing of the network must be given the IP address of at least one bitcoin node after which it can establish connections through further introductions. The command line argument +-seednode+ can be used to connect to one node just for introductions, using it as a DNS seed. After the initial seed node is used to form introductions, the client will disconnect from it and use the newly discovered peers.
Once one or more connections are established, the new node will send an +addr+ message containing its own IP address, to its neighbors. The neighbors will in turn forward the +addr+ message to their neighbors, ensuring that the newly connected node becomes well known and better connected. Additionally, the newly connected node can send +getaddr+ to the neighbors, asking them to return a list of IP addresses of other peers. That way, a node can find peers to connect to and advertise its existence on the network for other nodes to find it.
[[address_propagation]]
.Address Propagation and Discovery
image::images/AddressPropagation.png["AddressPropagation"]
A node must connect to a few different peers in order to establish diverse paths into the bitcoin network. Paths are not reliable, nodes come and go, and so the node must continue to discover new nodes as it loses old connections as well as assist other nodes when they bootstrap. Only one connection is needed to bootstrap, as the first node can offer introductions to its peer nodes and those peers can offer further introductions. It's also unnecessary and wasteful of network resources to connect to more than a handful of nodes. After bootstrapping, a node will remember its most recent successful peer connections, so that if it is rebooted it can quickly reestablish connections with its former peer network. If none of the former peers respond to its connection request, the node can use the seed nodes to bootstrap again.
On a node running the Bitcoin Core client, you can list the peer connections with the command +getpeerinfo+:
[source,bash]
----
$ bitcoin-cli getpeerinfo
----
[source,json]
----
[
{
"addr" : "85.213.199.39:8333",
"services" : "00000001",
"lastsend" : 1405634126,
"lastrecv" : 1405634127,
"bytessent" : 23487651,
"bytesrecv" : 138679099,
"conntime" : 1405021768,
"pingtime" : 0.00000000,
"version" : 70002,
"subver" : "/Satoshi:0.9.2.1/",
"inbound" : false,
"startingheight" : 310131,
"banscore" : 0,
"syncnode" : true
},
{
"addr" : "58.23.244.20:8333",
"services" : "00000001",
"lastsend" : 1405634127,
"lastrecv" : 1405634124,
"bytessent" : 4460918,
"bytesrecv" : 8903575,
"conntime" : 1405559628,
"pingtime" : 0.00000000,
"version" : 70001,
"subver" : "/Satoshi:0.8.6/",
"inbound" : false,
"startingheight" : 311074,
"banscore" : 0,
"syncnode" : false
}
]
----
To override the automatic management of peers and to specify a list of IP addresses, users can provide the option +-connect=<IPAddress>+ and specify one or more IP addresses. If this option is used, the node will only connect to the selected IP addresses, instead of discovering and maintaining the peer connections automatically.
If there is no traffic on a connection, nodes will periodically send a message to maintain the connection. If a node has not communicated on a connection for more than 90 minutes, it is assumed to be disconnected and a new peer will be sought. Thus, the network dynamically adjusts to transient nodes, network problems, and can organically grow and shrink as needed without any central control.
=== Full Nodes
Full nodes are nodes that maintain a full blockchain with all transactions. More accurately they probably should be called "full blockchain nodes". In the early years of bitcoin, all nodes were full nodes and currently the Bitcoin Core client is a full blockchain node. In the last two years, however, new forms of bitcoin clients have been introduced that do not maintain a full blockchain but run as lightweight clients. These are examined in more detail in the next section.
Full blockchain nodes maintain a complete and up-to-date copy of the bitcoin blockchain with all the transactions, which they independently build and verify, starting with the very first block (genesis block) and building up to the latest known block in the network. A full blockchain node can independently and authoritatively verify any transaction without recourse or reliance on any other node or source of information. The full blockchain node relies on the network to receive updates about new blocks of transactions, which it then verifies and incorporates into its local copy of the blockchain.
Running a full blockchain node gives you the pure bitcoin experience: independent verification of all transactions without the need to rely on, or trust, any other systems. It's easy to tell if you're running a full node because it requires 20 plus gigabytes of persistent storage (disk space) to store the full blockchain. If you need a lot of disk and it takes 2-3 days to "sync" to the network, you are running a full node. That is the price of complete independence and freedom from central authority.
There are a few alternative implementations of full-blockchain bitcoin clients, built using different programming languages and software architectures. However, the most common implementation is the reference client Bitcoin Core, also known as the Satoshi Client. More than 90% of the nodes on the bitcoin network run various versions of Bitcoin Core. It is identified as "Satoshi" in the sub-version string sent in the +version+ message and shown by the command +getpeerinfo+ as we saw above, for example +/Satoshi:0.8.6/+.
=== Exchanging "Inventory"
The first thing a full node will do once it connects to peers is try to construct a complete blockchain. If it is a brand-new node and has no blockchain at all, then it only knows one block (the genesis block), which is statically embedded in the client software. Starting with block #0, the genesis block, the new node will have to download hundreds of thousands of blocks to synchronize with the network and re-establish the full blockchain.
The process of "syncing" the blockchain starts with the +version+ message, as that contains +BestHeight+, a node's current blockchain height (number of blocks). A node will see the +version+ messages from its peers, know how many blocks they each have and be able to compare to how many blocks it has in its own blockchain. Peered nodes will exchange a +getblocks+ message that contains the hash (fingerprint) of the top block on their local blockchain. One of the peers will be able to identify the received hash as belonging to a block that is not at the top, but rather belongs to an older block, thus deducing that its own local blockchain is longer than its peer's.
The peer that has the longer blockchain has more blocks than the other node and can identify which blocks the other node needs in order to "catch up". It will identify the first 500 blocks to share and transmit their hashes using an +inv+ (inventory) message. The node missing these blocks will then retrieve them, by issuing a series of +getdata+ messages requesting the full block data and identifying the requested blocks using the hashes from the +inv+ message.
Let's assume for example that a node only has the genesis block. It will then receive an +inv+ message from its peers containing the hashes of the next 500 blocks in the chain. It will start requesting blocks from all of its connected peers, spreading the load and ensuring that it doesn't overwhelm any peer with requests. The node keeps track of how many blocks are "in transit" per peer connection, meaning blocks that it has requested but not received, checking that it does not exceed a limit (MAX_BLOCKS_IN_TRANSIT_PER_PEER). This way, if it needs a lot of blocks, it will only request new ones as previous requests are fulfilled, allowing the peers to control the pace of updates and not overwhelming the network. As each block is received, it is added to the blockchain as we will see in the next chapter <<blockchain>>. As the local blockchain is gradually built up, more blocks are requested and received and the process continues until the node catches up to the rest of the network.
This process of comparing the local blockchain with the peers and retrieving any missing blocks happens any time a node goes offline for any period of time. Whether a node has been offline for a few minutes and is missing a few blocks, or a month and is missing a few thousand blocks, it starts by sending +getblocks+, gets an +inv+ response, and starts downloading the missing blocks.
[[inventory_synchronization]]
.Node synchronizing the blockchain by retrieving blocks from a peer
image::images/InventorySynchronization.png["InventorySynchronization"]
[[spv_nodes]]
=== Simplified Payment Verification (SPV) Nodes
Not all nodes have the ability to store the full blockchain. Many bitcoin clients are designed to run on space- and power-constrained devices, such as smartphones, tablets or embedded systems. For such devices, a _simplified payment verification_ (SPV) method is used to allow them to operate without storing the full blockchain. These types of clients are called SPV clients or lightweight clients. As bitcoin adoption surges, the SPV node is becoming the most common form of bitcoin node, especially for bitcoin wallets.
SPV nodes download only the block headers and do not download the transactions included in each block. The resulting chain of blocks, without transactions, is 1,000 times smaller than the full blockchain. SPV nodes cannot construct a full picture of all the UTXOs that are available for spending, as they do not know about all the transactions on the network. SPV nodes verify transactions using a slightly different methodology that relies on peers to provide partial views of relevant parts of the blockchain on-demand.
As an analogy, a full node is like a tourist in a strange city, equipped with a detailed map of every street and every address. By comparison, an SPV node is like a tourist in a strange city asking random strangers for turn-by-turn directions while knowing only one main avenue. While both tourists can verify the existence of a street by visiting it, the tourist without a map doesn't know what lies down any of the side streets and doesn't know what other streets exist. Positioned in front of 23 Church Street, the tourist without a map cannot know if there are a dozen other "23 Church Street" addresses in the city and whether this is the right one. The map-less tourist's best chance is to ask enough people and hope some of them are not trying to mug the tourist.
Simplified Payment Verification verifies transactions by reference to their _depth_ in the blockchain instead of their _height_. Whereas a full-blockchain node will construct a fully verified chain of thousands of blocks and transactions reaching down the blockchain (back in time) all the way to the genesis block, an SPV node will verify the chain of all blocks and link that chain to the transaction of interest.
For example, when examining a transaction in block 300,000, a full node links all 300,000 blocks down to the genesis block and builds a full database of UTXO, establishing the validity of the transaction by confirming that the UTXO remains unspent. An SPV node cannot validate whether the UTXO is unspent. Instead, the SPV node will establish a link between the transaction and the block that contains it, using a Merkle Path (see <<merkle_trees>>). Then, the SPV node waits until it sees the six blocks 300,001 through 300,006 piled on top of the block containing the transaction and verifies it by establishing its depth under blocks 300,006 to 300,001. The fact that other nodes on the network accepted block 300,000 and then did the necessary work to produce 6 more blocks on top of it is proof, by proxy, that the transaction was not a double-spend.
An SPV node cannot be persuaded that a transaction exists in a block, when it does not in fact exist. The SPV node establishes the existence of a transaction in a block by requesting a merkle path proof and by validating the proof-of-work in the chain of blocks. However, a transaction's existence can be "hidden" from an SPV node. An SPV node can definitely prove that a transaction exists but cannot verify that a transaction, such as a double-spend of the same UTXO, doesn't exist because it doesn't have a record of all transactions. This type of attack can be used as a Denial-of-Service attack or as a double-spending attack against SPV nodes. To defend against this, an SPV node needs to connect randomly to several nodes, to increase the probability that it is in contact with at least one honest node. SPV nodes are therefore vulnerable to network partitioning attacks or Sybil attacks, where they are connected to fake nodes or fake networks and do not have access to honest nodes or the real bitcoin network.
For most practical purposes, well-connected SPV nodes are secure enough, striking the right balance between resource needs, practicality, and security. For the truly security conscious, however, nothing beats running a full blockchain node.
[TIP]
====
A full blockchain node verifies a transaction by checking the chain of thousands of blocks below it and checks that the UTXO is not spent, whereas an SPV node checks how deep the block is buried by a handful of blocks above it.
====
To get the block headers, SPV nodes use a +getheaders+ message instead of +getblocks+. The responding peer will send up to 2000 block headers using a single +headers+ message. The process is otherwise the same as that used by a full node to retrieve full blocks. SPV nodes also set a filter on the connection to peers, to filter the stream of future blocks and transactions sent by the peers. Any transactions of interest are retrieved using a +getdata+ request. The peer generates a +tx+ message containing the transactions, in response.
[[spv_synchronization]]
.SPV Node synchronizing the block headers
image::images/SPVSynchronization.png["SPVSynchronization"]
Because SPV nodes need to retrieve specific transactions in order to selectively verify them, they also create a privacy risk. Unlike full-blockchain nodes, which collect all transactions within each block, the SPV node's requests for specific data can inadvertently reveal the addresses in their wallet. For example, a third party monitoring a network could keep track of all the transactions requested by a wallet on an SPV node and use those to associate bitcoin addresses with the user of that wallet, destroying the user's privacy.
Shortly after the introduction of SPV/lightweight nodes, the bitcoin developers added a feature called _bloom filters_ to address the privacy risks of SPV nodes. Bloom filters allow SPV nodes to receive a subset of the transactions without revealing precisely which addresses they are interested in, through a filtering mechanism that uses probabilities rather than fixed patterns.
=== Bloom Filters
A bloom filter is a probabilistic search filter, a way to describe a desired pattern without specifying it exactly. Bloom filters offer an efficient way to express a search pattern while protecting privacy. They are used by SPV nodes to ask their peers for transactions matching a specific pattern, without revealing exactly which addresses they are searching for.
In our previous analogy, a tourist without a map is asking for directions to a specific address "23 Church St". If they asks strangers for directions to this street, they inadvertently reveal their destination. A bloom filter is like asking "Are there any streets in this neighborhood whose name ends in R-C-H". A question like that reveals slightly less about the desired destination, than asking for "23 Church St". Using this technique, a tourist could specify the desired address in more detail as "ending in U-R-C-H" or less detail as "ending in H". By varying the precision of the search, the tourist reveals more or less information, at the expense of getting more or less specific results. If they ask a less specific pattern, they get a lot more possible addresses and better privacy but many of the results are irrelevant. If they ask for a very specific pattern then they get fewer results but they lose privacy.
Bloom filters serve this function by allowing an SPV node to specify a search pattern for transactions that can be tuned towards precision or privacy. A more specific bloom filter will produce accurate results, but at the expense of revealing what addresses are used in the user's wallet. A less specific bloom filter will produce more data about more transactions, many irrelevant to the node, but will allow the node to maintain better privacy.
An SPV node will initialize a bloom filter as "empty" and in that state the bloom filter will not match any patterns. The SPV node will then make a list of all the addresses in its wallet and create a search pattern matching the transaction output that corresponds to each address. Usually, the search pattern is a Pay-to-Public-Key-Hash script that is the expected locking script that will be present in any transaction paying to the public-key-hash (address). If the SPV node is tracking the balance of a P2SH address, then the search pattern will be a Pay-to-Script-Hash script, instead. The SPV node then adds each of the search patterns to the bloom filter, so that the bloom filter can recognize the search pattern if it is present in a transaction. Finally, the bloom filter is sent to the peer and the peer uses it to match transactions for transmission to the SPV node.
Bloom filters are implemented as a variable-size array of N binary digits (a bit field) and a variable number of M hash functions. The hash functions are designed to always produce an output that is between 1 and N, corresponding to the array of binary digits. The hash functions are generated deterministically, so that any node implementing a bloom filter will always use the same hash functions and get the same results for a specific input. By choosing different length (N) bloom filters and a different number (M) of hash functions, the bloom filter can be tuned, varying the level of accuracy and therefore privacy.
In the example below, we use a very small array of 16 bits and a set of 3 hash functions to demonstrate how bloom filters work.
[[bloom1]]
.An example of a simplistic bloom filter, with 16 bit field and 3 hash functions
image::images/Bloom1.png["Bloom1"]
The bloom filter is initialized so that the array of bits is all zeros. To add a pattern to the bloom filter, the pattern is hashed by each hash function in turn. Applying the first hash function to the input results in a number between 1 and N. The corresponding bit in the array (indexed from 1 to N) is found and set to +1+, thereby recording the output of the hash function. Then, the next hash function is used to set another bit and so on and so forth. Once all M hash functions have been applied, the search pattern will be "recorded" in the bloom filter as M bits have been changed from +0+ to +1+.
Here's an example of adding a pattern "A" to the simple bloom filter shown above:
[[bloom2]]
.Adding a pattern "A" to our simple bloom filter
image::images/Bloom2.png["Bloom2"]
Adding a second pattern is as simple as repeating this process. The pattern is hashed by each hash function in turn and the result is recorded by setting the bits to +1+. Note that as a bloom filter is filled with more patterns, a hash function result may coincide with a bit that is already set to +1+ in which case the bit is not changed. In essence, as more patterns record on overlapping bits, the bloom filter starts to become saturated with more bits set to +1+ and the accuracy of the filter decreases. This is why the filter is a probabilistic data structure -- it gets less accurate as more patterns are added. The accuracy depends on the number of patterns added versus the size of the bit array (N) and number of hash functions (M). A larger bit array and more hash functions can record more patterns with higher accuracy. A smaller bit array or fewer hash functions will record fewer patterns and produce less accuracy.
Below is an example of adding a second pattern "B" to the simple bloom filter:
[[bloom3]]
.Adding a second pattern "B" to our simple bloom filter
image::images/Bloom3.png["Bloom3"]
To test if a pattern is part of a bloom filter, the pattern is hashed by each hash function and the resulting bit pattern is tested against the bit array. If all the bits indexed by the hash functions are set to +1+, then the pattern is _probably_ recorded in the bloom filter. Since the bits may be set because of overlap from multiple patterns, the answer is not certain, but is rather probabilistic. In simple terms, a bloom filter positive match is a "Maybe, Yes".
Below is an example of testing the existence of pattern "X" in the simple bloom filter. The corresponding bits are set to +1+, so the pattern is probably a match:
[[bloom4]]
.Testing the existence of pattern "X" in the bloom filter. The result is probabilistic positive match, meaning "Maybe"
image::images/Bloom4.png["Bloom4"]
On the contrary, if a pattern is tested against the bloom filter and any one of the bits is set to +0+, then this proves that the pattern was not recorded in the bloom filter. A negative result is not a probability, it is a certainty. In simple terms, a negative match on a bloom filter is a "Definitely No".
Below is an example of testing the existence of pattern "Y" in the simple bloom filter. One of the corresponding bits is set to +0+, so the pattern is definitely not a match:
[[bloom5]]
.Testing the existence of pattern "Y" in the bloom filter. The result is a definitive negative match, meaning "Definitely No"
image::images/Bloom5.png["Bloom5"]
Bitcoin's implementation of bloom filters is described in Bitcoin Improvement Proposal 37 (BIP0037). See <<bip0037>> or visit:
https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki.
=== Bloom Filters and Inventory Updates
Bloom filters are used to filter the transactions (and blocks containing them) that an SPV node receives from its peers. SPV nodes will create a filter that matches only the addresses held in the SPV node's wallet. The SPV node will then send a +filterload+ message to the peer, containing the bloom filter to use on the connection. After a filter is established, the peer will then test each transaction's outputs against the bloom filter. Only transactions which match the filter are sent to the node.
In response to a +getdata+ message from the node, peers will send a +merkleblock+ message that contains only block headers for blocks matching the filter and a merkle path (See <<merkle_trees>>) for each matching transaction. The peer will also then send +tx+ messages containing the transactions matched by the filter.
The node setting the bloom filter can interactively add patterns to the filter by sending a +filteradd+ message. To clear the bloom filter, the node can send a +filterclear+ message. Since it is not possible to remove a pattern from a bloom filter, a node has to clear and re-send a new bloom filter if a pattern is no longer desired.
[[transaction_pools]]
=== Transaction Pools
Almost every node on the bitcoin network maintains a temporary list of unconfirmed transactions called the memory pool or transaction pool. Nodes use this pool to keep track of transactions that are known to the network but are not yet included in the blockchain. For example, a node that holds a user's wallet will use the transaction pool to track incoming payments to the user's wallet that have been received on the network but are not yet confirmed.
As transactions are received and verified, they are added to the transaction pool and relayed to the neighboring nodes to propagate on the network.
Some node implementations also maintain a separate pool of orphaned transactions as detailed in <<orphan_transactions>>. If a transaction's inputs refer to a transaction that is not yet known, a missing parent, then the orphan transaction will be stored temporarily in the orphan pool until the parent transaction arrives.
When a transaction is added to the transaction pool, the orphan pool is checked for any orphans that reference this transaction's outputs (its children). Any matching orphans are then validated. If valid, they are removed from the orphan pool and added to the transaction pool, completing the chain that started with the parent transaction. In light of the newly added transaction which is no longer an orphan, the process is repeated recursively looking for any further descendants, until no more descendants are found. Through this process, the arrival of a parent transaction triggers a cascade reconstruction of an entire chain of interdependent transactions by re-uniting the orphans with their parents all the way down the chain.
Both the transaction pool and orphan pool (where implemented) are stored in local memory and are not saved on persistent storage, rather they are dynamically populated from incoming network messages. When a node starts, both pools are empty and are gradually populated with new transactions received on the network.
Some implementations of the bitcoin client also maintain a UTXO database or UTXO pool, which is the set of all unspent outputs on the blockchain. While the name "UTXO pool" sounds similar to the transaction pool, it represents a different set of data. Unlike the transaction and orphan pools, the UTXO pool is not initialized empty but instead contains millions of entries of unspent transaction outputs including some dating back to 2009. The UTXO pool may be housed in local memory or as an indexed database table on persistent storage.
Whereas the transaction and orphan pools represent a single node's local perspective and may vary significantly from node to node depending upon when the node was started or restarted, the UTXO pool represents the emergent consensus of the network and therefore will vary little between nodes. Furthermore, the transaction and orphan pools only contain unconfirmed transactions, while the UTXO pool only contains confirmed outputs.
=== Alert Messages
Alert messages are a seldom used function, which is nevertheless implemented in most nodes. Alert messages are bitcoin's "emergency broadcast system", a means by which the core bitcoin developers can send an emergency text message to all bitcoin nodes. This feature is implemented to allow the core developer team to notify all bitcoin users of a serious problem in the bitcoin network, such as a critical bug that requires user action. The alert system has only been used a handful of times, most notably early 2013 when a critical database bug caused a multi-block fork to occur in the bitcoin blockchain.
Alert messages are propagated by the +alert+ message. The alert message contains several fields, including:
* ID - An alert identified so that duplicate alerts can be detected
* Expiration - a time after which the alert expires
* RelayUntil - a time after which the alert should not be relayed
* MinVer, MaxVer - the range of bitcoin protocol versions that this alert applies to
* subVer - The client software version that this alert applies to
* Priority - An alert priority level, currently unused
Alerts are cryptographically signed by a public key. The corresponding private key is held by a few selected members of the core development team. The digital signature ensures that fake alerts will not be propagated on the network.
Each node receiving this alert message will verify it, check for expiration, and propagate it to all its peers, thus ensuring rapid propagation across the entire network. In addition to propagating the alert, each node may implement a user interface function to present the alert to the user.
In the Bitcoin Core client, the alert is configured with the command line option +-alertnotify+, which specifies a command to run when an alert is received. The alert message is passed as a parameter to the alertnotify command. Most commonly, the alertnotify command is set to generate an email message to the administrator of the node, containing the alert message. The alert is also displayed as a pop-up dialog in the graphical user interface (bitcoin-Qt) if it is running.
Other implementations of the bitcoin protocol may handle the alert in different ways. Many hardware-embedded bitcoin mining systems do not implement the alert message function, as they have no user interface. It is strongly recommended that miners running such mining systems subscribe to alerts via a mining pool operator or by running a lightweight node just for alert purposes.
[[bitcoin_network_ch06]]
== The Bitcoin Network
=== Peer-to-Peer Network Architecture
((("bitcoin network", id="ix_ch06-asciidoc0", range="startofrange")))((("bitcoin network","architecture of")))((("peer-to-peer networks")))Bitcoin is structured as a peer-to-peer network architecture on top of the Internet. The term peer-to-peer, or P2P, means that the computers that participate in the network are peers to each other, that they are all equal, that there are no "special" nodes, and that all nodes share the burden of providing network services. The network nodes interconnect in a mesh network with a "flat" topology. There is no server, no centralized service, and no hierarchy within the network. Nodes in a peer-to-peer network both provide and consume services at the same time with reciprocity acting as the incentive for participation. Peer-to-peer networks are inherently resilient, decentralized, and open. The preeminent example of a P2P network architecture was the early Internet itself, where nodes on the IP network were equal. Today's Internet architecture is more hierarchical, but the Internet Protocol still retains its flat-topology essence. Beyond bitcoin, the largest and most successful application of P2P technologies is file sharing with Napster as the pioneer and BitTorrent as the most recent evolution of the architecture.
Bitcoin's P2P network architecture is much more than a topology choice. Bitcoin is a peer-to-peer digital cash system by design, and the network architecture is both a reflection and a foundation of that core characteristic. Decentralization of control is a core design principle and that can only be achieved and maintained by a flat, decentralized P2P consensus network.
((("bitcoin network","defined")))The term "bitcoin network" refers to the collection of nodes running the bitcoin P2P protocol. In addition to the bitcoin P2P protocol, there are other protocols such as((("Stratum (STM) mining protocol"))) Stratum, which are used for mining and lightweight or mobile wallets. These additional protocols are provided by gateway routing servers that access the bitcoin network using the bitcoin P2P protocol, and then extend that network to nodes running other protocols. For example, Stratum servers connect Stratum mining nodes via the Stratum protocol to the main bitcoin network and bridge the Stratum protocol to the bitcoin P2P protocol. We use the term "extended bitcoin network" to refer to the overall network that includes the bitcoin P2P protocol, pool-mining protocols, the Stratum protocol, and any other related protocols connecting the components of the bitcoin system.
=== Nodes Types and Roles
((("bitcoin network","nodes")))((("nodes","roles of")))((("nodes","types of")))Although nodes in the bitcoin P2P network are equal, they may take on different roles depending on the functionality they are supporting. A bitcoin node is a collection of functions: routing, the blockchain database, mining, and wallet services. A full node with all four of these functions is shown in <<full_node_reference>>.
[[full_node_reference]]
.A bitcoin network node with all four functions: wallet, miner, full blockchain database, and network routing
image::images/msbt_0601.png["FullNodeReferenceClient_Small"]
All nodes include the routing function to participate in the network and might include other functionality. All nodes validate and propagate transactions and blocks, and discover and maintain connections to peers. In the full-node example in <<full_node_reference>>, the routing function is indicated by an orange circle named "Network Routing Node."
Some nodes, called full nodes, also maintain a complete and up-to-date copy of the blockchain. Full nodes can autonomously and authoritatively verify any transaction without external reference. Some nodes maintain only a subset of the blockchain and verify transactions using a method called((("simplified payment verification (SPV) nodes","defined"))) _simplified payment verification_, or SPV. These nodes are known as SPV or lightweight nodes. In the full-node example in the figure, the full-node blockchain database function is indicated by a blue circle named "Full Blockchain." In <<bitcoin_network>>, SPV nodes are drawn without the blue circle, showing that they do not have a full copy of the blockchain.
Mining nodes compete to create new blocks by running specialized hardware to solve the proof-of-work algorithm. Some mining nodes are also full nodes, maintaining a full copy of the blockchain, while others are lightweight nodes participating in pool mining and depending on a pool server to maintain a full node. The mining function is shown in the full node as a black circle named "Miner."
User wallets might be part of a full node, as is usually the case with desktop bitcoin clients. Increasingly, many user wallets, especially those running on resource-constrained devices such as smartphones, are SPV nodes. The wallet function is shown in <<full_node_reference>> as a green circle named "Wallet".
In addition to the main node types on the bitcoin P2P protocol, there are servers and nodes running other protocols, such as specialized mining pool protocols and lightweight client-access protocols.
<<node_type_ledgend>> shows the most common node types on the extended bitcoin network.
=== The Extended Bitcoin Network
((("bitcoin network","extended")))((("extended bitcoin network")))The main bitcoin network, running the bitcoin P2P protocol, consists of between 7,000 and 10,000 listening nodes running various versions of the bitcoin reference client (Bitcoin Core) and a few hundred nodes running various other implementations of the bitcoin P2P protocol, such as((("BitcoinJ library")))((("btcd")))((("libbitcoin library"))) BitcoinJ, Libbitcoin, and btcd. A small percentage of the nodes on the bitcoin P2P network are also mining nodes, competing in the mining process, validating transactions, and creating new blocks. Various large companies interface with the bitcoin network by running full-node clients based on the Bitcoin Core client, with full copies of the blockchain and a network node, but without mining or wallet functions. These nodes act as network edge routers, allowing various other services (exchanges, wallets, block explorers, merchant payment processing) to be built on top.
The extended bitcoin network includes the network running the bitcoin P2P protocol, described earlier, as well as nodes running specialized protocols. Attached to the main bitcoin P2P network are a number of((("mining pools","on the bitcoin network"))) pool servers and protocol gateways that connect nodes running other protocols. These other protocol nodes are mostly pool mining nodes (see <<ch8>>) and lightweight wallet clients, which do not carry a full copy of the blockchain.
<<bitcoin_network>> shows the extended bitcoin network with the various types of nodes, gateway servers, edge routers, and wallet clients and the various protocols they use to connect to each other.
[[node_type_ledgend]]
.Different types of nodes on the extended bitcoin network
image::images/msbt_0602.png["BitcoinNodeTypes"]
[[bitcoin_network]]
.The extended bitcoin network showing various node types, gateways, and protocols
image::images/msbt_0603.png["BitcoinNetwork"]
=== Network Discovery
((("bitcoin network","discovery", id="ix_ch06-asciidoc1", range="startofrange")))((("network discovery", id="ix_ch06-asciidoc2", range="startofrange")))((("nodes","network discovery and", id="ix_ch06-asciidoc3", range="startofrange")))((("peer-to-peer networks","discovery by new nodes", id="ix_ch06-asciidoc4", range="startofrange")))When a new node boots up, it must discover other bitcoin nodes on the network in order to participate. To start this process, a new node must discover at least one existing node on the network and connect to it. The geographic location of other nodes is irrelevant; the bitcoin network topology is not geographically defined. Therefore, any existing bitcoin nodes can be selected at random.
((("peer-to-peer networks","connections")))To connect to a known peer, nodes establish a TCP connection, usually to port 8333 (the port generally known as the one used by bitcoin), or an alternative port if one is provided. Upon establishing a connection, the node will start a "handshake" (see <<network_handshake>>) by transmitting a((("version message"))) +version+ message, which contains basic identifying information, including:
+PROTOCOL_VERSION+:: A constant that defines the bitcoin P2P protocol version the client "speaks" (e.g., 70002)
+nLocalServices+:: A list of local services supported by the node, currently just +NODE_NETWORK+
+nTime+:: The current time
+addrYou+:: The IP address of the remote node as seen from this node
+addrMe+:: The IP address of the local node, as discovered by the local node
+subver+:: A sub-version showing the type of software running on this node (e.g., "/Satoshi:0.9.2.1/")+
+BestHeight+:: The block height of this node's blockchain
(See http://bit.ly/1qlsC7w[GitHub] for an example of the +version+ network message.)
The peer node responds with +verack+ to acknowledge and establish a connection, and optionally sends its own +version+ message if it wishes to reciprocate the connection and connect back as a peer.
How does a new node find peers? Although there are no special nodes in bitcoin, there are some long-running stable nodes that are listed in the client as((("nodes","seed")))((("seed nodes"))) _seed nodes_. Although a new node does not have to connect with the seed nodes, it can use them to quickly discover other nodes in the network. In the Bitcoin Core client, the option to use the seed nodes is controlled by the option switch +-dnsseed+, which is set to 1, to use the seed nodes, by default. Alternatively, a bootstrapping node that knows nothing of the network must be given the IP address of at least one bitcoin node, after which it can establish connections through further introductions. The command-line argument +-seednode+ can be used to connect to one node just for introductions, using it as a DNS seed. After the initial seed node is used to form introductions, the client will disconnect from it and use the newly discovered peers.
[[network_handshake]]
.The initial handshake between peers
image::images/msbt_0604.png["NetworkHandshake"]
Once one or more connections are established, the new node will send an((("addr message"))) +addr+ message containing its own IP address to its neighbors. The neighbors will, in turn, forward the +addr+ message to their neighbors, ensuring that the newly connected node becomes well known and better connected. Additionally, the newly connected node can send +getaddr+ to the neighbors, asking them to return a list of IP addresses of other peers. That way, a node can find peers to connect to and advertise its existence on the network for other nodes to find it. <<address_propagation>> shows the address discovery protocol.
[[address_propagation]]
.Address propagation and discovery
image::images/msbt_0605.png["AddressPropagation"]
A node must connect to a few different peers in order to establish diverse paths into the bitcoin network. Paths are not reliable—nodes come and go—and so the node must continue to discover new nodes as it loses old connections as well as assist other nodes when they bootstrap. Only one connection is needed to bootstrap, because the first node can offer introductions to its peer nodes and those peers can offer further introductions. It's also unnecessary and wasteful of network resources to connect to more than a handful of nodes. After bootstrapping, a node will remember its most recent successful peer connections, so that if it is rebooted it can quickly reestablish connections with its former peer network. If none of the former peers respond to its connection request, the node can use the seed nodes to bootstrap again.
On a node running the Bitcoin Core client, you can list the peer connections with the command((("getpeerinfo command"))) +getpeerinfo+:
[source,bash]
----
$ bitcoin-cli getpeerinfo
----
[source,json]
----
[
{
"addr" : "85.213.199.39:8333",
"services" : "00000001",
"lastsend" : 1405634126,
"lastrecv" : 1405634127,
"bytessent" : 23487651,
"bytesrecv" : 138679099,
"conntime" : 1405021768,
"pingtime" : 0.00000000,
"version" : 70002,
"subver" : "/Satoshi:0.9.2.1/",
"inbound" : false,
"startingheight" : 310131,
"banscore" : 0,
"syncnode" : true
},
{
"addr" : "58.23.244.20:8333",
"services" : "00000001",
"lastsend" : 1405634127,
"lastrecv" : 1405634124,
"bytessent" : 4460918,
"bytesrecv" : 8903575,
"conntime" : 1405559628,
"pingtime" : 0.00000000,
"version" : 70001,
"subver" : "/Satoshi:0.8.6/",
"inbound" : false,
"startingheight" : 311074,
"banscore" : 0,
"syncnode" : false
}
]
----
((("peer-to-peer networks","automatic management, overriding")))To override the automatic management of peers and to specify a list of IP addresses, users can provide the option +-connect=<IPAddress>+ and specify one or more IP addresses. If this option is used, the node will only connect to the selected IP addresses, instead of discovering and maintaining the peer connections automatically.
If there is no traffic on a connection, nodes will periodically send a message to maintain the connection. If a node has not communicated on a connection for more than 90 minutes, it is assumed to be disconnected and a new peer will be sought. Thus, the network dynamically adjusts to transient nodes and network problems, and can organically grow and shrink as needed without any central control.(((range="endofrange", startref="ix_ch06-asciidoc4")))(((range="endofrange", startref="ix_ch06-asciidoc3")))(((range="endofrange", startref="ix_ch06-asciidoc2")))(((range="endofrange", startref="ix_ch06-asciidoc1")))
=== Full Nodes
((("blockchains","full nodes and")))((("full nodes")))((("nodes","full")))Full nodes are nodes that maintain a full blockchain with all transactions. More accurately, they probably should be called "full blockchain nodes." In the early years of bitcoin, all nodes were full nodes and currently the Bitcoin Core client is a full blockchain node. In the past two years, however, new forms of bitcoin clients have been introduced that do not maintain a full blockchain but run as lightweight clients. We'll examine these in more detail in the next section.
((("blockchains","on full nodes")))Full blockchain nodes maintain a complete and up-to-date copy of the bitcoin blockchain with all the transactions, which they independently build and verify, starting with the very first block (genesis block) and building up to the latest known block in the network. A full blockchain node can independently and authoritatively verify any transaction without recourse or reliance on any other node or source of information. The full blockchain node relies on the network to receive updates about new blocks of transactions, which it then verifies and incorporates into its local copy of the blockchain.
Running a full blockchain node gives you the pure bitcoin experience: independent verification of all transactions without the need to rely on, or trust, any other systems. It's easy to tell if you're running a full node because it requires 20+ gigabytes of persistent storage (disk space) to store the full blockchain. If you need a lot of disk and it takes two to three days to sync to the network, you are running a full node. That is the price of complete independence and freedom from central authority.
There are a few alternative implementations of full blockchain bitcoin clients, built using different programming languages and software architectures. However, the most common implementation is the reference client((("Bitcoin Core client","and full nodes"))) Bitcoin Core, also known as the Satoshi client. More than 90% of the nodes on the bitcoin network run various versions of Bitcoin Core. It is identified as "Satoshi" in the sub-version string sent in the +version+ message and shown by the command +getpeerinfo+ as we saw earlier; for example, +/Satoshi:0.8.6/+.
=== Exchanging "Inventory"
((("blockchains","creating on nodes")))((("blockchains","on new nodes")))((("blocks","on new nodes")))((("full nodes","creating full blockchains on")))The first thing a full node will do once it connects to peers is try to construct a complete blockchain. If it is a brand-new node and has no blockchain at all, it only knows one block, the genesis block, which is statically embedded in the client software. Starting with block #0 (the genesis block), the new node will have to download hundreds of thousands of blocks to synchronize with the network and re-establish the full blockchain.
((("syncing the blockchain")))The process of syncing the blockchain starts with the +version+ message, because that contains +BestHeight+, a node's current blockchain height (number of blocks). A node will see the +version+ messages from its peers, know how many blocks they each have, and be able to compare to how many blocks it has in its own blockchain. Peered nodes will exchange a%605.420%%% +getblocks+ message that contains the hash (fingerprint) of the top block on their local blockchain. One of the peers will be able to identify the received hash as belonging to a block that is not at the top, but rather belongs to an older block, thus deducing that its own local blockchain is longer than its peer's.
The peer that has the longer blockchain has more blocks than the other node and can identify which blocks the other node needs in order to "catch up." It will identify the first 500 blocks to share and transmit their hashes using an((("inv messages"))) +inv+ (inventory) message. The node missing these blocks will then retrieve them, by issuing a series of +getdata+ messages requesting the full block data and identifying the requested blocks using the hashes from the +inv+ message.
Let's assume, for example, that a node only has the genesis block. It will then receive an +inv+ message from its peers containing the hashes of the next 500 blocks in the chain. It will start requesting blocks from all of its connected peers, spreading the load and ensuring that it doesn't overwhelm any peer with requests. The node keeps track of how many blocks are "in transit" per peer connection, meaning blocks that it has requested but not received, checking that it does not exceed a limit((("MAX_BLOCKS_IN_TRANSIT_PER_PEER constant"))) (+MAX_BLOCKS_IN_TRANSIT_PER_PEER+). This way, if it needs a lot of blocks, it will only request new ones as previous requests are fulfilled, allowing the peers to control the pace of updates and not overwhelming the network. As each block is received, it is added to the blockchain, as we will see in <<blockchain>>. As the local blockchain is gradually built up, more blocks are requested and received, and the process continues until the node catches up to the rest of the network.
This process of comparing the local blockchain with the peers and retrieving any missing blocks happens any time a node goes offline for any period of time. Whether a node has been offline for a few minutes and is missing a few blocks, or a month and is missing a few thousand blocks, it starts by sending +getblocks+, gets an +inv+ response, and starts downloading the missing blocks. <<inventory_synchronization>> shows the inventory and block propagation protocol.
[[spv_nodes]]
=== Simplified Payment Verification (SPV) Nodes
((("nodes","SPV", id="ix_ch06-asciidoc5", range="startofrange")))((("nodes","lightweight", id="ix_ch06-asciidoc5a", range="startofrange")))((("simplified payment verification (SPV) nodes", id="ix_ch06-asciidoc6", range="startofrange")))Not all nodes have the ability to store the full blockchain. Many bitcoin clients are designed to run on space- and power-constrained devices, such as smartphones, tablets, or embedded systems. For such devices, a _simplified payment verification_ (SPV) method is used to allow them to operate without storing the full blockchain. These types of clients are called SPV clients or lightweight clients. As bitcoin adoption surges, the SPV node is becoming the most common form of bitcoin node, especially for bitcoin wallets.
((("blockchains","on SPV nodes")))SPV nodes download only the block headers and do not download the transactions included in each block. The resulting chain of blocks, without transactions, is 1,000 times smaller than the full blockchain. SPV nodes cannot construct a full picture of all the UTXOs that are available for spending because they do not know about all the transactions on the network. SPV nodes verify transactions using a slightly different methodology that relies on peers to provide partial views of relevant parts of the blockchain on demand.
[[inventory_synchronization]]
.Node synchronizing the blockchain by retrieving blocks from a peer
image::images/msbt_0606.png["InventorySynchronization"]
As an analogy, a full node is like a tourist in a strange city, equipped with a detailed map of every street and every address. By comparison, an SPV node is like a tourist in a strange city asking random strangers for turn-by-turn directions while knowing only one main avenue. Although both tourists can verify the existence of a street by visiting it, the tourist without a map doesn't know what lies down any of the side streets and doesn't know what other streets exist. Positioned in front of 23 Church Street, the tourist without a map cannot know if there are a dozen other "23 Church Street" addresses in the city and whether this is the right one. The mapless tourist's best chance is to ask enough people and hope some of them are not trying to mug him.
Simplified payment verification verifies transactions by reference to their _depth_ in the blockchain instead of their _height_. Whereas a full blockchain node will construct a fully verified chain of thousands of blocks and transactions reaching down the blockchain (back in time) all the way to the genesis block, an SPV node will verify the chain of all blocks (but not all transactions) and link that chain to the transaction of interest.
For example, when examining a transaction in block 300,000, a full node links all 300,000 blocks down to the genesis block and builds a full database of UTXO, establishing the validity of the transaction by confirming that the UTXO remains unspent. An SPV node cannot validate whether the UTXO is unspent. Instead, the SPV node will establish a link between the transaction and the block that contains it, using a((("merkle trees","SPV and"))) _merkle path_ (see <<merkle_trees>>). Then, the SPV node waits until it sees the six blocks 300,001 through 300,006 piled on top of the block containing the transaction and verifies it by establishing its depth under blocks 300,006 to 300,001. The fact that other nodes on the network accepted block 300,000 and then did the necessary work to produce six more blocks on top of it is proof, by proxy, that the transaction was not a double-spend.
An SPV node cannot be persuaded that a transaction exists in a block when the transaction does not in fact exist. The SPV node establishes the existence of a transaction in a block by requesting a merkle path proof and by validating the proof of work in the chain of blocks. However, a transaction's existence can be "hidden" from an SPV node. An SPV node can definitely prove that a transaction exists but cannot verify that a transaction, such as a double-spend of the same UTXO, doesn't exist because it doesn't have a record of all transactions. This vulnerability can be used in a denial-of-service attack or for a double-spending attack against SPV nodes. To defend against this, an SPV node needs to connect randomly to several nodes, to increase the probability that it is in contact with at least one honest node. This need to randomly connect means that SPV nodes also are vulnerable to network partitioning attacks or Sybil attacks, where they are connected to fake nodes or fake networks and do not have access to honest nodes or the real bitcoin network.
For most practical purposes, well-connected SPV nodes are secure enough, striking the right balance between resource needs, practicality, and security. For infallible security, however, nothing beats running a full blockchain node.
[TIP]
====
((("simplified payment verification (SPV) nodes","verification")))A full blockchain node verifies a transaction by checking the entire chain of thousands of blocks below it in order to guarantee that the UTXO is not spent, whereas an SPV node checks how deep the block is buried by a handful of blocks above it.
====
((("block headers","getting on SPV nodes")))To get the block headers, SPV nodes use a((("getheaders message"))) +getheaders+ message instead of +getblocks+. The responding peer will send up to 2,000 block headers using a single +headers+ message. The process is otherwise the same as that used by a full node to retrieve full blocks. SPV nodes also set a filter on the connection to peers, to filter the stream of future blocks and transactions sent by the peers. Any transactions of interest are retrieved using a +getdata+ request. The peer generates a((("tx messages"))) +tx+ message containing the transactions, in response. <<spv_synchronization>> shows the synchronization of block headers.
[[spv_synchronization]]
.SPV node synchronizing the block headers
image::images/msbt_0607.png["SPVSynchronization"]
Because SPV nodes need to retrieve specific transactions in order to selectively verify them, they also create a privacy risk. Unlike full blockchain nodes, which collect all transactions within each block, the SPV node's requests for specific data can inadvertently reveal the addresses in their wallet. For example, a third party monitoring a network could keep track of all the transactions requested by a wallet on an SPV node and use those to associate bitcoin addresses with the user of that wallet, destroying the user's privacy.
Shortly after the introduction of SPV/lightweight nodes, the bitcoin developers added a feature called _bloom filters_ to address the privacy risks of SPV nodes. Bloom filters allow SPV nodes to receive a subset of the transactions without revealing precisely which addresses they are interested in, through a filtering mechanism that uses probabilities rather than fixed patterns.(((range="endofrange", startref="ix_ch06-asciidoc6")))(((range="endofrange", startref="ix_ch06-asciidoc5a")))(((range="endofrange", startref="ix_ch06-asciidoc5")))
=== Bloom Filters
((("bitcoin network","bloom filters and", id="ix_ch06-asciidoc7", range="startofrange")))((("bloom filters", id="ix_ch06-asciidoc8", range="startofrange")))((("Simplified Payment Verification (SPV) nodes","bloom filters and", id="ix_ch06-asciidoc9", range="startofrange")))A bloom filter is a probabilistic search filter, a way to describe a desired pattern without specifying it exactly. Bloom filters offer an efficient way to express a search pattern while protecting privacy. They are used by SPV nodes to ask their peers for transactions matching a specific pattern, without revealing exactly which addresses they are searching for.
In our previous analogy, a tourist without a map is asking for directions to a specific address, "23 Church St." If she asks strangers for directions to this street, she inadvertently reveals her destination. A bloom filter is like asking, "Are there any streets in this neighborhood whose name ends in R-C-H?" A question like that reveals slightly less about the desired destination than asking for "23 Church St." Using this technique, a tourist could specify the desired address in more detail as "ending in U-R-C-H" or less detail as "ending in H." By varying the precision of the search, the tourist reveals more or less information, at the expense of getting more or less specific results. If she asks a less specific pattern, she gets a lot more possible addresses and better privacy, but many of the results are irrelevant. If she asks for a very specific pattern, she gets fewer results but loses privacy.
Bloom filters serve this function by allowing an SPV node to specify a search pattern for transactions that can be tuned toward precision or privacy. A more specific bloom filter will produce accurate results, but at the expense of revealing what addresses are used in the user's wallet. A less specific bloom filter will produce more data about more transactions, many irrelevant to the node, but will allow the node to maintain better privacy.
An SPV node will initialize a bloom filter as "empty" and in that state the bloom filter will not match any patterns. The SPV node will then make a list of all the addresses in its wallet and create a search pattern matching the transaction output that corresponds to each address. Usually, the search pattern is a((("pay-to-public-key-hash (P2PKH)","bloom filters and"))) pay-to-public-key-hash script that is the expected locking script that will be present in any transaction paying to the public-key-hash (address). If the SPV node is tracking the balance of a((("pay-to-script-hash (P2SH)","bloom filters and"))) P2SH address, the search pattern will be a pay-to-script-hash script, instead. The SPV node then adds each of the search patterns to the bloom filter, so that the bloom filter can recognize the search pattern if it is present in a transaction. Finally, the bloom filter is sent to the peer and the peer uses it to match transactions for transmission to the SPV node.
Bloom filters are implemented as a variable-size array of N binary digits (a bit field) and a variable number of M hash functions. The hash functions are designed to always produce an output that is between 1 and N, corresponding to the array of binary digits. The hash functions are generated deterministically, so that any node implementing a bloom filter will always use the same hash functions and get the same results for a specific input. By choosing different length (N) bloom filters and a different number (M) of hash functions, the bloom filter can be tuned, varying the level of accuracy and therefore privacy.
In <<bloom1>>, we use a very small array of 16 bits and a set of three hash functions to demonstrate how bloom filters work.
[[bloom1]]
.An example of a simplistic bloom filter, with a 16-bit field and three hash functions
image::images/msbt_0608.png["Bloom1"]
The bloom filter is initialized so that the array of bits is all zeros. To add a pattern to the bloom filter, the pattern is hashed by each hash function in turn. Applying the first hash function to the input results in a number between 1 and N. The corresponding bit in the array (indexed from 1 to N) is found and set to +1+, thereby recording the output of the hash function. Then, the next hash function is used to set another bit and so on. Once all M hash functions have been applied, the search pattern will be "recorded" in the bloom filter as M bits that have been changed from +0+ to +1+.
<<bloom2>> is an example of adding a pattern "A" to the simple bloom filter shown in <<bloom1>>.
Adding a second pattern is as simple as repeating this process. The pattern is hashed by each hash function in turn and the result is recorded by setting the bits to +1+. Note that as a bloom filter is filled with more patterns, a hash function result might coincide with a bit that is already set to +1+, in which case the bit is not changed. In essence, as more patterns record on overlapping bits, the bloom filter starts to become saturated with more bits set to +1+ and the accuracy of the filter decreases. This is why the filter is a probabilistic data structure—it gets less accurate as more patterns are added. The accuracy depends on the number of patterns added versus the size of the bit array (N) and number of hash functions (M). A larger bit array and more hash functions can record more patterns with higher accuracy. A smaller bit array or fewer hash functions will record fewer patterns and produce less accuracy.
[[bloom2]]
.Adding a pattern "A" to our simple bloom filter
image::images/msbt_0609.png["Bloom2"]
<<bloom3>> is an example of adding a second pattern "B" to the simple bloom filter.
[[bloom3]]
.Adding a second pattern "B" to our simple bloom filter
image::images/msbt_0610.png["Bloom3"]
To test if a pattern is part of a bloom filter, the pattern is hashed by each hash function and the resulting bit pattern is tested against the bit array. If all the bits indexed by the hash functions are set to +1+, then the pattern is _probably_ recorded in the bloom filter. Because the bits may be set because of overlap from multiple patterns, the answer is not certain, but is rather probabilistic. In simple terms, a bloom filter positive match is a "Maybe, Yes."
<<bloom4>> is an example of testing the existence of pattern "X" in the simple bloom filter. The corresponding bits are set to +1+, so the pattern is probably a match.
[[bloom4]]
.Testing the existence of pattern "X" in the bloom filter. The result is probabilistic positive match, meaning "Maybe."
image::images/msbt_0611.png["Bloom4"]
On the contrary, if a pattern is tested against the bloom filter and any one of the bits is set to +0+, this proves that the pattern was not recorded in the bloom filter. A negative result is not a probability, it is a certainty. In simple terms, a negative match on a bloom filter is a "Definitely Not!"
<<bloom5>> is an example of testing the existence of pattern "Y" in the simple bloom filter. One of the corresponding bits is set to +0+, so the pattern is definitely not a match.
[[bloom5]]
.Testing the existence of pattern "Y" in the bloom filter. The result is a definitive negative match, meaning "Definitely Not!"
image::images/msbt_0612.png[]
Bitcoin's implementation of bloom filters is described in Bitcoin Improvement Proposal 37 (BIP0037). See <<appdxbitcoinimpproposals>> or visit http://bit.ly/1x6qCiO[GitHub].
=== Bloom Filters and Inventory Updates
((("inventory updates, bloom filters and")))Bloom filters are used to filter the transactions (and blocks containing them) that an SPV node receives from its peers. SPV nodes will create a filter that matches only the addresses held in the SPV node's wallet. The SPV node will then send a((("filterload message"))) +filterload+ message to the peer, containing the bloom filter to use on the connection. After a filter is established, the peer will then test each transaction's outputs against the bloom filter. Only transactions that match the filter are sent to the node.
In response to a +getdata+ message from the node, peers will send a +merkleblock+ message that contains only block headers for blocks matching the filter and a merkle path (see <<merkle_trees>>) for each matching transaction. The peer will then also send +tx+ messages containing the transactions matched by the filter.
The node setting the bloom filter can interactively add patterns to the filter by sending a((("filteradd message"))) +filteradd+ message. To clear the bloom filter, the node can send a((("filterclear message"))) +filterclear+ message. Because it is not possible to remove a pattern from a bloom filter, a node has to clear and resend a new bloom filter if a pattern is no longer desired.(((range="endofrange", startref="ix_ch06-asciidoc9")))(((range="endofrange", startref="ix_ch06-asciidoc8")))(((range="endofrange", startref="ix_ch06-asciidoc7")))
[[transaction_pools]]
=== Transaction Pools
((("bitcoin network","transaction pools")))((("transaction pools")))((("transactions","unconfirmed, pools of")))((("unconfirmed transactions")))Almost every node on the bitcoin network maintains a temporary list of unconfirmed transactions called the _memory pool_, or _transaction pool_. Nodes use this pool to keep track of transactions that are known to the network but are not yet included in the blockchain. For example, a node that holds a user's wallet will use the transaction pool to track incoming payments to the user's wallet that have been received on the network but are not yet confirmed.
As transactions are received and verified, they are added to the transaction pool and relayed to the neighboring nodes to propagate on the network.
((("orphan transaction pool")))Some node implementations also maintain a separate pool of orphaned transactions. If a transaction's inputs refer to a transaction that is not yet known, such as a missing parent, the orphan transaction will be stored temporarily in the orphan pool until the parent transaction arrives.
When a transaction is added to the transaction pool, the orphan pool is checked for any orphans that reference this transaction's outputs (its children). Any matching orphans are then validated. If valid, they are removed from the orphan pool and added to the transaction pool, completing the chain that started with the parent transaction. In light of the newly added transaction, which is no longer an orphan, the process is repeated recursively looking for any further descendants, until no more descendants are found. Through this process, the arrival of a parent transaction triggers a cascade reconstruction of an entire chain of interdependent transactions by re-uniting the orphans with their parents all the way down the chain.
((("orphan transaction pool","storage")))((("transaction pools","storage")))Both the transaction pool and orphan pool (where implemented) are stored in local memory and are not saved on persistent storage; rather, they are dynamically populated from incoming network messages. When a node starts, both pools are empty and are gradually populated with new transactions received on the network.
Some implementations of the bitcoin client also maintain a UTXO database or UTXO pool, which is the set of all unspent outputs on the blockchain. Although the name "UTXO pool" sounds similar to the transaction pool, it represents a different set of data. Unlike the transaction and orphan pools, the UTXO pool is not initialized empty but instead contains millions of entries of unspent transaction outputs, including some dating back to 2009. The UTXO pool may be housed in local memory or as an indexed database table on persistent storage.
Whereas the transaction and orphan pools represent a single node's local perspective and might vary significantly from node to node depending upon when the node was started or restarted, the UTXO pool represents the emergent consensus of the network and therefore will vary little between nodes. Furthermore, the transaction and orphan pools only contain unconfirmed transactions, while the UTXO pool only contains confirmed outputs.
=== Alert Messages
((("alert messages")))((("bitcoin network","alert messages")))Alert messages are a seldom used function, but are nevertheless implemented in most nodes. Alert messages are bitcoin's "emergency broadcast system," a means by which the core bitcoin developers can send an emergency text message to all bitcoin nodes. This feature is implemented to allow the core developer team to notify all bitcoin users of a serious problem in the bitcoin network, such as a critical bug that requires user action. The alert system has only been used a handful of times, most notably in early 2013 when a critical database bug caused a multiblock fork to occur in the bitcoin blockchain.
Alert messages are propagated by the +alert+ message. The alert message contains several fields, including:
ID::
An alert identified so that duplicate alerts can be detected
Expiration::
A time after which the alert expires
RelayUntil::
A time after which the alert should not be relayed
MinVer, MaxVer::
The range of bitcoin protocol versions that this alert applies to
subVer::
The client software version that this alert applies to
Priority::
An alert priority level, currently unused
Alerts are cryptographically signed by a public key. The corresponding private key is held by a few select members of the core development team. The digital signature ensures that fake alerts will not be propagated on the network.
Each node receiving this alert message will verify it, check for expiration, and propagate it to all its peers, thus ensuring rapid propagation across the entire network. In addition to propagating the alert, the nodes might implement a user interface function to present the alert to the user.
((("Bitcoin Core client","alerts, configuring")))In the Bitcoin Core client, the alert is configured with the command-line option +-alertnotify+, which specifies a command to run when an alert is received. The alert message is passed as a parameter to the +alertnotify+ command. Most commonly, the +alertnotify+ command is set to generate an email message to the administrator of the node, containing the alert message. The alert is also displayed as a pop-up dialog in the graphical user interface (bitcoin-Qt) if it is running.
Other implementations of the bitcoin protocol might handle the alert in different ways. ((("mining","hardware, alerts and")))Many hardware-embedded bitcoin mining systems do not implement the alert message function because they have no user interface. It is strongly recommended that miners running such mining systems subscribe to alerts via a mining pool operator or by running a lightweight node just for alert purposes.(((range="endofrange", startref="ix_ch06-asciidoc0")))

505
ch07.asciidoc
Unescape View File

@ -1,249 +1,256 @@
[[ch7]]
[[blockchain]]
== The Blockchain
=== Introduction
The blockchain data structure is an ordered back-linked list of blocks of transactions. The blockchain can be stored as a flat file, or in a simple database. The bitcoin core client stores the blockchain metadata using Google's LevelDB database. Blocks are linked "back", each referring to the previous block in the chain. The blockchain is often visualized as a vertical stack, with blocks layered on top of each other and the first block ever serving as the foundation of the stack. The visualization of blocks stacked on top of each other results in the use of terms like "height" to refer to the distance from the first block, and "top" or "tip" to refer to the most recently added block.
Each block within the blockchain is identified by a hash, generated using the SHA256 cryptographic hash algorithm on the header of the block. Each block also references a previous block, known as the _parent_ block, through the "previous block hash" field in the block header. In other words, each block contains the hash of its parent inside its own header. The sequence of hashes linking each block to its parent, creates a chain going back all the way to the first block ever created, known as the _genesis block_.
While a block has just one parent, it can temporarily have multiple children. Each of the children refers to the same (parent) block and contains the same (parent) hash in the "previous block hash" field. Multiple children arise during a blockchain "fork", a temporary situation that occurs when different blocks are discovered almost simultaneously by different miners (see <<forks>>). Eventually, only one child block becomes part of the blockchain and the "fork" is resolved. Even though a block may have more than one child, each block can have only one parent. This is because a block has one single "previous block hash" field referencing its single parent.
The "previous block hash" field is inside the block header and thereby affects the _current_ block's hash. The child's own identity changes if the parent's identity changes. When the parent is modified in any way, the parent's hash changes. The parent's changed hash necessitates a change in the "previous block hash" pointer of the child. This in turn causes the child's hash to change, which requires a change in the pointer of the grandchild, which in turn changes the grandchild and so on. This cascade effect ensures that once a block has many generations following it, it cannot be changed without forcing a recalculation of all subsequent blocks. Because such a recalculation would require enormous computation, the existence of a long chain of blocks makes the blockchain's deep history immutable, a key feature of bitcoin's security.
One way to think about the blockchain is like layers in a geological formation, or glacier core sample. The surface layers may change with the seasons, or even be blown away before they have time to settle. But once you go a few inches deep, geological layers become more and more stable. By the time you look a few hundred feet down, you are looking at a snapshot of the past that has remained undisturbed for millennia or millions of years. In the blockchain, the most recent few blocks may be revised if there is a chain recalculation due to a fork. The top six blocks are like a few inches of topsoil. But once you go deeper into the blockchain, beyond 6 blocks, blocks are less and less likely to change. After 100 blocks back there is so much stability that the "coinbase" transaction, the transaction containing newly-mined bitcoins, can be spent. A few thousand blocks back (a month) and the blockchain is settled history. It will never change.
=== Structure of a Block
A block is a container data structure that aggregates transactions for inclusion in the public ledger, the blockchain. The block is made of a header, containing metadata, followed by a long list of transactions that make up the bulk of its size. The block header is 80 bytes, whereas the average transaction is at least 250 bytes and the average block contains more than 500 transactions. A complete block, with all transactions, is therefore 1000 times larger than the block header.
[[block_structure]]
.The structure of a block
[options="header"]
|=======
|Size| Field | Description
| 4 bytes | Block Size | The size of the block, in bytes, following this field
| 80 bytes | Block Header | Several fields form the block header (see below)
| 1-9 bytes (VarInt) | Transaction Counter | How many transactions follow
| Variable | Transactions | The transactions recorded in this block
|=======
[[block_header]]
=== Block Header
The block header consists of three sets of block metadata. First, there is a reference to a previous block hash, which connects this block to the previous block in the blockchain. We will examine this in more detail in <<blockchain>>. The second set of metadata, namely the difficulty, timestamp and nonce, relate to the mining competition, as detailed in <<mining>>. The third piece of metadata is the Merkle Tree root, a data structure used to efficiently summarize all the transactions in the block.
[[block_header_structure]]
.The structure of the block header
[options="header"]
|=======
|Size| Field | Description
| 4 bytes | Version | A version number to track software/protocol upgrades
| 32 bytes | Previous Block Hash | A reference to the hash of the previous (parent) block in the chain
| 32 bytes | Merkle Root | A hash of the root of the Merkle-Tree of this block's transactions
| 4 bytes | Timestamp | The approximate creation time of this block (seconds from Unix Epoch)
| 4 bytes | Difficulty Target | The proof-of-work algorithm difficulty target for this block
| 4 bytes | Nonce | A counter used for the proof-of-work algorithm
|=======
The Nonce, Difficulty Target, and Timestamp are used in the mining process and will be discussed in more detail in <<mining>>.
[[block_hash]]
=== Block Identifiers - Block Header Hash and Block Height
The primary identifier of a block is its cryptographic hash, a digital fingerprint, made by hashing the block header twice through the SHA256 algorithm. The resulting 32-byte hash, is called the _block hash_, but is more accurately the _block *header* hash_, as only the block header is used to compute it. For example, the block hash of the first bitcoin block ever created is +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+. The block hash identifies a block uniquely and unambiguously and can be independently derived by any node by simply hashing the block header.
Note that the block hash is not actually included inside the block's data structure, neither when the block is transmitted on the network, nor when it is stored on a node's persistence storage as part of the blockchain. Instead, the block's hash is computed by each node as the block is received from the network. The block hash may be stored in a separate database table as part of the block's metadata, to facilitate indexing and faster retrieval of blocks from disk.
A second way to identify a block is by its position in the blockchain, called the _block height_. The first block ever created is at block height 0 (zero) and is the same block that was referenced by the block hash +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+ above. A block can thus be identified two ways, either by referencing the block hash, or by referencing the block height. Each subsequent block added "on top" of that first block is one position "higher" in the blockchain, like boxes stacked one on top of the other. The block height on January 1st 2014 was approximately 278,000, meaning there were 278,000 blocks stacked on top of the first block created in January 2009.
Unlike the block hash, the block height is not a unique identifier. While a single block will always have a specific and invariant block height, the reverse is not true - the block height does not always identify a single block. Two or more blocks may have the same block height, competing for the same position in the blockchain. This scenario is discussed in detail in the section on <<forks>>. The block height is also not a part of the block's data structure; it is not stored within the block. Each node dynamically identifies a block's position (height) in the blockchain when it is received from the bitcoin network. The block height may also be stored as metadata in an indexed database table for faster retrieval.
[TIP]
====
A block's _block hash_ always identifies a single block uniquely. A block also always has a specific _block height_. However, it is not always the case that a specific block height can identify a single block. Rather, two or more blocks may compete for a single position in the blockchain.
====
=== The Genesis Block
The first block in the blockchain is called the _genesis block_ and was created in 2009. It is the "common ancestor" of all the blocks in the blockchain, meaning that if you start at any block and follow the chain backwards in time, you will eventually arrive at the _genesis block_.
Every node always starts with a blockchain of at least one block because the genesis block is statically encoded within the bitcoin client software, such that it cannot be altered. Every node always "knows" the genesis block's hash and structure, the fixed time it was created and even the single transaction within. Thus, every node has the starting point for the blockchain, a secure "root" from which to build a trusted blockchain.
See the statically encoded genesis block inside the Bitcoin Core client, in chainparams.cpp:
https://github.com/bitcoin/bitcoin/blob/3955c3940eff83518c186facfec6f50545b5aab5/src/chainparams.cpp#L123
The genesis block has the identifier hash +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+. You can search for that block hash in any block explorer website, such as blockchain.info, and you will find a page describing the contents of this block, with a URL containing that hash:
https://blockchain.info/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
https://blockexplorer.com/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
Using the Bitcoin Core reference client on the command-line:
[source,bash]
----
$ bitcoin-cli getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
----
[source,json]
----
{
"hash" : "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
"confirmations" : 308321,
"size" : 285,
"height" : 0,
"version" : 1,
"merkleroot" : "4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b",
"tx" : [
"4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b"
],
"time" : 1231006505,
"nonce" : 2083236893,
"bits" : "1d00ffff",
"difficulty" : 1.00000000,
"nextblockhash" : "00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048"
}
----
The genesis block contains a hidden message within it. The coinbase transaction input contains the text "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks". This message provides proof of the earliest date this block was created, by referencing the headline of the british newspaper _The Times_. It also serves as a tongue-in-cheek reminder of the importance of an independent monetary system, with bitcoin's launch occurring at the same time as an unprecedented worldwide monetary crisis. The message was embedded in the first block by Satoshi Nakamoto, bitcoin's creator.
=== Linking Blocks in the Blockchain
Bitcoin nodes maintain a local copy of the blockchain, starting at the genesis block. The local copy of the blockchain is constantly updated as new blocks are found and used to extend the chain. As a node receives incoming blocks from the network, it will validate these blocks and then link them to the existing blockchain. To establish a link, a node will examine the incoming block header and look for the "previous block hash".
Let's assume, for example, that a node has 277,314 blocks in the local copy of the blockchain. The last block the node knows about is block 277,314, with a block header hash of +00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249+.
The bitcoin node then receives a new block from the network, which it parses as follows:
[source,json]
----
{
"size" : 43560,
"version" : 2,
"previousblockhash" :
"00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249",
"merkleroot" :
"5e049f4030e0ab2debb92378f53c0a6e09548aea083f3ab25e1d94ea1155e29d",
"time" : 1388185038,
"difficulty" : 1180923195.25802612,
"nonce" : 4215469401,
"tx" : [
"257e7497fb8bc68421eb2c7b699dbab234831600e7352f0d9e6522c7cf3f6c77",
#[... many more transactions omitted ...]
"05cfd38f6ae6aa83674cc99e4d75a1458c165b7ab84725eda41d018a09176634"
]
}
----
Looking at this new block, the node finds the "previousblockhash" field, which contains the hash of its parent block. It is a hash known to the node, that of the last block on the chain at height 277,314. Therefore, this new block is a child of the last block on the chain and extends the existing blockchain. The node adds this new block to the end of the chain, making the blockchain longer with a new height of 277,315.
[[chain_of_blocks]]
.Blocks linked in a chain, by reference to the previous block header hash
image::images/ChainOfBlocks.png["chain_of_blocks"]
[[merkle_trees]]
=== Merkle Trees
Each block in the bitcoin blockchain contains a summary of all the transactions in the block, using a _Merkle Tree_.
A _Merkle Tree_, also known as a _Binary Hash Tree_ is a data structure used for efficiently summarizing and verifying the integrity of large sets of data. Merkle Trees are binary trees containing cryptographic hashes. The term "tree" is used in computer science to describe a branching data structure, but these trees are usually displayed upside down with the "root" at the top and the "leaves" at the bottom of a diagram, as you will see in the examples that follow.
Merkle trees are used in bitcoin to summarize all the transactions in a block, producing an overall digital fingerprint of the entire set of transactions, providing a very efficient process to verify if a transaction is included in a block. A merkle tree is constructed by recursively hashing pairs of nodes until there is only one hash, called the _root_, or _merkle root_. The cryptographic hash algorithm used in bitcoin's merkle trees is SHA256 applied twice, also known as double-SHA256.
When N data elements are hashed and summarized in a Merkle Tree, you can check to see if any one data element is included in the tree with at most +2*log~2~(N)+ calculations, making this a very efficient data structure.
The merkle tree is constructed bottom-up. In the example below, we start with four transactions A, B, C and D, which form the _leaves_ of the Merkle Tree, shown in the diagram at the bottom. The transactions are not stored in the merkle tree, rather their data is hashed and the resulting hash is stored in each leaf node as H~A~, H~B~, H~C~ and H~D~:
+H~A~ = SHA256(SHA256(Transaction A))+
Consecutive pairs of leaf nodes are then summarized in a parent node, by concatenating the two hashes and hashing them together. For example, to construct the parent node H~AB~, the two 32-byte hashes of the children are concatenated to create a 64-byte string. That string is then double-hashed to produce the parent node's hash:
+H~AB~ = SHA256(SHA256(H~A~ + H~B~))+
The process continues until there is only one node at the top, the node known as the Merkle Root. That 32-byte hash is stored in the block header and summarizes all the data in all four transactions.
[[simple_merkle]]
.Calculating the nodes in a Merkle Tree
image::images/MerkleTree.png["merkle_tree"]
The merkle tree is a binary tree with an even number of leaf nodes. If there is an odd number of transactions to summarize, the last transaction hash will be duplicated to create an even number of leaf nodes, also known as a _balanced tree_. This is shown in the example below, where transaction C is duplicated:
[[merkle_tree_odd]]
.An even number of data elements, by duplicating one data element
image::images/MerkleTreeOdd.png["merkle_tree_odd"]
The same method for constructing a tree from four transactions can be generalized to construct trees of any size. In bitcoin it is common to have several hundred to more than a thousand transactions in a single block, which are summarized in exactly the same way producing just 32 bytes of data as the single merkle root. In the diagram below, you will see a tree built from 16 transactions. Note that while the root looks bigger than the leaf nodes in the diagram, it is the exact same size, just 32 bytes. Whether there is one transaction or a hundred thousand transactions in the block, the merkle root always summarizes them into 32 bytes:
[[merkle_tree_large]]
.A Merkle Tree summarizing many data elements
image::images/MerkleTreeLarge.png["merkle_tree_large"]
To prove that a specific transaction is included in a block, a node only needs to produce +log~2~(N)+ 32-byte hashes, constituting an _authentication path_ or _merkle path_ connecting the specific transaction to the root of the tree. This is especially important as the number of transactions increases, because the base-2 logarithm of the number of transactions increases much more slowly. This allows bitcoin nodes to efficiently produce paths of ten or twelve hashes (320-384 bytes) which can provide proof of a single transaction out of more than a thousand transactions in a megabyte sized block. In the example below, a node can prove that a transaction K is included in the block by producing a merkle path that is only four 32-byte hashes long (128 bytes total). The path consists of the four hashes (noted in blue in the diagram below) H~L~, H~IJ~, H~MNOP~ and H~ABCDEFGH~. With those four hashes provided as an authentication path, any node can prove that H~K~ (noted in green in the diagram below) is included in the merkle root by computing four additional pair-wise hashes H~KL~, H~IJKL~ and H~IJKLMNOP~ (outlined in a dotted line in the diagram below) that lead to the merkle root.
[[merkle_tree_path]]
.A Merkle Path used to prove inclusion of a data element
image::images/MerkleTreePathToK.png["merkle_tree_path"]
The code in <<merkle_example>> demonstrates the process of creating a merkle tree from the leaf-node hashes up to the root, using the libbitcoin library for some helper functions:
[[merkle_example]]
.Building a merkle tree
====
[source, cpp]
----
include::code/merkle.cpp[]
----
====
Compiling and running the merkle code:
[[merkle_example_run]]
.Compiling and running the merkle example code
====
[source,bash]
----
$ # Compile the merkle.cpp code
$ g++ -o merkle merkle.cpp $(pkg-config --cflags --libs libbitcoin)
$ # Run the merkle executable
$ ./merkle
Current merkle hash list:
32650049a0418e4380db0af81788635d8b65424d397170b8499cdc28c4d27006
30861db96905c8dc8b99398ca1cd5bd5b84ac3264a4e1b3e65afa1bcee7540c4
Current merkle hash list:
d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
Result: d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
----
====
The efficiency of merkle trees becomes obvious as the scale increases. For example, proving that a transaction is part of a block requires:
[[block_structure]]
.Merkle Tree Efficiency
[options="header"]
|=======
|Number of Transactions| Approx. Size of Block | Path Size (Hashes) | Path Size (Bytes)
| 16 transactions | 4 kilobytes | 4 hashes | 128 bytes
| 512 transactions | 128 kilobytes | 9 hashes | 288 bytes
| 2048 transactions | 512 kilobytes | 11 hashes | 352 bytes
| 65,535 transactions | 16 megabytes | 16 hashes | 512 bytes
|=======
As you can see from the table above, while the block size increases rapidly, from 4KB with 16 transactions to a block size of 16 MB to fit 65,535 transactions, the merkle path required to prove the inclusion of a transaction increases much more slowly, from 128 bytes to only 512 bytes. With merkle trees, a node can download just the block headers (80 bytes per block) and still be able to identify a transaction's inclusion in a block by retrieving a small merkle path from a full node, without storing or transmitting the vast majority of the blockchain which may be several gigabytes in size. Nodes which do not maintain a full blockchain, called Simplified Payment Verification or SPV nodes use merkle paths to verify transactions without downloading full blocks.
=== Merkle Trees and Simplified Payment Verification (SPV)
Merkle trees are used extensively by Simplified Payment Verification nodes. SPV nodes don't have all transactions and do not download full blocks, just block headers. In order to verify that a transaction is included in a block, without having to download all the transactions in the block, they use an _authentication path_, or merkle path.
Consider for example an SPV node that is interested in incoming payments to an address contained in its wallet. The SPV node will establish a bloom filter on its connections to peers to limit the transactions received to only those containing addresses of interest. When a peer sees a transaction that matches the bloom filter, it will send that block using a +merkleblock+ message. The +merkleblock+ message contains the block header as well as a merkle path that links the transaction of interest to the merkle root in the block. The SPV node can use this merkle path to connect the transaction to the block and verify that the transaction is included in the block. The SPV node also uses the block header to link the block to the rest of the blockchain. The combination of these two links, between the transaction and block, and between the block and blockchain, proves that the transaction is recorded in the blockchain. All in all, the SPV node will have received less than a kilobyte of data for the block header and merkle path, an amount of data that is more than a thousand times less than a full block (about 1 megabyte currently).
[[blockchain]]
== The Blockchain
=== Introduction
((("blockchains", id="ix_ch07-asciidoc0", range="startofrange")))The blockchain data structure is an ordered, back-linked list of blocks of transactions. The blockchain can be stored as a flat file, or in a simple database. The Bitcoin Core client stores the blockchain metadata using((("LevelDB database (Google)"))) Google's LevelDB database. Blocks are linked "back," each referring to the previous block in the chain. The blockchain is often visualized as a vertical stack, with blocks layered on top of each other and the first block serving as the foundation of the stack. The visualization of blocks stacked on top of each other results in the use of terms such as "height" to refer to the distance from the first block, and "top" or "tip" to refer to the most recently added block.
Each block within the blockchain is identified by a hash, generated using the SHA256 cryptographic hash algorithm on the header of the block. Each block also references a previous block, known as the((("parent blocks"))) _parent_ block, through the "previous block hash" field in the block header. In other words, each block contains the hash of its parent inside its own header. The sequence of hashes linking each block to its parent creates a chain going back all the way to the first block ever created, known as the((("genesis block"))) _genesis block_.
Although a block has just one parent, it can temporarily have multiple children. Each of the children refers to the same block as its parent and contains the same (parent) hash in the "previous block hash" field. Multiple children arise during a blockchain "fork," a temporary situation that occurs when different blocks are discovered almost simultaneously by different miners (see <<forks>>). Eventually, only one child block becomes part of the blockchain and the "fork" is resolved. Even though a block may have more than one child, each block can have only one parent. This is because a block has one single "previous block hash" field referencing its single parent.
The "previous block hash" field is inside the block header and thereby affects the _current_ block's hash. The child's own identity changes if the parent's identity changes. When the parent is modified in any way, the parent's hash changes. The parent's changed hash necessitates a change in the "previous block hash" pointer of the child. This in turn causes the child's hash to change, which requires a change in the pointer of the grandchild, which in turn changes the grandchild, and so on. ((("security","immutability of blockchain and")))This cascade effect ensures that once a block has many generations following it, it cannot be changed without forcing a recalculation of all subsequent blocks. Because such a recalculation would require enormous computation, the existence of a long chain of blocks makes the blockchain's deep history immutable, which is a key feature of bitcoin's security.
One way to think about the blockchain is like layers in a geological formation, or glacier core sample. The surface layers might change with the seasons, or even be blown away before they have time to settle. But once you go a few inches deep, geological layers become more and more stable. By the time you look a few hundred feet down, you are looking at a snapshot of the past that has remained undisturbed for millions of years. In the blockchain, the most recent few blocks might be revised if there is a chain recalculation due to a fork. The top six blocks are like a few inches of topsoil. But once you go more deeply into the blockchain, beyond six blocks, blocks are less and less likely to change. After 100 blocks back there is so much stability that the coinbase transaction—the transaction containing newly mined bitcoins—can be spent. A few thousand blocks back (a month) and the blockchain is settled history. It will never change.
=== Structure of a Block
((("blocks","structure of")))A block is a container data structure that aggregates transactions for inclusion in the public ledger, the blockchain. The block is made of a header, containing metadata, followed by a long list of transactions that make up the bulk of its size. The block header is 80 bytes, whereas the average transaction is at least 250 bytes and the average block contains more than 500 transactions. A complete block, with all transactions, is therefore 1,000 times larger than the block header. <<block_structure1>> describes the structure of a block.
[[block_structure1]]
.The structure of a block
[options="header"]
|=======
|Size| Field | Description
| 4 bytes | Block Size | The size of the block, in bytes, following this field
| 80 bytes | Block Header | Several fields form the block header
| 1-9 bytes (VarInt) | Transaction Counter | How many transactions follow
| Variable | Transactions | The transactions recorded in this block
|=======
[[block_header]]
=== Block Header
((("block headers")))((("blocks","headers")))The block header consists of three sets of block metadata. First, there is a reference to a previous block hash, which connects this block to the previous block in the blockchain. The second set of metadata, namely the((("difficulty target","in block header")))((("nonce,","in block header")))((("timestamping blocks","in block header"))) _difficulty_, _timestamp_, and _nonce_, relate to the mining competition, as detailed in <<ch8>>. The third piece of metadata is the merkle tree root, a data structure used to efficiently summarize all the transactions in the block. <<block_header_structure_ch07>> describes the structure of a block header.
[[block_header_structure_ch07]]
.The structure of the block header
[options="header"]
|=======
|Size| Field | Description
| 4 bytes | Version | A version number to track software/protocol upgrades
| 32 bytes | Previous Block Hash | A reference to the hash of the previous (parent) block in the chain
| 32 bytes | Merkle Root | A hash of the root of the merkle tree of this block's transactions
| 4 bytes | Timestamp | The approximate creation time of this block (seconds from Unix Epoch)
| 4 bytes | Difficulty Target | The proof-of-work algorithm difficulty target for this block
| 4 bytes | Nonce | A counter used for the proof-of-work algorithm
|=======
The nonce, difficulty target, and timestamp are used in the mining process and will be discussed in more detail in <<ch8>>.
[[block_hash]]
=== Block Identifiers: Block Header Hash and Block Height
((("blocks","header hash")))((("blocks","height")))((("blocks","identifiers")))The primary identifier of a block is its cryptographic hash, a digital fingerprint, made by hashing the block header twice through the SHA256 algorithm. The resulting 32-byte hash is called the((("block hash")))((("block header hash"))) _block hash_ but is more accurately the _block header hash_, pass:[<phrase role="keep-together">because only the block header is used to compute it. For example,</phrase>] +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+ is the block hash of the first bitcoin block ever created. The block hash identifies a block uniquely and unambiguously and can be independently derived by any node by simply hashing the block header.
Note that the block hash is not actually included inside the block's data structure, neither when the block is transmitted on the network, nor when it is stored on a node's persistence storage as part of the blockchain. Instead, the block's hash is computed by each node as the block is received from the network. The block hash might be stored in a separate database table as part of the block's metadata, to facilitate indexing and faster retrieval of blocks from disk.
A second way to identify a block is by its position in the blockchain, called the((("block height"))) pass:[<phrase role="keep-together"><emphasis>block height</emphasis>. The first block ever created is at block height 0 (zero) and is the</phrase>] pass:[<phrase role="keep-together">same block that was previously referenced by the following block hash</phrase>] +000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f+. A block can thus be identified two ways: by referencing the block hash or by referencing the block height. Each subsequent block added "on top" of that first block is one position "higher" in the blockchain, like boxes stacked one on top of the other. The block height on January 1, 2014, was approximately 278,000, meaning there were 278,000 blocks stacked on top of the first block created in January 2009.
Unlike the block hash, the block height is not a unique identifier. Although a single block will always have a specific and invariant block height, the reverse is not true—the block height does not always identify a single block. Two or more blocks might have the same block height, competing for the same position in the blockchain. This scenario is discussed in detail in the section <<forks>>. The block height is also not a part of the block's data structure; it is not stored within the block. Each node dynamically identifies a block's position (height) in the blockchain when it is received from the bitcoin network. The block height might also be stored as metadata in an indexed database table for faster retrieval.
[TIP]
====
A block's _block hash_ always identifies a single block uniquely. A block also always has a specific _block height_. However, it is not always the case that a specific block height can identify a single block. Rather, two or more blocks might compete for a single position in the blockchain.
====
=== The Genesis Block
((("blockchains","genesis block")))((("genesis block")))The first block in the blockchain is called the genesis block and was created in 2009. It is the common ancestor of all the blocks in the blockchain, meaning that if you start at any block and follow the chain backward in time, you will eventually arrive at the genesis block.
Every node always starts with a blockchain of at least one block because the genesis block is statically encoded within the bitcoin client software, such that it cannot be altered. Every node always "knows" the genesis block's hash and structure, the fixed time it was created, and even the single transaction within. Thus, every node has the starting point for the blockchain, a secure "root" from which to build a trusted blockchain.
((("Bitcoin Core client","genesis block in")))See the statically encoded genesis block inside the Bitcoin Core client, in http://bit.ly/1x6rcwP[chainparams.cpp].
The following identifier hash belongs to the genesis block:
----
000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
----
You can search for that block hash in any block explorer website, such as blockchain.info, and you will find a page describing the contents of this block, with a URL containing that hash:
https://blockchain.info/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
https://blockexplorer.com/block/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
Using the Bitcoin Core reference client on the command line:
----
$ bitcoind getblock 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
----
[source,json]
----
{
"hash" : "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
"confirmations" : 308321,
"size" : 285,
"height" : 0,
"version" : 1,
"merkleroot" : "4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b",
"tx" : [
"4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b"
],
"time" : 1231006505,
"nonce" : 2083236893,
"bits" : "1d00ffff",
"difficulty" : 1.00000000,
"nextblockhash" : "00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048"
}
----
The genesis block contains a hidden message within it. The coinbase transaction input contains the text "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks." This message was intended to offer proof of the earliest date this block was created, by referencing the headline of the British newspaper _The Times_. It also serves as a tongue-in-cheek reminder of the importance of an independent monetary system, with bitcoin's launch occurring at the same time as an unprecedented worldwide monetary crisis. The message was embedded in the first block by Satoshi Nakamoto, bitcoin's creator.
=== Linking Blocks in the Blockchain
((("blockchains","linking blocks to")))((("blocks","linking to blockchain")))Bitcoin full nodes maintain a local copy of the blockchain, starting at the genesis block. The local copy of the blockchain is constantly updated as new blocks are found and used to extend the chain. As a node receives incoming blocks from the network, it will validate these blocks and then link them to the existing blockchain. To establish a link, a node will examine the incoming block header and look for the "previous block hash."
Let's assume, for example, that a node has 277,314 blocks in the local copy of the blockchain. The last block the node knows about is block 277,314, with a block header hash of +00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249+.
The bitcoin node then receives a new block from the network, which it parses as follows:
[source,json]
----
{
"size" : 43560,
"version" : 2,
"previousblockhash" :
"00000000000000027e7ba6fe7bad39faf3b5a83daed765f05f7d1b71a1632249",
"merkleroot" :
"5e049f4030e0ab2debb92378f53c0a6e09548aea083f3ab25e1d94ea1155e29d",
"time" : 1388185038,
"difficulty" : 1180923195.25802612,
"nonce" : 4215469401,
"tx" : [
"257e7497fb8bc68421eb2c7b699dbab234831600e7352f0d9e6522c7cf3f6c77",
#[... many more transactions omitted ...]
"05cfd38f6ae6aa83674cc99e4d75a1458c165b7ab84725eda41d018a09176634"
]
}
----
Looking at this new block, the node finds the +previousblockhash+ field, which contains the hash of its parent block. It is a hash known to the node, that of the last block on the chain at height 277,314. Therefore, this new block is a child of the last block on the chain and extends the existing blockchain. The node adds this new block to the end of the chain, making the blockchain longer with a new height of 277,315. <<chain_of_blocks>> shows the chain of three blocks, linked by references in the +previousblockhash+ field.
[[merkle_trees]]
=== Merkle Trees
((("blockchains","merkle trees and", id="ix_ch07-asciidoc1", range="startofrange")))((("merkle trees", id="ix_ch07-asciidoc2", range="startofrange")))Each block in the bitcoin blockchain contains a summary of all the transactions in the block, using a _merkle tree_.
A _merkle tree_, also known as a((("binary hash tree"))) _binary hash tree_, is a data structure used for efficiently summarizing and verifying the integrity of large sets of data. Merkle trees are binary trees containing cryptographic hashes. The term "tree" is used in computer science to describe a branching data structure, but these trees are usually displayed upside down with the "root" at the top and the "leaves" at the bottom of a diagram, as you will see in the examples that follow.
Merkle trees are used in bitcoin to summarize all the transactions in a block, producing an overall digital fingerprint of the entire set of transactions, providing a very efficient process to verify whether a transaction is included in a block. A((("Merkle trees","constructing"))) Merkle tree is constructed by recursively hashing pairs of nodes until there is only one hash, called the _root_, or _merkle root_. The cryptographic hash algorithm used in bitcoin's merkle trees is SHA256 applied twice, also known as double-SHA256.
When N data elements are hashed and summarized in a merkle tree, you can check to see if any one data element is included in the tree with at most +2*log~2~(N)+ calculations, making this a very efficient data structure.
[[chain_of_blocks]]
.Blocks linked in a chain, by reference to the previous block header hash
image::images/msbt_0701.png[]
The merkle tree is constructed bottom-up. In the following example, we start with four transactions, A, B, C and D, which form the _leaves_ of the Merkle tree, as shown in <<simple_merkle>>. The transactions are not stored in the merkle tree; rather, their data is hashed and the resulting hash is stored in each leaf node as H~A~, H~B~, H~C~, and H~D~:
----
H~A~ = SHA256(SHA256(Transaction A))
----
Consecutive pairs of leaf nodes are then summarized in a parent node, by concatenating the two hashes and hashing them together. For example, to construct the parent node H~AB~, the two 32-byte hashes of the children are concatenated to create a 64-byte string. That string is then double-hashed to produce the parent node's hash:
----
H~AB~ = SHA256(SHA256(H~A~ + H~B~))
----
The process continues until there is only one node at the top, the node known as the Merkle root. That 32-byte hash is stored in the block header and summarizes all the data in all four transactions.
[[simple_merkle]]
.Calculating the nodes in a merkle tree
image::images/msbt_0702.png["merkle_tree"]
Because the merkle tree is a binary tree, it needs an even number of leaf nodes. If there is an odd number of transactions to summarize, the last transaction hash will be duplicated to create an even number of leaf nodes, also known as a((("balanced trees"))) _balanced tree_. This is shown in <<merkle_tree_odd>>, where transaction C is duplicated.
[[merkle_tree_odd]]
.Duplicating one data element achieves an even number of data elements
image::images/msbt_0703.png["merkle_tree_odd"]
The same method for constructing a tree from four transactions can be generalized to construct trees of any size. In bitcoin it is common to have several hundred to more than a thousand transactions in a single block, which are summarized in exactly the same way, producing just 32 bytes of data as the single merkle root. In <<merkle_tree_large>>, you will see a tree built from 16 transactions. Note that although the root looks bigger than the leaf nodes in the diagram, it is the exact same size, just 32 bytes. Whether there is one transaction or a hundred thousand transactions in the block, the merkle root always summarizes them into 32 bytes.
[[merkle_tree_large]]
.A merkle tree summarizing many data elements
image::images/msbt_0704.png["merkle_tree_large"]
To prove that a specific transaction is included in a block, a node only needs to produce +log~2~(N)+ 32-byte hashes, constituting an((("authentication path")))((("merkle path"))) _authentication path_ or _merkle path_ connecting the specific transaction to the root of the tree. This is especially important as the number of transactions increases, because the base-2 logarithm of the number of transactions increases much more slowly. This allows bitcoin nodes to efficiently produce paths of 10 or 12 hashes (320–384 bytes), which can provide proof of a single transaction out of more than a thousand transactions in a megabyte-size block.
In <<merkle_tree_path>>, a node can prove that a transaction K is included in the block by producing a merkle path that is only four 32-byte hashes long (128 bytes total). The path consists of the four hashes (noted in blue in <<merkle_tree_path>>) H~L~, H~IJ~, H~MNOP~ and H~ABCDEFGH~. With those four hashes provided as an authentication path, any node can prove that H~K~ (noted in green in the diagram) is included in the merkle root by computing four additional pair-wise hashes H~KL~, H~IJKL~, H~IJKLMNOP~, and the merkle tree root (outlined in a dotted line in the diagram).
[[merkle_tree_path]]
.A merkle path used to prove inclusion of a data element
image::images/msbt_0705.png["merkle_tree_path"]
The code in <<merkle_example>> demonstrates the process of creating a merkle tree from the leaf-node hashes up to the root, using the libbitcoin library for some helper functions.
[[merkle_example]]
.Building a merkle tree
====
[source, cpp]
----
include::code/merkle.cpp[]
----
====
<<merkle_example_run>> shows the result of compiling and running the merkle code.
[[merkle_example_run]]
.Compiling and running the merkle example code
====
[source,bash]
----
$ # Compile the merkle.cpp code
$ g++ -o merkle merkle.cpp $(pkg-config --cflags --libs libbitcoin)
$ # Run the merkle executable
$ ./merkle
Current merkle hash list:
32650049a0418e4380db0af81788635d8b65424d397170b8499cdc28c4d27006
30861db96905c8dc8b99398ca1cd5bd5b84ac3264a4e1b3e65afa1bcee7540c4
Current merkle hash list:
d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
Result: d47780c084bad3830bcdaf6eace035e4c6cbf646d103795d22104fb105014ba3
----
====
The efficiency of merkle trees becomes obvious as the scale increases. <<block_structure2>> shows the amount of data that needs to be exchanged as a merkle path to prove that a transaction is part of a block.
[[block_structure2]]
.Merkle tree efficiency
[options="header"]
|=======
|Number of transactions| Approx. size of block | Path size (hashes) | Path size (bytes)
| 16 transactions | 4 kilobytes | 4 hashes | 128 bytes
| 512 transactions | 128 kilobytes | 9 hashes | 288 bytes
| 2048 transactions | 512 kilobytes | 11 hashes | 352 bytes
| 65,535 transactions | 16 megabytes | 16 hashes | 512 bytes
|=======
As you can see from the table, while the block size increases rapidly, from 4 KB with 16 transactions to a block size of 16 MB to fit 65,535 transactions, the merkle path required to prove the inclusion of a transaction increases much more slowly, from 128 bytes to only 512 bytes. With merkle trees, a node can download just the block headers (80 bytes per block) and still be able to identify a transaction's inclusion in a block by retrieving a small merkle path from a full node, without storing or transmitting the vast majority of the blockchain, which might be several gigabytes in size. Nodes that do not maintain a full blockchain, called simplified payment verification (SPV nodes), use merkle paths to verify transactions without downloading full blocks.(((range="endofrange", startref="ix_ch07-asciidoc2")))(((range="endofrange", startref="ix_ch07-asciidoc1")))
=== Merkle Trees and Simplified Payment Verification (SPV)
((("merkle trees","SPV and")))((("Simplified Payment Verification (SPV) nodes","merkle trees and")))Merkle trees are used extensively by SPV nodes. SPV nodes don't have all transactions and do not download full blocks, just block headers. In order to verify that a transaction is included in a block, without having to download all the transactions in the block, they use an authentication path, or merkle path.
Consider, for example, an SPV node that is interested in incoming payments to an address contained in its wallet. The SPV node will establish a bloom filter on its connections to peers to limit the transactions received to only those containing addresses of interest. When a peer sees a transaction that matches the bloom filter, it will send that block using a((("merkleblock message"))) +merkleblock+ message. The +merkleblock+ message contains the block header as well as a merkle path that links the transaction of interest to the merkle root in the block. The SPV node can use this merkle path to connect the transaction to the block and verify that the transaction is included in the block. The SPV node also uses the block header to link the block to the rest of the blockchain. The combination of these two links, between the transaction and block, and between the block and blockchain, proves that the transaction is recorded in the blockchain. All in all, the SPV node will have received less than a kilobyte of data for the block header and merkle path, an amount of data that is more than a thousand times less than a full block (about 1 megabyte currently).(((range="endofrange", startref="ix_ch07-asciidoc0")))

1738
ch08.asciidoc
View File

File diff suppressed because it is too large Load Diff

733
ch09.asciidoc
Unescape View File

@ -1,370 +1,363 @@
[[ch9]]
== Alternative Chains, Currencies, and Applications
Bitcoin was the result of twenty years of research in distributed systems and currencies and brought a revolutionary new technology into the space: the de-centralized consensus mechanism based on Proof-of-Work. This invention at the heart of bitcoin has ushered a wave of innovation in currencies, financial services, economics, distributed systems, voting systems, corporate governance, and contracts.
In this chapter we'll examine the many offshoots of the bitcoin and blockchain inventions, the alternative chains, currencies, and applications built since the introduction of this technology in 2009. Mostly, we will look at _alt-coins_, which are digital currencies implemented using the same design pattern as bitcoin, but with a completely separate blockchain and network.
For every alt-coin mentioned in this chapter, 50 or more will go unmentioned, eliciting howls of anger from their creators and fans. The purpose of this chapter is not to evaluate or qualify alt-coins, or to mention the most "significant" ones based on some subjective assessment. Instead, we will highlight a few examples that show the breadth and variety of the ecosystem, noting the first-of-a-kind for each innovation or significant differentiation. Some of the most interesting examples of alt-coins are in fact complete failures from a monetary perspective. That perhaps makes them even more interesting for study and highlights the fact that this chapter is not to be used as an investment guide.
With new coins introduced every day, it would be impossible not to miss some important coin, perhaps the one that changes history. The rate of innovation is what makes this space so exciting and guarantees this chapter will be incomplete and out-of-date as soon as it is published.
=== A taxonomy of alternative currencies and chains
Bitcoin is an open source project, and its code has been used as the basis for many other software projects. The most common form of software spawned from bitcoin's source code are alternative de-centralized currencies, or _alt-coins_, which use the same basic building blocks to implement digital currencies.
There are a number of protocol layers implemented on top of bitcoin's blockchain. These _meta-coins_, _meta-chains_, or _blockchain apps_ use the blockchain as an application platform or extend the bitcoin protocol by adding protocol layers. Examples include Colored Coins, Mastercoin, and Counterparty.
In the next section we will examine a few notable alt-coins, such as Litecoin, Dogecoin, Freicoin, Primecoin, Peercoin, Darkcoin, and Zerocoin. These alt-coins are notable for historical reasons or because they are good examples for a specific type of alt-coin innovation, not because they are the most valuable or "best" alt-coins.
In addition to the alt-coins, there are also a number of alternative blockchain implementations that are not really "coins", which I call _alt-chains_. These alt-chains implement a consensus algorithm and distributed ledger as a platform for contracts, name registration, or other applications. Alt-chains use the same basic building blocks and sometimes also use a currency or token as a payment mechanism, but their primary purpose is not currency. We will look at Namecoin, Ethereum, and NXT as examples of alt-chains.
In addition to the Proof-of-Work consensus mechanism used in bitcoin, alternatives include experimental protocols based on Proof-of-Resource and Proof-of-Publishing. We will examine Maidsafe and Twister as examples of these consensus mechanisms.
Finally, there are a number of bitcoin contenders that offer digital currency or digital payment networks, but without using a de-centralized ledger or consensus mechanism based on Proof-of-Work, such as Ripple and others. These non-blockchain technologies are outside of the scope of this book and will not be covered in this chapter.
=== Meta-Coin Platforms
Meta-coins and meta-chains are software layers implemented on top of bitcoin, either implementing a currency-inside-a-currency, or a platform/protocol overlay inside the bitcoin system. These function layers extend the core bitcoin protocol and add features and capabilities by encoding additional data inside bitcoin transactions and bitcoin addresses. The first implementations of meta-coins used various "hacks" to add meta-data to the bitcoin blockchain, such as using bitcoin addresses to encode data or using unused transaction fields (e.g. the transaction sequence field) to encode meta-data about the added protocol layer. Since the introduction of the OP_RETURN transaction scripting opcode, the meta-coins have been able to record meta-data more directly in the blockchain, and most are migrating to using that instead.
==== Colored Coins
Colored Coins is a meta-protocol that overlays information on small amounts of bitcoin. A "colored" coin is an amount of bitcoin repurposed to express another asset. Imagine for example taking a $1 USD note and putting a stamp on it that said "This is a 1 share certificate of Acme Inc.". Now the $1 serves two purposes: it is a currency note and also a share certificate. Because it is more valuable as a share, you would not want to use it to buy candy, so effectively it is no longer useful as currency. Colored coins work in the same way by converting a specific, very small, amount of bitcoin into a traded certificate that represents another asset. The term "color" refers to the idea of giving special meaning through the addition of an attribute such as a color - it is a metaphor not an actual color association. There are no colors in colored coins.
Colored coins are managed by specialized "wallets" that record and interpret the metadata attached to the "colored" bitcoins. Using such a wallet, the user will convert an amount of bitcoins from uncolored currency, into colored coins, by adding a label that has a special meaning. For example, a label could represent stock certificates, coupons, real property, commodities, collectible tokens, etc. It is entirely up to the users of colored coins to assign and interpret the meaning of the "color" associated with specific coins. To color the coins, the user defines the associated metadata, such as the type of issuance, whether it can be subdivided into smaller units, a symbol and description, and other related information. Once colored, these coins can be bought and sold, subdivided, aggregated and receive dividend payments. The colored coins can also be "uncolored" by removing the special association and redeem them for their face-value in bitcoin.
To demonstrate the use of colored coins, we have created a set of 20 colored coins with symbol "MasterBTC" that represent coupons for a free copy of this book. Each unit of MasterBTC, represented by these colored coins, can now be sold or given to any bitcoin user with a colored-coin-capable wallet, who can then transfer them to others or redeem them with the issuer for a free copy of the book. This example of colored coins can be seen here: https://cpr.sm/FoykwrH6UY
.The metadata profile of the colored coins recorded as a coupon for a free copy of the book
====
[source,json]
----
{
"source_addresses": [
"3NpZmvSPLmN2cVFw1pY7gxEAVPCVfnWfVD"
],
"contract_url": "https://www.coinprism.info/asset/3NpZmvSPLmN2cVFw1pY7gxEAVPCVfnWfVD",
"name_short": "MasterBTC",
"name": "Free copy of \"Mastering Bitcoin\"",
"issuer": "Andreas M. Antonopoulos",
"description": "This token is redeemable for a free copy of the book \"Mastering Bitcoin\"",
"description_mime": "text/x-markdown; charset=UTF-8",
"type": "Other",
"divisibility": 0,
"link_to_website": false,
"icon_url": null,
"image_url": null,
"version": "1.0"
}
----
====
==== Mastercoin
Mastercoin is a protocol layer on top of bitcoin that supports a platform for various applications extending the bitcoin system. Mastercoin uses the currency MST as a token for conducting Mastercoin transactions but it is not primarily a currency. Rather it is a platform for building other things, such as user currencies, smart property tokens, de-centralized asset exchanges, contracts, etc. Think of Mastercoin as an application-layer protocol on top of bitcoin's financial transaction transport-layer, just like HTTP runs on top of TCP.
Mastercoin operates primarily through transactions sent to and from a special bitcoin address called the "exodus" address (+1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P+), just like HTTP uses a specific TCP port (port 80) to differentiate its traffic from the rest of the TCP traffic. The Mastercoin protocol is gradually transitioning from using the specialized exodus address and multi-signatures to using the OP_RETURN bitcoin operator to encode transaction metadata.
==== Counterparty
Counterparty is another protocol layer implemented on top of bitcoin. Counterparty enables user currencies, tradable tokens, financial instruments, de-centralized asset exchanges, and other features. Counterparty is implemented primarily using the OP_RETURN operator in bitcoin's scripting language to record metadata enhancing bitcoin transactions with additional meaning. Counterparty uses the currency XCP as a token for conducting Counterparty transactions.
=== Alt-coins
The vast majority of alt-coins are derived from bitcoin's source code, also known as "forks". Some are implemented "from scratch" based on the blockchain model but without using any of bitcoin's source code. Alt-coins and alt-chains (in the next section) are both separate implementations of blockchain technology and both forms use their own blockchain. The difference in the terms is to indicate that alt-coins are primarily used as currency, whereas alt-chains are used for other purposes, not primarily currency.
The first alt-coins appeared in August of 2011 as forks of the bitcoin source code. Strictly speaking, the first major fork of bitcoin's code was not an alt-coin but the alt-chain _Namecoin_, which will be discussed in the next section.
Based on the date of announcement, the first alt-coin appears to be _IXCoin_, launched in August of 2011. IXCoin modified a few of the bitcoin parameters, specifically accelerating the creation of currency by increasing the reward to 96 coins per block.
In September of 2011, _Tenebrix_ was launched. Tenebrix was the first crypto-currency to implement an alternative Proof-of-Work algorithm, namely _scrypt_, an algorithm originally designed for password stretching (brute-force resistance). The stated goal of Tenebrix was to make a coin that was resistant to mining with GPUs and ASICs, by using a memory-intensive algorithm. Tenebrix did not succeed as a currency, but it was the basis for Litecoin, which has enjoyed great success and has spawned hundreds of clones.
_Litecoin_, in addition to using scrypt as the Proof-of-Work algorithm, also implemented a faster block generation time, targeted at 2.5 minutes instead of bitcoin's 10 minutes. The resulting currency is touted as "silver to bitcoin's gold" and is intended as a light-weight alternative currency. Due to the faster confirmation time and the 84 million total currency limit, many adherents of Litecoin believe it is better suited for retail transactions than bitcoin.
Alt-coins continued to proliferate in 2011 and 2012, either based on bitcoin or on Litecoin. In the beginning of 2013 there were 20 alt-coins vying for position in the market. By the end of 2013 however, this number had exploded to 200, with 2013 quickly becoming the "year of the alt-coins". The growth of alt-coins continued in 2014 with more than 500 alt-coins now in existence. More than half the alt-coins today are clones of Litecoin.
Creating an alt-coin is easy, which is why there are now more than 500 of them. Most of the alt-coins differ very slightly from bitcoin and do not offer anything worth studying. Many are in fact just attempts to enrich their creators. Among the copycats and pump-and-dump schemes, there are however some notable exceptions and very important innovations. These alt-coins take radically different approaches or add significant innovation to bitcoin's design pattern. There are three primary areas where alt-coins differentiate from bitcoin:
* Different monetary policy
* Different Proof-of-Work or consensus mechanism
* Specific features, such as strong anonymity
A graphical timeline of alt-coins and alt-chains can be found at http://mapofcoins.com.
==== Evaluating an alt-coin
With so many alt-coins out there, how does one decide which ones are worthy of attention? Some alt-coins attempt to achieve broad distribution and use as currencies. Others are laboratories for experimenting on different features and monetary models. Many are just get-rich-quick schemes by their creators. To evaluate alt-coins I look at their defining characteristics and their market metrics.
Here are some questions to ask about how well an alt-coin differentiates from bitcoin:
* Is the alt-coin introducing a significant innovation?
* Does the alt-coin differentiate sufficiently from bitcoin?
* Is the difference compelling enough to attract users away from bitcoin?
* Does the alt-coin address an interesting niche market or application?
* Can the alt-coin attract enough miners to be secured against consensus attacks?
Here are some of the key financial and market metrics to examine:
* What is the total market capitalization of alt-coin?
* How many estimated users/wallets does the alt-coin have?
* How many merchants accept the alt-coin?
* How many transactions (volume) are executed on the alt-coin?
* How much value is transacted daily?
In this chapter we will concentrate primarily on the technical characteristics and innovation potential of alt-coins, focusing on the first set of questions.
==== Monetary Parameter Alternatives: Litecoin, Dogecoin, Freicoin
Bitcoin has a few monetary parameters that give it distinctive characteristics of a deflationary fixed-issuance currency. It is limited to 21 million major currency units (or 21 quadrillion minor units), has a geometrically declining issuance rate and a 10-minute block "heartbeat" which controls the speed of transaction confirmation and currency generation. Many alt-coins have tweaked the primary parameters to achieve different monetary policies. Among the hundreds of alt-coins, some of the most notable examples include:
*Litecoin*
One of the first alt-coins, released in 2011, Litecoin is the second most successful digital currency after bitcoin. Its primary innovations were the use of _scrypt_ as the Proof-of-Work algorithm (inherited from Tenebrix) and the faster/lighter currency parameters.
* Block generation time: 2.5 minutes
* Total currency: 84 million coins by 2140
* Consensus Algorithm: scrypt Proof-of-Work
* Market capitalization: $160 million USD in mid-2014
*Dogecoin*
Dogecoin was released in December of 2013, based on a fork of Litecoin. Dogecoin is notable because it has a monetary policy of rapid issuance and a very high currency cap, to encourage spending and tipping. Dogecoin is also notable because it was started as a joke but became quite popular, with a large and active community, before declining rapidly in 2014.
* Block generation time: 60 seconds
* Total currency: 100,000,000,000 (100 billion) Doge by 2015
* Consensus algorithm: scrypt Proof-of-Work
* Market capitalization: $12 million USD in mid-2014
*Freicoin*
Freicoin was introduced in July 2012. It is a _demurrage currency_, meaning that it has a negative interest rate for stored value. Value stored in Freicoin is assessed a 4.5% APR fee, to encourage consumption and discourage hoarding of money. Freicoin is notable in that it implements a monetary policy that is the exact opposite of Bitcoin's deflationary policy. Freicoin has not seen success as a currency, but is an interesting example of the variety of monetary policies that can be expressed by alt-coins.
* Block generation: 10 minutes
* Total currency: 100 million coins by 2140
* Consensus algorithm: SHA256 Proof-of-Work
* Market capitalization: $130,000 USD in mid-2014
==== Consensus Innovation: Peercoin, Myriad, Blackcoin, Vericoin, NXT
Bitcoin's consensus mechanism is based on Proof-of-Work using the SHA256 algorithm. The first alt-coins introduced scrypt as an alternative Proof-of-Work algorithms, as a way to make mining more CPU-friendly and less susceptible to centralization with ASICs. Since then, innovation in the consensus mechanism has continued at a frenetic pace. Several alt-coins adopted a variety of algorithms such as scrypt, scrypt-N, Skein, Groestl, SHA3, X11, Blake, and others. Some alt-coins combined multiple algorithms for Proof-of-Work. In 2013 we saw the invention of an alternative to Proof-of-Work, called _Proof-of-Stake_, which forms the basis of many modern alt-coins.
Proof-of-Stake is a system by which existing owners of a currency can "stake" currency as interest-bearing collateral. Somewhat like a Certificate of Deposit (CD), participants can reserve a portion of their currency holdings, while earning an investment return in the form of new currency (issued as interest payments) and transaction fees.
*Peercoin*
Peercoin was introduced in August of 2012 and is the first alt-coin to use a hybrid Proof-of-Work and Proof-of-Stake algorithm for issuance of new currency.
* Block generation: 10 minutes
* Total currency: No limit
* Consensus algorithm: (Hybrid) Proof-of-Stake with initial Proof-of-Work
* Market capitalization: $14 million USD in mid-2014
*Myriad*
Myriad was introduced in February 2014 and is notable because it uses five different Proof-of-Work algorithms (SHA256d, Scrypt, Qubit, Skein or Myriad-Groestl) simultaneously, with difficulty varying for each algorithm depending on miner participation. The intent is to make Myriad immune to ASIC specialization and centralization as well as much more resistant to consensus attacks, as multiple mining algorithms would have to be attacked simultaneously.
* Block generation: 30 second average (2.5 minutes target per mining algorithm)
* Total currency: 2 billion by 2024
* Consensus algorithm: Multi-Algorithm Proof-of-Work
* Market capitalization: $120,000 USD in mid-2014
*Blackcoin*
Blackcoin was introduced in February 2014 and uses a Proof-of-Stake consensus algorithm. It is also notable for the introduction of "multipools", a type of mining pool that can switch between different alt-coins automatically, depending on profitability.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Stake
* Market capitalization: $3.7 million USD in mid-2014
*VeriCoin*
VeriCoin was launched in May 2014. It uses a Proof-of-Stake consensus algorithm with a variable interest rate that dynamically adjusts based on market forces of supply and demand. It also is the first alt-coin featuring auto-exchange to Bitcoin for payment in Bitcoin from the wallet.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Stake
* Market capitalization: $1.1 million USD in mid-2014
*NXT*
NXT (pronounced "Next") is a "pure" Proof-of-Stake alt-coin, in that it does not use Proof-of-Work mining. NXT is a from-scratch implementation of a crypto-currency, not a fork of bitcoin or any other alt-coins. NXT implements many advanced features, such as a name registry (similar to Namecoin), a de-centralized asset exchange (similar to Colored Coins), integrated de-centralized and secure messaging (similar to Bitmessage) and stake delegation (delegate Proof-of-Stake to others). NXT adherents call it a "next-generation" or 2.0 crypto-currency.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Stake
* Market capitalization: $30 million USD in mid-2014
==== Dual-Purpose Mining Innovation: Primecoin, Curecoin, Gridcoin
Bitcoin's Proof-of-Work algorithm has only one purpose: securing the bitcoin network. Compared to traditional payment system security, the cost of mining is not very high. However, it has been criticized by many as being “wasteful". The next set of alt-coins attempt to address this concern. Dual-purpose Proof-of-Work algorithms solve a specific "useful" problem, while producing Proof-of-Work to secure the network. The risk of adding an external use to the currency's security is that it also adds external influence to the supply/demand curve.
*Primecoin*
Primecoin was announced in July 2013. Its Proof-of-Work algorithm searches for prime numbers, computing Cunningham and bi-twin prime chains. Prime numbers are useful in a variety of scientific disciplines. The Primecoin blockchain contains the discovered prime numbers, thereby producing a public record of scientific discovery in parallel to the public ledger of transactions.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-Work with prime number chain discovery
* Market capitalization: $1.3 million USD in mid-2014
*Curecoin*
Curecoin was announced in May 2013. It combines a SHA256 Proof-of-Work algorithm with protein folding research through the Folding@Home project. Protein folding is a computationally intensive simulation of biochemical interactions of proteins, used to discover new drug targets for curing diseases.
* Block generation: 10 minutes
* Total currency: No limit
* Consensus algorithm: Proof-of-Work with protein folding research
* Market capitalization: $58,000 USD in mid-2014
*Gridcoin*
Gridcoin was introduced in October 2013. It supplements scrypt-based Proof-of-Work with subsidies for participation in BOINC open grid-computing. BOINC is an open protocol for scientific research grid-computing, which allows participants to share their spare computing cycles for a broad range of academic research computing. Gridcoin uses BOINC as a general purpose computing platform, rather than to solve specific science problems such as prime numbers or protein folding.
* Block generation: 150 seconds
* Total currency: No limit
* Consensus algorithm: Proof-of-Work with BOINC grid-computing subsidy
* Market capitalization: $122,000 USD in mid-2014
==== Anonymity-Focused Alt-Coins: CryptoNote, Bytecoin, Monero, Zerocash/Zerocoin, Darkcoin
Bitcoin is often mistakenly characterized as "anonymous" currency. In fact, it is relatively easy to connect identities to bitcoin addresses and, using big-data analytics, connect addresses to each other to form a comprehensive picture of someone's bitcoin spending habits. Several alt-coins aim to address this issue directly by focusing on strong anonymity. The first such attempt is most likely _Zerocoin_, a meta-coin protocol for preserving anonymity on top of bitcoin, introduced with a paper in the 2013 IEEE Symposium on Security and Privacy. Zerocoin will be implemented as a completely separate alt-coin called Zerocash, currently in development. An alternative approach to anonymity was launched with _CryptoNote_ in a paper published in October 2013. CryptoNote is a foundational technology that is implemented by a number of alt-coin forks discussed below. In addition to Zerocash and Cryptonotes, there are several other independent anonymous coins, such as Darkcoin that use stealth addresses or transaction re-mixing to deliver anonymity.
*Zerocoin/Zerocash*
Zerocoin is a theoretical approach to digital currency anonymity introduced in 2013 by researchers at Johns Hopkins. Zerocash is an alt-coin implementation of Zerocoin that is in development and not yet released.
*CryptoNote*
CryptoNote is a reference implementation alt-coin that provides the basis for anonymous digital cash that was introduced in October 2013. It is designed to be "forked" into different implementations and has a built-in periodic reset mechanism that makes it unusable as a currency itself. Several alt-coins have been spawned from CryptoNote, including Bytecoin (BCN), Aeon (AEON), Boolberry (BBR), duckNote (DUCK), Fantomcoin (FCN), Monero (XMR), MonetaVerde (MCN) and Quazarcoin (QCN). CryptoNote is also notable for being a complete ground-up implementation of a crypto-currency, not a fork of bitcoin.
*Bytecoin*
Bytecoin was the first implementation spawned from CryptoNote, offering a viable anonymous currency based on the CryptoNote technology. Bytecoin was launched in July of 2012. Note that there was a previous alt-coin named Bytecoin with currency symbol BTE, whereas the CryptoNote-derived Bytecoin has currency symbol BCN. Bytecoin uses the Cryptonight Proof-of-Work algorithm which requires access to at least 2 MB of RAM per instance, making it unsuitable for GPU or ASIC mining. Bytecoin inherits ring-signatures, unlinkable transactions and blockchain-analysis resistant anonymity from CryptoNote.
* Block generation: 2 minutes
* Total currency: 184 billion BCN
* Consensus algorithm: Cryptonight Proof-of-Work
* Market capitalization: $3 million USD in mid-2014
*Monero*
Monero is another implementation of CryptoNote. It has a slightly flatter issuance curve than Bytecoin, issuing 80% of the currency in the first 4 years. It offers the same anonymity features inherited from CryptoNote.
* Block generation: 1 minute
* Total currency: 18.4 million XMR
* Consensus algorithm: Cryptonight Proof-of-Work
* Market capitalization: $5 million USD in mid-2014
*Darkcoin*
Darkcoin was launched in January of 2014. Darkcoin implements anonymous currency using a re-mixing protocol for all transactions called DarkSend. Darkcoin is also notable for using 11 rounds of different hash functions (blake, bmw, groestl, jh, keccak, skein, luffa, cubehash, shavite, simd, echo) for the Proof-of-Work algorithm.
* Block generation: 2.5 minutes
* Total currency: maximum 22 million DRK
* Consensus algorithm: Multi-algorithm Multi-round Proof-of-Work
* Market capitalization: $19 million USD in mid-2014
=== Non-currency alt-chains
Alt-chains are alternative implementations of the blockchain design pattern, which are not primarily used as currency. Many include a currency, but the currency is used as a token for allocating something else, such as a resource or a contract. The currency, in other words, is not the main "point" of the platform, it is a secondary feature.
==== Namecoin
Namecoin was the first "fork" of the bitcoin code. Namecoin is a de-centralized key-value registration and transfer platform using a blockchain. It supports a global domain name registry similar to the domain-name registration system on the Internet. Namecoin is currently used as an alternative Domain Name Service (DNS) for the root-level domain +.bit+. Namecoin can also be used to register names and key-value pairs in other namespaces, for storing things like email addresses, encryption keys, SSL certificates, file signatures, voting systems, stock certificates and a myriad of other applications.
The Namecoin system includes the namecoin currency (symbol NMC), which is used to pay transaction fees for registration and transfer of names. At current prices, the fee to register a name is 0.01 NMC or approximately 1 US cent. As in bitcoin, the fees are collected by Namecoin miners.
Namecoin's basic parameters are the same as bitcoin's:
* Block generation: 10 minutes
* Total currency: 21 million NMC by 2140
* Consensus algorithm: SHA256 Proof-of-Work
* Market capitalization: $10 million USD in mid-2014
Namecoin's namespaces are not restricted, and anyone can use any namespace in any way. However, certain namespaces have an agreed upon specification so that when it is read from the blockchain, software knows how to read and proceed from there. If it is malformed, then whatever software you used to read from the specific namespace will throw an error. Some of the popular namespaces are:
* +d/+ is the domain-name namespace for +.bit+ domains
* +id/+ is the namespace for storing person identifiers such as email addresses, PGP keys etc.
* +u/+ is an additional, more structured specification to store identities (based on openspecs).
The Namecoin client is very similar to Bitcoin Core, as it is derived from the same source code. Upon installation, the client will download a full copy of the namecoin blockchain and then will be ready to query and register names. There are three main commands:
* +name_new+: Query or pre-register a name
* +name_firstupdate+: Register a name and make the registration public
* +name_update+: Change the details or refresh a name registration
For example, to register the domain +mastering-bitcoin.bit+, we use the command +name_new+ as follows:
[source,bash]
----
$ namecoind name_new d/mastering-bitcoin
----
[source,json]
----
[
"21cbab5b1241c6d1a6ad70a2416b3124eb883ac38e423e5ff591d1968eb6664a",
"a05555e0fc56c023"
]
----
The +name_new+ command registers a claim on the name, by creating a hash of the name with a random key. The two strings returned by +name_new+ are the hash and the random key (+a05555e0fc56c023+ in the example above) that can be used to make the name registration public. Once that claim has been recorded on the namecoin blockchain it can be converted to a public registration with the +name_firstupdate+ command, by supplying the random key:
----
$ namecoind name_firstupdate d/mastering-bitcoin a05555e0fc56c023 "{"map": {"www": {"ip":"1.2.3.4"}}}}"
b7a2e59c0a26e5e2664948946ebeca1260985c2f616ba579e6bc7f35ec234b01
----
The example above will map the domain name +www.mastering-bitcoin.bit+ to IP address 1.2.3.4. The hash returned is the transaction id that can be used to track this registration. You can see what names are registered to you by running the +name_list+ command:
----
$ namecoind name_list
----
====
[source,json]
----
[
{
"name" : "d/mastering-bitcoin",
"value" : "{map: {www: {ip:1.2.3.4}}}}",
"address" : "NCccBXrRUahAGrisBA1BLPWQfSrups8Geh",
"expires_in" : 35929
}
]
----
====
Namecoin registrations need to be updated every 36,000 blocks (approximately 200 to 250 days). The +name_update+ command has no fee and therefore renewing domains in Namecoin is free. Third party providers can handle registration, automatic renewal and updating via a web interface, for a small fee. With a third-party provider you avoid the need to run a namecoin client, but you lose the independent control of a de-centralized name registry offered by Namecoin.
==== Bitmessage
Bitmessage is a bitcoin alt-chain that implements a de-centralized secure messaging service, essentially a server-less encrypted email system. Bitmessage allows users to compose and send messages to each other, using a bitmessage address. The messages operate in much the same way as a bitcoin transaction, but they are transient - they do not persist beyond 2 days and if not delivered to the destination node in that time, they are lost. Senders and recipients are pseudonymous, they have no identifiers other than a bitmessage address, but are strongly authenticated, meaning that messages cannot be "spoofed". Bitmessages are encrypted to the recipient and therefore the bitmessage network is resistant to holistic surveillance - an eavesdropper has to compromise the recipient's device in order to intercept messages.
==== Ethereum
Ethereum is a Turing-complete contract processing and execution platform based on a blockchain ledger. It is not a clone of bitcoin, but a completely independent design and implementation. Ethereum has a built-in currency, called _ether_, which is required in order to pay for contract execution. Ethereum's blockchain records _contracts_, which are expressed in a low-level, byte-code like, Turing-complete language. Essentially, a contract is a program that runs on every node in the Ethereum system. Ethereum contracts can store data, send and receive ether payments, store ether and execute an infinite range (hence Turing-complete) of computable actions, acting as de-centralized autonomous software agents.
Ethereum can implement quite complex systems that are otherwise implemented as alt-chains themselves. For example, below is a Namecoin-like name registration contract written in Ethereum (or more accurately, written in a high-level language that can be compiled to Ethereum code):
[source,python]
----
if !contract.storage[msg.data[0]]: # Is the key not yet taken?
# Then take it!
contract.storage[msg.data[0]] = msg.data[1]
return(1)
else:
return(0) // Otherwise do nothing
----
=== Future of Currencies
The future of cryptographic currencies overall is even brighter than the future of bitcoin. Bitcoin introduced a completely new form of de-centralized organization and consensus that has spawned hundreds of incredible innovations. These inventions will likely affect broad sectors of the economy, from distributed systems science, to finance, economics, currencies, central banking, and corporate governance. Many human activities that previously required centralized institutions or organizations to function as authoritative or trusted points of control can now be de-centralized. The invention of the blockchain and consensus system will significantly reduce the cost of organization and coordination on large scale systems, while removing opportunities for concentration of power, corruption and regulatory capture.
[[ch9]]
== Alternative Chains, Currencies, pass:[<phrase role="keep-together">and Applications</phrase>]
Bitcoin was the result of 20 years of research in distributed systems and currencies and brought a revolutionary new technology into the space: the decentralized consensus mechanism based on proof of work. This invention at the heart of bitcoin has ushered a wave of innovation in currencies, financial services, economics, distributed systems, voting systems, corporate governance, and contracts.
In this chapter we'll examine the many offshoots of the bitcoin and blockchain inventions: the alternative chains, currencies, and applications built since the introduction of this technology in 2009. Mostly, we will look at alternative coins, or _alt coins_, which are digital currencies implemented using the same design pattern as bitcoin, but with a completely separate blockchain and network.
For every alt coin mentioned in this chapter, 50 or more will go unmentioned, eliciting howls of anger from their creators and fans. The purpose of this chapter is not to evaluate or qualify alt coins, or even to mention the most significant ones based on some subjective assessment. Instead, we will highlight a few examples that show the breadth and variety of the ecosystem, noting the first-of-a-kind for each innovation or significant differentiation. Some of the most interesting examples of alt coins are in fact complete failures from a monetary perspective. That perhaps makes them even more interesting for study and highlights the fact that this chapter is not to be used as an investment guide.
With new coins introduced every day, it would be impossible not to miss some important coin, perhaps the one that changes history. The rate of innovation is what makes this space so exciting and guarantees this chapter will be incomplete and out-of-date as soon as it is published.
=== A Taxonomy of Alternative Currencies and Chains
((("chains, alternative")))((("currencies, alternative")))Bitcoin is an open source project, and its code has been used as the basis for many other software projects. The most common form of software spawned from bitcoin's source code are alternative decentralized currencies, or _alt coins_, which use the same basic building blocks to implement digital currencies.
There are a number of protocol layers implemented on top of bitcoin's blockchain. These((("blockchain apps")))((("meta chains")))((("meta coin platforms"))) _meta coins_, _meta chains_, or _blockchain apps_ use the blockchain as an application platform or extend the bitcoin protocol by adding protocol layers. Examples include Colored Coins, Mastercoin, and Counterparty.
In the next section we will examine a few notable alt coins, such as Litecoin, Dogecoin, Freicoin, Primecoin, Peercoin, Darkcoin, and Zerocoin. These alt coins are notable for historical reasons or because they are good examples for a specific type of alt coin innovation, not because they are the most valuable or "best" alt coins.
In addition to the alt coins, there are also a number of alternative blockchain implementations that are not really "coins," which I call((("alt chains"))) _alt chains_. These alt chains implement a consensus algorithm and distributed ledger as a platform for contracts, name registration, or other applications. Alt chains use the same basic building blocks and sometimes also use a currency or token as a payment mechanism, but their primary purpose is not currency. We will look at Namecoin, Ethereum, and NXT as examples of alt chains.
In addition to the proof-of-work consensus mechanism used in bitcoin, alternatives include experimental protocols based on proof of resource and proof of publishing. We will examine Maidsafe and Twister as examples of these consensus mechanisms.
Finally, there are a number of bitcoin contenders that offer digital currency or digital payment networks, but without using a decentralized ledger or consensus mechanism based on proof of work, such as Ripple and others. These non–blockchain technologies are outside the scope of this book and will not be covered in this chapter.
=== Meta Coin Platforms
((("meta coin platforms", id="ix_ch09-asciidoc0", range="startofrange")))Meta coins and meta chains are software layers implemented on top of bitcoin, either implementing a currency-inside-a-currency, or a platform/protocol overlay inside the bitcoin system. These function layers extend the core bitcoin protocol and add features and capabilities by encoding additional data inside bitcoin transactions and bitcoin addresses. The first implementations of meta coins used various hacks to add metadata to the bitcoin blockchain, such as using bitcoin addresses to encode data or using unused transaction fields (e.g., the transaction sequence field) to encode metadata about the added protocol layer. Since the introduction of the +OP_RETURN+ transaction scripting opcode, the meta coins have been able to record metadata more directly in the blockchain, and most are migrating to using that instead.
==== Colored Coins
((("colored coins")))((("meta coin platforms","colored coins")))_Colored coins_ is a meta protocol that overlays information on small amounts of bitcoin. A "colored" coin is an amount of bitcoin repurposed to express another asset. ((("stock certificates","colored coins as")))Imagine, for example, taking a $1 note and putting a stamp on it that said, "This is a 1 share certificate of Acme Inc." Now the $1 serves two purposes: it is a currency note and also a share certificate. Because it is more valuable as a share, you would not want to use it to buy candy, so effectively it is no longer useful as currency. Colored coins work in the same way by converting a specific, very small amount of bitcoin into a traded certificate that represents another asset. The term "color" refers to the idea of giving special meaning through the addition of an attribute such as a color—it is a metaphor, not an actual color association. There are no colors in colored coins.
((("wallets","for colored coins")))Colored coins are managed by specialized wallets that record and interpret the metadata attached to the colored bitcoins. Using such a wallet, the user will convert an amount of bitcoins from uncolored currency into colored coins by adding a label that has a special meaning. For example, a label could represent stock certificates, coupons, real property, commodities, or collectible tokens. It is entirely up to the users of colored coins to assign and interpret the meaning of the "color" associated with specific coins. To color the coins, the user defines the associated metadata, such as the type of issuance, whether it can be subdivided into smaller units, a symbol and description, and other related information. Once colored, these coins can be bought and sold, subdivided, and aggregated, and receive dividend payments. The colored coins can also be "uncolored" by removing the special association and redeemed for their face value in bitcoin.
To demonstrate the use of colored coins, we have created a set of 20 colored coins with symbol "MasterBTC" that represent coupons for a free copy of this book shown in <<example_9-1>>. Each unit of MasterBTC, represented by these colored coins, can now be sold or given to any bitcoin user with a colored-coin-capable wallet, who can then transfer them to others or redeem them with the issuer for a free copy of the book. This example of colored coins can be seen https://cpr.sm/FoykwrH6UY[here].
[[example_9-1]]
.The metadata profile of the colored coins recorded as a coupon for a free copy of the book
====
[source,json]
----
{
"source_addresses": [
"3NpZmvSPLmN2cVFw1pY7gxEAVPCVfnWfVD"
],
"contract_url": "https://www.coinprism.info/asset/3NpZmvSPLmN2cVFw1pY7gxEAVPCVfnWfVD",
"name_short": "MasterBTC",
"name": "Free copy of \"Mastering Bitcoin\"",
"issuer": "Andreas M. Antonopoulos",
"description": "This token is redeemable for a free copy of the book \"Mastering Bitcoin\"",
"description_mime": "text/x-markdown; charset=UTF-8",
"type": "Other",
"divisibility": 0,
"link_to_website": false,
"icon_url": null,
"image_url": null,
"version": "1.0"
}
----
====
==== Mastercoin
((("meta-coin platforms","mastercoin protocol")))Mastercoin is a protocol layer on top of bitcoin that supports a platform for various applications extending the bitcoin system. Mastercoin uses the currency MST as a token for conducting Mastercoin transactions but it is not primarily a currency. Rather, it is a platform for building other things, such as user currencies, smart property tokens, de-centralized asset exchanges, and contracts. Think of Mastercoin as an application-layer protocol on top of bitcoin's financial transaction transport layer, just like HTTP runs on top of TCP.
Mastercoin operates primarily through transactions sent to and from a special bitcoin address called the((("exodus addresses"))) "exodus" address (+1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P+), just like HTTP uses a specific TCP port (port 80) to differentiate its traffic from the rest of the TCP traffic. The Mastercoin protocol is gradually transitioning from using the specialized exodus address and multi-signatures to using the OP_RETURN bitcoin operator to encode transaction metadata.
==== Counterparty
((("meta coin platforms","counterparty protocol")))Counterparty is another protocol layer implemented on top of bitcoin. Counterparty enables user currencies, tradable tokens, financial instruments, decentralized asset exchanges, and other features. Counterparty is implemented primarily using the +OP_RETURN+ operator in bitcoin's scripting language to record metadata that enhances bitcoin transactions with additional meaning. Counterparty uses the currency XCP as a token for conducting Counterparty transactions.(((range="endofrange", startref="ix_ch09-asciidoc0")))
=== Alt Coins
((("alt coins", id="ix_ch09-asciidoc1", range="startofrange")))((("currencies, alternative", id="ix_ch09-asciidoc2", range="startofrange")))The vast majority of alt coins are derived from bitcoin's source code, also known as "forks." Some are implemented "from scratch" based on the blockchain model but without using any of bitcoin's source code. Alt coins and alt chains (in the next section) are both separate implementations of blockchain technology and both forms use their own blockchain. The difference in the terms is to indicate that alt coins are primarily used as currency, whereas alt chains are used for other purposes, not primarily currency.
Strictly speaking, the first major "alt" fork of bitcoin's code was not an alt coin but the alt chain _Namecoin_, which we will discuss in the next section.
Based on the date of announcement, the first alt coin that was a fork of bitcoin appeared in August 2011; it was called _IXCoin_. IXCoin modified a few of the bitcoin parameters, specifically accelerating the creation of currency by increasing the reward to 96 coins per block.
In September 2011, _Tenebrix_ was launched. Tenebrix was the first cryptocurrency to implement an alternative proof-of-work algorithm, namely((("proof-of-work algorithm","alternative")))((("scrypt algorithm"))) _scrypt_, an algorithm originally designed for password stretching (brute-force resistance). The stated goal of Tenebrix was to make a coin that was resistant to mining with GPUs and ASICs, by using a memory-intensive algorithm. Tenebrix did not succeed as a currency, but it was the basis for Litecoin, which has enjoyed great success and has spawned hundreds of clones.
_Litecoin_, in addition to using scrypt as the proof-of-work algorithm, also implemented a faster block-generation time, targeted at 2.5 minutes instead of bitcoin's 10 minutes. The resulting currency is touted as "silver to bitcoin's gold" and is intended as a light-weight alternative currency. Due to the faster confirmation time and the 84 million total currency limit, many adherents of Litecoin believe it is better suited for retail transactions than bitcoin.
Alt coins continued to proliferate in 2011 and 2012, either based on bitcoin or on Litecoin.By 2013, there were 20 alt coins vying for position in the market. By the end of 2013, this number had exploded to 200, with 2013 quickly becoming the "year of the alt coins." The growth of alt coins continued in 2014, with more than 500 alt coins in existence at the time of writing. More than half the alt coins today are clones of Litecoin.
Creating an alt coin is easy, which is why there are now more than 500 of them. Most of the alt coins differ very slightly from bitcoin and do not offer anything worth studying. Many are in fact just attempts to enrich their creators. Among the copycats and pump-and-dump schemes, there are, however, some notable exceptions and very important innovations. These alt coins take radically different approaches or add significant innovation to bitcoin's design pattern. There are three primary areas where these alt coins differentiate from bitcoin:
* Different monetary policy
* Different proof of work or consensus mechanism
* Specific features, such as strong anonymity
For more information, see this http://mapofcoins.com[graphical timeline of alt coins and alt chains].((("alt chains","timeline of")))((("alt coins","timeline of")))
==== Evaluating an Alt Coin
((("alt coins","evaluating")))((("currencies, alternative","evaluating")))With so many alt coins out there, how does one decide which ones are worthy of attention? Some alt coins attempt to achieve broad distribution and use as currencies. Others are laboratories for experimenting on different features and monetary models. Many are just get-rich-quick schemes by their creators. To evaluate alt coins, I look at their defining characteristics and their market metrics.
Here are some questions to ask about how well an alt coin differentiates from bitcoin:
* Does the alt coin introduce a significant innovation?
* Is the difference compelling enough to attract users away from bitcoin?
* Does the alt coin address an interesting niche market or application?
* Can the alt coin attract enough miners to be secured against consensus attacks?
Here are some of the key financial and market metrics to consider:
* What is the total market capitalization of alt coin?
* How many estimated users/wallets does the alt coin have?
* How many merchants accept the alt coin?
* How many daily transactions (volume) are executed on the alt coin?
* How much value is transacted daily?
In this chapter, we will concentrate primarily on the technical characteristics and innovation potential of alt coins represented by the first set of questions.
==== Monetary Parameter Alternatives: Litecoin, Dogecoin, Freicoin
((("alt coins","monetary parameter alternatives")))((("currencies, alternative","monetary parameter alternatives")))((("monetary parameter alternatives")))Bitcoin has a few monetary parameters that give it distinctive characteristics of a deflationary fixed-issuance currency. It is limited to 21 million major currency units (or 21 quadrillion minor units), it has a geometrically declining issuance rate, and it has a 10-minute block "heartbeat," which controls the speed of transaction confirmation and currency generation. Many alt coins have tweaked the primary parameters to achieve different monetary policies. Among the hundreds of alt coins, some of the most notable examples include the following.
===== Litecoin
One of the first alt coins, released in 2011, Litecoin is the second most successful digital currency after bitcoin. Its primary innovations were the use of _scrypt_ as the proof-of-work algorithm (inherited from Tenebrix) and its faster/lighter currency parameters.
* Block generation time: 2.5 minutes
* Total currency: 84 million coins by 2140
* Consensus algorithm: Scrypt proof of work
* Market capitalization: $160 million in mid-2014
===== Dogecoin
Dogecoin was released in December 2013, based on a fork of Litecoin. Dogecoin is notable because it has a monetary policy of rapid issuance and a very high currency cap, to encourage spending and tipping. Dogecoin is also notable because it was started as a joke but became quite popular, with a large and active community, before declining rapidly in 2014.
* Block generation time: 60 seconds
* Total currency: 100,000,000,000 (100 billion) Doge by 2015
* Consensus algorithm: Scrypt proof of work
* Market capitalization: $12 million in mid-2014
===== Freicoin
Freicoin was introduced in July 2012. It is a((("demurrage currency"))) _demurrage currency_, meaning it has a negative interest rate for stored value. Value stored in Freicoin is assessed a 4.5% APR fee, to encourage consumption and discourage hoarding of money. Freicoin is notable in that it implements a monetary policy that is the exact opposite of Bitcoin's deflationary policy. Freicoin has not seen success as a currency, but it is an interesting example of the variety of monetary policies that can be expressed by alt coins.
* Block generation: 10 minutes
* Total currency: 100 million coins by 2140
* Consensus algorithm: SHA256 proof of work
* Market capitalization: $130,000 in mid-2014
==== Consensus Innovation: Peercoin, Myriad, Blackcoin, Vericoin, NXT
((("alt coins","consensus innovation")))((("consensus","innovation")))Bitcoin's consensus mechanism is based on proof of work using the SHA256 algorithm. The first alt coins introduced scrypt as an alternative proof-of-work algorithm, as a way to make mining more CPU-friendly and less susceptible to centralization with ASICs. Since then, innovation in the consensus mechanism has continued at a frenetic pace. Several alt coins adopted a variety of algorithms such as scrypt,((("Blake algorithm")))((("Groestl algorithm")))((("scrypt-N algorithm")))((("SHA3 algorithm")))((("Skein algorithm"))) scrypt-N, Skein, Groestl, SHA3, X11, Blake, and others. Some alt coins combined multiple algorithms for proof of work. In 2013, we saw the invention of an alternative to proof of work, called((("proof of stake"))) _proof of stake_, which forms the basis of many modern alt coins.
Proof of stake is a system by which existing owners of a currency can "stake" currency as interest-bearing collateral. Somewhat like a certificate of deposit (CD), participants can reserve a portion of their currency holdings, while earning an investment return in the form of new currency (issued as interest payments) and transaction fees.
===== Peercoin
Peercoin was introduced in August 2012 and is the first alt coin to use a hybrid proof-of-work and proof-of-stake algorithm to issue new currency.
* Block generation: 10 minutes
* Total currency: No limit
* Consensus algorithm: (Hybrid) proof-of-stake with initial proof-of-work
* Market capitalization: $14 million in mid-2014
===== Myriad
Myriad was introduced in February 2014 and is notable because it uses five different proof-of-work algorithms (SHA256d, Scrypt, Qubit, Skein, or Myriad-Groestl) simultaneously, with difficulty varying for each algorithm depending on miner participation. The intent is to make Myriad immune to ASIC specialization and centralization as well as much more resistant to consensus attacks, because multiple mining algorithms would have to be attacked simultaneously.
* Block generation: 30-second average (2.5 minutes target per mining algorithm)
* Total currency: 2 billion by 2024
* Consensus algorithm: Multi-algorithm proof-of-work
* Market capitalization: $120,000 in mid-2014
===== Blackcoin
Blackcoin was introduced in February 2014 and uses a proof-of-stake consensus algorithm. It is also notable for introducing "multipools," a type of mining pool that can switch between different alt coins automatically, depending on profitability.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-stake
* Market capitalization: $3.7 million in mid-2014
===== VeriCoin
VeriCoin was launched in May 2014. It uses a proof-of-stake consensus algorithm with a variable interest rate that dynamically adjusts based on market forces of supply and demand. It also is the first alt coin featuring auto-exchange to bitcoin for payment in bitcoin from the wallet.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-stake
* Market capitalization: $1.1 million in mid-2014
===== NXT
NXT (pronounced "Next") is a "pure" proof-of-stake alt coin, in that it does not use proof-of-work mining. NXT is a from-scratch implementation of a cryptocurrency, not a fork of bitcoin or any other alt coins. NXT implements many advanced features, including a name registry (similar to((("Namecoin"))) Namecoin), a decentralized asset exchange (similar to Colored Coins), integrated decentralized and secure messaging (similar to((("Bitmessage"))) Bitmessage), and stake delegation (to delegate proof-of-stake to others). NXT adherents call it a "next-generation" or 2.0 cryptocurrency.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof-of-stake
* Market capitalization: $30 million in mid-2014
==== Dual-Purpose Mining Innovation: Primecoin, Curecoin, Gridcoin
((("dual-purpose mining")))((("mining","dual-purpose")))Bitcoin's proof-of-work algorithm has just one purpose: securing the bitcoin network. Compared to traditional payment system security, the cost of mining is not very high. However, it has been criticized by many as being “wasteful." The next generation of alt coins attempt to address this concern. Dual-purpose proof-of-work algorithms solve a specific "useful" problem, while producing proof of work to secure the network. The risk of adding an external use to the currency's security is that it also adds external influence to the supply/demand curve.
===== Primecoin
Primecoin was announced in July 2013. Its proof-of-work algorithm searches for prime numbers, computing((("bi-twin prime chains")))((("Cunningham prime chains"))) Cunningham and bi-twin prime chains. Prime numbers are useful in a variety of scientific disciplines. The Primecoin blockchain contains the discovered prime numbers, thereby producing a public record of scientific discovery in parallel to the public ledger of transactions.
* Block generation: 1 minute
* Total currency: No limit
* Consensus algorithm: Proof of work with prime number chain discovery
* Market capitalization: $1.3 million in mid-2014
===== Curecoin
((("protein folding algorithms")))Curecoin was announced in May 2013. It combines a SHA256 proof-of-work algorithm with protein-folding research through the Folding@Home project. Protein folding is a computationally intensive simulation of biochemical interactions of proteins, used to discover new drug targets for curing diseases.
* Block generation: 10 minutes
* Total currency: No limit
* Consensus algorithm: Proof of work with protein-folding research
* Market capitalization: $58,000 in mid-2014
===== Gridcoin
Gridcoin was introduced in October 2013. It supplements scrypt-based proof of work with subsidies for participation in((("BOINC open grid computing"))) BOINC open grid computing. BOINC—Berkeley Open Infrastructure for Network Computing—is an open protocol for scientific research grid computing, which allows participants to share their spare computing cycles for a broad range of academic research computing. Gridcoin uses BOINC as a general-purpose computing platform, rather than to solve specific science problems such as prime numbers or protein folding.
* Block generation: 150 seconds
* Total currency: No limit
* Consensus algorithm: Proof-of-work with BOINC grid computing subsidy
* Market capitalization: $122,000 in mid-2014
==== Anonymity-Focused Alt Coins: CryptoNote, Bytecoin, Monero, Zerocash/Zerocoin, Darkcoin
((("alt coins","anonymity focused", id="ix_ch09-asciidoc3", range="startofrange")))((("currencies, alternative","anonymity focused", id="ix_ch09-asciidoc4", range="startofrange")))Bitcoin is often mistakenly characterized as "anonymous" currency. In fact, it is relatively easy to connect identities to bitcoin addresses and, using big-data analytics, connect addresses to each other to form a comprehensive picture of someone's bitcoin spending habits. Several alt coins aim to address this issue directly by focusing on strong anonymity. The first such attempt is most likely _Zerocoin_, a meta-coin protocol for preserving anonymity on top of bitcoin, introduced with a paper at the 2013 IEEE Symposium on Security and Privacy. Zerocoin will be implemented as a completely separate alt coin called Zerocash, in development at time of writing. An alternative approach to anonymity was launched with _CryptoNote_ in a paper published in October 2013. CryptoNote is a foundational technology that is implemented by a number of alt coin forks discussed next. In addition to Zerocash and CryptoNotes, there are several other independent anonymous coins, such as Darkcoin, that use stealth addresses or transaction re-mixing to deliver anonymity.
===== Zerocoin/Zerocash
Zerocoin is a theoretical approach to digital currency anonymity introduced in 2013 by researchers at Johns Hopkins. Zerocash is an alt-coin implementation of Zerocoin that is in development and not yet released.
===== CryptoNote
CryptoNote is a reference implementation alt coin that provides the basis for anonymous digital cash. It was introduced in October 2013. It is designed to be forked into different implementations and has a built-in periodic reset mechanism that makes it unusable as a currency itself. Several alt coins have been spawned from CryptoNote, including Bytecoin (BCN), Aeon (AEON), Boolberry (BBR), duckNote (DUCK), Fantomcoin (FCN), Monero (XMR), MonetaVerde (MCN), and Quazarcoin (QCN). CryptoNote is also notable for being a complete ground-up implementation of a crypto-currency, not a fork of bitcoin.
===== Bytecoin
((("Application Specific Integrated Circuit (ASIC)")))((("Graphical Processing Units (GPUs)")))Bytecoin was the first implementation spawned from CryptoNote, offering a viable anonymous currency based on the CryptoNote technology. Bytecoin was launched in July 2012. Note that there was a previous alt coin named Bytecoin with currency symbol BTE, whereas the CryptoNote-derived Bytecoin has the currency symbol BCN. Bytecoin uses the Cryptonight proof-of-work algorithm, which requires access to at least 2 MB of RAM per instance, making it unsuitable for GPU or ASIC mining. Bytecoin inherits ring signatures, unlinkable transactions, and blockchain analysis–resistant anonymity from CryptoNote.
* Block generation: 2 minutes
* Total currency: 184 billion BCN
* Consensus algorithm: Cryptonight proof of work
* Market capitalization: $3 million in mid-2014
===== Monero
Monero is another implementation of CryptoNote. It has a slightly flatter issuance curve than Bytecoin, issuing 80% of the currency in the first four years. It offers the same anonymity features inherited from CryptoNote.
* Block generation: 1 minute
* Total currency: 18.4 million XMR
* Consensus algorithm: Cryptonight proof of work
* Market capitalization: $5 million in mid-2014
===== Darkcoin
Darkcoin was launched in January 2014. Darkcoin implements anonymous currency using a re-mixing protocol for all transactions called DarkSend. Darkcoin is also notable for using 11 rounds of different hash functions((("proof-of-work algorithm","for Darkcoin"))) (blake, bmw, groestl, jh, keccak, skein, luffa, cubehash, shavite, simd, echo) for the proof-of-work algorithm.
* Block generation: 2.5 minutes
* Total currency: Maximum 22 million DRK
* Consensus algorithm: Multi-algorithm multi-round proof of work
* Market capitalization: $19 million in mid-2014(((range="endofrange", startref="ix_ch09-asciidoc4")))(((range="endofrange", startref="ix_ch09-asciidoc3")))(((range="endofrange", startref="ix_ch09-asciidoc2")))(((range="endofrange", startref="ix_ch09-asciidoc1")))
=== Noncurrency Alt Chains
((("chains, alternative","noncurrency", id="ix_ch09-asciidoc5", range="startofrange")))((("non-currency alt chains", id="ix_ch09-asciidoc6", range="startofrange")))Alt chains are alternative implementations of the blockchain design pattern, which are not primarily used as currency. Many include a currency, but the currency is used as a token for allocating something else, such as a resource or a contract. The currency, in other words, is not the main point of the platform; it is a secondary feature.
==== Namecoin
Namecoin was the first fork of the bitcoin code. Namecoin is a decentralized key-value registration and transfer platform using a blockchain. It supports a global domain-name registry similar to the domain-name registration system on the Internet. Namecoin is currently used as an alternative((("domain name service (DNS)"))) domain name service (DNS) for the root-level domain +.bit+. Namecoin also can be used to register names and key-value pairs in other namespaces; for storing things like email addresses, encryption keys, SSL certificates, file signatures, voting systems, stock certificates; and a myriad of other applications.
The Namecoin system includes the Namecoin currency (symbol NMC), which is used to pay transaction fees for registration and transfer of names. At current prices, the fee to register a name is 0.01 NMC or approximately 1 US cent. As in bitcoin, the fees are collected by namecoin miners.
Namecoin's basic parameters are the same as bitcoin's:
* Block generation: 10 minutes
* Total currency: 21 million NMC by 2140
* Consensus algorithm: SHA256 proof of work
* Market capitalization: $10 million in mid-2014
Namecoin's namespaces are not restricted, and anyone can use any namespace in any way. However, certain namespaces have an agreed-upon specification so that when it is read from the blockchain, application-level software knows how to read and proceed from there. If it is malformed, then whatever software you used to read from the specific namespace will throw an error. Some of the popular namespaces are:
* +d/+ is the domain-name namespace for +.bit+ domains
* +id/+ is the namespace for storing person identifiers such as email addresses, PGP keys, and so on
* +u/+ is an additional, more structured specification to store identities (based on openspecs)
((("blockchains","Namecoin")))The Namecoin client is very similar to Bitcoin Core, because it is derived from the same source code. Upon installation, the client will download a full copy of the Namecoin blockchain and then will be ready to query and register names. There are three main commands: ((("Namecoin","commands")))
+name_new+:: Query or preregister a name
+name_firstupdate+:: Register a name and make the registration public
+name_update+:: Change the details or refresh a name registration
For example, to register the domain +mastering-bitcoin.bit+, we use the command +name_new+ as follows:
[source,bash]
----
$ namecoind name_new d/mastering-bitcoin
----
[source,json]
----
[
"21cbab5b1241c6d1a6ad70a2416b3124eb883ac38e423e5ff591d1968eb6664a",
"a05555e0fc56c023"
]
----
The +name_new+ command registers a claim on the name, by creating a hash of the name with a random key. The two strings returned by +name_new+ are the hash and the random key (+a05555e0fc56c023+ in the preceding example) that can be used to make the name registration public. Once that claim has been recorded on the Namecoin blockchain it can be converted to a public registration with the +name_firstupdate+ command, by supplying the random key:
----
$ namecoind name_firstupdate d/mastering-bitcoin a05555e0fc56c023 "{"map": {"www": {"ip":"1.2.3.4"}}}}"
b7a2e59c0a26e5e2664948946ebeca1260985c2f616ba579e6bc7f35ec234b01
----
This example will map the domain name +www.mastering-bitcoin.bit+ to IP address 1.2.3.4. The hash returned is the transaction ID that can be used to track this registration. You can see what names are registered to you by running the +name_list+ command:
----
$ namecoind name_list
----
====
[source,json]
----
[
{
"name" : "d/mastering-bitcoin",
"value" : "{map: {www: {ip:1.2.3.4}}}}",
"address" : "NCccBXrRUahAGrisBA1BLPWQfSrups8Geh",
"expires_in" : 35929
}
]
----
====
Namecoin registrations need to be updated every 36,000 blocks (approximately 200 to 250 days). The +name_update+ command has no fee and therefore renewing domains in Namecoin is free. Third-party providers can handle registration, automatic renewal, and updating via a web interface, for a small fee. With a third-party provider you avoid the need to run a Namecoin client, but you lose the independent control of a decentralized name registry offered by Namecoin.
==== Bitmessage
((("messages, sending in blockchain")))Bitmessage is a bitcoin alt chain that implements a decentralized secure messaging service, essentially a server-less encrypted email system. Bitmessage allows users to compose and send messages to each other, using a Bitmessage address. The messages operate in much the same way as a bitcoin transaction, but they are transient—they do not persist beyond two days and if not delivered to the destination node in that time, they are lost. Senders and recipients are pseudonymous—they have no identifiers other than a bitmessage address—but are strongly authenticated, meaning that messages cannot be "spoofed." Bitmessages are encrypted to the recipient and therefore the Bitmessage network is resistant to holistic surveillance—an eavesdropper has to compromise the recipient's device in order to intercept messages.
==== Ethereum
((("contracts, in Ethereum")))Ethereum is a Turing-complete contract processing and execution platform based on a blockchain ledger. It is not a clone of Bitcoin, but a completely independent design and implementation. Ethereum has a built-in currency, called _ether_, which is required in order to pay for contract execution. Ethereum's blockchain records _contracts_, which are expressed in a low-level, byte code–like, Turing-complete language. Essentially, a contract is a program that runs on every node in the Ethereum system. Ethereum contracts can store data, send and receive ether payments, store ether, and execute an infinite range (hence Turing-complete) of computable actions, acting as decentralized autonomous software agents.
Ethereum can implement quite complex systems that are otherwise implemented as alt chains themselves. For example, the following is a Namecoin-like name registration contract written in Ethereum (or more accurately, written in a high-level language that can be compiled to Ethereum code): (((range="endofrange", startref="ix_ch09-asciidoc6")))(((range="endofrange", startref="ix_ch09-asciidoc5")))
[source,python]
----
if !contract.storage[msg.data[0]]: # Is the key not yet taken?
# Then take it!
contract.storage[msg.data[0]] = msg.data[1]
return(1)
else:
return(0) // Otherwise do nothing
----
=== Future of Currencies
The future of cryptographic currencies overall is even brighter than the future of bitcoin. Bitcoin introduced a completely new form of decentralized organization and consensus that has spawned hundreds of incredible innovations. These inventions will likely affect broad sectors of the economy, from distributed systems science to finance, economics, currencies, central banking, and corporate governance. Many human activities that previously required centralized institutions or organizations to function as authoritative or trusted points of control can now be decentralized. The invention of the blockchain and consensus system will significantly reduce the cost of organization and coordination on large-scale systems, while removing opportunities for concentration of power, corruption, and regulatory capture.

157
ch10.asciidoc
Unescape View File

@ -1,78 +1,79 @@
[[ch10]]
== Bitcoin Security
Securing bitcoin is challenging because bitcoin is not an abstract reference to value, like a balance in a bank account. Bitcoin is very much like digital cash or gold. You've probably heard the expression "Possession is nine tenths of the law". Well, in bitcoin, possession is ten tenths of the law. Possession of the keys to unlock the bitcoin is equivalent to possession of cash or a chunk of precious metal. You can lose it, misplace it, have it stolen or accidentally give the wrong amount to someone. In every one of those cases, the end-user would have no recourse, just as if they dropped cash on a public sidewalk.
However, bitcoin has capabilities that cash, gold and bank accounts do not. A bitcoin wallet, containing your keys, can be backed up like any file. It can be stored in multiple copies, even printed on paper for hard-copy backup. You can't "backup" cash, gold or bank accounts. Bitcoin is different enough from anything that has come before that we need to think about bitcoin security in a novel way too.
=== Security principles
The core principle in bitcoin is de-centralization and it has important implications for security. A centralized model, such as a traditional bank or payment network, depends on access control and vetting to keep bad actors out of the system. By comparison, a de-centralized system like bitcoin pushes the responsibility and control to the end-users. Since security of the network is based on Proof-of-Work, not access control, the network can be open and no encryption is required for bitcoin traffic.
On a traditional payment network, such a credit card system, the "payment" is really open-ended because it contains the user's private identifier (the credit card number). After the initial charge, anyone with access to the identifier can "pull" funds and charge the owner again and again. Thus, the payment network has to be secured end-to-end with encryption and must ensure that no eavesdroppers or intermediaries can compromise the payment traffic, in transit or when it is stored (at rest). If a bad actor gains access to the system, they can compromise current transactions _and_ payment tokens that can be used to create new transactions. Worse, when customer data is compromised, the customers are exposed to identity theft and must take action to prevent fraudulent use of the compromised accounts.
Bitcoin is dramatically different. A bitcoin transaction authorizes only a specific value to a specific recipient and cannot be forged or modified. It does not reveal any private information, such as the identities of the parties and cannot be used to authorize additional payments. Therefore, a bitcoin payment network does not need to be encrypted or protected from eavesdropping. In fact, you can broadcast bitcoin transactions over an open public channel, such as unsecured Wifi or Bluetooth, with no loss of security.
Bitcoin's de-centralized security model puts a lot of power in the hands of the end-users. With that power comes responsibility for maintaining the secrecy of the keys. For most users that is not easy to do, especially on general purpose computing devices, such as Internet-connected smartphones or laptops. Whereas bitcoin's de-centralized model prevents the type of mass compromise seen with credit cards, many end-users are not able to adequately secure their keys and get hacked one by one.
==== Developing Bitcoin Systems Securely
The most important principle for bitcoin developers is de-centralization. Most developers will be familiar with centralized security models and may be tempted to apply these models to their bitcoin applications, with disastrous results.
Bitcoin's security relies on decentralized control over keys and on independent transaction validation by miners. If you want to leverage bitcoin's security, you need to ensure that you remain within the bitcoin security model. In simple terms: don't take control of keys away from users and don't take transactions off the blockchain.
For example, many early bitcoin exchanges concentrated all user funds in a single "hot" wallet with keys stored on a single server. Such a design removes control from users and centralizes control over keys to a single system. Many such systems have been hacked with disastrous consequences for their customers.
Another common mistake is to take transactions "off blockchain" in a misguided effort to reduce transaction fees or accelerate transaction processing. An "off blockchain" system will record transactions on an internal, centralized ledger and only occasionally synchronize them to the bitcoin blockchain. This practice, again, substitutes de-centralized bitcoin security with a proprietary and centralized approach. When transactions are off blockchain, improperly secured centralized ledgers can be falsified, diverting funds and depleting reserves, unnoticed.
Unless you are prepared to invest heavily in operational security, multiple layers of access control, and audits (as the traditional banks do) you should think very carefully before taking funds outside of bitcoin's de-centralized security context. Even if you have the funds and discipline to implement a robust security model, such a design merely replicates the fragile model of traditional financial networks, plagued by identity theft, corruption, and embezzlement. To take advantage of bitcoin's unique decentralized security model, you have to avoid the temptation of centralized architectures which may feel familiar but ultimately subvert bitcoin's security.
==== The Root of Trust
Traditional security architecture is based upon a concept called the _root of trust_, which is a trusted core used as the foundation for the security of the overall system or application. Security architecture is developed around the root of trust as a series of concentric circles, like layers in an onion, extending trust outwards from the root. Each layer builds upon the more-trusted inner layer using access controls, digital signatures, encryption, and other security primitives. As software systems become more complex, they are more likely to contain bugs, which make them vulnerable to security compromise. As a result the more complex a software system becomes, the harder it is to secure. The root of trust concept ensures that most of the trust is placed within the least complex part of the system, and therefore least vulnerable, parts of the system while more complex software is layered around it. This security architecture is repeated at different scales, first establishing a root of trust within the hardware of a single system, then extending that root of trust through the operating system to higher-level system services, and finally across many servers layered in concentric circles of diminishing trust.
Bitcoin security architecture is different. In bitcoin the consensus system creates a trusted public ledger which is completely de-centralized. A correctly validated blockchain uses the genesis block as the root of trust, building a chain of trust up to the current block. Bitcoin systems can and should use the blockchain as their root of trust. When designing a complex bitcoin application that consists of services on many different systems, you should carefully examine the security architecture in order to ascertain where trust is being placed. Ultimately the only thing that should be explicitly trusted is a fully validated blockchain. If your application explicitly or implicitly vests trust in anything but the blockchain, that should be a source of concern as it introduces points of vulnerability. A good method to evaluate the security architecture of your application is to consider each individual component and evaluate a hypothetical scenario where that component is completely compromised and under the control of a malicious actor. Take each component of your application, in turn, and assess the impacts on the overall security if that component is compromised. If your application is no longer secure when components are compromised that shows that you have implicitly misplaced trust in those components. A bitcoin application without vulnerabilities should be vulnerable only to a compromise of the bitcoin consensus mechanism. Meaning that its root of trust is based on the strongest part of the bitcoin security architecture.
The numerous examples of bitcoin exchange failures serve to underscore this point as their security architecture and design fails even under the most casual scrutiny. Their centralized implementations invested trust explicitly in numerous components outside the bitcoin blockchain such as hot wallets, centralized ledger databases, vulnerable encryption keys, etc. Worse yet, in most cases those trusted components lacked even the most rudimentary security controls.
=== User Security Best Practices
Humans have used physical security controls for thousands of years. By comparison, our experience with digital security is less than fifty years old. Modern general-purpose operating systems are not very secure and not particularly suited to storing digital money. Our computers are constantly exposed to external threats via always-on Internet connections. They run thousands of software components from hundreds of authors, often with unconstrained access to the user's files. A single piece of rogue software, among the many thousands installed on your computer, can compromise your keyboard and files, stealing any bitcoin stored on wallet applications. The level of computer maintenance required to keep a computer virus-free and trojan-free is beyond the skill level of all but a tiny minority of computer users.
Despite decades of research and advancements in information security, digital assets are still woefully vulnerable to a determined adversary. Even the most highly protected and very restricted systems, in financial services companies, intelligence agencies, and defense contractors are frequently breached. Bitcoin creates digital assets that have intrinsic value and can be stolen and diverted to new owners instantly and irrevocably. This creates a massive incentive for hackers. Until now, hackers had to convert identity information or account tokens - like credit cards, bank accounts, etc. - into value after compromising them. Despite the difficulty of fencing and laundering financial information, we have seen ever escalating thefts. Bitcoin escalates this problem because it doesn't need to be fenced or laundered, it is intrinsic value within a digital asset.
Fortunately, bitcoin also creates the incentives to improve computer security. Whereas previously, the risk of computer compromise was vague and indirect, bitcoin makes these risks clear and obvious. Holding bitcoin on a computer serves to focus the user's mind on the need for improved computer security. As a direct result of the proliferation and increased adoption of bitcoin and other digital currencies we have seen an escalation in both hacking techniques and security solutions. In simple terms, hackers now have a very juicy target and users have a clear incentive to defend themselves.
Over the last three years, as a direct result of bitcoin adoption, we have seen tremendous innovation in the realm of information security in the form of hardware encryption, key storage and hardware wallets, multi-signature technology, and digital escrow. In the following sections we will examine various best practices for practical user security.
==== Physical Bitcoin Storage
Since most users are far more comfortable with physical security than information security, a very effective method for protecting bitcoin is to convert them into physical form. Bitcoin keys are nothing more than long numbers. This means that they can be stored in a physical form, such as printed on paper or etched on a metal coin. Securing the keys then becomes as simple as physically securing the printed copy of the bitcoin keys. A set of bitcoin keys that is printed on paper is called a "paper wallet" and there are many free tools that can be used to create them. I personally keep the vast majority of my bitcoins (99% or more) stored on paper wallets, encrypted with BIP0038, with multiple copies locked in safes. Keeping bitcoin offline is called _cold storage_ and it is one of the most effective security techniques. A cold storage system is one where the keys are generated on an offline system (one never connected to the Internet) and stored offline either on paper or on digital media, such as a USB memory stick.
==== Hardware Wallets
In the longer term, bitcoin security will increasingly be implemented with hardware tamper-proof wallets. Unlike a smartphone or desktop computer, a purpose-built bitcoin hardware wallet has only one purpose and function - holding bitcoins securely. Without general purpose software to compromise and with limited interfaces, hardware wallets can deliver an almost foolproof level of security to non-expert users. I expect to see hardware wallets becoming the predominant method of bitcoin storage. For an example of such a hardware wallet, see the Trezor (http://www.bitcointrezor.com/).
==== Balancing Risk (loss vs. theft)
While most users are, rightly, concerned about theft, there is an even bigger risk of loss. Data files get lost all the time, but if they contain bitcoin the loss is much more painful. In the effort to secure their bitcoin wallets, users must be very careful not to go too far and end up losing the bitcoin. In the summer of 2010, a well known bitcoin awareness and education project lost almost 7,000 bitcoins. In an effort to prevent theft, the owners had implemented a complex series of encrypted backups. In the end they accidentally lost the encryption keys, making the backups worthless and losing a fortune. Like hiding money by burying it in the desert, if you do it too well you might not be able to find where you buried it.
==== Diversifying Risk
Would you carry your entire net-worth in cash in your wallet? Most people would consider that reckless, yet bitcoin users often keep all their bitcoin in a single wallet. Instead, users should spread the risk among multiple and diverse bitcoin wallets. The prudent user will keep only a small fraction, perhaps less than 5%, of their bitcoins in an online or mobile wallet as "pocket change". The rest should be split between a few different storage mechanisms, such as a desktop wallet and offline (cold storage).
==== Multi-sig and Governance
Whenever a company or individual stores large amounts of bitcoin, they should consider using a multi-signature bitcoin address. Multi-signature addresses secure funds by requiring more than one signature to make a payment. The signing keys should be stored in a number of different locations and under the control of different people. In a corporate environment, for example, the keys should be generated independently and held by several company executives, to ensure no single person can compromise the funds. Multi-signature addresses can also offer redundancy, where a single person holds several keys that are stored in different locations.
==== Survivability
One important security consideration that is often overlooked is availability, especially in the context of incapacity or death of the key holder. Bitcoin users are told to use complex passwords and keep their keys secure and private, not sharing them with anyone. Unfortunately, that practice makes it almost impossible for the user's family to recover any funds if the user is not available to unlock them. In most cases in fact, the families of bitcoin users may be completely unaware of the existence of bitcoin funds.
If you have a lot of bitcoin, you should consider sharing access details with a trusted relative or lawyer. A more complex survivability scheme can be set up with multi-signature access and estate planning through a lawyer specialized as a "digital asset executor".
=== Conclusion
Bitcoin is a completely new, unprecedented and complex technology. Over time we will develop better security tools and practices that are easier to use by non-experts. For now, bitcoin users can use many of the tips above to enjoy a secure and trouble-free bitcoin experience.
[[ch10]]
== Bitcoin Security
((("security", id="ix_ch10-asciidoc0", range="startofrange")))Securing bitcoin is challenging because bitcoin is not an abstract reference to value, like a balance in a bank account. Bitcoin is very much like digital cash or gold. You've probably heard the expression, "Possession is nine-tenths of the law." Well, in bitcoin, possession is ten-tenths of the law. Possession of the keys to unlock the bitcoin is equivalent to possession of cash or a chunk of precious metal. You can lose it, misplace it, have it stolen, or accidentally give the wrong amount to someone. In every one of these cases, users have no recourse, just as if they dropped cash on a public sidewalk.
However, bitcoin has capabilities that cash, gold, and bank accounts do not. A bitcoin wallet, containing your keys, can be backed up like any file. It can be stored in multiple copies, even printed on paper for hard-copy backup. You can't "back up" cash, gold, or bank accounts. Bitcoin is different enough from anything that has come before that we need to think about bitcoin security in a novel way too.
=== Security Principles
((("security","principles of")))The core principle in bitcoin is decentralization and it has important implications for security. A centralized model, such as a traditional bank or payment network, depends on access control and vetting to keep bad actors out of the system. By comparison, a decentralized system like bitcoin pushes the responsibility and control to the users. Because security of the network is based on proof of work, not access control, the network can be open and no encryption is required for bitcoin traffic.
On a((("credit card payment system")))((("payment networks, traditional"))) traditional payment network, such as a credit card system, the payment is open-ended because it contains the user's private identifier (the credit card number). After the initial charge, anyone with access to the identifier can "pull" funds and charge the owner again and again. Thus, the payment network has to be secured end-to-end with encryption and must ensure that no((("eavesdroppers"))) eavesdroppers or intermediaries can compromise the payment traffic, in transit or when it is stored (at rest). If a bad actor gains access to the system, he can compromise current transactions _and_ payment tokens that can be used to create new transactions. Worse, when customer data is compromised, the customers are exposed to identity theft and must take action to prevent fraudulent use of the compromised accounts.
Bitcoin is dramatically different. A bitcoin transaction authorizes only a specific value to a specific recipient and cannot be forged or modified. It does not reveal any private information, such as the identities of the parties, and cannot be used to authorize additional payments. Therefore, a bitcoin payment network does not need to be encrypted or protected from eavesdropping. In fact, you can broadcast bitcoin transactions over an open public channel, such as unsecured WiFi or Bluetooth, with no loss of security.
Bitcoin's decentralized security model puts a lot of power in the hands of the users. With that power comes responsibility for maintaining the secrecy of the keys. For most users that is not easy to do, especially on general-purpose computing devices such as Internet-connected smartphones or laptops. Although bitcoin's decentralized model prevents the type of mass compromise seen with credit cards, many users are not able to adequately secure their keys and get hacked, one by one.
==== Developing Bitcoin Systems Securely
((("bitcoin","system security")))((("security","centralized controls and")))The most important principle for bitcoin developers is decentralization. Most developers will be familiar with centralized security models and might be tempted to apply these models to their bitcoin applications, with disastrous results.
Bitcoin's security relies on decentralized control over keys and on independent transaction validation by miners. If you want to leverage Bitcoin's security, you need to ensure that you remain within the Bitcoin security model. In simple terms: don't take control of keys away from users and don't take transactions off the blockchain.
For example, many early bitcoin exchanges concentrated all user funds in a single "hot" wallet with keys stored on a single server. Such a design removes control from users and centralizes control over keys in a single system. Many such systems have been hacked, with disastrous consequences for their customers.
((("transactions","taking off blockchain")))Another common mistake is to take transactions "off blockchain" in a misguided effort to reduce transaction fees or accelerate transaction processing. An "off blockchain" system will record transactions on an internal, centralized ledger and only occasionally synchronize them to the bitcoin blockchain. This practice, again, substitutes decentralized bitcoin security with a proprietary and centralized approach. When transactions are off blockchain, improperly secured centralized ledgers can be falsified, diverting funds and depleting reserves, unnoticed.
Unless you are prepared to invest heavily in operational security, multiple layers of access control, and audits (as the traditional banks do) you should think very carefully before taking funds outside of Bitcoin's decentralized security context. Even if you have the funds and discipline to implement a robust security model, such a design merely replicates the fragile model of traditional financial networks, plagued by identity theft, corruption, and embezzlement. To take advantage of Bitcoin's unique decentralized security model, you have to avoid the temptation of centralized architectures that might feel familiar but ultimately subvert Bitcoin's security.
==== The Root of Trust
((("root of trust")))((("security","root of trust")))Traditional security architecture is based upon a concept called the _root of trust_, which is a trusted core used as the foundation for the security of the overall system or application. Security architecture is developed around the root of trust as a series of concentric circles, like layers in an onion, extending trust outward from the center. Each layer builds upon the more-trusted inner layer using access controls, digital signatures, encryption, and other security primitives. As software systems become more complex, they are more likely to contain bugs, which make them vulnerable to security compromise. As a result, the more complex a software system becomes, the harder it is to secure. The root of trust concept ensures that most of the trust is placed within the least complex part of the system, and therefore least vulnerable, parts of the system, while more complex software is layered around it. This security architecture is repeated at different scales, first establishing a root of trust within the hardware of a single system, then extending that root of trust through the operating system to higher-level system services, and finally across many servers layered in concentric circles of diminishing trust.
Bitcoin security architecture is different. In Bitcoin, the consensus system creates a trusted public ledger that is completely decentralized. A correctly validated blockchain uses the genesis block as the root of trust, building a chain of trust up to the current block. Bitcoin systems can and should use the blockchain as their root of trust. When designing a complex bitcoin application that consists of services on many different systems, you should carefully examine the security architecture in order to ascertain where trust is being placed. Ultimately, the only thing that should be explicitly trusted is a fully validated blockchain. If your application explicitly or implicitly vests trust in anything but the blockchain, that should be a source of concern because it introduces vulnerability. A good method to evaluate the security architecture of your application is to consider each individual component and evaluate a hypothetical scenario where that component is completely compromised and under the control of a malicious actor. Take each component of your application, in turn, and assess the impacts on the overall security if that component is compromised. If your application is no longer secure when components are compromised, that shows you have misplaced trust in those components. A bitcoin application without vulnerabilities should be vulnerable only to a compromise of the bitcoin consensus mechanism, meaning that its root of trust is based on the strongest part of the bitcoin security architecture.
The numerous examples of hacked bitcoin exchanges serve to underscore this point because their security architecture and design fails even under the most casual scrutiny. These centralized implementations had invested trust explicitly in numerous components outside the bitcoin blockchain, such as hot wallets, centralized ledger databases, vulnerable encryption keys, and similar schemes.
=== User Security Best Practices
((("security","user", id="ix_ch10-asciidoc1", range="startofrange")))((("user security", id="ix_ch10-asciidoc2", range="startofrange")))Humans have used physical security controls for thousands of years. By comparison, our experience with digital security is less than 50 years old. ((("operating systems, bitcoin security and")))Modern general-purpose operating systems are not very secure and not particularly suited to storing digital money. Our computers are constantly exposed to external threats via always-on Internet connections. They run thousands of software components from hundreds of authors, often with unconstrained access to the user's files. A single piece of rogue software, among the many thousands installed on your computer, can compromise your keyboard and files, stealing any bitcoin stored in wallet applications. The level of computer maintenance required to keep a computer virus-free and trojan-free is beyond the skill level of all but a tiny minority of computer users.
Despite decades of research and advancements in information security, digital assets are still woefully vulnerable to a determined adversary. Even the most highly protected and restricted systems, in financial services companies, intelligence agencies, and defense contractors, are frequently breached. Bitcoin creates digital assets that have intrinsic value and can be stolen and diverted to new owners instantly and irrevocably. ((("hackers")))This creates a massive incentive for hackers. Until now, hackers had to convert identity information or account tokens—such as credit cards, and bank accounts—into value after compromising them. Despite the difficulty of fencing and laundering financial information, we have seen ever-escalating thefts. Bitcoin escalates this problem because it doesn't need to be fenced or laundered; it is intrinsic value within a digital asset.
Fortunately, bitcoin also creates the incentives to improve computer security. Whereas previously the risk of computer compromise was vague and indirect, bitcoin makes these risks clear and obvious. Holding bitcoin on a computer serves to focus the user's mind on the need for improved computer security. As a direct result of the proliferation and increased adoption of bitcoin and other digital currencies, we have seen an escalation in both hacking techniques and security solutions. In simple terms, hackers now have a very juicy target and users have a clear incentive to defend themselves.
Over the past three years, as a direct result of bitcoin adoption, we have seen tremendous innovation in the realm of information security in the form of hardware encryption, key storage and hardware wallets, multi-signature technology, and digital escrow. In the following sections we will examine various best practices for practical user security.
==== Physical Bitcoin Storage
((("backups","cold-storage wallets")))((("bitcoin","storage, physical")))((("cold-storage wallets")))((("paper wallets")))((("user security","physical bitcoin storage")))Because most users are far more comfortable with physical security than information security, a very effective method for protecting bitcoins is to convert them into physical form. Bitcoin keys are nothing more than long numbers. This means that they can be stored in a physical form, such as printed on paper or etched on a metal coin. Securing the keys then becomes as simple as physically securing the printed copy of the bitcoin keys. A set of bitcoin keys that is printed on paper is called a "paper wallet," and there are many free tools that can be used to create them. I personally keep the vast majority of my bitcoins (99% or more) stored on paper wallets, encrypted with BIP0038, with multiple copies locked in safes. Keeping bitcoin offline is called _cold storage_ and it is one of the most effective security techniques. A cold storage system is one where the keys are generated on an offline system (one never connected to the Internet) and stored offline either on paper or on digital media, such as a USB memory stick.
==== Hardware Wallets
((("hardware wallets")))((("user security","hardware wallets")))((("wallets","hardware")))In the long term, bitcoin security increasingly will take the form of hardware tamper-proof wallets. Unlike a smartphone or desktop computer, a bitcoin hardware wallet has just one purpose: to hold bitcoins securely. Without general-purpose software to compromise and with limited interfaces, hardware wallets can deliver an almost foolproof level of security to nonexpert users. I expect to see hardware wallets become the predominant method of bitcoin storage. For an example of such a hardware wallet, see the((("Trezor wallet"))) http://www.bitcointrezor.com/[Trezor].
==== Balancing Risk
((("risk, security")))((("user security","risk, balancing")))Although most users are rightly concerned about bitcoin theft, there is an even bigger risk. Data files get lost all the time. If they contain bitcoin, the loss is much more painful. In the effort to secure their bitcoin wallets, users must be very careful not to go too far and end up losing the bitcoin. In the summer of 2010, a well-known bitcoin awareness and education project lost almost 7,000 bitcoins. In their effort to prevent theft, the owners had implemented a complex series of encrypted backups. In the end they accidentally lost the encryption keys, making the backups worthless and losing a fortune. Like hiding money by burying it in the desert, if you secure your bitcoin too well you might not be able to find it again.
==== Diversifying Risk
((("user security","risk, diversifying")))Would you carry your entire net worth in cash in your wallet? Most people would consider that reckless, yet bitcoin users often keep all their bitcoin in a single wallet. Instead, users should spread the risk among multiple and diverse bitcoin wallets. Prudent users will keep only a small fraction, perhaps less than 5%, of their bitcoins in an online or mobile wallet as "pocket change." The rest should be split between a few different storage mechanisms, such as a desktop wallet and offline (cold storage).
==== Multi-sig and Governance
((("corporations, multi-sig governance and")))((("governance")))((("multi-signature addresses","security and")))((("security","governance")))((("security","multi-signature addresses and")))Whenever a company or individual stores large amounts of bitcoin, they should consider using a multi-signature bitcoin address. Multi-signature addresses secure funds by requiring more than one signature to make a payment. The signing keys should be stored in a number of different locations and under the control of different people. In a corporate environment, for example, the keys should be generated independently and held by several company executives, to ensure no single person can compromise the funds. Multi-signature addresses can also offer redundancy, where a single person holds several keys that are stored in different locations.
==== Survivability
((("bitcoin","death of owner and")))((("death of owners")))((("security","death of owner and")))((("security","survivability")))((("survivability")))One important security consideration that is often overlooked is availability, especially in the context of incapacity or death of the key holder. Bitcoin users are told to use complex passwords and keep their keys secure and private, not sharing them with anyone. Unfortunately, that practice makes it almost impossible for the user's family to recover any funds if the user is not available to unlock them. In most cases, in fact, the families of bitcoin users might be completely unaware of the existence of the bitcoin funds.
If you have a lot of bitcoin, you should consider sharing access details with a trusted relative or lawyer. A more complex survivability scheme can be set up with multi-signature access and estate planning through a lawyer specialized as a "digital asset executor."
=== Conclusion
Bitcoin is a completely new, unprecedented, and complex technology. Over time we will develop better security tools and practices that are easier to use by nonexperts. For now, bitcoin users can use many of the tips discussed here to enjoy a secure and trouble-free bitcoin experience.(((range="endofrange", startref="ix_ch10-asciidoc2")))(((range="endofrange", startref="ix_ch10-asciidoc1")))(((range="endofrange", startref="ix_ch10-asciidoc0")))

40
code/pycoin_example.py
Unescape View File

@ -0,0 +1,40 @@
#!/usr/bin/env python
from pycoin.key import Key
from pycoin.key.validate import is_address_valid, is_wif_valid
from pycoin.services import spendables_for_address
from pycoin.tx.tx_utils import create_signed_tx
def get_address(which):
while 1:
print("enter the %s address=> " % which, end='')
address = input()
is_valid = is_address_valid(address)
if is_valid:
return address
print("invalid address, please try again")
src_address = get_address("source")
spendables = spendables_for_address(src_address)
print(spendables)
while 1:
print("enter the WIF for %s=> " % src_address, end='')
wif = input()
is_valid = is_wif_valid(wif)
if is_valid:
break
print("invalid wif, please try again")
key = Key.from_text(wif)
if src_address not in (key.address(use_uncompressed=False), key.address(use_uncompressed=True)):
print("** WIF doesn't correspond to %s" % src_address)
print("The secret exponent is %d" % key.secret_exponent())
dst_address = get_address("destination")
tx = create_signed_tx(spendables, payables=[dst_address], wifs=[wif])
print("here is the signed output transaction")
print(tx.as_hex())

61
glossary.asciidoc
Unescape View File

@ -0,0 +1,61 @@
[preface]
== Quick Glossary
This quick glossary contains many of the terms used in relation to bitcoin. These terms are used throughout the book, so bookmark this for a quick reference.
address::
A bitcoin address looks like +1DSrfJdB2AnWaFNgSbv3MZC2m74996JafV+. It consists of a string of letters and numbers starting with a "1" (number one). Just like you ask others to send an email to your email address, you would ask others to send you bitcoin to your bitcoin address.((("bitcoin address")))((("address", see="bitcoin address")))((("public key", see="bitcoin address")))
bip::
Bitcoin Improvement Proposals. A set of proposals that members of the bitcoin community have submitted to improve bitcoin. For example, BIP0021 is a proposal to improve the bitcoin uniform resource identifier (URI) scheme.((("bip")))
bitcoin::
The name of the currency unit (the coin), the network, and the software.((("bitcoin")))
block::
A grouping of transactions, marked with a timestamp, and a fingerprint of the previous block. The block header is hashed to produce a proof of work, thereby validating the transactions. Valid blocks are added to the main blockchain by network consensus.((("block")))
blockchain::
A list of validated blocks, each linking to its predecessor all the way to the genesis block.((("blockchain")))
confirmations::
Once a transaction is included in a block, it has one confirmation. As soon as _another_ block is mined on the same blockchain, the transaction has two confirmations, and so on. Six or more confirmations is considered sufficient proof that a transaction cannot be reversed.((("confirmations")))
difficulty::
A network-wide setting that controls how much computation is required to produce a proof of work.((("difficulty")))
difficulty target::
A difficulty at which all the computation in the network will find blocks approximately every 10 minutes.((("target difficulty")))
difficulty retargeting::
A network-wide recalculation of the difficulty that occurs once every 2,106 blocks and considers the hashing power of the previous 2,106 blocks.((("difficulty retargeting")))
fees::
The sender of a transaction often includes a fee to the network for processing the requested transaction. Most transactions require a minimum fee of 0.5 mBTC.((("fees")))
hash::
A digital fingerprint of some binary input.((("hash")))
genesis block::
The first block in the blockchain, used to initialize the cryptocurrency.((("genesis block")))
miner::
A network node that finds valid proof of work for new blocks, by repeated hashing.((("miner")))
network::
A peer-to-peer network that propagates transactions and blocks to every bitcoin node on the network.((("network")))
Proof-Of-Work::
A piece of data that requires significant computation to find. In bitcoin, miners must find a numeric solution to the SHA256 algorithm that meets a network-wide target, the difficulty target. ((("proof-of-work")))
reward::
An amount included in each new block as a reward by the network to the miner who found the Proof-Of-Work solution. It is currently 25BTC per block.((("reward")))
secret key (aka private key)::
The secret number that unlocks bitcoins sent to the corresponding address. A secret key looks like +5J76sF8L5jTtzE96r66Sf8cka9y44wdpJjMwCxR3tzLh3ibVPxh+.((("secret key")))((("private key", see="secret key")))
transaction::
In simple terms, a transfer of bitcoins from one address to another. More precisely, a transaction is a signed data structure expressing a transfer of value. Transactions are transmitted over the bitcoin network, collected by miners, and included into blocks, made permanent on the blockchain.((("transaction")))
wallet::
Software that holds all your bitcoin addresses and secret keys. Use it to send, receive, and store your bitcoin.((("wallet")))

BIN
images/740px-Total_bitcoins_over_time_graph.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 156 KiB

BIN
images/AddressPropagation.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 9.0 KiB

BIN
images/AliceCoffeeTransaction.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 71 KiB

BIN
images/Alices_Transaction_Chain.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 25 KiB

BIN
images/BIP32-derivation.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 162 KiB

BIN
images/Base58CheckEncoding.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 50 KiB

BIN
images/BitcoinDifficulty.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 34 KiB

BIN
images/BitcoinMoneySupply.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 15 KiB

BIN
images/BitcoinNetwork.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 350 KiB

BIN
images/BitcoinNodeTypes.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 139 KiB

BIN
images/Bitcoin_Overview.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 129 KiB

BIN
images/Bitcoin_Transaction_Structure_Aggregating.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 19 KiB

BIN
images/Bitcoin_Transaction_Structure_Common.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 26 KiB

BIN
images/Bitcoin_Transaction_Structure_Distribution.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 20 KiB

BIN
images/BlockChainWithForks.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 17 KiB

BIN
images/BlockRain2.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 113 KiB

BIN
images/Blockchain_height_and_depth.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 28 KiB

BIN
images/Bloom1.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 22 KiB

BIN
images/Bloom2.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 23 KiB

BIN
images/Bloom3.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 26 KiB

BIN
images/Bloom4.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 30 KiB

BIN
images/Bloom5.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 31 KiB

BIN
images/ChainOfBlocks.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 92 KiB

BIN
images/ChildHardPrivateDerivation.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 36 KiB

BIN
images/ChildPrivateDerivation.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 37 KiB

BIN
images/ChildPublicDerivation.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 28 KiB

BIN
images/ForkedChain1.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 4.2 KiB

BIN
images/ForkedChain2a.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 5.1 KiB

BIN
images/ForkedChain2b.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 5.0 KiB

BIN
images/ForkedChain3b.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.3 KiB

BIN
images/FullNodeReferenceClient_Small.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 22 KiB

BIN
images/GlobalFork.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 132 KiB

BIN
images/GlobalFork1.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 130 KiB

BIN
images/GlobalFork2.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 126 KiB

BIN
images/GlobalFork3.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 136 KiB

BIN
images/GlobalFork4.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 132 KiB

BIN
images/GlobalFork5.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 133 KiB

BIN
images/HDWalletFromRootSeed.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 31 KiB

BIN
images/HD_wallet.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 30 KiB

BIN
images/InventorySynchronization.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 19 KiB

BIN
images/MerkleTree.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 20 KiB

BIN
images/MerkleTreeLarge.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 26 KiB

BIN
images/MerkleTreeOdd.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 21 KiB

BIN
images/MerkleTreePathToK.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 29 KiB

BIN
images/MultibitReceive.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 27 KiB

BIN
images/MultibitWallet.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 45 KiB

BIN
images/MultibitWelcome.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 60 KiB

BIN
images/NetworkHandshake.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.3 KiB

BIN
images/NetworkHashingRate.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 51 KiB

BIN
images/PubKey_to_Bitcoin_Address.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 39 KiB

BIN
images/SPVSynchronization.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 12 KiB

BIN
images/Transaction_Chain.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 67 KiB

BIN
images/Transaction_Double_Entry.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 69 KiB

BIN
images/TxScriptSimpleMathExample.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 61 KiB

BIN
images/Tx_Script_P2PubKeyHash_1.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 48 KiB

BIN
images/Tx_Script_P2PubKeyHash_2.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 74 KiB

BIN
images/bitcoin-choose-client.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 55 KiB

BIN
images/bitcoin-price-android.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 169 KiB

BIN
images/bitcoin-qt-1.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 89 KiB

BIN
images/bitcoin-qt-2.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 111 KiB

BIN
images/bitcoin-qt-3.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 99 KiB

BIN
images/bitcoin-qt-dl-options.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 60 KiB

BIN
images/bitcoin-qt-firstload.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

BIN
images/blockchain-home.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 82 KiB

BIN
images/blockchain-mnemonic.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 49 KiB

BIN
images/blockchain-mobile-send.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 24 KiB

BIN
images/blockchain-registration.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 84 KiB

BIN
images/blockchain.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 139 KiB

BIN
images/chained_wallet.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 17 KiB

BIN
images/cover.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 122 KiB

After

Width:  |  Height:  |  Size: 1.5 MiB

BIN
images/ecc-addition.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 24 KiB

BIN
images/ecc-curve.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.3 KiB

BIN
images/ecc-over-F17-math.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 21 KiB

BIN
images/ecc-over-F37-math.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 18 KiB

BIN
images/ecc_illustrated.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 50 KiB

BIN
images/msbt_0101.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

BIN
images/msbt_0102.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

BIN
images/msbt_0103.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 21 KiB

BIN
images/msbt_0104.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

BIN
images/msbt_0201.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 140 KiB

BIN
images/msbt_0202.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.0 KiB

Some files were not shown because too many files have changed in this diff Show More

Write Preview
Loading…
Cancel
Save