From 52045782de95d7d575039797656ebf16a617322c Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Thu, 22 May 2014 20:19:51 -0400 Subject: [PATCH 1/3] chapter 4: re-organized flow, expanded sections - private key, ecc, public key, address, base58, base58check. --- ch04.asciidoc | 126 ++++++++++++++++++++++++++++++++++---------------- 1 file changed, 85 insertions(+), 41 deletions(-) diff --git a/ch04.asciidoc b/ch04.asciidoc index 59cdc065..e1e0e186 100644 --- a/ch04.asciidoc +++ b/ch04.asciidoc @@ -36,26 +36,24 @@ image::images/privk_to_pubK_to_addressA.png["privk_to_pubK_to_addressA"] ==== Private Keys -In the most simple form, the +private key+ is a number. The private key can be used to create a corresponding +public key+. The public key can then be converted into a +bitcoin address+, which is shared with anyone who we want to send us bitcoin. Ownership and control over the private key is the root of user control over all funds associated with the corresponding bitcoin address. +A +private key+ is simply a number, picked at random. Ownership and control over the private key is the root of user control over all funds associated with the corresponding bitcoin address. The private key is used to create signatures that are required to spend bitcoins, by proving ownership of funds used in a transaction. The private key must remain secret at all times, as revealing it to a third party is equivalent to giving them control over the bitcoins secured by that key. The private key must also be backed up and protected from accidental loss, since if lost it cannot be recovered and the funds secured by it are forever lost too. ===== Generating a private key from a random number -A private key is a number between +1+ and +n - 1+ where latexmath:[\(n = 1.158 * 10^\(77\) \)] is the order of the elliptic curve used in bitcoin (See <>). To create such a key, we just pick a 256-bit random number and check that it is less than +n - 1+. The constant +n+ is defined in any elliptic curve cryptography library. In programming terms, this is usually achieved by feeding a larger string of random bits, collected from a cryptographically-secure source of randomness, into the SHA-256 hash algorithm which will conveniently produce a 256-bit number. - +A private key is a number between +1+ and +n - 1+, where n is a constant defined in the elliptic curve standard (latexmath:[\(n = 1.158 * 10^\(77\) \)], n is the order of the elliptic curve used in bitcoin. See <>). To create such a key, we randomly pick a 256-bit number and check that it is less than +n - 1+. In programming terms, this is usually achieved by feeding a larger string of random bits, collected from a cryptographically-secure source of randomness, into the SHA-256 hash algorithm which will conveniently produce a 256-bit number. If the result is less than +n - 1+, we have a suitable private key. If it is greater than +n - 1+, we simply try again with another random number. [TIP] ==== Do not try and design your own pseudo random number generator (PRNG). Use a cryptographically-secure pseudo-random number generator (CSPRNG) with a seed from a source of sufficient entropy, the choice of which depends on the operating-system. Correct implementation of the CSPRNG is critical to the security of the keys. DIY is highly discouraged unless you are a professional cryptographer. ==== - Below is a randomly generated private key shown in hexadecimal format (256 binary digits shown as 64 hexadecimal digits, each 4 bits): ---- 1E99423A4ED27608A15A2616A2B0E9E52CED330AC530EDCC32C8FFC6A526AEDD ---- -To generate a new key with bitcoind, use the +getnewaddress+ command. For security reasons it displays the public key only, not the private key. To ask bitcoind to expose the private key, use the +dumpprivkey+ command. Here's an example of both commands: +To generate a new key with bitcoind, use the +getnewaddress+ command. For security reasons it displays the public key only, not the private key. To ask bitcoind to expose the private key, use the +dumpprivkey+ command. The +dumpprivkey+ shows the private key in a base-58 checksum encoded format called the Wallet Import Format (WIF), which we will examine in more detail in <>. Here's an example of generating and displaying a private key using these two commands: ---- $ bitcoind getnewaddress @@ -64,21 +62,27 @@ $ bitcoind dumpprivkey 1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ ---- -The +dumpprivkey+ command is opening the wallet and extracting the private key that was generated by the +getnewaddress+ command. It is not otherwise possible for bitcoind to know the private key from the public key, unless they are both stored in the wallet. In the example above, we see that the private key has a "K" prefix, indicating it is encoded as a WIF-compressed format. This means that the key-pair is stored in the wallet with both keys compressed, saving 31 bytes of space. If the prefix had been "5", indicating the WIF format, we would know that the key-pair is uncompressed. +The +dumpprivkey+ command is opening the wallet and extracting the private key that was generated by the +getnewaddress+ command. It is not otherwise possible for bitcoind to know the private key from the public key, unless they are both stored in the wallet. -You can also use +sx tools+ to generate keys and convert them between formats: +You can also use +sx tools+ to generate and display private keys: -===== New key +===== New key with sx tools ---- $ sx newkey 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn ---- +[TIP] +==== +A private key is just a number. A public key can be generated from any number, up to 256 bits long. You can pick your keys randomly using a method as simple as tossing a coin, pencil and paper. Toss a coin 256 times and you have the binary digits of a random private key you can use in a bitcoin wallet. Keys really are just a pair of numbers, one calculated from the other. +==== + ==== Public Keys -The public key is calculated from the private key using elliptic curve multiplication, which is irreversible: latexmath:[\(K = k * G\)]+ where +k+ is the private key, +G+ is a constant point called the _Generator Point_ and +K+ is the resulting public key. The reverse (division), or calculating +k+ if you know +K+ is as difficult as trying all possible values of +k+, i.e. a brute-force search. +The public key is calculated from the private key using elliptic curve multiplication, which is irreversible: latexmath:[\(K = k * G\)]+ where +k+ is the private key, +G+ is a constant point called the _Generator Point_ and +K+ is the resulting public key. The reverse (division), or calculating +k+ if you know +K+ is as difficult as trying all possible values of +k+, i.e. a brute-force search. Before we demonstrate how to generate a public key from a private key, let's look at Elliptic Curve Cryptography in a bit more detail. -==== Elliptic Curve Cryptography +[[elliptic_curve]] +==== Elliptic Curve Cryptography Explained ((("elliptic curve cryptography", "ECC"))) Elliptic Curve Cryptography is a type of asymmetric or public-key cryptography based on the discrete logarithm problem as expressed by addition and multiplication on the points of an elliptic curve. @@ -88,7 +92,8 @@ Below we see an exaple of an elliptic curve, similar to that used by bitcoin: .An Elliptic Curve image::images/ecc-curve.png["ecc-curve"] -Bitcoin specifically uses a specific curve and a set of constants, defined as a standard called +secp256k1+, by the National Institute of Standards and Technology (NIST). The +secp256k1+ is defined by the following function, which produces an elliptic curve: +Bitcoin specifically uses a specific curve and a set of constants, defined as a standard called +secp256k1+, by the National Institute of Standards and Technology (NIST). The +secp256k1+ curve is defined by the following function, which produces an elliptic curve: + ((("secp256k1"))) [latexmath] ++++ @@ -107,15 +112,16 @@ or The +mod p+ indicates that this curve is over a finite field of prime order +p+, also written as latexmath:[\(\mathbb{F}_p\)], where latexmath:[\(p = 2^256 - 2^32 - 2^9 - 2^8 - 2^7 - 2^6 - 2^4 - 1\)], a very large prime number. -The curve looks like a pattern of dots scattered in two dimensions, which makes it difficult to visualize. However, the math is identical as that of an elliptic curve over the real numbers shown above. Below is the same elliptic curve over a much smaller finite field of prime order 17, showing a pattern of dots on a grid. The bitcoin elliptic curve can be thought of as a much more complex pattern of dots on a unfathomably large grid. +Because this curve is defined over a finite field of prime order instead of over the real numbers it looks like a pattern of dots scattered in two dimensions, which makes it difficult to visualize. However, the math is identical as that of an elliptic curve over the real numbers shown above. As an example, below is the same elliptic curve over a much smaller finite field of prime order 17, showing a pattern of dots on a grid. The +secp256k1+ bitcoin elliptic curve can be thought of as a much more complex pattern of dots on a unfathomably large grid. [[ecc-over-F17-math]] .Elliptic Curve Cryptography: Visualizing an elliptic curve over F(p), with p=17 image::images/ecc-over-F17-math.png["ecc-over-F17-math"] +[[public_key_derivation]] ==== Generating a public key -Starting with a private key in the form of a randomly generated number +k+, we multiply it with a predetermined point on the curve called the _generator point_ +G+ to produce another point somewhere else on the curve, which is the corresponding public key +K+. The generator point is specified as part of the +secp256k1+ standard and is always the same for all keys in bitcoin. +Starting with a private key in the form of a randomly generated number +k+, we multiply it with a predetermined point on the curve called the _generator point_ +G+ to produce another point somewhere else on the curve, which is the corresponding public key +K+. The generator point is specified as part of the +secp256k1+ standard and is always the same for all keys in bitcoin. [latexmath] ++++ @@ -124,39 +130,16 @@ Starting with a private key in the form of a randomly generated number +k+, we m \end{equation} ++++ -[[key_derivation]] where +k+ is the private key, +G+ is a fixed point on the curve called the _generator point_, ((("generator point"))) and +K+ is the resulting public key, another point on the curve. Since the generator point is always the same, a private key k multiplied with G will always produce the same public key K. -To visualize multiplication of a point with an integer, we will use the simpler elliptic curve over the real numbers - remember, the math is the same. Starting with the generator point G, we take the tangent of the curve at G until it crosses the curve again at another point. This new point is the negative of G+G, or -2G. Reflecting that point across the x-axis gives us 2G. If we take the tangent at 2G, it crosses the curve at -3G, which we can reflect on the x-axis to find 3G. Continuing this process, we can bounce around the curve finding the multiples of G, 2G, 3G, 4G etc. As you can see, a randomly selected large number k, when multiplied against the generator point G is like bouncing around the curve until we land on the point kG which is the public key. This process is irreversible, meaning that it is infeasible to find the factor k (the secret k) in any way other than trying all multiples of G (1G, 2G, 3G etc) in a brute-force search for k. Since k can be an enormous number, that brute-force search would take forever. +To visualize multiplication of a point with an integer, we will use the simpler elliptic curve over the real numbers - remember, the math is the same. Our goal is to find the multiple kG of the generator point G. That is the same as adding G to itself, k times in a row. In elliptic curves, adding a point to itself is the equivalent of drawing a tangent line on the point and finding where it intersects the curve again, then reflecting that point on the x-axis. +Starting with the generator point G, we take the tangent of the curve at G until it crosses the curve again at another point. This new point is -2G. Reflecting that point across the x-axis gives us 2G. If we take the tangent at 2G, it crosses the curve at -3G, which again we reflect on the x-axis to find 3G. Continuing this process, we can bounce around the curve finding the multiples of G, 2G, 3G, 4G etc. As you can see, a randomly selected large number k, when multiplied against the generator point G is like bouncing around the curve k times, until we land on the point kG which is the public key. This process is irreversible, meaning that it is infeasible to find the factor k (the secret k) in any way other than trying all multiples of G (1G, 2G, 3G etc) in a brute-force search for k. Since k can be an enormous number, that brute-force search would take an infeasible amount of computation and time. [[ecc_illustrated]] .Elliptic Curve Cryptography: Visualizing the multiplication of a point G by an integer k on an elliptic curve image::images/ecc_illustrated.png["ecc_illustrated"] - -[TIP] -==== -The private key is just a number. A public key can be generated from any private key. Therefore, a public key can be generated from any number, up to 256 bits long. You can pick your keys randomly using a method as simple as tossing a coin, pencil and paper. Toss a coin 256 times and you have the binary digits of a random private key you can use in a bitcoin wallet. Keys really are just a pair of numbers, one calculated from the other. -==== - -The public key is a point on the elliptic curve, and consists of a pair of coordinates +(x,y)+, normally represented by a 512-bit number with the added prefix +04+. - -Here's the public key generated by the private key we created above, shown as the coordinates +(x,y)+ - -.Public Key K defined as a point +K = (x,y)+ ----- -x = 32 5D 52 E3 B7 ... E5 D3 78 -y = 7A 3D 41 E6 70 ... CD 90 C2 ----- - -Here's the same public key shown as a 512-bit number (130 hex digits) with the prefix +04+ followed by +x+ and then +y+ - -.Uncompressed Public Key K shown in hex (130 hex digits) as +04 x y+ ----- -K = 04 32 5D 52 E3 B7 ... CD 90 C2 ----- - [TIP] ==== A private key can be converted into a public key, but a public key cannot be converted back into a private key because the math only works one way. @@ -164,18 +147,58 @@ A private key can be converted into a public key, but a public key cannot be con ==== Addresses -An address is a string of digits and characters that can be shared with anyone who wants to send you money. In bitcoin, addresses begin with the digit "1". This is an address made by hashing the public key twice through two different hashing algorithms. +An address is a string of digits and characters that can be shared with anyone who wants to send you money. In bitcoin, addresses begin with the digit "1". The bitcoin address is what appears most commonly in a transaction as the "recipient" of the funds. If we were to compare a bitcoin transaction to a paper cheque, the bitcoin address is the beneficiary, which is what we write on the line after "Pay to the order of". On a paper cheque, that beneficiary can sometimes be the name of a bank account holder, but can also include corporations, institutions or even simply "cash". Because paper cheques do not need to specify an account, but rather use an abstract name as the recipient of funds, that makes paper cheques very flexible as payment instruments. Bitcoin transactions use a similar abstraction, the bitcoin address, to make them very flexible. A bitcoin address can represent the owner of a private/public key pair, or it can represent something else, such as a payment script, as we will see in <>. For now, let's examine the simple case, a bitcoin address that represents, and is derived from, a public key. + +A bitcoin address derived from a public key is a string of numbers and letters that begins with the number one, such as +1J7mdg5rbQyUHENYdx39WVWK7fsLpEoXZy+. The bitcoin address is derived from the public key through the use of one-way cryptographic hashing. A "hashing algorithm" or simply "hash algorithm" is a one-way function that produces a fingerprint or "hash" of an arbitrary sized input. Cryptographic hash functions are used extensively in bitcoin: in bitcoin addresses, script addresses and in the mining "Proof-of-Work" algorithm. The algorithms used to make a bitcoin address from a public key are the Secure Hash Algorithm (SHA) and the RACE Integrity Primitives Evaluation Message Digest (RIPEMD), specifically SHA256 and RIPEMD160. + +Starting with the public key K, we compute the SHA256 hash and then compute the RIPEMD160 hash of the result, producing a 160 bit (80 byte) number: +[latexmath] +++++ +\begin{equation} +{A = RIPEMD160(SHA256(K))} +\end{equation} +++++ +where K is the public key and A is the resulting bitcoin address. + +Bitcoin addresses are almost always presented to users in an encoding called "Base58Check", which uses 58 characters (a base-58 nunber system) and a checksum to help human readability, avoid ambiguity and protect against errors in address transcription and entry. Base58Check is also used in many other ways in bitcoin, whenever there is a need for a user to read and correctly transcribe a number, such as a bitcoin address, a private key, an encrypted key, or a script hash. In the next section we will examine the mechanics of Base58Check encoding and decoding, and the resulting representations. ===== Base58 and Base58Check Encoding +[[base58]] +====== Base58 +Base58Check is a format developed for the use of bitcoin and used in many other crypto-currencies. It offers a balance between compact representation, readbility, disambiguity and error detection and prevention. In order to represent long numbers in a compact way, many computer systems use mixed-alphanumeric representations with a base (or radix) higher than 10. For example, whereas the traditional decimal system uses the ten numerals 0 through 9, the hexadecimal system uses six additional symbols, the letters A through F. A number represented in hexadecimal format is shorter than the equivalent decimal representation. Even more compact, base64 representation uses 26 lower case letters, 26 capital letters, 10 numerals and two more characters such as "+" and "/" to transmit binary data over text-based media such as email. Base64 is most commonly used to add binary attachments to email. Base58 is a subset of base64, which uses the upper and lower case letters and numbers but ommits some characters that are frequently mistaken for one another and can appear identical when displayed in certain fonts. Specifically, base58 is base64 without the 0 (number zero), O (capital o), l (lower L), I (capital i) and the symbols "+" and "/". Or, more simply, it is a set of lower and capital letters and numbers without the four (0, O, l, I) mentioned above. + +[[base58alphabet]] +---- +Bitcoin's Base-58 Alphabet: +123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz +---- + +[[base58check]] +====== Base58Check + +To add extra security against typos or transcription errors, Base58Check is a format that uses the Base58 encoding but has a built-in error-checking code (checksum) and version identifier. The checksum is an additional four bytes of data added to the end of the number that is being encoded. The checksum is derived from the hash of the encoded data and can therefore be used to detect and prevent transcription and typing errors. If a single symbol is accidentally changed or mistyped in a Base58Check encoded string, the checksum will not compute correctly, allowing for the detection of the error. + +To convert data (a number) into a Base58Check format, we first add a prefix to the data, called the "version byte", which serves to easily identify the type of data that is encoded. For example, in the case of a bitcoin address the prefix is zero (0x00 in hex), whereas the prefix used when encoding a private key is 128 (0x80 in hex). + +Next compute the checksum by "double-SHA", meaning we apply the SHA256 hash-algorithm twice on the previous result (prefix and data): +checksum = SHA256(SHA256(prefix\+data))+ From the resulting 32-byte hash (hash-of-a-hash), we take only the last four bytes. These four bytes serve as the error-checking code, or checksum. The checksum is concatenated (appended) to the end. + +The result of the above is now a prefix, the data and a checksum, concatenated (bytewise). This result is encoded using the base-58 alphabet described in the section above. + +Let's look at each of these steps, using the sx tools and other command-line tools, starting with a public key and producing a bitcoin address. The public key is : + + +---- + +---- [[base58check_encoding]] .Base58Check Encoding: A base-58, versioned and checksummed format for unambiguously encoding bitcoin data image::images/Base58CheckEncoding.png["Base58CheckEncoding"] - ==== Key Formats and Addresses +[[priv_formats]] ===== Private Key Formats The private key can be represented in a number of different formats, all of which correspond to the same 256-bit number. These formats include: @@ -221,7 +244,26 @@ $ sx base58check-encode 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6 KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ ---- -===== Compressed Keys +===== Public Key Formats + +The public key is a point on the elliptic curve, and consists of a pair of coordinates +(x,y)+, normally represented by a 512-bit number with the added prefix +04+. + +Here's the public key generated by the private key we created above, shown as the coordinates +(x,y)+ + +.Public Key K defined as a point +K = (x,y)+ +---- +x = 32 5D 52 E3 B7 ... E5 D3 78 +y = 7A 3D 41 E6 70 ... CD 90 C2 +---- + +Here's the same public key shown as a 512-bit number (130 hex digits) with the prefix +04+ followed by +x+ and then +y+ + +.Uncompressed Public Key K shown in hex (130 hex digits) as +04 x y+ +---- +K = 04 32 5D 52 E3 B7 ... CD 90 C2 +---- + +===== Compressed Public Keys The +y+ coordinate can be deduced from the +x+ coordinate, since they both lie on the same curved line defined by the elliptic curve equation. This makes it possible to store the public key _compressed_, with the +y+ omitted. A +compressed public key+ has the prefix +02+ if the +y+ is above the x-axis, and +03+ if it is below the x-axis, allowing the software to calculate it from +x+. @@ -287,6 +329,8 @@ image::images/HD_wallet.png["HD wallet"] ===== Paper Wallets ==== Advanced Keys and Addresses ===== Encrypted Keys (BIP0038) + +[[p2sh]] ===== Pay To Script Hash Addresses (P2SH) ===== Multi-Signature Addresses ===== Vanity Addresses \ No newline at end of file From 89fb743be53d256b4ed6f287aa22a03c376a3e80 Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 24 May 2014 12:13:28 -0400 Subject: [PATCH 2/3] base58check encodings, compressed public keys, compressed WIF --- ch04.asciidoc | 73 ++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 57 insertions(+), 16 deletions(-) diff --git a/ch04.asciidoc b/ch04.asciidoc index e1e0e186..7f982578 100644 --- a/ch04.asciidoc +++ b/ch04.asciidoc @@ -77,6 +77,7 @@ $ sx newkey A private key is just a number. A public key can be generated from any number, up to 256 bits long. You can pick your keys randomly using a method as simple as tossing a coin, pencil and paper. Toss a coin 256 times and you have the binary digits of a random private key you can use in a bitcoin wallet. Keys really are just a pair of numbers, one calculated from the other. ==== +[[pubkey]] ==== Public Keys The public key is calculated from the private key using elliptic curve multiplication, which is irreversible: latexmath:[\(K = k * G\)]+ where +k+ is the private key, +G+ is a constant point called the _Generator Point_ and +K+ is the resulting public key. The reverse (division), or calculating +k+ if you know +K+ is as difficult as trying all possible values of +k+, i.e. a brute-force search. Before we demonstrate how to generate a public key from a private key, let's look at Elliptic Curve Cryptography in a bit more detail. @@ -186,12 +187,7 @@ Next compute the checksum by "double-SHA", meaning we apply the SHA256 hash-algo The result of the above is now a prefix, the data and a checksum, concatenated (bytewise). This result is encoded using the base-58 alphabet described in the section above. -Let's look at each of these steps, using the sx tools and other command-line tools, starting with a public key and producing a bitcoin address. The public key is : - ----- - ----- [[base58check_encoding]] .Base58Check Encoding: A base-58, versioned and checksummed format for unambiguously encoding bitcoin data image::images/Base58CheckEncoding.png["Base58CheckEncoding"] @@ -208,11 +204,11 @@ The private key can be represented in a number of different formats, all of whic |======= |Type|Prefix|Description | Hex | None | 64 hexadecimal digits -| WIF | 5 | Base-58 encoding with version prefix of 128 and 32-bit checksum +| WIF | 5 | Base58Check encoding: Base-58 with version prefix of 128 and 32-bit checksum | WIF-compressed | K or L | As above, with added suffix 0x01 before encoding |======= -The key above, for example can be represented as: +The private key we generated earlier, for example can be represented as: .Example: Same Key, Different Formats [options="header"] @@ -226,29 +222,39 @@ The key above, for example can be represented as: All of the above representations are different ways of showing the same number, the same private key. They look different, but any one format can easily be converted to any other format. -===== Decoded from the Base58Check encoding to Hex +===== Decoded from Base58Check to Hex + +The sx-tools package (See <>) makes key conversion easy on the command line. We use the base58check-decode command: ---- $ sx base58check-decode 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd 128 ---- -===== Encode from Hex back to Base58Check encoding, with the version prefix "128" +The result is the hexadecimal key, followed by the Wallet Import Format (WIF) version prefix 128 + +===== Encode from Hex to Base58Check + +To encode into Base58Check, we provide the hex private key, followed by the Wallet Import Format (WIF) version prefix 128 ---- $ sx base58check-encode 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd 128 5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn ---- -===== Encode from Hex with a suffix of "01" to Base58Check encoding, with the version prefix "128" +===== Encode from Hex (Comrpessed Key) to Base58Check encoding + +To encode into Base58Check as a "compressed" private key, we add the suffix +01+ to the end of the hex key and then encode as above: ---- $ sx base58check-encode 1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd01 128 KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ ---- +The resulting WIF-compressed format, starts with a "K". This denotes that the private key within has a suffic of "01" and will be used to produce compressed public keys only (See <> below) + ===== Public Key Formats -The public key is a point on the elliptic curve, and consists of a pair of coordinates +(x,y)+, normally represented by a 512-bit number with the added prefix +04+. +The public key is a point on the elliptic curve, and consists of a pair of coordinates +(x,y)+, usually represented by a 512-bit number with the added prefix +04+. That's two 256-bit numbers, one for the x-coordinate of the point, the other for the y-coordinate. The prefix +04+ is used to distinguish uncompressed public keys. -Here's the public key generated by the private key we created above, shown as the coordinates +(x,y)+ +Here's the public key generated by the private key we created above, shown as the coordinates +x+ and +y+ .Public Key K defined as a point +K = (x,y)+ ---- @@ -256,24 +262,59 @@ x = 32 5D 52 E3 B7 ... E5 D3 78 y = 7A 3D 41 E6 70 ... CD 90 C2 ---- -Here's the same public key shown as a 512-bit number (130 hex digits) with the prefix +04+ followed by +x+ and then +y+ +Here's the same public key shown as a 512-bit number (130 hex digits) with the prefix +04+ followed by +x+ and then +y+ coordinates, as +04xy+: -.Uncompressed Public Key K shown in hex (130 hex digits) as +04 x y+ +.Uncompressed Public Key K shown in hex (130 hex digits) as +04xy+ ---- K = 04 32 5D 52 E3 B7 ... CD 90 C2 ---- ===== Compressed Public Keys -The +y+ coordinate can be deduced from the +x+ coordinate, since they both lie on the same curved line defined by the elliptic curve equation. This makes it possible to store the public key _compressed_, with the +y+ omitted. A +compressed public key+ has the prefix +02+ if the +y+ is above the x-axis, and +03+ if it is below the x-axis, allowing the software to calculate it from +x+. +Compressed public keys were introduced to bitcoin to reduce the size of transaction and conserve disk space on nodes that store the bitcoin blockchain database. Most transactions include a signature and public key, required to spend the funds and "transfer out" of one bitcoin address and into another. Each public key requires 513 bytes (prefix \+ x \+ y) to store, which when multiplied by several hundred transactions per block, or tens of thousands of transactions per day can add a significant amount of data to the blockchain. -Here's the same public key above, shown as a +compressed public key+ stored in 264-bits (66 hex digits) with the prefix +02+ indicating the +y+ coordinate has a positive sign: +As we saw in the section <> above, a public key is a point (x,y) on an elliptic curve. Since the curve expresses a mathematical function, a point on the curve represents a solution to the equation and therefore if we know the x coordinate we can calculate the y coordinate by solving the euqation y^2 mod p = (x^3 + 7) mod p. That allows us to store only the x-coordinate of the public key point, ommitting the y-coordinate and reducing the size of the key and the space required to store it by 256 bits. A 50% reduction in size in every transaction adds up to a lot of data saved over time! + +There's one small detail to consider: since the left side of the equation is y^2, that means the solution for y is a square root, which can have a positive or negative value. Visually, this means that the resulting y-coordinate can be above the x-axis or below the x-axis. As you can see from the graph of the elliptic curve, the curve is symmetric, meaning it is reflected like a mirror by the x-axis. So, while we can ommit the y-coordinate we have to store the _sign_ of y (positive or negative), or in other words we have to remember if it was above or below the x-axis, as each of those options represents a different point and a different public key. When calculating the elliptic curve in binary arithmetic on the finite field of prime order p, the y coordinate is either even or odd, which corresponds to the positive/negative sign as explained above. Therefore, to distinguish between the two possible values of y, we store a +compressed public key+ with the prefix +02+ if the +y+ is even (same as above-the-axis), and +03+ if it is odd (same as below the axis), allowing the software to correctly deduce the y-coordinate from the x-coordinate. + +Here's the same public key above, shown as a +compressed public key+ stored in 264-bits (66 hex digits) with the prefix +02+ indicating the +y+ coordinate is even: .Compressed Public Key K shown in hex (66 hex digits) as +K = {02 or 03} x+ ---- K = 02 32 5D 52 E3 B7 ... E5 D3 78 ---- +The compressed public key, above, corresponds to the same private key, meaning that it is generated from the same private key. However it looks different from the uncompressed public key. More importantly, if we convert this compressed public key to a bitcoin address using the double-hash function (RIPEMD160(SHA256(K))) it will produce a _different_ bitcoin address. This can be confusing, because it means that a single private key can produce a public key expressed in two different formats (compressed and uncompressed) which produce two different bitcoin addresses. But the private key is identical and therefore can produce valid signatures that authorize spending funds from _either_ bitcoin address. + +Compressed public keys are gradually becoming the default across bitcoin clients, which is having a significant impact on reducing the size of transactions and therefore the blockchain. However, not all clients support compressed public keys yet. Newer clients that support compressed public keys have to account for transactions and older clients which do not support compressed public keys. This is especially important when a wallet application is importing private keys from another bitcoin wallet application, because the new wallet needs to scan the blockchain to find transactions corresponding to these imported keys. Which bitcoin addresses should the bitcoin wallet scan for? The bitcoin addresses produced by uncompressed public keys, or the bitcoin addresses produced by compressed public keys? Both are valid bitcoin addresses, both can be signed for by the private key, but they are different addresses! + +To resolve this issue, when private keys are exported from a wallet, the Wallet Import Format that is used to represent them is implemented differently in newer bitcoin wallets, to indicate that these private keys have been used to produce _compressed_ public keys and therefore _compressed_ bitcoin addresses. This allows the importing wallet to distinguish between private keys originating from older or newer wallets and search the blockchain for transactions with bitcoin addresses corresponding to the compressed, or the uncompressed public keys. Let's look at how this works in more detail, in the next section. + +===== Compressed Private Keys + +If a bitcoin wallet is able to implement compressed public keys, then it will use those in all transactions. The private keys in the wallet will be used to derive the public key points on the curve, which will be compressed. The compressed public keys will be used to produce bitcoin addresses and those will be used in transactions. When exporting private keys from a new wallet that implements compressed public keys, the Wallet Import Format is modified, with the addition of a one-byte suffix +01+to the private key. The resulting base58check encoded private key is called a "Compressed WIF" and starts with the letter K or L, instead of starting with "5" as is the case with WIF encoded (non-compressed) keys from older wallets. + +Here's the same key, encoded in WIF and WIF-compressed formats: + +.Example: Same Key, Different Formats +[options="header"] +|======= +|Format | Private Key +| Hex | +1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd+ +| Hex-compressed | +1e99423a4ed27608a15a2616a2b0e9e52ced330ac530edcc32c8ffc6a526aedd_01_+ +| WIF | +5J3mBbAH58CpQ3Y5RNJpUKPE62SQ5tfcvU2JpbnkeyhfsYB1Jcn+ +| WIF-compressed | +KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ+ +|======= + +Remember, these formats are _not_ used interchangeably. In a newer wallet that implements compressed public keys, the private keys will only ever be exported as WIF-compressed (K/L prefix). If the wallet is an older implementation and does not use compressed public keys, the private keys will only ever be exported as WIF (5 prefix). The goal here is to signal to the wallet importing these private keys whether it must search the blockchain for compressed or uncompressed public keys and addresses. + +Ironically, the name "compressed private key" is misleading, because when a private key is exported as WIF-compressed it is actually one byte _longer_ than an "uncompressed" private key. That is because it has the added 01 suffix which signifies it comes from a newer wallet and should only be used to produce compressed public keys. Private keys are not compressed and cannot be compressed. The term "compressed private key" really means "private key from which compressed public keys should be derived", whereas "uncompressed private key" really means "private key from which uncompressed public keys should be derived". + +[TIP] +==== +"Compressed private keys" is a misnomer! They are not compressed, rather they signify that they should only be used to derive compressed public keys and their corresponding bitcoin addresses. Ironically, a "compressed" private key is one byte longer because it has the added 01 suffix to distinguish it from an "uncompressed" one. +==== + ==== Wallets There are many ways to generate keys for use in bitcoin. The simplest is to pick a large random number and turn it into a key pair (See <>). A random key can be generated with very simple hardware or even manually with pen, paper and dice. The disadvantage of random keys is that if you generate many of them you must keep copies of all of them. Another method for making keys is _deterministic key generation_. Here you generate each new key as a function of the previous key, linking them in a sequence. As long as you can re-create that sequence, you only need the first key to generate them all. In this section we will examine the different methods for key generation. From 4d78debacca7d4c2fd010436aba14e5f800c30cd Mon Sep 17 00:00:00 2001 From: "Andreas M. Antonopoulos" Date: Sat, 24 May 2014 12:16:12 -0400 Subject: [PATCH 3/3] further explain WIF-compressed --- ch04.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ch04.asciidoc b/ch04.asciidoc index 7f982578..f45fc94c 100644 --- a/ch04.asciidoc +++ b/ch04.asciidoc @@ -308,11 +308,11 @@ Here's the same key, encoded in WIF and WIF-compressed formats: Remember, these formats are _not_ used interchangeably. In a newer wallet that implements compressed public keys, the private keys will only ever be exported as WIF-compressed (K/L prefix). If the wallet is an older implementation and does not use compressed public keys, the private keys will only ever be exported as WIF (5 prefix). The goal here is to signal to the wallet importing these private keys whether it must search the blockchain for compressed or uncompressed public keys and addresses. -Ironically, the name "compressed private key" is misleading, because when a private key is exported as WIF-compressed it is actually one byte _longer_ than an "uncompressed" private key. That is because it has the added 01 suffix which signifies it comes from a newer wallet and should only be used to produce compressed public keys. Private keys are not compressed and cannot be compressed. The term "compressed private key" really means "private key from which compressed public keys should be derived", whereas "uncompressed private key" really means "private key from which uncompressed public keys should be derived". +Ironically, the name "compressed private key" is misleading, because when a private key is exported as WIF-compressed it is actually one byte _longer_ than an "uncompressed" private key. That is because it has the added 01 suffix which signifies it comes from a newer wallet and should only be used to produce compressed public keys. Private keys are not compressed and cannot be compressed. The term "compressed private key" really means "private key from which compressed public keys should be derived", whereas "uncompressed private key" really means "private key from which uncompressed public keys should be derived". You should only refer to the export format as "WIF-compressed" or "WIF" and not refer to the private key as "compressed" to avoid further confusion. [TIP] ==== -"Compressed private keys" is a misnomer! They are not compressed, rather they signify that they should only be used to derive compressed public keys and their corresponding bitcoin addresses. Ironically, a "compressed" private key is one byte longer because it has the added 01 suffix to distinguish it from an "uncompressed" one. +"Compressed private keys" is a misnomer! They are not compressed, rather the WIF-compressed format signifies that they should only be used to derive compressed public keys and their corresponding bitcoin addresses. Ironically, a "WIF-compressed" encoded private key is one byte longer because it has the added 01 suffix to distinguish it from an "uncompressed" one. ==== ==== Wallets