Redeeming a raw transaction step by step example required

22

27

I am looking, specifically, for a step by step example transaction of sending Bitcoin which uses an actual transaction (How To Redeem A Basic Txn, from a few years back, does a great job of outlining most steps of sending a raw txn but does not use a real TxID).

I have played around with both the Python pybitcointools library, the SX library and JSON output from the Bitcoin Core client (Bitcoin-QT, Bitcoin-cli, Bitcoin daemon) but have yet to find an actual step by step guide using raw transaction output (single input / single output Bitcoin txn, ie not multisig).

Hopefully the bounty will bring a step by step example transaction (with private keys for the sending address) will bring an answer showing how it's done and, specifically:

  • how the ScriptPubKey element fits into the raw txn
  • how the ScriptSig element fits into the raw txn
  • how the txn is signed (with DER encoding).

EDIT: I think the best resource is Ken Shirriff's Bitcoins the hard way: Using the raw Bitcoin protocol, but again, there's no single source online that answers my question without skimming past areas like scriptPubKey, signing etc.

When I activate the bounty, if you can answer this refer to this Tx as it can serve as the actual example (ie I'll provide ~$1 in BTC and private keys for 1From/1MBngSqZbMydscpzSoehjP8kznMaHAzh9y if interested)

EDIT 2: The RoyalFork Blog: Deconstructing Txns provides an unbelievably good reference for interactive Txn creation

Wizard Of Ozzie

Posted 2014-11-20T04:36:43.433

Reputation: 4 535

3

You should read Bitcoin the hard way.

– Nick ODell – 2014-11-20T04:56:13.600

1Are you looking for step by step with RPC calls, or step by step with byte manipulation by hand? – morsecoder – 2014-11-20T06:38:47.077

@NickODell - yes, have seen that, Nick, and you're right, it's a fantastic resource! The issue still remains though even after reading the Python code... I just don't get the whole ScriptPubKey / scriptSig part and every resource I read is very opaque at that part. I'll edit the question to provide more clarity once I can put up the bounty – Wizard Of Ozzie – 2014-11-21T03:28:42.587

@StephenM347 Aiming for a step by step example which is as close to how would I create a Bitcoin transaction with pencil and paper? (similar to: http://bitcoin.stackexchange.com/q/808/9382 ). I am certainly avoiding RPC calls which "do the work" so to speak

– Wizard Of Ozzie – 2014-11-21T03:33:48.953

Do you care that it uses the exact txid you mentioned? I just think it would be easier to do it with a tx that I own. I wouldn't reveal the private key, but I would make the signatures that it calculates known. – morsecoder – 2014-11-21T05:17:00.213

@stephenm347 it doesn't have to be that tx. If you'd like to reference your own txn that's fine too but I've setup this because the outputs are from 1From and the 1MB address is a brainwallet which happens to reference letters the same as 1MB (so private key is easy to link to the address) – Wizard Of Ozzie – 2014-11-24T05:43:09.020

@StephenM347 For the bounty I need the private keys so I can repeat the process myself. Even Nick's suggestion to check out "Bitcoin the hard way" references the stack exchange How To Redeem A Raw TX so there is no canonical answer to this online; https://github.com/shirriff/bitcoin-code/blob/master/txnUtils.py

– Wizard Of Ozzie – 2014-11-24T05:54:50.277

Note that even if you repeat the process provided, the signature produced may not be identical to the one in the instructions. That's because DSA/ECDSA signatures need to be produced with a secret value (often called k) and one method for producing a secret value is a one-time random number. If you want perfectly reproducible steps, you should specify either that the value of k be provided in the instructions or that RFC6979 deterministic generation of k be used.

– David A. Harding – 2014-11-24T06:20:20.317

Answers

39

Step-by-step description:

We start creating a new transaction which we hash and sign.

  1. Add four-byte version field: 01000000
  2. One-byte varint specifying the number of inputs: 01
  3. 32-byte hash of the transaction from which we want to redeem an output (reverse order): be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396
  4. Four-byte field denoting the output index we want to redeem from the transaction with the above hash (counting from zero): 00000000
  5. Now comes the scriptSig. For the purpose of signing the transaction, this is temporarily filled with the scriptPubKey of the output we want to redeem. First we write a one-byte varint which denotes the length of the scriptSig (0x19 = 25 bytes): 19
  6. Then we write the actual scriptSig (which is the scriptPubKey of the output we want to redeem): 76 a9 14 dd6cce9f255a8cc17bda8ba0373df8e861cb866e 88 ac (look to the bottom line line on https://blockchain.info/tx/96534da2f213367a6d589f18d7d6d1689748cd911f8c33a9aee754a80de166be?show_adv=true )
  7. Then we write a four-byte field denoting the sequence. This is currently always set to 0xffffffff: ffffffff
  8. Next comes a one-byte varint containing the number of outputs in our new transaction. We will set this to 1 in this example: 01
  9. We then write an 8-byte field (64 bit integer, little-endian) containing the amount we want to redeem from the specified output. I will set this to the total amount available in the output minus a fee of 0.0001 BTC (128307 - 10000): 23ce010000000000
  10. Then we start writing our transaction's output. We start with a one-byte varint denoting the length of the output script (0x19 or 25 bytes): 19
  11. Then the actual output script: 76 a9 14 a2fd2e039a86dbcf0e1a664729e09e8007f89510 88 ac ( this is transferring funds back to address 1FromKBPAS8MWsk1Yv1Yiu8rJbjfVioBHc )
  12. Then we write the four-byte "lock time" field: 00000000
  13. And at last, we write a four-byte "hash code type" (1 in our case): 01000000

OK, the result is

01000000
01
be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396
00000000
19 76 a9 14 dd6cce9f255a8cc17bda8ba0373df8e861cb866e 88 ac
ffffffff
01
23ce010000000000
19 76 a9 14 a2fd2e039a86dbcf0e1a664729e09e8007f89510 88 ac
00000000
01000000

  1. Now we double-SHA256 hash this entire structure, which yields the hash 1cde0239b55717cca8003104abc2ec2673d4f6fabea0b74351940e382e88486f

  2. Now we should create ECDSA signature... 1MBngSqZbMydscpzSoehjP8kznMaHAzh9y is a brainwallet of "mrbubbymrbubbymrbubby!", which just happens to encode an address starting in 'MB' (making linking the 2 quite easy; see @WizardOfAussie comment below for phrase origins). Private key in WIF: 5HvofFG7K1e2aeWESm5pbCzRHtCSiZNbfLYXBvxyA57DhKHV4U3

In hex, the private key is 0ecd20654c2e2be708495853e8da35c664247040c00bd10b9b13e5e86e6a808d. There is a sign (key, digest) method in every crypto lib. It will return an array of bytes. This array will be no more than 72 bytes, and start with hex code 30. Lets imagine that the signature is 3046022100cf4d7571dd47a4d47f5cb767d54d6702530a3555726b27b6ac56117f5e7808fe0221008cbb42233bb04d7f28a715cf7c938e238afde90207e9d103dd9018e12cb7180e To this signature we append the one-byte hash code type: 01. The public key for 1MBngSqZbMydscpzSoehjP8kznMaHAzh9y is: 042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9

  1. We construct the final scriptSig by concatenating: <One-byte script OPCODE containing the length of the DER-encoded signature plus the one-byte hash code type>|< The actual DER-encoded signature plus the one-byte hash code type>|< One-byte script OPCODE containing the length of the public key>|<The actual public key>

scriptSig will be

49 3046022100cf4d7571dd47a4d47f5cb767d54d6702530a3555726b27b6ac56117f5e7808fe0221008cbb42233bb04d7f28a715cf7c938e238afde90207e9d103dd9018e12cb7180e 01
41 042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9

first line is 'push signature concatenated with 01', second line is 'push pubkey'. The length of scriptSig is 140 bytes (0x8c in hex)

  1. We then replace the one-byte, varint length-field from step 5 with the length of the data from step 16. The length is 140 bytes, or 0x8C bytes: 8c

  2. And we replace the actual scriptSig with the data structure constructed in step 16.

  3. We finish off by removing the four-byte hash code type we added in step 13, and we end up with the following stream of bytes, which is the final transaction:

    01000000 01 be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396 00000000 8c 49 3046022100cf4d7571dd47a4d47f5cb767d54d6702530a3555726b27b6ac56117f5e7808fe0221008cbb42233bb04d7f28a715cf7c938e238afde90207e9d103dd9018e12cb7180e 01 41 042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9 ffffffff 01 23ce010000000000 19 76 a9 14 a2fd2e039a86dbcf0e1a664729e09e8007f89510 88 ac 00000000

amaclin

Posted 2014-11-20T04:36:43.433

Reputation: 5 763

It's great post. Thanks. I wrote some code based on your great answer but my code doesn't run. Could you help take a look at https://bitcoin.stackexchange.com/questions/92005/verify-bitcoin-transaction ? Thanks a lot.

– YongJiang Zhang – 2019-11-28T20:32:43.627

Isn't this the steps outlined in the link in the question? I don't this this qualifies for the bounty. @wizardofozzie one thing you have to understand about bitcoin is that testing transactions is unlike any other computer program in history: it is impossible to replicate the same situation twice as is normally the case with testing. ie you can never go back a re-run the exact same situation again. – T9b – 2014-11-24T14:37:15.173

2I do not pretend for bounty. I've just answered a question. And yes, most of my answer is pure copy-paste. May be very small additions. English is not my native language and it is hard for me to write the whole answer myself :) – amaclin – 2014-11-24T14:53:19.080

I understand now! @amaclin if you want the private keys to edit your answer I can provide it (or I can edit the answer and give the privkey). Fyi it was the DER encoding (the part starting with a 0x30 byte) which I couldn't for the life of me place. I knew there was the 130 char public key starting with 0x04 within the byte array but I couldn't place the first part. I do understand it's going to be signed differently each time (one would hope!) but this is exactly the clarification I needed. I'll likely have to wait until I'm not using the iOS app to award the bounty but amaclin thank you! – Wizard Of Ozzie – 2014-11-25T09:22:31.910

You should not publish your private keys. You even should not send me your keys (my email is in profile). I think that my answer is full enough, nobody needs that the result been correct transaction existing in blockchain. – amaclin – 2014-11-25T09:28:47.187

I appreciate what you're getting at (and the fabulous answer!) but the addresses are not holding BTC. The whole point of this was to have a canonical answer which is not testnet or regtest or abstract. So I'm going to edit your answer as I've successfully signed the txn and transmitted it. https://blockchain.info/tx/211b8fb30990632751a83d1dc4f0323ff7d2fd3cad88084de13c9be2ae1c6426

– Wizard Of Ozzie – 2014-11-28T10:08:32.627

I'll add an answer outlining the process using sx tools but for those interested the brainwallet for 1MB... is "mrbubbymrbubbymrbubby!" (no quotes) => 1MBngSqZbMydscpzSoehjP8kznMaHAzh9y ... in case you're wondering it's what the neighbor yells each night to call her cat. It just so happens to hash to 1MB :) – Wizard Of Ozzie – 2014-11-28T10:18:19.697

1Waiting for your next question: how to redeem two outputs in one tx (how to create & sign transaction with more than one input) – amaclin – 2014-11-28T11:09:38.313

@amaclin Found this AWESOME reference; this blog is unbelievably good for visual ppl like myself... http://www.royalforkblog.com/2014/11/20/txn-demo/?#the-ins-and-outs-of-transactions

– Wizard Of Ozzie – 2015-01-05T02:26:17.673

This is great! However, how is the raw transaction constructed when there are more than one input / output? – user2298995 – 2017-12-20T18:47:18.390

Is there a bip that describes the above? – Malone – 2019-03-01T08:27:51.283

3

This is an excellent answer by amaclin and Wizard of Ozzie. I'd just like to add some more detail for the Wizard of Ozzie's transaction at blockchain.info.

  1. The correct input to the signature process for this transaction is

    01000000
01
be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396
00000000
19 76 a9 14 dd6cce9f255a8cc17bda8ba0373df8e861cb866e 88 ac
ffffffff
01
23ce010000000000
19 76 a9 14 2bc89c2702e0e618db7d59eb5ce2f0f147b40754 88 ac
00000000
01000000

  2. The double-SHA256 hash of this is computed by bx as

    d304448dff517bcf677cd36f3491e9ef2ccfdf40fb63af5782d9b768640af130

but this is written in "reverse" order for whatever peculiar historical reason (I've heard it was to be compatible with M*soft CRAPi). So when using this with the sign(key, digest) in most sensible crypto toolkits you may need to reverse the hash value to the "proper" order for SHA256(SHA256(m))

30f10a6468b7d98257af63fb40dfcf2cefe991346fd37c67cf7b51ff8d4404d3

  1. The above input data will indeed validate against the signature given in the actual blockchain transaction

     3045022100da43201760bda697222002f56266bf65023fef2094519e13077f777baed553b102205ce35d05eabda58cd50a67977a65706347cc25ef43153e309ff210a134722e9e

using the given public key

042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9

  1. To generate a reproducible signature over the same data, the following signature has been made by the deterministic method of RFC6979.

    30450220587ce0cf0252e2db3a7c3c91b355aa8f3385e128227cd8727c5f7777877ad772022100edc508b7c14891ed15ab38c687019d7ebaf5c12908cf21a83e8ae57e8c47e95c

using the matching private key provided by Wizard of Ozzie

0ecd20654c2e2be708495853e8da35c664247040c00bd10b9b13e5e86e6a808d

This signature also validates over the given data using the public key and should be reproducible.

David I

Posted 2014-11-20T04:36:43.433

Reputation: 31

Asked: 2014-11-20T04:36:43.433

Viewed: 15 312 times

Active: 2017-02-19T21:37:43.760