How is the whitepaper decoded from the blockchain (Tx with ~1000x m of n multisig outputs)

This is a fun little puzzle on the blockchain, basically. First, you need to know a little about pdf's and how they're structured, which you can find here.

Second, you'll note from section 3.4.1 that all pdf's start with this string:

%PDF-

In hex, that is 255044462d. And indeed that is in the very first output in the very first bare multisig pubkey:

<e4cf0200><067daf13>**255044462d**312e340a25c3a4c3bcc3b6c39f0a322030206f626a0a3c3c2f4c656e6774682033203020522f46696c7465722f466c6174654465

I haven't figured out what the first 8 bytes are for (♦edit: e4cf0200067daf13 = 2x 4byte little Endian "checksums", see @WizardOfOzzie comment below), but the rest of the bare multisig keys (everything in between 1 and 3 OP_CHECKMULTISIG in each output -- note the last one is a 1 of 1, so it's 1 OP_CHECKMULTISIG) are pieces of data for the pdf and they are in order. If you can put all the hex digits of the bare multisig keys into a single file (no whitespaces) called "fromblockchain.hex", you can run this very simple program to extract the pdf:

contents = open('fromblockchain.hex').read()
bytes = contents[16:].decode('hex')
f = open("bitcoin.pdf")
f.write(bytes)
f.close()

This should create a bitcoin.pdf which is the actual satoshi whitepaper. I've tested this and indeed it is the whitepaper. Good to know it's literally in the blockchain.

Alternatively, if you have bitcoind running on your machine, you can run this python script to grab the bitcoin whitepaper:

import subprocess

# raw = full hex of raw Tx using Bitcoin-cli
raw = subprocess.check_output(["bitcoin-cli", "getrawtransaction", "54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713"])

outputs = raw.split("0100000000000000")

pdf = ""
for output in outputs[1:-2]:
    # there are 3 65-byte parts in this that we need
    cur = 6
    pdf += output[cur:cur+130].decode('hex')
    cur += 132
    pdf += output[cur:cur+130].decode('hex')
    cur += 132
    pdf += output[cur:cur+130].decode('hex')

pdf += outputs[-2][6:-4].decode("hex")
f = open("bitcoin.pdf", "wb")
f.write(pdf[8:])
f.close()

Jimmy Song

Posted 2015-02-09T08:51:45.377

Reputation: 7 067

Ahh ha! Another fantastic answer! I was trying to get a script like the above working and couldn't parse the hex after 0x5141 and between the 0x41s. What is up with the last two outputs being standard 76a914____88ac? – Wizard Of Ozzie – 2015-02-10T06:19:09.790

Looked to me like the person spent 1 satoshi on each output and wanted the change back for the last one. Not sure about the second to last. – Jimmy Song – 2015-02-10T06:31:50.860

Perhaps the it's a dual purpose: allows output to be collected and the data length can't be padded, since 20 bytes x2 % 8 whereas doing that with 65 bytes is much harder (actually 65x3) – Wizard Of Ozzie – 2015-02-10T08:02:30.263

Did you want to edit the bottom script to add the output at outputs[-2]? The script needs this after the for loop: pdf += outputs[-2][6:-4].decode('hex'). Also open needs to be using open("bitcoin.pdf", 'wb') (binary mode). I'll have to look into multisig more b/c I still fail to see why the rest of the multisigs are pushing 0x41 bytes Vs the last only pushing 0x21 bytes – Wizard Of Ozzie – 2015-02-10T10:32:35.557

edited it for you – Wizard Of Ozzie – 2015-02-10T14:12:46.277

Can you edit your fantastic answer to also show reverse functionality (ie bitcoin.pdf => rawhextx) using Python? Python-specific forums/subreddits are having a hard time understanding the Bitcoin aspect. EDIT: yes I am aware this is a non_standard Tx; 'tis for intellectual curiosity only – Wizard Of Ozzie – 2015-03-25T01:49:01.557

Re: 0xe4cf0200067daf13 ("I haven't figured out what the first 8 bytes are for"). It's actually 2x 4 byte "checksums". More specifically, little Endian hex representations of: <total pdf size> <crc32 for pdf> ie 184292 330267910 ;) – Wizard Of Ozzie – 2015-03-26T05:26:57.123

Nice catch @WizardOfOzzie. I'll try to write something up to do the opposite direction (any pdf, not just bitcoin whitepaper) soon. – Jimmy Song – 2015-03-26T19:37:49.397

this code - https://gist.github.com/shirriff/bfc4df70a02732493a28#file-bitcoin-insertion-tool-py - is what was used, I believe. The problem is it depends on actual UTXOs and counts values, checking both, whereas I'm just trying to create a raw unsigned Tx, and sign it (ie no need for Bitcoincore RPC, I'll provide the input data like txid, VOUT etc)

– Wizard Of Ozzie – 2015-03-26T23:15:28.013

I should add its 945x ? of 3(?) txns and 2x paytopublickey outputs – Wizard Of Ozzie – 2015-02-09T12:41:38.903

scriptsig limit of 20kB! I was aware of a 10 kB limit, enforced in EvalScript, but what 20 kB limit are you referring to? – Nick ODell – 2015-02-09T16:18:09.730

@NickODell isn't the limit for scripts 20kb? – Wizard Of Ozzie – 2015-02-10T06:20:27.663

How is the whitepaper decoded from the blockchain (Tx with ~1000x m of n multisig outputs)

Answers