How to access Bitcoin's transaction database?

I want to process the Bitcoin network's transactions. As I understand it, the Bitcoin-Qt client downloads and stores all that information locally.

Since version 0.8 LevelDB has been used to access this data. I found two databases, in blocks/index and in chainstate. However, I cannot make sense of the content. The first entry looks like this:

B = ??????{?:???????`?5g??

Am I on the right track and where can I find a specification of the data?

msteiger

Posted 2013-04-19T10:58:50.707

Reputation: 211

The data is stored in an application-specific format optimized for compact storage, and wasn't really intended to be easily parsed by other applications. You can access the data in the chainstate through the gettxout RPC call, and the data in the block index through getblock. I'll try to find time for a write-up of the exact format. – Pieter Wuille – 2013-07-29T07:17:48.337

@PieterWuille Your comment would be a great start for an answer. – Murch – 2014-10-03T10:47:26.873

Answers

The best way to write a parser of your own for the blockchain is to find the source code of one that already exists and from that deduce the precise underlying data structure and how to parse and interpret it. e.g. here in C++ and here in C# - and there are many others.

The data in the blockchain is stored in a custom binary format that is a little tricky to untangle without some help; this link seems as good as any to get started on the overall structure.

There are many"gotcha's" to watch out for. For example:

'endianess' i.e. having to reverse binary digits before interpretation and
some of the interesting values associated with a transaction have to be calculated e.g. for addresses
'big numbers' where the numbers used internally are so huge you may need custom code/libraries to deal with them

Comparing your results to those that are freely available via the blockchain and blockexplorer as well as the bitcoind.exe API would be a must. Otherwise it is easy to pull out seemingly valid transactions that are just nonsense. I speak from experience.

While writing your own parser for the blockchain is a worthwhile learning exercise you would be much better off IMHO using the bitcoin API and ignoring the underlying blockchain database for any 'real' code.

Fred Tingey

Posted 2013-04-19T10:58:50.707

Reputation: 181

The best way to access transaction database for analysis is probably through the blockchain.info API. You can also use the Bitcoin Block Explorer; in fact, this is what a study of the Bitcoin network did.

katriel

Posted 2013-04-19T10:58:50.707

Reputation: 213