What's the best practice for creating and maintaining an address index?

I am working on a Blockchain.info-like block explorer, and I have some questions regarding best practice for data retrieval. Bitcoin Core stores and serves all blocks and, if txindex is set to 1, all transactions too. It doesn't, however, have an option where you can tell it to maintain an index of all addresses.

I wrote a script that iterates through all blocks and inserts the address/txId key pairs into a MySQL table indexed on address. It maintains the index by polling for new blocks. The program doesn't yet take into account unconfirmed transactions, and chain reorganizations can result in invalid data. Is there a better way to do it?

user4042

Posted 2013-06-04T12:55:38.433

Reputation:

Answers

I'm not sure that your question matches your title, but anyway:

Regarding polling the blockchain, you should take a look at the '-blocknotify' option available in the 0.8.2 daemon. This will send you a block hash every time that the latest block changes. You will need to handle the situation where the valid chain changes for any reason within whatever script you write (check block height and go from there).

Unconfirmed transactions are another matter; there's nothing right now in the official daemon which will help you here so if you do want these you don't have much choice other than to poll or to patch the official daemon to send you details of unconfirmed transactions. What you do with them is very much dependent on what you are trying to achieve, however.

jgm

Posted 2013-06-04T12:55:38.433

Reputation: 1 487

There are a number of open-source softwares for creating an address index. Coinbase's Toshi syncs the data to an indexed PostgreSQL database. BitPay's Insight writes an index to local storage using LevelUp. There's also a patch release of Bitcoin Core where it itself maintains an address index.

chrisarnesen

Posted 2013-06-04T12:55:38.433

Reputation: 442

This answer is a bit old already. Mind refreshing it? BitPay came to questionable fame in the SegWit2x drama and bitcore had no commit in two months. Toshi is 404 here. BtcDrak's patch was last updated a year ago, although others patched the missing two versions of bitcoin core.

– Giszmo – 2018-01-16T16:53:17.370

Like what jgm has said. In addition, you can call 'getrawtransaction TXID 1' on each transaction from the block you detected, via jsonrpc. That way, you can look at each transaction and check whether the in/out address matches the one you care about.

zfei

Posted 2013-06-04T12:55:38.433

Reputation: 41