What would be the fastest way to query for a certain txid to get its inputs?

I am working in an artistic/visualization project that analyses Bitcoin transactions. I have a Bitcore instance with insight API and from a Python script I query through the REST API so the queries are returned in json format. For the project I use a macbook pro and with its internal ssd drive I get around 70 queries per second, using an external NTFS (fuse drivers) driver I get as little as 5 per second.

I would like to know if there is a faster way to query transactions by transaction's hash even if it is more complex to code.

muimota

Posted 2016-12-19T09:33:31.940

Reputation: 192

this question was asked previously in stackoverflow but didn't get any answer http://stackoverflow.com/q/41092857/2205297

– muimota – 2016-12-19T09:34:27.003

By an external driver do you mean one connected via USB? – Shabahat M. Ayubi – 2016-12-21T16:31:24.843

@shabahat-m-ayubi yes a WD Passport ultra. – muimota – 2016-12-26T17:03:37.740

Answers

A quick and relatively fast way (probably much faster then Bitcore) is using a recent version of Bitcoin Core and run it with -txindex and -rest.

Then use the REST interface to query for a txid.

Example: curl localhost:18332/rest/tx/0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098.json

You can even bypass the JSON encoding/decoding if you look for faster processing by using .bin instead of .json in your request URI.

Internally, Bitcoin Core uses levelDB for the txindex which tends to be blazing fast and once the file & disk position has been loaded from the levelDB, the according block file will be opened, then the file pointer seeks to the tx-position and deserialises only the requested tx.

Also don't forget, that Bitcore uses Bitcoin Core under the hood (very likely an outdated version since Bitcore is AFAIK no longer maintained).

Using a blockexplorer (or say web API) will make you trust a third party and will pretty sure be much slower.

Jonas Schnelli

Posted 2016-12-19T09:33:31.940

Reputation: 5 465

Since hardware is acting as a bottleneck, how about using APIs offered by services or using a fast VPS to run bitcore?

API services

Websites such as Blockchain.info offer free developer APIs. Since they have limits to the number of queries per given time, you can use multiple services at the same time to satisfy your needs while not getting blocked for abuse. Also, services such as blockr.io recommend contacting them if you have a large number of queries to make.
VPS

You can also spend on a VPS which uses SSDs and run bitcore on them to attempt to get more queries per second.

MrFregg

Posted 2016-12-19T09:33:31.940

Reputation: 101

Using a API service could be a option but the it wouldn't be a solution since it depends on external company limitations. – muimota – 2016-12-25T12:53:01.683

VPS would need 200Gb to store the blockchain it would be much more than 10$ month – muimota – 2016-12-25T12:55:15.350

@muimota Will this visualization be real-time and require data on new transactions or will this be a one-time thing? – MrFregg – 2016-12-26T10:19:10.297

A one-time thing. Just checking previous transactions in time, not realtime. – muimota – 2016-12-26T17:02:06.683

@muimota Try increasing rpcthread. – MrFregg – 2016-12-26T17:42:27.103