4
1
I am data mining the blockchain using JSON/RPC on a bitcoind full node with txindex (0.9.1).
I have 64 RPC connections simultaneously going to bitcoind from a remote client. My workload is mostly getrawtransaction and decoderawtransaction. I have bumped rpcthreads to 64.
What I am observing though is that bitcoind is taking a very large amount of time (from 30 seconds to 2-3 minutes) to respond to each call.
I have verified that there are about 66 connections opened on the RPC port, and the bitcoind process has 73 threads, so that seems to confirm that both my load and the rpcthreads setting are correct.
If I query something like "help", I get the same slow response time, so it seems to be related to the RPC server rather than actually accessing the data from disk/memory.
Am I missing a setting to make it scale?
2What is the cpu load on the server? How about the memory usage; are you swapping? Bitcoind is not particularly optimized for performance so it's conceivable that it is just that slow. – Nate Eldredge – 2014-04-29T13:47:43.200
CPU is ok, but the memory usage is high, I see quite a bit of swapping. I will allocate more RAM to the machine and see if that makes a difference. – Flavien – 2014-04-29T13:51:16.180
1When you have a use case that is very different from what everyone else uses a piece of software for, it's not unusual to see pathological performance behavior. You may want to profile the running instance and see if there's any obvious optimizations that help with your particular use case. – David Schwartz – 2014-04-29T20:55:57.673