bitcoind JSON-RPC interface timeouts under unit tests

3

I am running unit tests against bitcoind which do JSON-RPC requests in high rate. Random failures are an issue. E.g. if you have something like this in printconsole logs:

ThreadRPCServer method=gettransaction
ThreadRPCServer method=gettransaction
ThreadRPCServer method=gettransaction
ThreadRPCServer method=gettransaction
....
ThreadRPCServer method=gettransaction
ThreadRPCServer method=gettransaction
sending: inv (37 bytes)
ThreadRPCServer method=gettransaction
ThreadRPCServer method=listtransactions
sending: inv (37 bytes)

You start getting:

synced with network?".format(name)) from e
cryptoassets.core.backend.bitcoind.BitcoindJSONError: Got timeout when doing bitcoin RPC call listtransactions. 

I have tried with both reusing HTTP connection and not reusing HTTP connection with various results.

Are there any low hanging fruits to make bitcoind RPC interface more stable (increase threads, concurrency, etc.?)

I am using the following Python JSON-RPC implementation:

https://github.com/jgarzik/python-bitcoinrpc/blob/master/bitcoinrpc/authproxy.py

UPDATE: Looks like the problematic call might be listtransactions. When called at the same the time from command line using curl there is no response either:

[~]% curl --user foo:bar --data-binary '{"id":"t0", "method": "listtransactions", "params": ["cryptoassets"] }' http://127.0.0.1:8332/ |python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:--100    70    0     0  100    70      0     58  

Mikko Ohtamaa

Posted 2014-12-02T13:44:40.080

Reputation: 2 417

rpcthreads=64 seems to work around the issue, but doesn't explain what's happening.Mikko Ohtamaa 2014-12-02T14:10:31.203

Answers

1

I used BitcoinLib to perform 15,000 consecutive listtransactions calls and the result was:

0 timeouts
Fastest response: 0.1133479 sec
Slowest response: 0.6475277 sec

bitcoind was started with the default parameters and was receiving blocks during the test.

enter image description here

George Kimionis

Posted 2014-12-02T13:44:40.080

Reputation: 2 824

Thanks a lot! I'll keep digging what causes bitcoind to go haywire. I'll try to make a bash script to repeat the problem. Maybe it's the pattern of calls made.Mikko Ohtamaa 2014-12-02T23:47:16.810

1

@MikkoOhtamaa here's the gist for the above test: https://gist.github.com/GeorgeKimionis/7532fe23a5ad74c9c628

George Kimionis 2014-12-03T10:24:43.563