1
Based upon any standard you think relevant, how quickly are current blocks being verified for rule-following such as correct balance tallies, true transactions, correct rewards & fees?
Is this an onerous process? Would a database quicken this process?
Do you have any suggestions for improving block verification speed?
2
The bulk of the work lies in verifying each transaction within the block. This means that the signatures need to be checked for each transaction input, which requires a lot of CPU processing. So, one way to improve the verification speed would be to have a hardware-assist for the verification process (such as a specialized ASIC chip). I don't know if a miner ASIC could be used, but it is an interesting thought.
1
The transaction verification process is not particularly CPU or memory intensive. The CPU intensive part is the network protocol needed to get a global consensus on a single total order of transactions. That is called "mining" and that is what the specialized ASICs from various companies such as kncminer are designed to help with.
It depends on what part of the process you are looking at. Definitely, if you are looking at the mining end, then finding the correct block hash is the most CPU intensive. However, if you are looking at the peer nodes which receive blocks, then the transaction verification is the most CPU intensive. This is one reason it takes so long for a new peer to process the block chain. – ScripterRon – 2013-12-31T20:37:37.520
Thank you Neal Gafter! Would you mind explaining why verification isn't expensive and why the network portion is? It seems that every individual part could be fast but will become less so with higher verification frequency. Thank you so much in advance! – None – 2014-01-03T15:55:09.440
I've downloaded the whole blockchain, copied it to an off-line, recently-formatted machine, and still had to wait (an hour? 2? can't remember now) before everything was ready - so clearly verifying takes a lot of time and CPU. I once read the process could be accelerated should the QT client use the GPU for verifying, just as it's used for mining, but so far I've not seen any implementation of this idea... – Joe Pineda – 2014-01-10T20:34:32.587
That is not correct. See my answer. – Neal Gafter – 2013-12-31T19:53:55.573
Thank you ScripterRon! What about searching for past transactions to make sure the new ones balance? Is that not a terribly disk intensive process? Thank you very much in advance! – None – 2013-12-31T21:26:03.767
2The bitcoin client uses the LevelDB database to store the transaction outputs. I'm using the H2 database in my own application to store both the block chain and the transaction outputs. I'm not sure there is much more to be done other than performance tuning in high-usage areas (better SQL search arguments, indexes, caches, etc). – ScripterRon – 2014-01-01T01:04:18.483
Thank you so much again ScripterRon! Thank you for answering me many of my questions and giving me this crash course! I was able to lazily hash at about 1M/sec, so I don't understand how simple block verification is so intensive. Either something conforms or it doesn't, right? Do you have any statistics on the average block verification time vs hash power? I definitely can understand how a DDoS attack with bad blocks could bog one down, but I don't understand how verifying good actors' blocks is onerous except on the disk. Thank you so much in advance! – None – 2014-01-01T07:11:31.653
1
Verifying the signature is CPU-intensive since you need to first derive the public key from the address and then use the public key to verify the signature. The digital verification process uses very large numbers and complex mathematical formulas. Check https://en.bitcoin.it/wiki/Transactions for more information.
– ScripterRon – 2014-01-01T17:59:06.6171I should add that transaction verification isn't a problem when you get a new block once every 10 minutes. It only becomes a problem when you are downloading the initial block chain (either from bootstrap.dat or a peer node). It then becomes a gating factor in how fast the new node can synchronize with the network. And you are right, matching transaction inputs with unspent outputs is disk-intensive. So it is a combination of checking signatures, matching inputs to outputs, and creating new database entries. – ScripterRon – 2014-01-01T18:09:38.577
Thank you as usual ScripterRon! You've helped me so much with my undestanding of bitcoin! Isn't the public key merely concatted to the signature (or have I misread yet again?)? Can't a single hash or "decryption" be performed very cheaply? Thank you so much again and again in advance! – None – 2014-01-03T16:07:32.993
I am just getting into this area, so I can't tell you for sure. But I think the public key is not in the input, so all they have is the address from the referenced output. So the code has to first derive the public key using the address and the signed data. Once it has the public key, it can then perform the signature verification. I think it was done this way to reduce the size of the transaction (public keys are very long) – ScripterRon – 2014-01-03T19:07:01.177