1
Considering how large the total blockchain is relative to average download speeds, why were the blocks never data compressed before being sent?
Why could this not be implemented on a new alt?
Why aren't blocks "zipped"?
Answers
3
Because the data is already serialized in a pretty compact way and large parts of that data are hashes which are effectively random numbers as far as compressability is concerned. (Random data is not compressable.)
Also when you search for for example a transaction id in an index, it will point you to the exact location in the blockchain. Having to read a whole bunch of data around that area and decompress it before being able to access the transaction that you wanted, adds disk I/O, CPU overhead and memory consumption. That severely limits the dictionary size any compression algorithm could use, and therefore the compression ratio.
Give it a try by zipping the whole blockchain file, but remember that any usable live compression would have a lower ratio than what you would find.
Thank you Jannes! I definitely agree that they should not be stored compressed, and thank you especially for showing me the futility of compressing hashes, but considering that the vast bulk of the data in a block is the transaction array, do you still think that that section can't be compressed? Thank you so much in advance! – None – 2013-12-27T05:21:35.853
Nevermind. Transactions are sent as signed hashes, not raw data. You're right, there probably isn't too much benefit in compressing the json keys. Thank you! – None – 2013-12-29T03:16:15.243
It's not clear what exactly you're asking. Do you mean compressed before being sent over the network? Compressed for storage somewhere? – David Schwartz – 2013-12-27T04:54:09.610
@DavidSchwartz Thank you David Schwartz! Before being sent. – None – 2013-12-27T05:15:46.457