Blockchain synchronization is painfully slow / Best way to download blockchain

I'm trying to setup my own Bitcoin node on Ubuntu 16.04.1 LTS.

Right now, I've stuck with blockchain synchronization. I'm letting bitcoind to run every day for at least 10-12 hours while I work and it already takes three days to download 83% of the entire data.

I have very good Internet connection, it should take no more than two hours to download 80 GB of data, however, it's taking at least 30 hours already. My connection is practically free from other downloads most of the time.

I've googled this problem: some people say that current version of the bitcoind is so fast that it doesn't matter how blockchain is downloaded throught the Bitcoin network or by torrent. Other people say, that it takes a week to download the entire blockchain.

Is there a way to optimize my setup to make it download blockchain much-much faster?
Or do I better download it via some torrent and then synchronize the differences from the network? What is the best place to find such torrent file or magnet link? Also, is it safe to download blockchain from some third-party, will bitcoind validate it?

SO, generally speaking, what is the best way to download the blockchain in order to setup a working up-to-date Bitcoin node?

$ cat /etc/issue
Ubuntu 16.04.1 LTS \n \l

$ uname -a
Linux destiny 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:39:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

$ bitcoind -version
Bitcoin Core Daemon version v0.13.1.0-g03422e5

Slava Fomin II

Posted 2016-11-11T19:01:50.460

Reputation: 133

Answers

What usually takes time is the validation of the transactions in the blockchain, not downloading it. Bitcoin Core implements a fully validating node, which does not trust any of the information other peers give it. The only way to accomplish that is by validating everything itself.

30 hours sounds painfully long though. If you have memory to spare on your system, you absolutely should increase the database cache size. The default in 0.13 is 300 MB, but you'll see very significant speedups from using a higher value, up to 4000 or so (which would require an extra 4GB of memory).

Pieter Wuille

Posted 2016-11-11T19:01:50.460

Reputation: 54 032

Thanks Pieter, that makes perfect sense. I will try to increase the cache size then and will see if it helps! – Slava Fomin II – 2016-11-11T19:08:21.390

By the way, is it safe to download blockchain from some third-party, will bitcoind validate it? – Slava Fomin II – 2016-11-11T19:16:34.267

You can download a bootstrap.dat file (which contains just blocks to process at startup, as if they were received from network). Do not download a full database copy, bitcoind does not and cannot know you copied from a potentially untrustworthy party. – Pieter Wuille – 2016-11-11T20:35:42.783

Thanks! Please consider adding this info to the answer, so other users could also easily find it. – Slava Fomin II – 2016-11-11T20:55:50.540

Ugh, its already been taking me days and it still says 3 days to go. – B T – 2017-06-03T01:27:10.993

How do we increase the database cache size? – Shamoon – 2017-10-24T20:37:51.127

Using the -dbcache command line option, the dbcache config file option, or in the settings menu in Bitcoin-Qt. – Pieter Wuille – 2017-10-25T00:16:02.823

If anyone else has a similar problem: what really made a difference for me was moving the chainstate directory to a ssd. Now it's about 10x faster. – Jure1873 – 2017-12-03T09:01:26.380

@Jure1873 Did you try increasing the dbcache size? Disk speed should be almost irrelevant if you can give it a few GB of cache. – Pieter Wuille – 2017-12-03T13:48:16.023

Yes I did: 4000 dbcache with 8gb ram - no difference, but moving the chainstate dir. to my ssd and symlinking it back - the speed went from 0,5% to 6% and then to 10%. So if anyone else has the same problem it's definitely worth trying. – Jure1873 – 2017-12-04T18:40:30.890

1@Jure1873 Do you have pruning enabled? There is currently a bug that makes pruning effectively reduce the dbcache effectiveness. – Pieter Wuille – 2017-12-04T18:42:20.173

No, I didn't have pruning. Apparently disk was the bottleneck in my situation. – Jure1873 – 2017-12-05T21:24:45.240

downloading 150 Gigabyte will always be very slow. Some suggest to set "listen" to "off" but this has no effect on speed. You might, at intervals, ban the slowest 3 nodes, so you get pretty fast nodes after a while.

dotbit

Posted 2016-11-11T19:01:50.460

Reputation: 101

The problem was not with network bandwidth, but with the fact that Bitcoin verifies all the blocks and it takes a lot of CPU resources. The 150 GB could be downloaded within 2 hours on my network, but it will take much more time to verify all those blocks. And the good CPU will really help in speeding up this process. – Slava Fomin II – 2018-12-25T09:13:29.610