5
I would like to keep the latest 512 MB of blocks on my SSD (using pruning?) and the rest of the blockchain on an external non-SSD hard-drive. Is there a way to do this already, or do I need to write a script to accomplish this?
Keeping old blocks on an external hard drive and latest blocks on an SSD
Answers
2
Blocks are not accessed under normal operation, except: When a peer fetches one (and the most recent block is usually served out of an in memory cache), when there is a reorginization that must undo the effect of a block (which is pretty rare), or when you use an RPC to look up a historic block.
With that in mind, perhaps you can just put your blocks directory on the other disk.
If you set your node to be pruned but set the pruned amount much larger than the amount of block data, you will avoid peers fetching large numbers of old blocks from you (but still keep the data around).
If you found high disk accesses while in this configuration, I think thats something that the project would like to fix.
0
You could have two datadirs, one locally and one on the external hard drive where, every so often, you connect the external hard drive and start Bitcoin Core to use that as a datadir and sync it up. The local one could be pruned.
However if you wanted to have most blocks stored on the external hard drive and have Bitcoin Core automatically move them there and then be able to handle when it can't find those blocks, then that is not possible.
0
If you are already synced, you could call a simple script using blocknotify=script.sh in your bitcoin.conf to check for a higher numbered block file, and if one is found, move the lowest one out and the newest one in.
But a better way would probably be doing this at the block level with some fs/raid abstraction, or an inotify watcher if you have to do it at the file level.
1I don't think this will work, as bitcoind may try to access the file afterwards. Even if you replace it with a symlink, it may be accessed in between deleting the old file and creating the link. – Pieter Wuille – 2017-10-31T06:37:23.483
Yeah, you're probably right. I would just raid the ssd & hdd together, but if that's not possible and you have to do it by moving blk.dat files around, probably the safest bet is to (1) watch the blocks directory for new files with inotify; (2) stop the bitcoind service, (3) move the newest file to the ssd, (4) move the oldest file on the ssd to the hdd, (5) resymlink those two files to blocks, (6) restart bitcoind. – Luke Mlsna – 2017-10-31T06:45:35.933
Are you using Bitcoin Core or some other client? Please add an appropriate tag. – Nate Eldredge – 2017-01-23T21:06:32.693
1Note that you might be able to achieve a similiar performance gain by putting all the blocks on the external hard drive (e.g. by making
.bitcoin/blocksa symlink) and leaving the chainstate and other heavily-accessed databases on the SSD. – Nate Eldredge – 2017-01-23T21:09:18.043@NateEldredge Yes, Core. I've added that tag. – Geremia – 2017-01-24T02:11:35.340
@NateEldredge Yes, performance-wise that would be good, but I don't want the external HD running constantly, as it would need to be if the latest blocks were stored on it. (I'm on a laptop, so there are energy limitations.) – Geremia – 2017-01-24T02:13:06.050
1I see. But I would not be surprised if you get fairly frequent requests for old blocks, from nodes trying to sync for the first time. Bitcoin Core would try to satisfy those requests and spin up the external drive. So depending on your timeout settings, I expect you will either be running that drive a large fraction of the time, or spinning it up and down a lot (not good for its lifespan). – Nate Eldredge – 2017-01-24T02:16:32.620
@NateEldredge Yeah, that's what I was wondering about: what percentage of nodes, on average, would be requesting old blocks anyways – Geremia – 2017-01-24T15:37:00.670
1you can remove the block serving service flag I think so that peers don't ask for historical blocks from you. – renlord – 2017-01-26T09:58:01.667
Or just set maxuploadtarget really low. – Luke Mlsna – 2017-10-31T06:47:54.077
I wonder if running two instances of bitcoind would work, one with full blocks, the other pruned, and allow the full block instance to only serve to the pruned one. – Bobbi Bennett – 2017-12-30T22:55:48.993