Do we need to maintain the ability to do a reorg over 100 blocks?

1

1

The protocol requires that freshly generated coins wait 100 blocks before they can be spent in order to avoid polluting bitcoin addresses with bitcoins that are "uncreated" by a reorg that invalidates the block that created them.

If we decide that "the chain with the most difficulty" means "the chain with the most difficulty over the last N blocks" and pick a good N, it could make some things simpler AND make transactions at a depth of at least N immune to a 51% attack. Thus, it limits the value of cornering the market in hashing power. Such "cornering" could happen without being noticed, so it has begun to worry me.

If I remember correctly, we've already done this by (sometimes?) making checkpoints in the Satoshi client at sufficiently large depths when new clients got compiled.

N = 4032 would ensure that two difficulty levels are spanned.
N = 100 would change the pain of a 100-block reorg from something barely tolerable to something impossible.

If the protocol were changed to use this, then a UTXO database based on the results of the blockchain up to the checkpoint block could replace the gigabytes of blockchain that we currently use for validation.

EDIT: Why I asked: It would be nice to validate a UTXO database so that one didn't have to build it from (always increasing GBs of) downloaded block data. We validate transactions before passing them on, but do we validate them when we download a(n old enough) block? I think we trust that whatever (old enough) block we get from a peer contains good transactions, so we don't hunt all previous blocks to validate them. If this trust is already being used, as I suspect, perhaps it is more useful than we know. I asked a related question: Would a normalized, hashed, protected chainstate reduce miner storage costs?. I'll have to explore the work on making a UTXO Merkle tree as that seems promising.

Dave Scotese

Posted 2015-07-30T05:44:13.803

Reputation: 719

Clients today only store the UTXO, the blocks are optional and completely unused for anything other than serving to other peers. Reorganization is served by undo blocks which are stored separately. You always need to have all the blocks at some point in time to build your UTXO though, there's no getting around that. – Anonymous – 2015-07-30T06:59:06.310

What makes you think it's a bad thing that coinbase transactions need to be mature before they can be spent? It also serves as a (small) protection against a miner doing something really bad. The evil block might expose a big problem and tank the BTC price, making the reward worthless as well. Or other miners might quickly come to a consensus to fork the chain and orphan the evil block. – Jannes – 2015-07-30T13:29:40.700

I don't think it's a bad thing. I think it's a great idea. What's bad is pain, and waiting for coinbase transactions to mature is a good way to avoid that pain. I suspect I wrote something that puzzled you and I haven't addressed it yet so if you can explain a bit more, I'll do my best. – Dave Scotese – 2015-07-31T14:33:07.013

A) Coinbase maturation is a consensus rule, and would require a hardfork to change. B) 100 blocks is less than a day. Hell, exchanging your bitcoins for fiat will take you longer than that. – Nick ODell – 2015-07-31T18:50:47.850

@Nick, I still think one of us is not understanding the other. I didn't mean to propose changing the coinbase maturation. I intended to suggest that making it impossible to do a reorg at some depth ("N or greater") would solve more problems than it would create - if N is big enough.

I like revising my strategy for rhetoric, so if you don't mind quoting what I wrote that seemed to propose a change to the maturation, I will thank you. – Dave Scotese – 2015-08-01T00:55:10.427

Answers

1

If the protocol were changed to use this, then a UTXO database based on the results of the blockchain up to the checkpoint block could replace the gigabytes of blockchain that we currently use for validation.

You can already do something similar to this by turning on -prune. That can cause your node to stop working if the reorg is more than 288 blocks, though as you noted, that doesn't happen often.

That still requires you to download the entire blockchain in the first place, of course.

If we decide that "the chain with the most difficulty" means "the chain with the most difficulty over the last N blocks" and pick a good N, it could make some things simpler AND make transactions at a depth of at least N immune to a 51% attack. Thus, it limits the value of cornering the market in hashing power. Such "cornering" could happen without being noticed, so it has begun to worry me.

Suppose a node joins later, after the new chain has been published. From its perspective, the two chains were created at the same time. Because of that, it would naturally choose the one with more work attached to it.

So how do we avoid that? This is a problem that's a lot more complicated than it looks.

The protocol requires that freshly generated coins wait 100 blocks before they can be spent in order to avoid polluting bitcoin addresses with bitcoins that are "uncreated" by a reorg that invalidates the block that created them.

Sure, but we're talking about two levels of consequences. In one circumstance, one person on the network loses money. In the other, the blockchain is forked forever.

Nick ODell

Posted 2015-07-30T05:44:13.803

Reputation: 26 536

Mind that you can also enable prune with a higher target like -prune=2000 (=2GiB) and that the amount of possible blocks/undos is only an estimation (mind the variable block size).

If a reorg with more then 100 blocks would be a dramatic (bitcoin-killing?) event. Blocks nowadays containing transactions summed to a value of ~3000BTC. Reorging 100 blocks would put around 3'000'000 transactions back to the mempool (a value of 300'000BTC). – Jonas Schnelli – 2015-07-30T06:58:13.810

@JonasSchnelli Due to a regression in syncing logic until very recently the majority of nodes in the network couldn't handle very deep reorganizations either. Hundreds of nodes still run <=0.9.x unfortunately, though they have other problems at present. – Anonymous – 2015-07-30T07:05:38.580

@Nick All forks appear to have been created when block one was solved. One of us (probably me) doesn't understand something the other one knows. It seems to me that there is a critical number of blocks beyond which a reorg would be so painful that we could choose to just accept the problem - grandfather it in or whatever. – Dave Scotese – 2015-07-31T14:44:31.560

What you're describing could be implemented using broadcast checkpoints, or something similar. You now have two problems: ensuring that no one gains a majority of hashpower, and protecting the broadcast checkpoint key. Either can be used to do evil stuff. – Nick ODell – 2015-07-31T18:36:17.283

We already had the first problem, but yeah, about that new problem: That is why I figured it would be a good idea to include the merkle root of the UTXO database (not exactly a checkpoint, as you said, but same idea), once it's structured as a merkle tree, which I think is being worked on (see the discussion on the question I posed, which is displayed in the new paragraph I added to this question). – Dave Scotese – 2015-08-01T01:00:10.807

It seems like you're trying to fix two unrelated problems (sync time, big reorgs) with two unrelated solutions. (utxo commitments, refusing big reorgs) It will be a lot easier to discuss your suggestions if you stick to one topic per question. – Nick ODell – 2015-08-01T02:10:31.297

Asked: 2015-07-30T05:44:13.803

Viewed: 418 times

Active: 2015-08-31T12:28:24.323