How can Segwit increase transaction throughput if the same amount of data is stored in the blockchain?

I'm posting this after reading the answers to this: Where is the Signature stored in a SegWit transaction?

I've found out from reading that, that in Segwit, the "witnesses" (or signatures) that verify a transaction's authenticity are still stored on the blockchain. In which case, the same amount of bits ought to be used up by a given transaction as if Segwit weren't being used. If the block size remains the same, let's say 1MB, then it ought to hold the exact same number of transactions.

I've got an alternative possibility. Let's say that what actually gets sent to the nodes is a transaction with an empty signatures field (I think that such a field doesn't quite exist because of multi-sig stuff and scripts, but bear with me). Then the signatures can be sent through a separate channel. What actually gets stamped onto the blockchain is a transaction with an empty signatures field. And then because of the longest-blockchain-always-wins rule, the non-Segwit nodes will simply accept that the blockchain with the most number of transactions, which includes some funny signature-less transactions, is the authoritative blockchain.

But apparently reality is more complicated.

Also, there's a question about how much the longest-blockchain-always-wins rule can be abused. (The word "abused" does not imply a negative opinion). For instance, let's say I come up with FoobarWit. FoobarWit is the same as Bitcoin but with a larger blocksize (no Segwit). Once enough nodes adopt FoobarWit, then more such nodes will win the mining race, and so other nodes will effectively accept bigger blocks because the history is longer. I know that's not the case, but why?

jkabrg

Posted 2018-12-07T10:25:33.747

Reputation: 137

– Murch – 2018-12-07T17:45:52.153

@Murch I read that. Didn't really help – jkabrg – 2018-12-07T20:02:51.040

Answers

The answer to the question "Are the segwit witnesses part of the blockchain" depends on what you define as the blockchain:

According to old pre-segwit nodes, the answer is no, as they don't care or receive the witnesses.
According to new segwit nodes, the answer is yes; the witnesses are as much part of the chain as everything else, and subject to just as many validation rules.

So segwit increases throughput slightly, at the cost of increasing the size of the blockchain. There is no magic bullet here, and this aspect of segwit is simply a block size increase. It is not a scalability improvement.

Segwit was primarily a solution for the transaction malleability problem, opening the door for various higher-level protocols that were handicapped by being unable to reason about future on-chain transactions that weren't published yet. In addition it also improved he incentive structure a bit (making spending relatively cheaper w.r.t. creating outputs). As a final side effect of that, it also increased the block size a bit in a backward compatible way.

As for your question about another fork that goes even further: what your software does has no effect on those who don't adopt it. Even if it has all the hashrate in the world on board, old nodes won't care or see your additional data.

Pieter Wuille

Posted 2018-12-07T10:25:33.747

Reputation: 54 032

But what makes this change able to not trigger a hard fork? And how can something both be and not be on the blockchain? I thought there was only one chain. – jkabrg – 2018-12-07T20:07:50.030

I can't just take what you say on faith – jkabrg – 2018-12-07T20:09:21.393

The only real explanation would be a story of a segwit transaction between Alice and Bob making it through the network. Everything else is "It magically can do this!" Though James C.'s explanation is starting to help – jkabrg – 2018-12-07T20:10:16.150

I'm reading the BIP. If I have a breakthrough, I'll post an answer – jkabrg – 2018-12-07T20:16:22.293

Segwit nodes simply strip out the witnesses when relaying to non-segwit peers. As the witnesses do not affect the txid, there is nothing in a transaction that alludes or refers to its witness in any way, so nothing is broken for nodes who don't care about those witnesses. However, the block does contain a hash of all witnesses (in a way that looks like a piece of random data to old nodes), making it part of the chain for those who care about it. – Pieter Wuille – 2018-12-07T20:29:14.447

1It helps to realize that the blockchain is not just a blob of bytes. It's a structure with pieces of data that refers to other pieces of data with hashes. Whether a node requires the data pointed to by those hashes is its own decision (for example, light nodes have the headers but don't require the transactions). Segwit simply added a new piece of data, which new nodes care about and others don't. – Pieter Wuille – 2018-12-07T20:32:44.183

So it's almost like saying that we make a backwards-compatible dialect of C where some comments (which older compilers would ignore) become executable. I kind of get that. But then you're saying that the block contains only the hash of the witnesses – jkabrg – 2018-12-07T20:54:48.957

My understanding is that hashing is an irreversible operation. So you only put a scrambled version of the witness data onto the blockchain – jkabrg – 2018-12-07T20:55:20.217

1It's wrong to think of blocks to "contain" anything. They're all just hashes. In the same way you can also say transactions are not "part" of the blockchain, because block headers just contain a hash of the transactions. Whether you think of transactions being an essential part of the chain depends on the context: full nodes do (they won't accept a block with tx hashes without the actual transactions), light nodes don't (they see the hashes and are happy with that). The same is true for segwit witnesses. – Pieter Wuille – 2018-12-07T21:06:40.590

What is the point of a light node then? What on Earth can it do? I'm lost – jkabrg – 2018-12-07T21:13:06.120

1It can verify that transactions were included in the chain, selectively, without verifying that their signatures are valid. This is not as secure as a full node, but assuming enough others do run full node, it has much better usability with a security level that's acceptable to some use cases. Note that I'm not saying that segwit is related to this security tradeoff (it isn't), but it's a similar principle. You need to stop thinking about the chain being just a blob of data that contains some things and not other things. It's a structure with commitments, which some care about. – Pieter Wuille – 2018-12-07T21:32:40.243

The 1MB block limit still stands after segwit activation, but it solely refers to the block transactions serialised in the pre-segwit format, which includes no witnesses. This 1MB is still consensus today.

With segwit activated, there is an additional block size constraint, which DOES apply to segwit data: It is a "weight" limit of 4M per block.

The weight is computed as follows:

Transaction Bytes (Pre-segwit serialisation) x 3
Plus Transaction Bytes with Segwit (Segwit serialisation) x 1

As the witness portion of the total transaction data increases towards 100%, the effective block size limit in Bytes approaches 4MB.

James C.

Posted 2018-12-07T10:25:33.747

Reputation: 2 183

So the witnesses are not stamped onto the blockchain? – jkabrg – 2018-12-07T11:58:22.260

The block size constraint is to avoid DDOSes via the Segwit channel? – jkabrg – 2018-12-07T11:59:37.467

Good question: they are committed as witness root(like the merkleroot) together with the coinbase witness in a coinbase op_return output, which is validated for correctness. – James C. – 2018-12-07T12:00:37.300

Does the op_return field not count towards the classical 1MB block size? – jkabrg – 2018-12-07T12:02:08.977

There is no separate channel. If a node is not segwit ready, peers recognize this on the P2P level and refrain from sending witness data to the older peer. Witness data is “stripped” from the tx data. – James C. – 2018-12-07T12:02:23.640

Yes op_return is not witness data and therefore contributes to the 1MB block limit. – James C. – 2018-12-07T12:03:08.800

1"committed as witness root". Onto the blockchain? Thanks. – jkabrg – 2018-12-07T12:06:44.553

Yes in the coinbase transaction, which is the first tx in every block :) you can find details in one of the segwit BIPs. Something like dsha256(witnessroot|coinbasewitness). – James C. – 2018-12-07T12:12:57.913

Let us continue this discussion in chat.

– jkabrg – 2018-12-07T20:24:05.880