Orphaned (non referenced) blocks list

1

Orphan block is a block that doesn't have a known parent in the longest block chain.

As I understand, this mean that orphaned block does not have a reference on it as "previous block hash" in any newest block. It is correct?

If yes, I make some research and does not understand why I get some strange results.

So, I extract all block hashes from raw dat files. Then I extract all "previous block hash" data from raw dat files. As a result I've got 2 arrays: block hashes (array A) and parent references (array B). Then if I subtract B from A, I would get a list of orphaned blocks.

Is it right way to get orphaned block list, or not?

P.S. I get this results after parsing dat files from blk00000.dat to blk00953.dat (I chose two blocks from the compiled list):

000000000000000003D57B69D1AC77F64287C893C16ADBC1816C6D7386CCC3C0 – orphaned 0000000000000000011523D7477DD274B7E0DCC2C616B2E2F584FFDEC20237D3 - main chain

main chain and orphaned are the status based on block explorer sites.

On this two blocks does not any reference as "previous" in raw dat files. I'am confused – "why?"

D L

Posted 2017-06-30T06:08:22.840

Reputation: 478

The term 'orphan block' was commonly used for at least two very different things: see https://bitcoin.stackexchange.com/a/5869/208. It seems you're referring to the second meaning here, which is not relevant anymore. Since the introduction of headers-first sync, no such orphans exist anymore.

– Pieter Wuille – 2017-06-30T06:17:19.063

@PieterWuille thank's for help) But the first block from my example was mined 2016-11-29 (after release bitcoin core v0.10) and this block is in dat file that on disk \blocks dir – D L – 2017-06-30T06:49:16.573

Block explorer websites use the term 'orphaned' to mean "no longer in the main chain", not "no known parent". – Pieter Wuille – 2017-08-22T04:03:04.053

here is about the blocks that has no references after. I'v got em. – D L – 2017-08-24T00:13:40.810

Answers

1

As a result I've got 2 arrays: block hashes (array A) and parent references (array B). Then if I subtract B from A, I would get a list of orphaned blocks.

No, doing that will get you a list of stale blocks (which are commonly referred to as orphaned blocks). What you are doing is getting all blocks which are not parents of anything, not blocks which do not have parents.

000000000000000003D57B69D1AC77F64287C893C16ADBC1816C6D7386CCC3C0 – orphaned

The term "orphaned" here means that the block does not have any children. This is the second meaning of the term "orphaned block". This result is expected since you are removing from your list of block hashes the list of parent blocks, and this block is not a parent.

0000000000000000011523D7477DD274B7E0DCC2C616B2E2F584FFDEC20237D3 - main chain

This block is in the main chain. It is probably in your list because that is what your node was synced up to and this was its most recent block.

Andrew Chow

Posted 2017-06-30T06:08:22.840

Reputation: 40 910

I think my English is not so good. I mean that every block have reference on previous, but for example in blk00234.dat present block that has no reference on it in whole database. This block calls orphaned. Is it true? – D L – 2017-08-22T18:28:28.057

blocks can come to your node not in right order, so, it is possible to have a block without parent in your local storage – amaclin – 2017-08-22T18:37:30.133

Yes, a block which has no children is commonly referred to as orphaned. But that is not the definition of orphaned that you are using. – Andrew Chow – 2017-08-22T18:38:43.617

(...but for example in blk00234.dat...) the mainchain is equal on all nodes, but blk-files are not byte-to-byte equal to each other. your version has orphan block in file #234, mine does not have – amaclin – 2017-08-22T18:40:55.650

@amaclin so you tell that a full bitcoin database (\blocks\blk*.dat files) is not equal for all who install bitcoin client? – D L – 2017-08-22T18:43:29.840

yes. my file is different from yours. and my copy does not have 000000000000000003D57B69D1AC77F64287C893C16ADBC1816C6D7386CCC3C0 – amaclin – 2017-08-22T19:06:22.787

But I think that the main principal of blockchain is to keep the same database on every node. – D L – 2017-08-23T14:59:25.473

@Denis The goal is for everyone to have the same blockchain. However the blockchain itself is not a database, and not everyone will have exactly the same blocks in exactly the same order. But the node software will be able to parse the blockchain data and figure out which blocks to use so all nodes eventually end up using the same blockchain. Nodes can have different blockchain data, but the vast majority of it will be the same. The only differences are from orphaned blocks since new nodes won't know about them and those orphaned blocks may propagate slowly. – Andrew Chow – 2017-08-23T20:08:01.913

@AndrewChow you mean that local database is not equal for all nodes? – D L – 2017-08-23T22:30:52.787

Yes, the databases are not equal for all nodes. – Andrew Chow – 2017-08-23T22:51:20.830

@AndrewChow please feel the basic principal of p2p tech) Database is equal for every node. That is true. Especially for Bitcoin chain. So -1. – D L – 2017-08-23T23:00:47.977

@Denis, No, that is completely wrong. Please understand how the P2P protocol works before jumping to such conclusions. The entire system is asynchronous, which means that peers will receive blocks at different times and complete validating blocks at different times. There is latency in block relay, and that causes nodes to disagree. – Andrew Chow – 2017-08-23T23:09:52.890

For example, two miners (miner A and miner B) can find a block at the same height at roughly the same time. When the block is broadcasted, peers will receive one block before receiving the other. This means that some peers will have A's block first and B's block second. Their databases now disagree. If some other miner mines a block on top of A's block and there is no race at that time, A's block becomes part of the main chain and B's block is orphaned. – Andrew Chow – 2017-08-23T23:10:02.320

Now suppose C decides to set up a new full node. He will only be downloading blocks in the main chain, so his database will only contain A's block, not B's, since B's block was orphaned and no longer relevant. C is still using the same blockchain as everyone else, but his database will be different from other people's. – Andrew Chow – 2017-08-23T23:10:05.843

But files blk*.dat in blocks dir are the same for everyone that's connected to bitcoin? – D L – 2017-08-23T23:13:32.440

No, they are not. Blocks are not downloaded and stored to disk in order, so the block files will likely be different just due to ordering. Furthermore, as in my example above, if you have a new node, you won't have downloaded and stored to disk an orphan block, but other nodes that were online at the time the block was mined will have downloaded and stored the orphan block. That will make your block files different from other nodes. – Andrew Chow – 2017-08-23T23:15:29.610

@AndrewChow database is equal for every node, byte by byte. – D L – 2017-08-23T23:18:42.320

@Denis no they are not. Why do you keep insisting that they are? I maintain multiple nodes and I can prove that they have different block files and databases. If you have a node, then I can prove that my nodes have different block files and databases from yours. – Andrew Chow – 2017-08-23T23:21:51.790

@AndrewChow if this is so, then the main blockchain principle collapses. – D L – 2017-08-23T23:24:56.150

Let us continue this discussion in chat.

– Andrew Chow – 2017-08-23T23:26:03.607

Asked: 2017-06-30T06:08:22.840

Viewed: 629 times

Active: 2017-08-22T18:38:52.360