Would it be possible to add an RPC call to be able to determine if a node is fully synced?

After reading this answer, where Peter Wuille talks about how a node can't really tell if it is synced or not, it got me wondering if there is a way we can tell that a node is as synced as it possibly can be given the peers it is connected to.

Could we add an RPC call like getsyncstatus that would have the node ask its peers for their highest known blocks and if a majority (or some percentage) of their peers all agree on the best known block height, then it could use that info to be able to say (with not perfect reliability) how synced the node is?

So, for example, if a node has 8 peers, and 7 of them are returning 330448 as the best block height, and the client has information about block 330448, then it could give a status code and message signalling that it is up to date. If the peers give different answers (which could happen if you connect to a malicious node that is falsely reporting it's best known height OR if other peers are not fully synced), then it could give a status code and a message indicating that the search was inconclusive and they should try again later. And lastly, if 87.5% of nodes are saying 330448 and the node is only synced up to block 1000, then it can give a status code and message indicating that the node is likely not sufficiently synchronized.

I think this would mainly be useful as a way to determine whether servers are sufficiently synchronized with the network for use on some application. The responses to this question describes a way to see if nodes are up to date, but they all seem kind of hacky, and I'm not sure they would be reliable enough.

morsecoder

Posted 2014-11-17T18:56:21.730

Reputation: 12 624

Answers

Existing Methods

The Bitcoin client does have a way of detecting if it is not sufficiently synchronized. However, to my knowledge, it's not really exposed to RPC in a nice way.

It's called IsInitialBlockDownload(), and it will consider itself not synchronized if any of the following are true:

The client is importing from bootstrap.dat or reindexing the blocks it has.
The client has not reached all of its block checkpoints.
The client has received and verified a block in the last 10 seconds AND that block has a timestamp of more than 24 hours ago.

You can call getblocktemplate, and if IsInitialBlockDownload() is true, it will come back with an error of RPC_CLIENT_IN_INITIAL_DOWNLOAD (or -10).

Drawbacks

If your Bitcoin node is down for less than 24 hours, this won't detect anything wrong when it comes back up.
It's a bit ugly.
When your client gets within 24 hours of the current block, it'll think it's up to date, when it probably needs to get ~144 blocks.
If your client stops downloading blocks while it's still synchronizing, this won't detect anything wrong.

New RPC call

Let's suppose that we're designing a new RPC call from scratch. As other posters have noted, it's not possible to determine this perfectly in the presence of malicious actors. What factors could we measure to make an educated guess?

Node uptime. Has the node been up for at least five minutes?
Node downtime. Has the node been down recently? If so, take the amount of time that it was down, and divide it a thousand. If your node has not been up at least that long, then it is not synchronized.
Block timestamp. The network is fairly liberal about enforcing this rule, but my unscientific guess is that it's usually within half an hour or so. You could use this information in a similar way to how IsInitialBlockDownload() does.
Block height of central node. This is the "Look at blockheight of blockexplorer," solution mentioned in the other question.
Your method would probably work. The only issue I can think of is that if you connected to another node that was still syncronizing, you'd fail to fail.

Nick ODell

Posted 2014-11-17T18:56:21.730

Reputation: 26 536

What if the client has received a block recently but it is not done processing it, what will getblocktemplate do in this case? – morsecoder – 2014-11-17T19:57:27.327

@StephenM347 It looks at when it finished validating the tip of the best blockchain. (So, it doesn't take blocks that haven't been validated into account.) – Nick ODell – 2014-11-17T20:00:03.123

Okay, thanks. I wish this were exposed in the RPC interface in a less hacky way. – morsecoder – 2014-11-17T20:02:45.783

So won't this method fail if the client is synced past all of the checkpoints, but has been offline for like 8 hours? – morsecoder – 2014-11-17T20:06:15.440

2@StephenM347 Well, it'll fail to fail. :) In all seriousness, you're correct. – Nick ODell – 2014-11-17T21:20:16.600

So, unfortunately this doesn't really solve the problem then... :/ – morsecoder – 2014-11-17T21:27:55.767

1But it does answer my question, so if you update it to include the stuff you told me in this commenting, I'll mark is as the right answer. And adding a new RPC call could fix this, you think, right? – morsecoder – 2014-11-17T21:33:55.063

No, there is no way to tell, because there is no definition of "being synchronized". Are you still synchronized at the very moment some miner finds a new block? How would you know?

What you describe has no benefit over the already implemented methods of detecting whether the node is far behind (also it is already implemented as Pieter Wuille explains in the answer you linked, though it is not available in the API).

What is it exactly what to try to achieve with this information? Why do you need to know if a server is sufficiently synchronized? You will encounter the real blocks much sooner than any reasonable attacker can generate them.

Jori

Posted 2014-11-17T18:56:21.730

Reputation: 1 522

1Example usage: You have an payment processing application, and you want it to disable itself if it's blockchain download is not up to date, rather than some features mysteriously not working. – Nick ODell – 2014-11-17T19:54:26.290

Just like Nick said. – morsecoder – 2014-11-17T19:55:30.833

@StephenM347 How many payment processing application do not run continuously? Even then, you can still just check the time stamp in the last block and compare it to the system time. I don't see the advantage. – Jori – 2014-11-17T19:59:34.510

Those timestamps in blocks are pretty unreliable, so I don't think that's a possibility. We have a more complicated use case in mind, but I don't really want to go too far into it. What we really want is a way to know if a server is too far behind to be reliable. – morsecoder – 2014-11-17T20:22:33.213

And even if it runs continuously, you need to know when it's ready after the initial download. – morsecoder – 2014-11-17T20:26:10.473

@StephenM347 Why are they not reliable? They are at least equally reliable as querying random unverifiable nodes for their block height. – Jori – 2014-11-17T20:46:00.647

They're not reliable because miners change them frequently to get more work, and the network allows for a certain amount of flexibility when interpreting these block times. I don't think they would be as reliable as querying the nodes who you are connected to, since bitcoin only really works if most nodes are honest nodes. – morsecoder – 2014-11-17T21:31:53.817

Well bitcoind is opensource so it is entirely possible for you to try to integrate something like that yourself. However, rather than make changes to the base code, you might be able to get similar functionality using the rpc calls that are already implemented. Essentially, the bitcoin client already somewhat does what you're explaining. When you connect to a peer, part of the handshake process is sharing each other's current block; if one is higher than the other's and is valid, the other starts to download from the more current peer. If you look at the debug.log file you can see this happening, along with a running "progress" field that gets logged showing you the percentage you have downloaded.

However for your idea, where you aggregate information, you can try using the "getpeerinfo" rpc call. You could take the information from there and analyse it in anyway you wish without having to to make a native rpc call.

CJ-Coinsetter

Posted 2014-11-17T18:56:21.730

Reputation: 151

May or may not be of use to you, but just to check sometimes I have a little script/webpage that gets the current block height from each of my bitcoind nodes - if any of them are lower than the majority then they are out of sync. Obviously you'd need nodes on different networks and machines else an error could mean all nodes are out of sync. Works for me though, it's simple and uses an existing RPC call.

George

Posted 2014-11-17T18:56:21.730

Reputation: 586