6
2
A transaction is confirmed if 80% of the nodes in the UNL accept it. Where does this 80% number come from? Could it just as well have been 79% or 81%? What about 51%?
Can different nodes have different thresholds (for faster transactions, say), or must they all follow the same rule?
5
If validators waited until other validators declared consensus to declare consensus themselves, no validator would ever declare consensus. So you must declare consensus and hope you turn out to be right. Most of the time you will be. If not, you will later not be mathematically certain of the results and will not pass them on to clients.
If you use a 75% threshold for mathematical certainty, many times you wouldn't be certain and would have to repeat the process. If you picked a 90% threshold for mathematical certainty, you'd be certain almost all of the time, but you'd be wasting time.
The video is a bit misleading. In fact, any threshold could be used and you would still have mathematical certainty if you actually did reach consensus. However, if you set the threshold too low, you would only rarely have certainty. 80% is the threshold we've chosen for mathematical certainty because it's a compromise that results in certainty being reached often and quickly.
Maybe that's not clear enough. Let me try it another way. When we report a transaction as fully validated, we want to be mathematically certain of that. We will not make such a report until we are. Choosing 80% as the consensus threshold allows us to reach consensus quickly, requires at least 20% of validators to be completely malicious to delay consensus, and allows us to (in a later step) be mathematically certain of the result almost all of the time.
Note that almost all of the time is not always. Sometimes, we reach 80% and later are not certain of the results. In that case, we wait until the next consensus round completes and hope we fare better in that one.
Say for some stupid reason you set your threshold to 10%. You would think you had consensus all the time. But equally obviously, a lot of the time there would be no actual consensus. If there were, you could be certain that it had happened (because you would see it). But it often wouldn't happen, and then validators would have to resynchronize and clients would be waiting patiently for the network to fix itself so their transactions can be confirmed.
Ledgers that pass the consensus process are still held up by the "validation gate". They are not reported to clients as "fully validated" until they pass this gate.
80% worked very, very well in simulation. Validators are free to change this threshold if they want. Perhaps in a future network topology, a different threshold will give better performance characteristics.
It's hard to write clearly at 3AM. Maybe I'll rewrite this when I'm more awake.
"Sometimes, we reach 80% and later are not certain of the results." Why would you be "not certain" when 80% is the very definition of certainty? Do you mean you wouldn't be certain "in a later step" (which you allude to)? "... requires at least 20% of validators to be completely malicious to delay consensus." That's 20% + 1, right? And what is the definition of "fully validated?" (By the way, your detailed answers are much appreciated. I'm sure this will be useful to anyone taking an interest in Ripple.) – Manish – 2013-04-27T21:45:57.757
Once you reach the 80% threshold, you move on to mathematical certainty. If you didn't actually have a consensus, the certainty step would fail. A transaction is "Fully validated" if a majority of validators you trust have signed off on a ledger that includes it. If there's no actual consensus, then the ledger won't fully validate. (Which is fine, it just wastes time.) – David Schwartz – 2013-04-27T23:19:40.230
So there are two steps: (1) consensus (80%); (2) validation (majority of validators). But why two steps? And how much is the "majority" in the second step? – Manish – 2013-04-27T23:32:33.157
1There need to be two steps because someone has to build a ledger first or nobody ever will. Think of a crowd of people walking towards two doors and they want to walk through the same door. Once 80% are closer to one door, you walk through it. Then you look back to see if everyone follows you (if so, you are mathematically certain you walked through the right door). If not, you failed and try again. The majority in the second step is currently 50+%. – David Schwartz – 2013-04-27T23:36:33.067
Why ripple needs 80% of threshold in consensus? What is the mathematical prove behind that? Why not 66% threshold? Because i think 66% threshold in consensus is enough to validate a transaction in a distributed environment. Because (2m+1)/(3m+1) equation gives a valid threshold in a Byzantine fault, where 2m+1 represents the genuine validation nodes and 3m+1 represents the entire validation nodes. – tdumidu – 2014-05-18T07:54:03.537
With a lower percentage, the required overlap in trust lists is greater, making it harder to avoid catastrophic failure (double spends). – David Schwartz – 2014-05-19T13:12:56.857
@tdumidu To make this a little clearer. Ripple needs 80% to reach the 66% required for the algorithm to work because people are taking 80% of different things and we need them to be taking 66% of the same thing. Say you and I have lists of names with lots of overlap. If we each require 80% of the people on our list to agree, that gives us a good chance that we will have 66% overlapping agreement. That's what we need to still make safe forward progress with the highest number of malicious nodes. – David Schwartz – 2017-11-13T17:31:50.897
@DavidSchwartz Where can we read more about how the final step works that ensures "mathematical certainty"? Sure, if you demand 80% local "pre-commit" certainty, there will be a higher chance that the final commit succeeds. But starting from this, how do you actually guarantee that > 2/3 or > 1/2 of the total participants agree on the same state and thus pass the "validation gate" you mention? It seems that to achieve that, knowing the commitment of UNL servers is not enough, as you need to know if n% of ALL nodes have committed. – nh2 – 2018-01-22T15:58:56.817
@nh2 You don't need to be certain to commit. You just need to be certain that if you do commit, no other honest node will commit to a different ledger. – David Schwartz – 2018-01-22T16:03:32.457
@DavidSchwartz But how do you achieve that certainty? In your example of going through the door, what you just said would be analogous to saying "you don't need to be certain to go through the door. You just need to be certain that if you go through the door, no other person will go through another door". But in your example you highlighted that if not everybody followes you, you failed and try again (you go back through the door). So that sounds like no single node can be certain that everybody will follow if they go through the door. [continuing in next comment ...] – nh2 – 2018-01-22T16:11:35.330
[... continued] In other Byzantine consensus algorithms, they obtain hard guarantees on the agreed value by collecting a supermajority (typically > 1/2 or > 2/3) of all nodes (not only some subset of the network graph). As soon as more than this many yes-votes have been received, guarenteed finality is achieved (the node receiving this many votes is 100% certain that the network will not accept a different value instead, and any form of rollback is impossible). Does this concept exist like this in Ripple? – nh2 – 2018-01-22T16:17:25.463
80% is a supermajority of all nodes that you consider to be participating in the consensus process. There has to be sufficient agreement on who is participating to ensure that no two participants can see an 80% supermajority for a different ledger. You can think of sufficient agreement on participants as a network rule, akin to block size in bitcoin. Agree on it with others or you risk/create a fork. – David Schwartz – 2018-01-22T16:22:07.253
Or, to put it another way, instead of being forced to pay three companies we didn't choose millions of dollars a day to decide which transactions go in a block, let's just agree on those three companies subject to them following deterministic, fair rule for transaction inclusion and retain the right to gradually change the companies if they don't behave as we wish them to. – David Schwartz – 2018-01-22T16:24:48.030
The explanation video says it is "the threshold for mathematical certainty"... – Steven Roose – 2013-04-26T01:31:33.523
@StevenRoose Yeah, but is it? I mean, why not 90% or 75%? – Manish – 2013-04-26T02:52:29.493