Introduction Optimistic Generic Broadcast Conclusion
Optimistic Generic Broadcast Piotr Zieli´ nski Computer Laboratory University of Cambridge
September 28, 2005
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
Hotel booking system
Protocol 1
client → server: “book room 5”
2
server → client: “room booked”
client
book room 5
client
room booked
server server
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
Fault tolerance by replication Problem a single server crash blocks the entire system
A
A
A
A
B
B
C
C
Solution introduce many servers system still usable despite some servers being down
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
Consistency problems Problem messages might reach the replicas in different orders A and B book the room to
client , replica C to client . results: unpredictable
A B C
Solution ensure that replicas receive requests in the same order by using Atomic Broadcast to disseminate requests
A B C
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
Atomic Broadcast atomic broadcast
Atomic Broadcast clients atomically broadcast messages, such as and replicas atomically deliver them
A
replicas atomically deliver all messages in the same order
B
Atomic Broadcast
C
fault-tolerant
Goal: Minimizing latency in common scenarios
Piotr Zieli´ nski
Optimistic Generic Broadcast
atomic delivery
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
System assumptions
Messages message passing: communication by messages reliable channels: no message loss between correct processes asynchrony: no time bounds for messages, no clocks Processes crash-stop model: only crash failures, no malicious processes Ω leader elector: leader eventually correct and fixed n > 3f: less than a third of the servers can crash Consensus implementable in one step
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
Goal: minimizing latency
Goal Atomic Broadcast with minimum latency if the leader is correct and does not change.
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
Generic Broadcast read x write to x r w r w r r w w
A B
Atomic Broadcast
r r w w r r w w
C
Observations ordering all messages is expensive (Atomic Broadcast) not all messages have to be ordered (Generic Broadcast) r r w w = r r w w 6= r r w w Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Replication example Our goal: minimizing latency Generic Broadcast
Generic Broadcast read x write to x r w r w r r w w
A B
Generic Broadcast
r r w w r r w w
C
Observations ordering all messages is expensive (Atomic Broadcast) not all messages have to be ordered (Generic Broadcast) r r w w = r r w w 6= r r w w Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Generic Broadcast r w
w r
Meta-solution 1
Define the conflict relation“ ”. Only conflicting messages must be delivered in the same order.
2
Determine the partial order “ of conflicting messages.
3
Deliver messages in any total order consistent with “ ”.
r w
w r
r
r w w
r
r w w Piotr Zieli´ nski
Optimistic Generic Broadcast
”
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2 ,m3
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2 ,m3
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2 ,m3
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2 ,m3
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2 ,m3 ,m1
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2 ,m3 ,m1 ,m4
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1 order 2:
order 1: m2 ,m3 ,m1 ,m4
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5
order 2:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2:
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2: m1
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2: m1
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2: m1
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2: m1 ,m2
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
order 2: m1 ,m2
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
m1
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
m1
order 2: m1 ,m2 ,m3
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
m1
order 2: m1 ,m2 ,m3
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
m1
order 2: m1 ,m2 ,m3 ,m4
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
m1
order 2: m1 ,m2 ,m3 ,m4
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
m1
order 2: m1 ,m2 ,m3 ,m4 ,m5
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
m1
order 2: m1 ,m2 ,m3 ,m4 ,m5 ,m6
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m4
m4
m5
m3
m6
m2
m3
m6
m2
m1
order 1: m2 ,m3 ,m1 ,m4 ,m5 ,m6
m1
order 2: m1 ,m2 ,m3 ,m4 ,m5 ,m6
Delivery rule Deliver a message when all undelivered conflicting messages are its successors Piotr Zieli´ nski
m5
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Problems m1
m2 1
m2 m3
use a separate Consensus instance for each pair of messages, all executed in parallel 2
Cycles if no failures, the leader dictates the order, no cycles if failures occur, a cycle-resolution algorithm used
m1 3
m1
Different processes perceive different orders
m2 m3 m4 m5 m6
The graph contains all possible messages infinitely many parallel instances of Consensus most of them identical, only finitely many different implementable with finite resources
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Agreement on message order: two messages Problem messages
and
m1
conflict
m2
different processes see different orders Solution Consensus to decide on the order each replica proposes the first message received
A
if decision
, then
→
B
if decision
, then
→
C
Piotr Zieli´ nski
Optimistic Generic Broadcast
?
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Agreement on message order: many messages
m1 m2
m3 m4
m1
m2
m3
m4
m1
no conflict
m1 ↔ m2
m1 ↔ m3
m1 ↔ m4
m2
m1 ↔ m2
no conflict
no conflict
m2 ↔ m4
m3
m1 ↔ m3
no conflict
no conflict
m3 ↔ m4
m4
m1 ↔ m4
m2 ↔ m4
m3 ↔ m4
no conflict
Comments many parallel Consensus instances mi ↔ mj one instance for each pair of conflicting messages mi and mj message pairs are unordered: mi ↔ mj ≡ mj ↔ mi Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Agreement on message order: many messages
m1 m2
m3 m4
m1
m2
m3
m4
m1
no conflict
m1
m1
m1
m2
m1
no conflict
no conflict
m2
m3
m1
no conflict
no conflict
?
m4
m1
m2
?
no conflict
Comments many parallel Consensus instances mi ↔ mj one instance for each pair of conflicting messages mi and mj message pairs are unordered: mi ↔ mj ≡ mj ↔ mi Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Infinitely many instances at the same time Example. All processes receive only m1 , m2 , m3 , in this order
m1
m2
m3
m4 m5 m6 m7 m8
m1 m2 m3 m4 m5 m6 .. .
m1
m2
m3
m4
m5
m6
···
— m1 m1 m1 m1 m1 .. .
m1 — m2 m2 m2 m2 .. .
m1 m2 — m3 m3 m3 .. .
m1 m2 m3 — ? ? .. .
m1 m2 m3 ? — ? .. .
m1 m2 m3 ? ? — .. .
··· ··· ··· ··· ··· ··· .. .
infinitely many instances mi ↔ mj
identical instances share state
finitely many different instances
finite resources sufficient
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Infinitely many instances at the same time Example. All processes receive only m1 , m2 , m3 , in this order
m1
m2
m3
m4 m5 m6 m7 m8
m1 m2 m3 m4 m5 m6 .. .
m1
m2
m3
m4
m5
m6
···
— m1 m1 m1 m1 m1 .. .
m1 — m2 m2 m2 m2 .. .
m1 m2 — m3 m3 m3 .. .
m1 m2 m3 — ? ? .. .
m1 m2 m3 ? — ? .. .
m1 m2 m3 ? ? — .. .
··· ··· ··· ··· ··· ··· .. .
infinitely many instances mi ↔ mj
identical instances share state
finitely many different instances
finite resources sufficient
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Latency 1
1
1
A B
2
A
?
B
C
?
C
Different orders
Same order conflicting messages received in the same order
conflicting messages received in different orders
same Consensus proposals
different Consensus proposals
delivery in 1 + 1 = 2 steps
delivery in 1 + 2 = 3 steps
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Related work
Generic Broadcast
same order
Chandra and Toueg [1996] Pedone and Schiper [1998] Pedone and Schiper [1999] Aguilera et al. [2000] This work
3 2 4 4 2
steps steps steps steps steps
no conflicts 3 4 2 2 2
steps steps steps steps steps
The latency of our algorithm is provably optimal.
Piotr Zieli´ nski
Optimistic Generic Broadcast
other 3 4 4 4 3
steps steps steps steps steps
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Cycles Observation If the leader does not change, cycles do not appear
m1
m3
m5
m2
m4
m6
Proof by contradiction 1
Consensus instances adopt leader’s proposal, so
2
the leader proposed m3 → m4 → m5 → m3 , so
3
the leader received m3 before m4 before m5 before m3 . Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Cycles Observation If cycles appear, they must be resolved
m1
m3
m5
m2
m4
m6
Cycle resolution messages in cycles and their successors are blocked (grey) break cycles by delivering the first message (m3 ) use Atomic Broadcast to agree on a total order on messages Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6 Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Example m1
m3
m5
m1
m3
m5
m2
m4
m6
m2
m4
m6
m1 , m2 , m3 , m4 , m5 , m6
m3 , m1 , m4 , m5 , m6 , m2
Delivery rule Deliver a message when all undelivered conflicting messages 1
succeed it in the partial order, or
2
succeed it in the total order and are blocked Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Problem 1: Agreement on message order Problem 2: Infinitely many instances of Consensus Problem 3: Cycle resolution
Problems m1
m2 1
m2 m3
use a separate Consensus instance for each pair of messages, all executed in parallel 2
Cycles if no failures, the leader dictates the order, no cycles if failures occur, a cycle-resolution algorithm used
m1 3
m1
Different processes perceive different orders
m2 m3 m4 m5 m6
The graph contains all possible messages infinitely many parallel instances of Consensus most of them identical, only finitely many different implementable with finite resources
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Frequently asked questions Summary of the talk
Frequently Asked Questions Is it possible to . . . 1
deliver some messages faster than in two steps? No, this means delivery before feedback from others.
2
drop the requirement n > 3f ? No, 2-step delivery requires n > 3f . [Pedone and Schiper, 2004]
3
not use the oracle for 2-step deliveries (thriftiness)? No, at least not with Consensus-based implementations. [Guerraoui and Raynal, 2003]
4
deliver all messages in two steps in all runs? Yes, but only in closed groups, with no failures, and perfectly synchronized clocks. Otherwise, no. [Zieli´ nski, 2005]
Piotr Zieli´ nski
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Frequently asked questions Summary of the talk
Summary of the talk m4
Optimistic Generic Broadcast
m5
m3
based on agreed partial order delivery in 2 or 3 steps
m6
latency provably optimal m2
m1
m2 m1
m3
m2
subsumes Generic Broadcast and Optimistic Atomic Broadcast
m1
m2 m3 m4 m5 m6
Contributions
m1
Piotr Zieli´ nski
1
1-2-step Consensus
2
cycle resolution
3
infinitely many instances
Optimistic Generic Broadcast
Introduction Optimistic Generic Broadcast Conclusion
Frequently asked questions Summary of the talk
References Marcos Kawazoe Aguilera, Carole Delporte-Gallet, Hugues Fauconnier, and Sam Toueg. Thrifty Generic Broadcast. Lecture Notes in Computer Science, 1914:268–282, 2000. Tushar Deepak Chandra and Sam Toueg. Unreliable failure detectors for reliable distributed systems. Journal of the ACM, 43(2):225–267, 1996. Rachid Guerraoui and Michel Raynal. The information structure of indulgent Consensus. Technical Report PI-1531, IRISA, April, 2003. Fernando Pedone and Andr´e Schiper. On the inherent cost of Generic Broadcast. Technical Report IC/2004/46, Swiss Federal Institute of Technology (EPFL), May 2004. Fernando Pedone and Andr´e Schiper. Optimistic Atomic Broadcast. In Proceedings of the 12th International Symposium on Distributed Computing, September 1998. Fernando Pedone and Andr´e Schiper. Generic Broadcast. In Proceedings of the 13th International Symposium on Distributed Computing, 1999. Piotr Zieli´ nski
Optimistic Generic Broadcast
