\documentclass[11pt,twoside]{article}
\usepackage{macros-course-lectures}
%\renewcommand{\bar}[1]{\overline{#1}}

\begin{document}

\lecture{notes12}{More Consensus}{October 17, 2002}

\begin{tabbing}
XXX\=XXX\=XXX\=XXX\= \kill
Bring:  Notes for scribe.  Chalk
\end{tabbing}

\noindent
Today's reading: \\
Aguilera, Toueg.  A simple bivalency proof that t-resilient consensus
requires t+1 rounds. \\
*** Keidar, Rajsbaum.  On the cost of fault-tolerant consensus when
there are no faults---a tutorial. \\
*** Gafni, Round-by-round fault detectors:  Unifying synchrony and
asynchrony \\
Datta, Guerraoui.  The inherent price of indulgence (skim) \\
\\
Next:  Data replication and data consistency: \\
Schneider \\
*** Attiya, Welch \\
*** Ladin, Liskov, Shrira, Ghemawat \\
Fekete, Gupta, Luchangco, Lynch, Shvartsman \\
Frolund, Guerraoui 

\section{Introduction}

\noindent
Today we have several results dealing with optimal consensus
algorithms, with corresponding lower bounds and impossibility
results. \\
Some new results, and some new ways of obtaining old results. \\
Especially interesting are some newly-observed connections between
synchronous and asynchronous models.

\section{Aguilera, Toueg}

\subsection{Overview}

\noindent
A really neat 4-page paper! \\
It gives a very simple proof for the known lower bound of $f+1$ rounds
for $f$ stopping failures in the (round-based) synchronous model, by
using the technique of FLP for impossibility in the asynchronous case. \\
\\
Historically, the lower bound on rounds was proved first.  \\
It was proved by constructing a ``chain'' of executions, where 
consecutive exections in the chain look the same to some nonfaulty
process. \\
The chain can be constructed to span from a failure-free execution
where everyone starts with $0$ to a failure-free execution where
everyone starts with $1$.  \\
Agreement and validity lead to a contradiction. \\
\\
The first such proof was by Lynch, Fischer, for the Byzantine case. \\
Around 1979 or 1980. \\
A more complicated chain construction evolved, not attributed to anyone
in particular.  Dolev, Strong?  Merritt?  \\
A version in my book.  \\
The construction is a little complicated. \\
\\
The impossibility for consensus in the asynchronous model was proved
a little later---the result was actually obtained in 1982. \\
It introduced a new method based on ``bivalence'' and ``univalence''. \\
Until recently, no one seems to have tried out the new method on the
former problem.  \\
It works much better than the old method. \\
\\
The new proof seems to have been found not just by AT, but also by
Moses and Rajsbaum.  A similar argument was also used by Bar-Joseph (a
student here, before he was here) and Ben-Or, but they applied it to
the setting of randomized algorithms.

\subsection{Model and problem}

\noindent
Synchronous round model.  Known set of processes.  In each round, each
process sends messages to all other processes, receives messages,
changes state.  \\
\\
Stopping failures.  When a process stops, it may send
any subset of its messages for the round in which it fails. \\
Furthermore, at most one process fails per round. \\
(Makes the impossibility result stronger.) \\
\\
Consensus:  Agreement, validity (decision value must be the initial
value of some process), termination.  \\
The uniformity of agreement isn't needed. 

\subsection{The proof}

\noindent
Assumes that $n > f+1$. \\
\\
By contradiction.  Assume there is such an algorithm $A$, tolerating
$f$ failures and always terminating in exactly $f$ rounds. \\
\\
Restrict attention to executions with at most $f$ failures. \\
Define a execution that completes some finite number of rounds to be
$0$-valent ($1$-valent) if the only decision that can follow from
extensions of the execution is $0$ ($1$). \\
\\
Univalent:  $0$-valent or $1$-valent. \\
Bivalent:  Both decisions are possible. \\
\\
Lemma 1:  There is some initial state ($0$-round execution) that is
bivalent. \\
\\
Proof:  As in FLP.  Assume all are univalent, note that $C^0$, where
everyone starts with $0$, is $0$-valent and $C^1$ is $1$-valent. \\
Then there must be two initial configurations with Hamming distance
$1$ (differ in state of one process) such that one is $0$-valent and
one is $1$-valent.  \\
But this can't be, because if that process fails, no one else can
distinguish the two configs. \\
\\
Lemma 2:  There is a bivalent $k$-round execution for every $k$, 
$0 \leq k \leq f-1$. \\
\\
Proof:  By induction.  The base, $k = 0$, follows from lemma 1. \\
Inductive step:  Assume true for $k$:  we have a bivalent $k$-round
bivalent execution $r$.  \\
Show for $k+1$, where $0 \leq k \leq f-2$. \\
Assume not---then every one-round extension of $r$ is univalent. \\
\\
Let $r^*$ be the one-round extension of $r$ in which no new failures
occur in round $k+1$. \\
This is univalent by assumption, wlog $1$-valent. \\
Since $r$ is bivalent, there must be another 1-round extension of $r$,
call it $r^0$, that is $0$-valent. \\
In $r^0$, some single process $p$ fails in round $k+1$.  \\
Let $q_1,\ldots,q_m$ be the processes to which $p$ fails to send a
message in round $k+1$. \\
\\
Now define a little chain of runs $r^1, r^2,\ldots,r^m$.  \\
Each $r^i$ is the same as $r^0$ except that $p$ sends a message to
$q_1,\ldots,q_i$ in addition to the others that it sends to in $r^0$. \\
That is, this chain represents adding in one message from $p$ at a
time.  \\
In all these executions, $p$ still fails at round $k+1$; however, in
the case of $r^m$, it fails (think of a special ``fail'' event
occurring) after sending all its messages. \\
\\
Each $r^i$, $0 \leq i \leq m$, is univalent, by assumption. \\
Since $r^0$ is assumed to be $0$-valent, there are two possibilities:
\begin{enumerate}
\item
At least one is $1$-valent. \\
Then there must some $j$ such that $r^{j-1}$ is $0$-valent and $r^j$
is $1$-valent.  \\
But these two differ after round $k+1$ only in the state of process
$q_j$.  \\
This can't be possible, because we can extend both executions by
simply failing $q_j$ at the beginning of round $k+2$ (recall we have
$k+2 \leq f$), and no one who is left could tell the difference. 
\item
All of them are $0$-valent. \\
Then compare $r^m$, in which $p$ sends all its messages at round $k+1$
and then fails, with $r^*$, in which $p$ sends all its messages at
round $k+1$ and does not fail. \\
$r^m$ is $0$-valent and $r^*$ is $1$-valent. \\
But we can fail $p$ at the beginning of round $k+1$ in $r^*$, and then
no one else can tell the difference, again a contradiction.
\end{enumerate}
So, we can keep the execution bivalent through $f-1$ rounds, while
expending at most $f-1$ failures. \\
\\
Now, what about the final round?  The argument is essentially the same
as the inductive argument.  But instead of arguing that there is a bivalent
$f$-round execution (which doesn't exactly make sense, since we've
required that everyone decide by the end of $f$ rounds), we argue that
there is an $f$-round execution in which two nonfaulty processes
decide differently: \\
\\
Lemma 3:  There is an $f$-round execution in which two nonfaulty
processes decide differently (which is a contradiction to the problem
definition). \\
\\
Proof:  Use Lemma $2$ to get a bivalent $f-1$-round execution $r$ (with at
most $f-1$ failures). \\
We know that in every one-round extension of $r$, everyone who hasn't
failed decides (and agrees). \\
\\
Let $r^*$ be the one-round extension of $r$ in which no new failures
occur in round $f$. \\
Everyone decides in this extension. \\
If two decide differently we are done, so assume they all decide the
same.  Wlog they decide $1$. \\
Since $r$ is bivalent, there must be another 1-round extension of $r$,
call it $r^0$, in which some (nonfaulty) process $q$ decides $0$. \\
\\
In $r^0$, some single process $p$ fails in round $f$.  \\
And $p$ must fail to sends a message to $q$ in round $f$ (otherwise
the execution would look to $q$ just like $r^*$ and it would decide
$1$). \\
\\
Now define a very small chain:  an ``intermediate'' execution
$r^{0,1}$ that is identical to $r^0$ but in which $p$ sends its
message to $q$. \\
Then $r^{0,1}$ looks to $q$ like $r^*$, so it decides $1$. \\
But  $r^{0,1}$ looks to every other nonfaulty process like $r^0$, so
they decide $1$. \\
Thus, we have disagreement, as needed.

\subsection{Discussion}

\noindent
The final round argument is really like the earlier round arguments.
There should be an even shorter proof that treats this as a special
case. \\
\\
Note the slight trickiness that arises in these proofs:  Every failure
is apportioned to some round.  There is an ambiguity when someone
fails ``between'' rounds---can be assigned to either round.  Has to be
handled carefully in the proofs. \\
\\
The key seems to be what happens in a round that happens right after a
bivalent execution. \\
If a single failure can occur at the next round, then the execution
can be forced to remain bivalent. \\
And if the decision must be made after that additional round, then
disagreement is possible. \\
\\
What can we say about the situation where the maximum number of
failures is some value $f$, but the actual number that occur in some
particular execution may be smaller? \\
\\
Note that this proof assumed right at the beginning that everyone
decides right at the end. \\
We could generalize this to allow processes to decide earlier if
possible. \\
There are ``early stopping'' algorithms in the literature that
terminate early in better cases, where fewer failures occur. \\
\\
This lower bound doesn't show that more than $f$ rounds are needed
in every execution---just that this is necessary in the worst case.
The proof assumes we are free to expend the maximum number of
failures $f$.  \\
The next paper considers what is possible for early stopping
algorithms.

\section{Keidar, Rajsbaum}

\subsection{Overview}

\noindent
On the cost of fault-tolerant consensus when there are no faults---a
tutorial. \\
This 20-pager overviews many results about time bounds on consensus,
but it also contains some new results, and new proofs of old results. \\
I'll just present the new results/proofs. \\
They appear in Section 4. \\
\\
Their motivation starts with partially synchronous models (models with
timing assumptions, which may hold from some point on), and
asynchronous models with unreliable failure detectors (diamond-S). \\
But the actual technical results they get are for the round-based
synchronous model.  They then use those to get results about the other
models as simple corollaries. \\
\\
So, I'll focus on the results about the synchronous model. \\
Process stopping failures. \\
Assume a worst-case bound of $t$ on the number of failures, but they
also consider the number $f$ that occur in particular executions. \\
\\
The results I will describe deal with uniform consensus:
\begin{enumerate}
\item
A 2-round lower bound for the time for all nonfaulty processes to
decide, in executions with no failures. \\
\\
This matches what can be obtained from Mostefaoui, Raynal's algorithm,
when executed in the synchronous model (implement your own reliable
failure detector by observing when messages don't arrive).
\item
%2.  Ordinary consensus:  A $f+1$-round lower bound for the time for
%every nonfaulty process to decide, in executions with at most $f$
%failures. \\
%\\
%This follows from the standard bound:  For any fixed $f$, we can
%restrict attention to the set of runs with at most $f$ failures.
%These all occur in a similar system in which $f$ is an actual upper
%bound on failures.  The standard lower bound yields $f+1$ for this
%case, so that carries over to the current, more general setting. \\
%\\
%However, they give a new proof. \\
%
\item
An $f+2$-round lower bound for the time for every nonfaulty process to
decide, in executions with at most $f$ failures. \\
\\
Charron-Bost and Schiper have an earlier version of this, slightly less
general (assumes $f \geq 1$). \\
\item
Unsolvable in the ``mobile failures'' model, which simply allows
messages from at most one process to be lost at
each round.  (Can change from round to round.). \\
This is a re-proving of a result previously proved by Santoro,
Widmayer and re-proved earlier by Moses, Rajsbaum. 
\end{enumerate}
Compare:  Dolev, Reischuk, Strong 90 algorithm:  min(f+2,t+1) rounds
(?). \\
That is, if $f \leq t-2$ then it decides within $f+2$ rounds (else
within the max, $t+1$). \\
\\
KR emphasize their technique.  It's pretty simple, like that of
Aguilera, Toueg's.  But it doesn't use bivalence.  In fact, they claim
that a bivalency argument can't be used to get the $f+2$-round lower
bound.  Not exactly clear how one would state/prove that precisely. 

\subsection{Layering, similarity, and connectivity}

\noindent
Everything in this section applies to general algorithms in the
round-based synchronous model---not just consensus. \\
\\
I don't see a careful definition of ``layering'' here.  I'll try to
fill in as needed. \\
Restrict attention to executions in which at most one process fails
for the first time at each round. \\
Moreover, when it fails, it fails to send to some ``prefix'' of the
set of processes, $[k] = p_1,p_2,\ldots,p_k$ for some $k$. \\
\\
An execution is determined by an assignment $I$ of initial states to 
all the processes plus a sequence $E$ of ``environment actions'' of
the form $(i,k)$, where $p_i$ is a process that fails to send to all
of the processes in $[k]$.  \\
If $k = 0$, then $p_i$ sends to everyone.  In this case, no failure
occurs at this step. \\
\\
It's clear how such an initialization and sequence of actions entirely
determines an execution of the standard process-stopping-failure model.  \\
The process designated for each round indicates a new failure, except
in the special case where $k=0$. \\
The convention is that a failed process doesn't send any messages at later
rounds either. \\
\\
However, they ``overload'' their notation a bit by using it also for
the ``mobile failure'' model.  \\
Here, the convention is different:  process failures are for only one
round. \\
By this convention, their notation determines an execution for the
mobile failures model.  \\
\\
Let's restrict attention to the stopping failure model, to avoid
having to go back and forth thinking about the two different cases. \\
Then we can check that the results apply in the mobile model. \\
\\
Define an environment sequence $E$ to be {\em well-formed\/} if it
includes at most $t$ process failures and the same process fails at
most once in the sequence. \\
\\
I don't understand exactly what a ``layer'' is.  I think their ideas
are better understood in terms of executions and extensions of those
executions, as usual. \\
\\
Recast the $L^k$ definitions slightly: \\
Let $\cal E$ be a set of well-formed finite environment sequences.  
Then $L^k(\cal E)$ is the set of well-formed environment sequences that are
$k$-step extensions of sequences in $\cal E$. \\
\\
Define a {\em run\/} to be a pair $(I,E)$:  initialization and
well-formed environment sequence. \\
\\
Similarity:  Two runs $(I,E)$ and $(I',E')$ of the same length are
{\em similar\/} provided that there are two processes, $p_i$ and
$p_j$, such that: \\
(a) $p_i$ and $p_j$ are both non-failed in both of the executions
generated by these runs, and \\
(b) These two executions look identical to all processes except
$p_j$. \\
\\
Q:  Why would we care if $p_j$ is faulty or not? \\
Could we just say:  \\
These two executions look identical to all but one process, and \\
Some process is non-failed in both executions. \\
??? \\
\\
Thus, if two runs are similar, at least one nonfaulty process
$p_i$ can't distinguish them. \\
\\
A set $\rho$ of runs is said to be {\em similarity-connected\/} if every
pair is in the reflexive transitive closure of the similarity
relation. \\
\\
Lemma 4.1:  The set of ``initial'' runs, in which the envt-seqs have
length $0$ is similarity-connected.  \\
\\
Proof:  Change the initial state of one process at a time. 

\subsection{The 2-round lower bound for failure-free executions}

\noindent
Now assume that the algorithm is solving uniform consensus. \\
\\
Coloring:  For any run $\rho = (I,E)$, we define $val(\rho)$ to be the
unique decision value that results in the extension of $\rho$ in which
no failures occur. \\
\\
Lemma 4.2:  Let $\Rho$ be a similarity-connected set of runs.  Suppose
$\rho$ and $\rho'$ are two runs in the set such that $val(\rho) \neq
val(\rho')$. \\
(Also assume there are ``enough'' failures left to use.) \\
Then there is a run $\sigma \in \Rho$ such that, in the (a?) failure-free
extension of $\sigma$, some non-failed process has not decided within
one additional round. \\
\\
This implies the first main theorem:  the $2$-round lower bound for
uniform consensus when there are no failures.  \\
Uses the fact, in Lemma 4.1, that the set of initial states is
similarity-connected. \\
\\
Proof of Lemma 4.2:  WLOG, we can assume that $\rho$ and $\rho'$ are
actually similar, not just similarity connected (by the usual
argument---the two assumed runs are connected by some chain, so we
have a difference at some point in the chain). \\
Then we know that $\rho$ and $\rho'$ look identical to everyone except
for a particular $p_j$.  And there are enough failures left over for
us to play around with. \\
Choose $p_m$ to be some other process that is nonfaulty in both
runs---in fact, the one with the largest index that isn't faulty in
both runs. \\
(We seem to be using the fact here that the same processes are faulty
in both.  That seems to follow...) \\
\\
Assume for the sake of contradiction that everyone decides within one
round in all failure-free extensions of $\rho$ and $\rho'$. \\
Assume wlog that $val(\rho) = 1$; thus, all processes in the
failure-free extension of $\rho$ decide $1$, within one more step. \\
And $val(\rho') = 0$. \\
\\
Consider the execution constructed by extending $\rho$ with $(j,m-1)$,
that is, $j$ (who hasn't failed yet) now fails and misses sending to
everyone up to and not including $m$. \\
To process $m$, this looks the same as the failure-free extension of
$\rho$, so $m$ must decide $1$ after one round. \\
Then everyone else must also decide $1$ in {\em any extension\/} of
this extended run, though because they know this is not a failure-free
extension (they can see missing messages), they are allowed to take
more rounds to do it.  \\
\\
Likewise, extend $\rho'$ with $(j,m-1)$, conclude that $m$ must decide
$0$ after one round, so everyone else decides $0$ in any extension of
that extended run. \\
\\
But, among the extensions, there are two---one extending 
$\rho \cdot (j,m-1)$ and one extending $\rho' \cdot (j,m-1)$---
in which $m$ fails immediately after the decision round and no one
else ever fails.  \\
No remaning nonfaulty process can tell the difference between these
two runs.  \\
But they are forced to decide differently in these runs. \\
Contradiction.

\subsection{The $f+2$-round lower bound for $f$-failure executions}

\noindent
The previous result is like the special-case result used in the
Aguilera-Toueg proof.  This paper also has an analog of the
earlier-rounds argument (bivalence preserved for one round when a
failure occurs) of AT. \\
Stated in terms of preserving similarity. \\
\\
Lemma 4.4:  Let $\Rho$ be a similarity-connected set of states.  \
(Assume there are enough failure left to use.) \\
Then $L(\Rho)$ (the set of one-round extensions of $\Rho$) is also
similarity-connected. \\
\\
Proof: This isn't hard---add lots of extra intermediate ``links'' in
the chain, removing one message at a time. \\
LTTR. \\
\\
Now consider uniform consensus again.  \\
Let $\Rho_0$ be the set of initializations for the processes. \\
\\
Lemma 4.5, part 1: \\
$L^k(\Rho)$ (the set of $k$-round runs) is similarity connected
for all $k$, $0 \leq q \leq t$. \\
\\
This follows because $\Rho$ is similarity-connected and the previous
lemma says this is preserved (since we have ``enough'' failures to
play with). \\
\\
Lemma 4.5, part 2: \\
For every $k \geq 0$, there exist $\rho$, $\rho'$ in $L^k(\Phi_0)$
such that $val(\rho) \neq val(\rho')$. \\
\\
Just take non-failing runs that start with all $0$ or all $1$. \\
\\
Finally, we can prove the $f+2$-round lower bound on uniform consensus
with $f$ actual failures (out of a possible $t$): \\
\\
Theorem 4.9: ...\\
Proof:  Fix $f$, $0 \leq f \leq t-2$. \\
Use Lemma 4.5 to obtain runs $\rho, \rho'$ in $L^f(\Rho_0)$ such that
$val(\rho) \neq val(\rho')$. \\
Also by Lemma 4.5, $L^f(\Rho_0)$ is similarity-connected. \\
Then we use Lemma 4.2 (noting that there are ``enough'' failures
left), and conclude that there is a run $\sigma \in L^f(\Rho_0)$ such
that in the failure-free extension of $\sigma$, some non-failed
process hasn't decided within one additional round.  \\
That is, it takes at least $f+2$ rounds to decide. 

\subsection{Mobile failures}

\noindent
They use the same results to show impossibility of ever deciding in
the (uniform or nonuniform---the difference isn't meaningful here)
consensus problem, in the mobile failures model. \\
\\
This uses Lemma 4.5 again, but with no upper bound $t$---so after
any finite number of rounds, the set of runs is
similarity-connected. \\
Since each such set has runs with distinct vals, a simple argument
shows there's another run with no decision. \\
Use Konig's lemma to show that there's an infinite branch in which
some nonfaulty process doesn't decide.

\subsection{Discussion}

\noindent
Results have nice topological flavor. \\
\\
The definitions of layers, etc. are probably not the cleanest---could
use some technical polishing.  \\
\\
Open questions?  \\
What's the minimum time to terminate after stabilization time, in
synchronous models where: \\
(a)  For any amount of time processes may exhibit omission failures,
any number, and may stop and restart. 
(b)  After stabilization time, no more processes stop or restart, and
no messages are lost. \\
\\
What's the exact (optimal) bound?



\section{Gafni}


\section{Guerraoui}

\end{document}
