\documentclass[11pt,twoside]{article}
\usepackage{macros-course-lectures}
%\renewcommand{\bar}[1]{\overline{#1}}

\begin{document}

\lecture{notes5}{Fault-Tolerant Objects}{September 19, 2002}

\begin{tabbing}
XXX\=XXX\=XXX\=XXX\= \kill
Bring:  \\
Extra notes for scribe. 
\end{tabbing}

\noindent
Today's papers: \\
Carrying over from last time:  \\
Merritt, Taubenfeld \\
Gafni, Merritt, Taubenfeld \\
New: \\
*** Afek, Greenberg, Merritt, Taubenfeld, Computing with Faulty Shared
Objects \\
*** Jayanti, Chandra, Toueg, Fault-Tolerant Wait-Free Shared Objects \\
\\
Next time:  Disk Paxos papers

\section{Introduction}

\noindent
Last time, we were working on the Merritt, Taubenfeld paper on
computing with infinitely many processes. \\
Recall this was in the model with reliable shared variables. \\
We covered an impossibility result saying that, even if we know how
many processes will participate (out of a bigger set of processes), we
can't solve a simple leader election problem with bounded memory
(read/write shared variables).  \\
Then for contrast, we did an algorithm that solved leader election
with only two read/write shared variables, but they were allowed to
take on unboundedly many values.  \\
In particular, they could record process ids, in fact, sets of at most
$c$ process identifiers, where $c$ is a lower bound on participation
and upper bound on concurrency. \\
\\
Today, I want to describe one more result from this paper, because it
connects with the mutual exclusion results we talked about on the
first day of class. \\
\\
This is a very powerful result for mutual exclusion using read/write
memory. \\
It guarantees lockout-freedom. \\
It tolerates unbounded concurrency. \\
It's adaptive (in the point contention sense), and symmetric
(processes are identical except for ids, and use their ids only in
comparisons for equality). \\
No participation is required (as usual for mutual exclusion). \\
\\
After than, I'd like to present one result from the Gafni, Merritt,
Taubenfeld paper---a snapshot algorithm that works for unbounded
concurrency. \\
\\
Then go on to the two papers on computing with faulty objects.

\section{Merritt, Taubenfeld}

\subsection{Adaptive, symmetric mutual exclusion algorithm with
read/write registers}

\noindent
Theorem 3.2, p. 12. \\
\\
They first present a deadlock-free m.e. algorithm and then sketch how
to modify it to make it lockout-free.  I'm not completely sure of the
adaptivity claim here. \\
\\
\subsubsection{A new kind of splitter}

\noindent
They use a slightly modified splitter, p. 13. \\
Prop. 3.1 lists the properties they now want;  they are slightly
stronger than what we have seen so far. \\
Namely, they divide the processes into ``early'' and ``late'', where
``late'' means that they invoke the splitter after the first time when
some process exits it. \\
``Early'' are the others---those who enter it before the first exit. \\
\\
Now instead of just saying ``not all processes go right'', they
make the stronger statement: ``not all the early processes go right''. \\
\\
And instead of just saying ``not all processes go down'', they
strengthen this to say (1) the latecomers all move right, and (2)
either at least one latecomer moves right, or else at most n-1 (early)
processes move down. \\
\\
(Actually, I think this should say:  (2) either at least one latecomer
does its first assignment, or else at most n-1 (early) processes move down.
Since the splitter is wait-free, processes are supposed to be allowed
to stop anytime.  Therefore, someone could do the assignment to $y$
but never actually move right.) \\
\\
Lamport's algorithm still satisfies this slightly stronger
statements. \\
\\
Now they modify the splitter, weakening the liveness guarantee to
lockout-freedom, rather than wait-freedom. \\
This is weaker because it counts on everyone continuing to take steps,
that is, it assumes no failures. \\
\\
What they gain from this is an additional property:  If any process
wins then no one moves down. \\
\\
See code, p. 15. \\
\\
The detailed algorithms isn't exciting. \\
They add a couple of new flags, $b$, and $z$. \\
The await statement adds some dependencies, so termination is only
ensured if everyone keeps taking steps.  LTTR. \\
\\
The new flags ensure the new safety property:  If anyone wins then no one
moves down. \\
(1)  We can't have someone winning while someone goes down as a
result of the final ``else move down'', on line 9. \\
That's because this works like Lamport's splitter,
and no 2 processes get to the corresponding point in Lamport's
splitter. \\
\\
(2)  We can't have someone winning while another goes down as a result
of the first ``move down'' statement, on line 6. \\
That's because a process can only win if it sees that 
$z$ is set to $1$ before $b$ is first set to $1$. \\
But a process can only move down on line 6 if $b$ is set to $1$ before
$z$ is first set to $1$. \\
Incompatible. \\
\\
The other properties needed for the Lamport splitter carry over. \\
LTTR. 

\subsubsection{The mutual exclusion algorithm, using the splitters}

\noindent
A linear array of new-splitters (going downward). \\
\\
See code, p. 16, figure 6. \\
\\
It's just what you'd expect, except that moving right is transformed
into another await statement, which waits for a shared variable
``next'' to get set to a new splitter, larger than the one where
you're currently contending. \\
At that point, the process goes back to the beginning of its code,
but starting with the new splitter indicated in ``next''. \\
Thus, as time goes on, the splitters that are used for competition
progress down the array. \\
\\
At the first splitter of a competition, all the latecomers move right,
so only finitely many processes move down. \\
Each splitter either has a winner (who is the process who gets the
critical section), or else eliminates at least one process who moves
right. \\
So, within finitely many levels, a process wins. \\
\\
After the winner of a competition leaves the CS, it sets the shared
var next to the next unused level, to indicate where the next
competition should start. \\
\\
Thus, successive competitions begin at later and later splitters. \\
The properties of the new-splitter ensure that when a process wins at
a level, no process is already active in any splitter at a greater
level. \\
\\
Satisfies mutual exclusion, no deadlock. \\
\\
Time complexity:  Measure from when no one is in CS and someone is
waiting until someone enters CS. \\
At worst, someone has to progress through $O(n)$ splitters before
entering CS, where $n$ is the number of early processes at the
``starting splitter'' for the new competition. \\
Time $O(n)$. \\
Since $n \leq$ point contention, that gives an adaptive time bound. \\
\\
Q:  Does this work with ordinary Lamport splitters, if we use a
2-dimensional array and start each competition at the top of a new
column? \\
\\
A:  Not directly.  Nothing would stop a second process from winning in
a column, below the first splitter in that column where someone wins. \\
\\
This algorithm doesn't guarantee lockout-freedom as is. \\
To add lockout-freedom, they hand-wave about adding some flags
and having a process that leaves the CS examine the flags to help
select the next process. \\
Are these per-process flags?  \\
If so, then checking all of these sems to require some kind of global
diagonalization over these flags, which seems to incur time delays
that depend on the process ids.  
That wouldn't be adaptive. 
??? \\
Perhaps they are assuming use of renaming first...\\
Q: What about storing names in splitters?
\\
A: Maybe could work, but the solution would still need some work.
\\
Q: What about unbounded registers?
\\
A: Remember that the algorithm needs to not depend on total number competing, so this might not work.
\\
Anyway...Even deadlock-free m.e. is enough to solve leader election or
consensus. \\
Using m.e., the first process selected can be the leader, can use its
value for consensus.  \\
This does seem to preserve the adaptivity and symmetry properties. \\
\\
This result doesn't quite subsume Theorem 2.4, because that used only
two registers, whereas this uses infinitely many.  \\
(M.e. is a harder problem than leader election.)

\subsection{Significance}

\noindent
They also revisit the Friedberg-Peterson result, which (recall) gives
a lockout-free mutex algorithm using 2 weak semaphores and 2
r/w/shared vars. \\
MT just observe that, according to their new defs, FP allows unbounded
concurrency, and has no required participation. \\
It is adaptive and symmetric.
\\
The adaptive, symmetric read/write register results for leader
election (Theorem 2.4) and m.e. (Theorem 3.2) seem quite powerful. \\
\\
MT are trying not only to get individual algorithms but also general
classification results. \\
They have set up a nice set of definitions to use for classification:
concurrency and participation bounds, adaptivity, symmetry. \\
They would like results that say exactly which problems can be solved
under which of these assumptions, with which properties. 

\subsection{Further work}

\noindent
Needs polishing.  Lots of things don't quite work or aren't well
explained. \\
\\
Continue working on the classification theory:
\begin{itemize}
\item
Exactly which problems can be solved under which sets of
concurrency/participation assumptions, with which properties?
Which results imply which other results? 
\item
Extend this to fault-tolerant algorithms, for problems for which we
normally consider this, such as renaming, lattice agreement, snapshot,
collect, consensus. 
\item
Get separation theorems for different sets of requirements.
E.g., are there interesting problems that can be solved for any finite,
known number of processes but can't be solved for a finite, unknown
number of processes?
\item
General transformations to convert algorithms for finite numbers of
processes to algorithms for infinitely many processes (perhaps under
certain assumptions, like symmetry)?
\end{itemize}

\section{Gafni, Merritt, Taubenfeld}

\subsection{Overview}

\noindent
Continuing on the same theme.  \\
Unlike the other paper, they are now allowing process stopping faults,
in fact, they are looking for wait-free algorithms. \\
Now the problems are things like snapshot, renaming, etc., for which
wait-free algorithms are usually considered.  \\
They consider read/write registers only. \\
\\
The paper has two different foci:
\begin{enumerate}
\item
They are trying to get a general classification theory for problems,
based on the kind of concurrency assumptions that are required for
solving the problems. 
The most interesting case is ``unbounded concurrency'', as defined in
the MT paper.  Any number of processes might participate at once. 
\item
They try to obtain algorithms that work with unbounded concurrency.
\end{enumerate}
This paper does not deal with participation lower bounds. \\
\\
Section 2 contains hierarchy results. \\
They show that ``there exist problems'' that can be solved with
bounded concurrency but not unbounded concurrency, and also problems
that can be solved with $k$-bounded concurrency but not $k+1$-bounded. \\
These are bounded versions of snapshot problems.  \\
Somewhat contrived. \\
\\
Section 3 gives algorithms that work with unbounded concurrency.  \\
In particular, Section 3.1 focuses on a cute snapshot algorithm that
handles unbounded concurrency.  \\
Its time complexity, in the initial presentation, depends not 
only on the actual contention but on the process indices, so it's not
``adaptive''.  \\
Then they describe how to make it adaptive by doing renaming. \\
\\
The kinds of restrictions on concurrency they consider are: 
\begin{itemize}
\item
$k$-bounded:  A finite bound $k$ on the number of processes that are
active at any point in any execution (point contention).
\item
bounded:  In each execution, the concurrency is bounded by some $k$.
\item
unbounded:  No bound on the concurrency.
\end{itemize}

\subsection{The hierarchy results}

\noindent
They show the existence of a problem that can be solved with bounded
concurrency but not unbounded. \\
And for each $k$, a problem that can be solved with $k$ concurrency
but not $k+1$. \\
\\
Bounded snapshot problem:  They fomulate the atomic snapshot problem
in a somewhat simplified, unusual way.  In fact, it looks to me just
like lattice agreement. \\
\\
Consider one-shot only---everyone invokes at most once. \\
\\
Only one kind of operation, rather than separate update and scan
operations as in the Collect service. \\
The operation is supposed to return a set of processes, all of which
have invoked. \\
Each process' set should include itself, and the sets returned should
be totally ordered by containment. \\
\\
New twist:  For each execution, there is some bound $r$ on the
difference between the sizes of two consecutive sets in the nesting
order. \\
\\
(I'm not sure exactly what this means.  The setting is wait-free, so
processes can fail.  Are we allowed to assign sets to processes that
don't finish their operations?  Probably so. ) \\
\\
Theorem 2.1:  The bounded snapshot problem isn't solvable is the
unbounded concurrency model. \\
\\
(It's trivially solvable with bounded concurrency---just use an
ordinary snapshot algorithm.  \\
If concurrency is bounded in an execution by, say, $c$, then in that
execution, the difference between the sizes of two consecutive
snapshots will be at most $c$. \\
\\
Proof:  Enumerate the sets returned in nondecreasing order, counting
repeats. \\
The smallest set in this sequence must have size between $1$ and $c$,
inclusive.  \\
That's because someone terminates before the $c+1$st process starts. \\
The second smallest must have size between 2 and $c+1$, because two
must terminate before the $c+2$nd process starts. \\
(And the second must contain the first.) \\
Etc. \\
The $k$th must have size between $k$ and $c+k-1$, inclusive. \\
\\
Then the difference between two consecutive entries in this list is at
most $(c+k-1) - (k-1) = c$.) \\
\\
To see that it's not solvable with unbounded concurrency: \\
Divide the processes into batches, $G_1, G_2, \ldots$, each with more
processes than the preceding one. \\
They claim that if a batch comes in all together, it must be possible
for them all to return the entire batch. \\
Since the batches get larger and larger, this isn't bounded. \\
\\
The claim about them all returning the entire batch is ``proved'' in a
separate lemma. \\
The proof seems rather odd to me, and I'm not convinced (though I'd
bet the result is correct). \\
The proof they give is based on the lower bound of $2k-1$ for
renaming, and uses a sort of reduction. \\
I think this needs some work.  LTTR. \\
QA: A clarification about the issue is that a reduction argument should produce an algorithm and the authors don't do that here. 
\\
A similar construction is used to ``separate'' $k$-concurrency from
$k+1$-concurrency, based on a $k$-bounded version of the snapshot
problem. \\
This is similar to ``bounded snapshot'', except the difference is
always at most $k$. 

\subsection{One-shot snapshot algorithm whose latency depends on
contention and process indices}

\noindent
Theorem 3.1 gives an (ordinary, not bounded) snapshot algorithm 
that handles unbounded concurrency.  Interesting. \\
\\
The time complexity of the basic algorithm depends on both the
concurrency that happens to occur, and the maximum index of a participant. \\
But then they show, using various tricks, how to remove the dependency
on the max index. \\
\\
One-shot version, but any number of processes.  \\
(Could allow repeated visits simply by letting each process use
multiple names.) \\
\\
Code, p. 5. \\
\\
The main data structures used by this code are an infinite array of
start bits and an infinite array of flag bits. \\
There's also an infinite array of ``snap'' sets, but let's ignore
these for the moment. \\
(Basically, they provide short-cuts---answers that someone has already
calculated, that another process can borrow rather than completing the
work of doing its own snapshot.) \\
\\
Trace through the code without looking at the snap values. \\
\\
Process $i$ arrives, sets its start[i] flag, then proceeds to
collect the names of other processes who have set their start
flags. \\
\\
How does it know where to look?  It can't just read all the
registers. \\
It traverses registers from the smallest as long as the flag bits are
set.  \\
It accumulates, in its set ``collect'', the indices of processes whose
start bits are set. \\
\\
It does this again. \\
The second time (and subsequent times) it saves the result of its
previous collect in its set ``doublecollect''. \\
Keeping two sets will allow it to compare the results it obtains on
two successive attempts. \\
\\
A stopping condition is that collect = doublecollect (so nothing new
has been found on the last pass), and also $i$ itself is in the
set. \\
Until this condition is reached, the process just keeps making
attempts. \\
\\
In one situation, it takes some positive action before beginning the
next attempt: \\
(line 10) If collect = doublecollect but $i$ itself isn't in the
set. \\
That means that the traversal didn't get far enough---didn't reach the
$i$ entry. \\
That must be because not enough flags were set. \\
So in this case, before beginning the next attempt, process $i$ sets
its own flag[i] and all smaller flag variables. \\
Then tries again. \\
\\
The next time it sees collect = doublecollect, it will also have $i$
itself in the set, because it is guaranteed to reach $i$ the next time
(enough flags are set). \\
\\
From what has been said so far, it is still possible that process $i$
might continue making attempts forever, and on each pass, learn about
more new entrants.  \\
That could cause nontermination. \\
To get around this, we use the same trick as in the earlier algorithm
of Afek, Attiya,..., and let a process that determines a good snapshot
record it someplace for other processes to use. \\
\\
In this case, a ``good'' snapshot, worth recording, is one that sees
collect = doublecollect, and where the process itself isn't in the set. \\
(Q:  Why make that last restriction?) \\
So in this case, just before process $i$ sets the flags, it also
records the ``good'' snapshot. \\
\\
Now look how the snapshots are used in the code:  in line 5, 9, 15, as
a stopping condition---when $i$ sees someone else's snap that already
contains $i$, $i$ can just take that snapshot and stop. \\
\\
The safety arguments aren't given in this paper but are apparently
(seems believable) analogous to those for standard snapshot. \\
\\
Q:  Work this out?  The argument from the standard case doesn't seem
to carry over immediately.  First, the snapshot that someone writes
isn't necessarily a ``true'' snapshot at any time in its interval.
And, it's not obvious that the interval of the operation that produces
a snap is included in the interval of an operation that uses it. ???\\
\\
Termination: \\
Process $i$ can't loop forever reading a fixed number of shared
variables. \\
(Or would eventually see collect = doublecollect, set flags, see
again, terminate.) \\
So the only problem is if a process, over time, reads all the flag
variables, 1,2,3,...\\
Suppose it does. \\
\\
After some point, we have $flag[k] = 1$ for all $k \leq i$. \\
At that point, let $max \geq i$ be the maximum id of any process that
has ever been active up to this point. \\
Now, eventually $flag[max + 1]$ has to get set to $1$, and this has to
be done by some process with index $p \geq max + 1$. \\
{\em Before\/} it does this, it also sets $snap[p]$ to a snapshot set. \\
That set must contain $i$, because the snapshot that produces this set
starts after the flags are already set through $i$. \\
So eventually $i$ sees this. \\
\\
Key idea:  The flag bits are set only after a new snap has been
written. \\
Thus, a process helps other processes out (by supplying a snapshot for
them) before extending the flag prefix (which could cause other
processes to go on forever). \\
\\
Can extend this easily to a long-lived algorithm, with multiple
snapshots per process, by allocating multiple indices per process. \\
\\
Not adaptive, latency depends on process ids.

\subsection{Making the snapshot adaptive}

\noindent
They claim that they can modify this to get the algorithm to be
adaptive---latency depending on actual total number of participants.  \\
Still handles unbounded concurrency, still one-shot. \\
\\
Do this by using a one-shot $f(k)$-renaming algorithm that is adaptive
(latency depends on $k$), and handles potentially unbounded number of
participants, arriving in any concurrency pattern. \\
\\
First apply the renaming algorithm to reduce the set of
names so that they depend on the actual number of arrivals.  \\
Then use the resulting names in the snapshot algorithm just
sketched. \\
\\
Getting such a renaming algorithm: \\
Take a standard one-shot adaptive renaming algorithm like
Attiya-Fouren's, which uses an upper bound $n$ on the size of the
namespace, and modify it so it doesn't assume such an upper bound. \\
\\
Algorithms like AF (they say) can be expressed in a form where they
compete for names in order, 1,2,3,...; when they fail to get one, they
try to get the next one.  (Check this---that isn't exactly how AF's
algorithm is described.) \\
\\
``Interleaving trick'':  Let the processes in AF compete for only the
odd names. \\
If process $i$ fails to acquire any odd number through $2i-1$, it just
chooses $2i$. \\
Thus, it defaults to twice its own index, a name that is reserved for
it.  \\
Although this depends on $i$, it's OK here because the fact that $i$
reaches this point means that at least (approx) $i$ processes are
participating.  \\
\\
Alternatively, they give a second trick, which works for all adaptive
one-shot renaming algorithms, but causes the namespace to be squared. \\
LTTR. \\
\\
Go over if time, from p. 6. \\
\\
Other results in the paper extend these to long-lived algorithms for
renaming, collect, and snapshot, all for unbounded concurrency.

\subsection{Significance}

\noindent
The algorithms seem much more interesting than the hierarchy results. \\
Unbounded processes, with unbounded concurrency, can solve all kinds
of problems in a wait-free manner. \\
Using just (infinitely many) read/write registers. \\
Costs can be made adaptive, based on number of actual participants. \\

\subsection{Future work}

\noindent
Lots of details and clean-up needed. \\
\\
Some more digesting, to turn this into a general theory with the best
results presented in the best way, the best modularity, etc. \\
\\
What does all this have to do with real distributed computing?

\section{Afek, Greenberg, Merritt, and Taubenfeld}

\noindent
Cover:  Sections 1-4, 5.1-2, 6.1, 6.4

\subsection{Overview}

\noindent
This and the Jayanti paper have a lot of overlap.  It seems to me that
the overlapping part is the most interesting.  I'll emphasize that. \\
\\
AGMT consider faulty objects instead of reliable shared variables. \\
Faults:  Objects can return completely wrong results.  However, they
aren't allowed to not respond.  So, this is a kind of ``synchronous
Byzantine'' model. \\
Processes are also allowed to fail by stopping---they consider
wait-free algorithms. \\
\\
Interesting results:
\begin{enumerate}
\item
Theorem 2.  A ``fault-tolerance-boosting'' result, showing how
implementations that tolerate small numbers of faulty objects can be
combined to tolerate any larger number of faulty objects (but more
total objects are needed).
\item
Corollary 8.  A reliable atomic read/write register from atomic
read/write registers, some of which may be faulty. 
\item
Theorem 10.  A reliable test-and-set register from test-and-set
registers, some of which may be faulty, plus reliable atomic
read/write registers.
\item
Theorem 12.  A reliable consensus object from RMW registers, some of
which may be faulty. 
\item
Theorem 18.  A ``universality'' result, which say that ``all
objects'' can be implemented reliably in terms of certain kinds of
basic objects (consensus objects or RMW objects), where some of the
basic object can be faulty.
\end{enumerate}

\noindent
They consider situations where the number of faults that can occur in
some objects is bounded (their $k$ parameter).  \\
But I don't see any interesting results about this. \\
So I'll just restrict attention to the bound $f$, on the number of
faulty objects. \\
\\
They use the notation $CONST(X,(f,\infty),Y)$ to denote the minimum
number of $n$-process objects of type $X$ that are needed to implement
a reliable $n$-process object of type $Y$, if at most $f$ of the
objects may be faulty. \\
QA: The $\infty$ refers to the number of arbitrary state changes a faulty object can make. \\
Most of their results are stated in terms of this CONST measure. \\
\\
The results in this paper seem basically easy.

\subsection{Composition results}

\noindent
p. 8, Theorem 2 describes a way of ``boosting'' fault-tolerance. \\
\\
It bounds the number of objects of type $X$ needed to implement a
reliable object of type $X$, assuming $f$ of the lower-level objects
may be faulty. \\
\\
This bound is expressed in terms of the number of objects that are 
needed to implement $X$ from $X$ tolerating $t$ faulty objects, where 
$t < f$.  \\
Namely, if $(t+1)^L$ objects are sufficient to implement $X$,
tolerating $t$ faults, then $((t+1)f)^L$ are enough to implement it,
tolerating $f$ faults. \\
\\
The construction is recursive. \\
\\
E.g., consider an arbitrary $f$ that we want to tolerate, and assume
for simplicity that $f$ is a multiple of $t+1$. \\
Use $C = (t+1)^L$ ``intermediate objects'' of type $X$, each of which
tolerates up to $f/(t+1)$ faulty objects in its own implementation. \\
(These will be constructed recursively.) \\
Use the assumed $t$-fault-tolerant construction at the top level. \\
This will work if at most $t$ of the $C$ intermediate objects are
faulty. \\
\\
Suppose that at least $t+1$ of the intermediate objects are faulty. \\
Since each of these tolerates $f/(t+1)$ faults among its implementing
objects, that means there must be at least $(t+1) (f/(t+1) + 1)$ total
faulty objects among all the low-level objects. \\
This is greater than the $f$ failures that the entire implementation
needs to be able to tolerate. \\
\\
This argument provides the inductive step for fault-tolerance of the
entire algorithm. \\
It all bottoms out when we reach $f = 1$. \\
\\
How many objects are needed?  \\
If $f = 2^k$, this expands exponentially to give $C^{k+1}$. 

\subsection{Atomic register constructions}

\noindent
The main result here is Corollary 8 (err, 4.4), p. 12.
Says that an atomic read/write register can be implemented from 
$20f + 8$ atomic read/write registers, tolerating $f$ faulty
registers. \\
\\
The direct construction for this isn't actually in this paper, but
they refer to the conference version. \\
\\
Here, they outline various approaches to implementing
atomic-from-faulty-atomic, though I don't think they get exactly the
same number in this way.  But here's the general idea: \\
\\
Rely on standard results that show how to implement reliable
atomic registers in terms of weaker forms of registers---not atomic,
but ``safe'' (defined by Lamport). \\
All the lower-level registers are assumed to be reliable in these
constructions. \\
These now-standard constructions are well known. \\
\\
In this paper, they ``bootstrap'' the process by showing explicitly
how to implement a safe register in terms of $2f+1$ safe registers
where $f$ may be faulty. \\
Basically, a writer writes all the registers, and a reader reads all
the registers.  \\
The reader sees if there's a majority for one value and if so returns
it.  If not, it returns an arbitrary value. \\
\\
Now, they use a standard construction implementing an atomic register
from reliable safe registers at the top level. \\
Now implement each reliable safe register as above---using unreliable
safe registers. \\
Each individual implementation of a reliable safe register has to be
able to tolerate $f$ failures. \\
\\
For example, they mention a construction by Tromp, which implements an
atomic $V$-register using $4$ safe $V$-registers and $8$ safe binary
registers. \\
As above, each of the safe registers can be implemented using $2f+1$
unreliable safe registers. \\
The total number of registers is then $12 (2f + 1) = 24f + 12$.  

\subsection{Test-and-set constructions}

\noindent
Now, to see an actual example or two of their algorithms. \\
Section 5.1 and 5.2 construct a reliable ``single-use test-and-set''
object from unreliable SUT\&S objects, where at most $1$ may be
unreliable. \\
They can use the boosting idea to boost the fault-tolerance to any
amount. \\
Just consider $1$ fault. \\
\\
SUT\&S:  Very simple primitive, of the sort available in hardware. \\
Binary values, 0 and 1, initially 0. \\
One operation, T\&S. \\
Sequential specification says that any T\&S operation writes 1 into the
register but returns the value the register previously had. \\
Thus, the first process should get $0$ and everyone else should get
$1$. \\
Of course, this is an atomic version, so several processes can invoke
it concurrently. \\
The results are supposed to be serialized at some points in their
intervals. \\
\\
2-process algorithm:  \\
Uses $7$ low-level registers, tolerates $1$ of them being faulty. \\
Cute, see Figure 3, inner box. \\
\\
See code top of figure 2: \\
Process does T\&S on the three A registers, and counts how many he
``wins'' ($0 =$ win here). \\
If he doesn't win at least $2$, he tests the single register $B$; if
he wins that, he returns 1 (loses). \\
\\
Otherwise, that is, if he either wins at least $2$ in the $A$ array or
else loses (gets $1$) in $B$, he goes on to compete using the $C$
array.  That decides wheter he wins or loses. \\
\\
Since we are assuming that there is only one faulty register out of
the $7$, we can argue that this works using a case analysis based on
which is the faulty register. \\
We have to show that one gets a $0$ and one gets a $1$. \\
Moreover, for this to be linearizable, we also have to argue that a
process can't return a $1$ before the other process (the one that
returns $0$) even starts. \\
First, a couple of claims: \\
\\
Claim 1:  If no faulty register appears in $A$ and two processes reach
$A$, then one wins $A$ and one loses $A$.  Likewise for $C$. \\
\\
Claim 2:  If only one process reaches $A$ then it wins.  (Regardless of
whether the faulty register is in $A$.) \\
Likewise for $C$. \\
This is because only one of the registers in $A$ can be faulty. 
\\
Cases: \\
\\
1. The faulty register (if any) is in $A$. \\
Then it could cause the two processes to take the same branch (either)
or different branches. \\
\\
If both take the ``win'' branch, then one wins and one loses in $C$,
by Claim 1 applied to $C$. \\
\\
If they both take the ``lose'' branch, then one wins at $B$ and returns
$1$ for the whole algorithm, and the other goes to the $C$ array.  But
since he's the only one in the $C$ array, Claim 2 implies that he wins
and outputs $0$. \\
\\
If one takes each branch, then the one that goes to $B$ must win at $B$
(he's alone, and $B$ is nonfaulty), and so outputs $1$, whereas the one
that goes to $C$ must output $0$ by Claim 2. \\
\\
2. The faulty register is $B$. \\
Then by Claim 1, $A$ sends one process to $C$ and one to $B$. \\
$B$ can send its one participant either way.  If it lets it win $B$, it
moves down and outputs $1$.  But then the other process is alone in
$C$, which is correct, so by Claim 2, it wins $C$ and outputs $0$. \\
\\
If it sends its participant to $C$, then two compete in a correct $C$,
so Claim 1 implies that one wins and one loses. \\
\\
3. The faulty register is in $C$. \\
Then $A$ and $B$ are correct, which means that one of the processes
goes to $B$, wins there and outputs $1$. \\
The other is alone in $C$.  \\
Then Claim 2 implies that it wins $C$ and outputs $0$. \\
\\
\\
We still have to argue that a process can't return $1$ before the
other process arrives. \\
A process that arrives alone will win at $A$ and $C$, by Claim 2, and
return $0$. \\
\\
\\
This is extended to an $n$-process algorithm still tolerating only one
fault, by adding a doorway that allows at least one and at most two
processes though. \\
Basically, to enter the main algorithm, you have to win some register
at each of two preliminary stages. \\
\\
That means at most two processes can do this, because at least one of
the stages has both registers nonfaulty, and each process that wins
must grab at least one of these registers. \\
\\
Claim at least one process will win one at both stages: \\
Because at most one of the four registers is faulty. \\
Thus, some process wins one of the registers at stage one and enters
stage two. \\
Then some process wins one of the registers at stage two. \\
\\
\\
The $n$-process algorithm can be boosted to an algorithm for T\&S from
T\&S that tolerates any number of failures.

\end{document}

\subsection{RMR register constructions}

\noindent
Now jump ahead to Section 6. \\
Show one result, Theorem 12. \\
Consensus from RMW, using only $f+1$ RMW registers to tolerate $f$ faults. \\
Well, also needs a bunch ($4f^2 + 6f + 2$) of atomic binary r/w registers. \\
The $f$ limitation applies to all the low-level registers---both the
RMW and the r/w registers. \\
\\
See code p. 29. \\
\\
The algorithm keeps track of input values in r/w registers $v$. \\
It uses the RMW registers $r$ to decide on the actual value. \\
\\
The algorithm operates in $f+1$ identical stages. \\
Each stage has $1$ associated RMW register and $2f+1$ $v$ bits. \\
\\
For each stage, the process first sets all the validation flags for 
its current estimate $x$, for that stage. \\
\\
Then it atomically accesses the RMW register for that stage, seeing if
it has already been set by anyone.  If not, it sets it to its own
value $x$.  In either case, it remembers the final value of this
register in a temporary variable. \\
Note since this is a RMW variable, it can do all this at once. \\
\\
Next, it checks whether ``enough'' ($f+1$) of the validation bits for
this stage, for this temporary value, have already been set. \\
If so, it uses this as its new value for the next stage.  Otherwise,
it doesn't change. \\
\\
At the end of $f+1$ stages, just output the final value. \\
\\
A pretty standard-looking $f+1$-stage consensus algorithm. \\
\\
Correctness: \\
Validity:  Claim (stronger) that the value of anyone's $x$ is always a
valid value. \\
Because no one adopts a new value without having read it in $f+1$ out
registers for a stage.  \\
If it appears there, at least one register was written by a nonfaulty
process.  \\
(Can turn this into an induction.) \\
\\
Agreement:  At least one stage, say $g$, has no faulty variables
(RMW or bits). \\
At that stage, the first process to attempt a RMW succeeds in writing
its value, say $x_g$. \\
Has already written this value to the validity bits for this stage. \\
They persist (all nonfaulty), so all other processes can confirm the
validity. \\
Every other process will also read $x_g$ as the value of this RMW
variable, and will therefore adopt the value $x_g$ at stage $g$. \\
\\
Claim they all keep this value for the remaining stages. \\
E.g., consider stage $g+1$.  
All processes set validity bits for $x_g$ at that stage, and none for
the other value. 
Then no one will see enough evidence to switch its value. \\
\\
Some slack in this proof---I think it shows that $f$ faulty RMW
registers and $f$ faulty r/w registers can be tolerated.  But their
notation doesn't express this. 

\subsection{Universality}

\noindent
Finally, some discussion of universality results for the new
setting. \\
Recall Herlihy defined the notion of a ``universal object type'',
which allows construction of a wait-free implementation of any other
object type. \\
This assumes the basic objects are nonfaulty, of course. \\
Some examples of universal objects:  RMW, and consensus. \\
The construction for consensus uses some atomic r/w registers also. \\
\\
AGMT extend this result to the case where some of the base objects may
be faulty, in Theorem 18. \\
\\
Part 1:  RMW registers can be used to construct any object type, even
if $f$ of them may be faulty. \\
\\
Part 2:  Consensus objects plus r/w registers (they use safe, I'll say
atomic) are universal, even if some ($f$ out of all the objects, both
kinds) may be faulty. \\
\\
Proof:  Part 1:  
Use Herlihy's non-fault-tolerant universality construction, which
implements an atomic object in terms of nonfaulty consensus objects
and nonfaulty r/w registers. \\
Implement the consensus objects in terms of possibly-faulty RMW
objects, using the fault-tolerant implementation of consensus we
just described (Theorem 12). \\
(This uses possibly-faulty RMW registers and r/w registers, but of
course we can use RMW registers in place of r/w gegisters.  AGMT use a
variant, which uses only RMW registers, $2f+1$ of them). \\
Implement the r/w registers using possibly-faulty RMW registers
(Corollary 8). \\
Combine everything, using composition theorems to express the
fault-tolerance (Theorems 3, 4). \\
\\
Part 2: 
Use Part 1 to get a fault-tolerant implementation of the arbitrary
object in terms of RMW objects. \\
Now use Herlihy's universality construction to implement the RMW
objects in terms of consensus objects and atomic r/w registers, in a
non-fault-tolerant way. \\
Combine using a composition theorem (Theorem 4). 
(If the top level tolerates $f$ failures, then $f$ failures among the
lowest-level objects can kill only $f$ of the intermediate
objects---even if the lower-level implementation isn't
fault-tolerant.  The top level can tolerate this.)

\subsection{Significance}

\noindent
The model seems similar to the Byzantine quorums work, and the basic
strategies are similar. \\
\\
The main idea is that you can tolerate a limited number of faulty
objects, in shared-memory algorithms.  The basic technique is to
use enough redundancy to ``swamp out'' the bad results. \\
\\
This only works in a kind of ``synchronous'' model, in which objects
always respond.

\subsection{Future work}

\noindent
Make the constructions more ``modular'', systematic? 

\section{Jayanti, Chandra, and Toueg}

Cover:  Section 5.1 only, with whatever is needed to lead up to this. \\
Chapters 1-4 are just the intro, definitions, and simple composition
results like those in AGMT. \\
Chapter 5 deals with responsive failures, like AGMT. \\
Results to cover:  5.1.2.1, for crash failures, and 5.1.3.1, for
Byzantine failures.

\subsection{Overview}

\noindent
Another paper on fault-tolerant shared objects, with many of the same
results as AGMT. \\
Came out about the same time, concurrent work. \\
Differences: \\
They consider other failure modes, like ``nonresponsive'' failures,
and a ``graceful degradation'' property. \\
The main results about nonresponsive failures are impossibility
results based on FLP and Abu-Amara impossibility results. \\
\\
They also consider better behaved kinds of failures for objects, e.g.,
a form of crash failure, where an object just reverts at some point to
responding with a known $\bot$ response, which it will use
consistently from that point onward. \\
\\
I read the paper, and decided to not cover the non-responsive and
graceful degradation results.  Then a lot of the rest is overlap.  So
I'm left with Section 5.1 only, p. 467-475. \\
Two results:
\begin{enumerate}
\item
Theorem 5.1.2.1.  An implementation of reliable consensus from
consensus (only), tolerating $f$ faults, using only $f+1$ base
objects.  But, the faults are limited to crash (actually, also
omission, where the object can mix up default and correct responses,
but not actually give wrong responses).
\item
Theorem 5.1.3.1.  Another implementation of reliable consensus from
consensus (only), this time tolerating $f$ Byzantine faulty objects.
\end{enumerate}

\subsection{Crash-tolerant algorithm}

\noindent
Code, p. 469 \\
\\
Use $f+1$ consensus objects. \\
Start with your input as your ``estimate''. \\
Go through the sequence of base consensus objects in order. \\
For each, propose your latest estimate. \\
If the response you get back isn't the default $\bot$, change your
estimate to the new response. \\
After $f+1$ stages, return your final estimate. \\
\\
Similar to the AGMT multi-stage algorithm, but now we don't need to
use validity bits---we can just accept the value as correct. \\
\\
Validity is obvious.  \\
Agreement:  One of the base consensus objects must be nonfaulty, so it
gives the same response to everyone.  Then no one ever submits a
different value to later objects. 

\subsection{Byzantine-tolerant algorithm}

\noindent
Code p. 471, picture p. 472. \\
\\
Now consensus from (just) consensus, with Byzantine failures.  \\
Has a lot of similarity to consensus from RMW, in AGMT paper. \\
\\
Construction is recursive; what's shown here is the top level.  To
tolerate $f$ faulty base objects, use two intermediate consensus
objects $O_1$ and $O_2$ that tolerate approximarly $f/2$ faulty base
objects in their implementations. \\
Also use three arrays of consensus objects, $A_1$ and $A_2$ with 
$3f + 1$ objects each, and $B$ with $4f+1$ objects. \\
None of these objects tolerate any failures (that is, a failure within
their implementations would just kill the object). \\
\\
A process with initial value $0$ follows the arrows in the diagram,
going through $A_0$, then $O_1$, then $B$, then $A_1$, then $O_2$. \\
One with initial value $1$ would follow the arrows ``symmetrically'',
starting with $A_1$, then $O_1$, $B$, $A_0$, $Q_2$. \\
\\
E.g., if initial value $0$: \\
\\
$A_0$:  Go through the entire array, proposing your initial value and
counting (in $count[0]$) the number of times the response value agrees
with your initial value.  \\
\\
$O_1$:  Propose your initial value to $O_1$ and record the response,
in $ans_1$. \\
\\
$B$:  Go through the entire array, this time proposing your $ans_1$
value, and counting (in $witness-count[0]$ and $witness-count[1]$) the
number of responses you get for each of the two values. \\
\\
$A_1$:  Now go through the entire array, again proposing your initial
value.  This time, count (in $count[1]$) the number of times the
response value {\em disagrees\/} with your initial value. \\
\\
At this point, you might have enough information to return a value. \\
Choose $belief$ to be the value with the larger $witness-count$. \\
If the $witness-count[belief]$ is sufficiently large ($3f+1$), and your
$count[belief]$ is also large ($2f+1$) then simply return $belief$. \\
\\
Otherwise, rely on the final consensus object, $O_2$. \\
Propose to $O_2$ either your $belief$ (if $witness-count[belief] \geq
2f+1$ and $count[belief] \geq f+1$), or else your initial value. \\
Whatever $O_2$ returns is your final decision. \\
\\
\\
Proof:  \\
\\
Lemma 1:  If $O_1$ is nonfaulty and returns $0$ to any process $p$,
then $count[0]_p \geq 2f + 1$. \\
(And analogously for $1$.) \\
\\
Proof:  Since $O_1$ returns $0$ to $p$, it must be (by validity) that
some process must have previously proposed $0$ to $O_1$; let $q$ be
the first. \\
Then before $q$ proposes $0$ to $O_1$: \\
(1) $q$ must have submitted its $0$ to all the consensus objects in array
$A_0$. \\
(2) no process submitted $1$ to any of the consensus objects in $A_0$
(because they haven't heard from $O_1$ yet). \\
(3) all the $A_0$ objects have returned decisions to $q$. \\
Therefore, all the nonfaulty $A_0$ objects must have returned $0$
decisions. \\
\\
Therefore, any process that reads $A_0$ sees at least $2f + 1$
$0$'s. \\
\\
\\
Lemma 2: $O_1$ and $O_2$ can't both be faulty.  \\
Because there aren't enough faults in the base objects (of the
recursively expanded implementation) to kill both intermediate
objects. \\
\\
Main proof, agreement and validity:  Use cases, first assuming that
$O_1$ is nonfaulty and then $O_2$. \\
\\
Case 1:  $O_1$ nonfaulty: \\
\\
Then $O_1$ gives out the same $v$ to every process.  \\
So every process proposes $v$ to all the $B$ objects, and all 
the nonfaulty $B$ objects return $v$. \\
That gives everyone a high enough $witness-count[v]$. \\
They also have high enough $count[v]$, by Lemma 1. \\
So they all decide $v$. \\
\\
Case 2:  $O_2$ is nonfaulty: \\
\\
If someone returns his $belief$ on line 14, then there is a clear
enough majority for this value $v$ that eeryone will satisfy the
condition in line 15 and submit the same value $v$ to the final
consensus object $O_2$. \\
Since $O_2$ is nonfaulty, it returns $v$, so everyone agrees. \\
\\
On the other hand, if no one returns his $belief$ on line 14, then all
return the results of consensus using $O_2$, which must agree. \\
Validity also follows, because validity of $O_2$ implies that the
value $v$ was submitted by someone. \\
It was either that process' initial value, which immediately gives
validity, or that process belief, then the condition on line 15 says
that the count for belief was sufficiently high that someone must have
originally submitted it.

\end{document}










