% Begin your document as any latex file
%\documentclass{article}
% Use package 6895
%\usepackage{6895}
%\usepackage{epsfig}


% The following are just for the sake of this document.  Ignore them
% for the purposes of generating scribe notes.
% --------------------------------------------------------------------
%\newcommand{\pc}[1]{{\tt\begin{description} \item [\TT] #1
%\end{description}}}

%\newcommand{\com}[1]{$\backslash$#1} 
%\newcommand{\comone}[2]{$\backslash$#1\{#2\}}
%\newcommand{\comtwo}[3]{$\backslash$#1\{#2\}\{#3\}}
%\newcommand{\comthree}[4]{$\backslash$#1\{#2\}\{#3\}\{#4\}}
%\newcommand{\comfour}[5]{$\backslash$#1\{#2\}\{#3\}\{#4\}\{#5\}}
%\newcommand{\comfive}[6]{$\backslash$#1\{#2\}\{#3\}\{#4\}\{#5\}\{#6\}}

%\newcommand{\sh}[2]{\com{#1} & $#2$ \\} \newcommand{\shn}[2]{\com{#1} & #2 \\}

%\newenvironment{todolist}{\begin{description}}{\end{description}}
%\newcommand{\todoitem}[2]{\item {\em #1} \hfill \\ #2}
% ---------------------------------------------------------------------

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Begin the lecture with a line like the following:
% This replaces the usual \begin{document} that a latex
% file begins with.
\begin{guestlecture}{5}{Consistency of the Memory Sub-System}{Alexandru Caracas, Rui
    Fan}{Bradley C. Kuszmaul}
  
% Now summarize the lecture.  This should be like an abstract
% for the lecture, broken down into a few bullet points.
  \begin{summary}
  % The first item of each \sumitem is the title of the part of the
  % lecture, and the second part is a description (i.e. abstract) of
  % that part.
    \sumitem{Mutual Exclusion} {Introduction to the problems of shared-memory parallel
      systems and definition of mutual exclusion.}  \sumitem{Mutual Exclusion
      Algorithms}{Description of the mutual exclusion algorithms of Dekker and Dijkstra.}
    \sumitem{Memory Consistency} {Definition of memory consistency in terms of atomic
      operations.} \sumitem {Memory Subsystem of Hardware Architectures} {Description of
      various memory operation primitives such as Load-Link-Store-Conditional and
      Transactional Memory.}
  \end{summary}

% Now organize your lecture into sections.
  \section{Mutual Exclusion}
  The underlying problem of shared memory parallel systems is that we have more than one
  thread and shared memory locations, accessed by all concurrent executing threads.  The
  question is how do the threads interact?
  
  \begin{example}
    One example Cilk code which contains a simple determinacy race, already discussed in
    the previous lectures, %(see \ref{})
    is the following:

\begin{verbatim}
   cilk int foo (void) { x++; }
   cilk int main (int argc, char *argv[]) { 
      spawn foo();
      spawn foo();
      sync;
   }
\end{verbatim}
    
  \end{example}
  
  Once we detect the determinacy race, e.g. by using the Nondeterminator, our goal is to
  eliminate it. In order to avoid the determinacy race we have to make sure that only one
  thread at a time can perform the operation \texttt{x++}.  As a start we could consider
  the following idea. Simply prevent context switching, for example by turning off the
  interrupts as in the following MIPS assembler code:
  
\begin{verbatim}
   DI            ; turn off interrupts
   LW R1, (0)R2
   ADD R1,R1,1
   SW R1, (0)R2
   EI
\end{verbatim}
  
  However, there are a number of downfalls with using interrupts for mutual exclusion:
  \begin{itemize}
  \item It only works on a uniprocessor machine. On a multiprocessor machine the other
    processors are not affected by the \texttt{DI} instruction.
  \item It only works in kernel (not user) mode. On most architectures there is no
    user-level call that can disable interrupts.
  \item And it also prevents other threads from doing non-critical work, which is
    extremely dangerous, since the operating system time slice interrupt will never take
    place, and the operating system would fail gaining control.
  \end{itemize}

  \begin{definition}
    In a more general conceptual setting, mutual exclusion can be described in the
    following algorithmic setup:
    \label{mutex-algorithm}
\begin{verbatim}
   while (1){
      trying();
      critical_section()
      releasing();
      noncrit();
   }
\end{verbatim}
  \end{definition}
  In an infinite loop each thread tries to execute \texttt{critical\_section()}, that is
  the sequence of operations which need to be executed atomically. In order to be allowed
  to execute the critical section the thread first needs to get permission. In the above
  example the task of allocating the right to enter the critical section is given by the
  part \texttt{trying()}.  Moreover the thread needs to be cooperative and release hold of
  its right to enter the critical section in \texttt{releasing()}.  Consequently it
  performs the rest of its non-critical task \texttt{noncrit()}.
  
  Now the problem is to write the functions \texttt{trying()} and \texttt{releasing()} so
  that at most one thread is in \texttt{critical\_section} at a time. One extremely
  trivial solution would be:

  \begin{example}
\begin{verbatim}
   trying(void) { halt(); }
\end{verbatim}
  \end{example}
  
  The trivial solution is not good since it simply stops the currently executing thread.
  This illustrates a problem with our specification of mutual exclusion. In addition to
  the algorithmic description of mutual exclusion from Definition \ref{mutex-algorithm} we
  now introduce some additional definitions.

  \begin{definition}
    Mutual exclusion requires that there is {\bf Progress}. If no thread is executing {\em
      \texttt{critical\_section}}\ and there are some threads which are trying to enter
    {\em \texttt{critical\_section}} (in our example by executing {\em \texttt{trying()}})
    then some thread will do {\em \texttt{critical\_section}}.
  \end{definition}
  
  Apart from the definition of mutual exclusion with guaranteed progress there are other
  aspects which are desirable, however not strictly required for {\em Mutual Exclusion},
  such as:
  
  \begin{definition}
    {\bf No Starvation}. Every executing thread gets a fair chance of executing in the
    {\em \texttt{critical\_section}}.
  \end{definition}
  
  \begin{example}
    An alternate idea for the mutual exclusion problem in the case of two threads would
    be:
\begin{verbatim}
   int turn=1;
   int run(int i) {
      // i is the thread number for the currently executing thread,
      // 0 for one thread and 1 for the other
      while (turn!=i);
      crit();
      turn=1-i;  // it is the other thread's turn
      noncrit();
   }
\end{verbatim}
  
  \end{example}
  
  However, the problem with the previous approach is that it does not guarantee progress.
  If, for example, \texttt{noncrit()} is an infinite loop in thread 1, then thread 0 would
  get stuck.


  \section{Mutual Exclusion Algorithms}
  
  This section presents two algorithms that solve the mutual exclusion problem.  The first
  one is due to Dekker and it solves the mutual exclusion problem for two threads. The
  second is Dijkstra's algorithm for mutual exclusion, which is a generalization
  applicable to any number of threads.
  
  \subsection{Dekker's Algorithm}
  
  A solution which works for two threads and uses turns was described by Dekker. The
  following C code presents his algorithm:

\begin{verbatim}
   int turn;
   int wants[2];

   // i is the current thread, j=1-i is the other thread
   while(1) {                 // trying
      wants[i] = TRUE;
      while (wants[j]) {
         if (turn==j) {
            wants[i] = FALSE;
            while (turn==j) ; // empty loop
            wants[i] = TRUE;
         }
     }
     critical_section();
     turn=j;                  // release
     wants[i] = FALSE;
     noncrit();
   }
\end{verbatim}
  
  As a visualization of Dekker's algorithm consider the following railroad from Figure
  \ref{dekker-railroad}. Each thread has its own railroad system which intersects with the
  railroad of the other thread only on the \texttt{critical\_section}. There are a number
  of switches which the threads activate as they traverse the railroad. Whenever a thread
  encounters a semaphore it can change its course according to the indication of the
  semaphore. The threads ``move'' in the direction shown by the arrows.

  As an illustration, assume that thread 0 starts at the top of its railway system and
  wants to do \texttt{critical\_section}. It will first set \texttt{turn=1}, and
  then \texttt{wants[0]=TRUE}. Next it will encounter a semaphore which tells it
  whether thread 1 wants to enter the critical section. Suppose that thread 1 also wants
  to enter, then thread 0 will switch tracks, and ``move'' downwards. On this new track
  it will set \texttt{wants[0]=FALSE} and go and check the next semaphore. Assume that now
  \texttt{turn=0}, then thread 0 will remain on this track, and set
  \texttt{wants[0]=TRUE}. Also assume that now thread 1 does not want to enter the
  \texttt{critical\_section}. Then thread 0 can go and execute \texttt{critical\_section}.

  \begin{figure}
    \centering \includegraphics[scale=0.5]{5/lecture-5-fig1.eps}
    \caption{Railroad visualization of Dekker's algorithm for mutual exclusion. 
      The threads ``move'' in the direction shown by the arrows.}
    \label{dekker-railroad}
  \end{figure}

  \subsection{Dijkstra's Algorithm} 
 
  Dijkstra devised a mutual exclusion algorithm for any number of threads. The following are the conditions for
  Dijkstra's mutual exclusion problem:
  
  \begin{itemize}
  \item Only 1 thread may execute \texttt{critical\_section()} at a time.
  \item The threads execute in an infinite loop trying to enter
    \texttt{critical\_section}.
  \item {\em Atomic read} and {\em atomic write} are available, but no other instructions
    (no locking instructions).
  \item Progress condition is satisfied.
  \item No assumptions about relative speeds of threads.
  \end{itemize}
  
  The following is the C code for Dijkstra's Algorithm:
  
\begin{verbatim}
   char b[N], c[N];  // b[i] and c[i] are only be written by thread i, initialize true
   int  k;           // 0 <= k < N

   while (1) {
      // i is my own thread number
      b[i] = FALSE;
  L:  if (k!=i) {
         c[i] = TRUE;
         if (b[k]) k = i;
         goto L;
      }
       else {
         c[i] = FALSE;
         for (j=0; j<N; j++) {
            if (j!=i && !c[j]) {
               goto L;
            }
         }
      }
      crit();
      c[i] = TRUE;  
      b[i] = TRUE;
      noncrit();
   }
\end{verbatim}

\section{Proof of Dijkstra's Algorithm}


We first show that Dijkstra's algorithm satisfies mutual exclusion.  Suppose for
contradiction that there are two processes $i$ and $j$ in the critical section at the same
time.  Then $c[i] = c[j] = \textrm{FALSE}$.  Suppose, without loss of generality, that $i$
was the first to set $c[i] = \textrm{FALSE}$.  Then, $j$ must see $c[i] = \textrm{FALSE}$
in the for loop after assigning $c[j]$, hence $j$ goes to $L$.  Then, $j$ does not enter
the critical section, which is a contradiction.  Thus, there is at most one process in the
critical section at once.

To show that progress is always made, suppose several processes try to enter the critical
section.  Then each such process $i$ sets $b[i] = \textrm{FALSE}$.  We claim that at least
one trying processes advances into the else clause.  Consider all the trying processes for
which the test ``$\textrm{if} \ (b[k])$'' succeeds.  There is at least one such process.
Consider the last process $i$ among this set to set $k \gets i$.  Then $i$ will enter the
else clause.

Suppose only one process $i$ enters the else clause.  Then $i$ finds $c[j] =
\textrm{TRUE}$ for all $j \neq i$, and enters the critical section.  Next, suppose a set
$P$ of more than one process enters the else clause.  They may all find $c[j] =
\textrm{FALSE}$ for some other $j \in P$, and goto $L$.  But now, for every process $i \in
P$, $b[i] = \textrm{FALSE}$, and so $i$ cannot reset $k \gets i$.  On the other hand, for
some $i \in P$, we have $k = i$.  Then $i$ will be the only process to go into the else
clause\footnote{Actually, another (slow) process $j$ can set $k \gets j$ and enter the
  else clause with $i$.  But then we can repeat the previous argument.  Each time we
  repeat the previous argument, we eliminate at least one trying process from going into
  the else clause, while ensuring there is still at least one process which can go into
  the else clause.  Since the number of trying processes is finite, we are eventually left
  with exactly one process which can go into the else clause.}, where it finds $c[j] =
\textrm{TRUE}$ for all other $j$, and so can enter the critical section.  Thus, Dijkstra's
algorithm ensures progress.

\section{Memory Consistency}
Notice that Dijkstra's algorithm depends on a property of the memory system, which is that
all the processes read the last value written to a memory location.  This is called
\emph{atomic read} and \emph{atomic write}.  Here is an example to illustrate the idea:

\begin{example}
\label{ex-seq-con}
\begin{verbatim}

  main {
    X = 0; Y = 0;
    spawn thread0();
    spawn thread1();
    sync }

 thread0() {             thread1() {
    X = 1;                  while (Y == 0);
    Y = 1 }                  print(X) }
\end{verbatim}
\end{example} 

If the reads and writes are atomic, then the value that is printed for $X$ must be 1.
This is because to exit the while loop in thread 1, $Y$ must be set to 1.  But if writes
are atomic, this means that $X$ is already set to 1.  We can draw each computation as a
DAG, where the arrows point from earlier to later events.  For example, one DAG for an
execution of the above example is shown in the Figure 2.%\ref{fig-dag}.

\begin{figure}
  \label{fig-dag}
  \begin{center}
    \includegraphics[scale=0.5]{5/lecture-5-fig2.eps}
    \caption{A DAG for the multithreaded program in Example
      \ref{ex-seq-con}.  The solid arrows represent dependencies induced by the program order.
      The dashed arrow represents an implicit dependency of the execution.}
  \end{center}
\end{figure}

As a side note, we define an \emph{observer function} as a mapping that gives, at each
point in the execution, for each memory location, which write operation wrote the value
that a process sees if it reads that location.  A \emph{consistency model} is a set of
allowable observer functions.  An execution is consistent with the consistency model if it
can be observed by an observer function of the model.  A hardware implementation of memory
is consistent with a consistency model if all its executions are consistent with the
model.  We now define a consistency model for memory called \emph{sequential consistency}.

\begin{definition}
\label{def-seq-con}
Let $G$ be a DAG of an execution $\alpha$ of a multithreaded program.  Then $\alpha$ is
\emph{sequentially consistent} if and only if there is a topological sort $S$ of $G$ such
that for each read, the most recent write to that memory location according to $S$ wrote
the value that the read received.
\end{definition}

Note that there can be more than one topological sort of $G$ demonstrating consistency.
All that Definition \ref{def-seq-con} requires is the existence of some sort consistent
with the execution.  Also, note that for sequential consistency, the allowable observer
functions are the ones for which a topological sort exists on the DAG such that the value
for a location seen is the one most recently written to that location.

\section{Memory Subsystem of Hardware Architectures}
The MIPS multiprocessors guarantee sequential consistency.  However, the x86
multiprocessors do not.  The reason is that CPU's pipeline instructions and can execute
them out of order.  Therefore, even though a read may occur later than a write in code,
the read can actually be executed before the write.  It is complicated to do out of order
execution and guarantee strong consistency.

Instead of sequential consistency, Intel offers a \emph{barrier} operation.  Specifically,
the instruction MFENCE forces all reads and writes before an MFENCE to finish before any
reads and writes after the MFENCE can start.  However, the Intel MFENCE instruction is
very slow.  On a 2.5Ghz Pentium 4, one MFENCE instruction takes 120ns.  Interestingly, the
barrier instruction on a 1.5Ghz AMD64 Opteron takes only 8ns.

\subsection{Additional Memory Primitives}
As we saw, mutual exclusion is very complicated to do using only atomic read and write
memory.  Thus, most machines provide additional memory instructions to make mutual
exclusion easier.  One common instruction is \emph{test-and-set}.  The following
pseudocode demonstrates what test-and-set does:

\begin{example}
\begin{verbatim}

  test_and_set(int *l) {
    atomically {
      int old_value = *l;
      *l = l;
      return old_value;
    }
  }      
\end{verbatim}
\end{example}

As the example shows, test-and-set atomically reads the old value from a memory location
and writes a 1 to that location.  To demonstrate its usefulness, consider the following
simple algorithm for mutual exclusion using test-and-set.

\begin{example}
\begin{verbatim}

  while (test_and_set(&lock));
  critical_section();
  lock = 0;
\end{verbatim}
\end{example}
 
The first process which gets a 0 from test-and-set can enter the critical section.  All
the other processes have to wait till that process is done and resets lock.

Another popular memory instruction is \emph{load-linked store-conditional
  (LLSC)}.\hspace{2pt} This is provided, for example, on SGI.\hspace{2pt} LLSC is
illustrated in the following code:

\begin{example}
\begin{verbatim}

  again:
    LL R1, R2      // R1 = *R2
    compute
    SC R1, R2      // *R2 = R1 but only if *R2 has not changed since LL
    bz again:      // the zero flag is set so we can do it again if it failed
\end{verbatim}
\end{example}

As the example shows, LLSC lets a process read a memory location, do some computation,
then write a value back to that memory location.  But the write only succeeds if the
location didn't change since the read.  If the location did change, LLSC sets the zero
flag to let the process know to try again.  LLSC lets us increment a variable without a
lock, as shown in the following example.  Note that this code performs the increment
directly, without attempting to get mutual exclusion on the variable.  Also note that the
algorithm may not make progress, if for example, two processes continually perform LLSC
alternately.

\begin{example}
\begin{verbatim}

  again:
    LL R1, R2
    ADD R1, R1, 1
    SC R1, R2
    bz again
\end{verbatim}
\end{example}

To end this lecture, we mention a few other mutual-exclusion type memory operations
offered by many architectures.  The most important among these is \emph{compare-and-swap
  (CAS)}.  The operation CAS($x, y, v$) does the following.  If $x = y$, then the
operation sets $x \gets v$.  Otherwise, it returns an error code saying the CAS failed.
Some architectures also offer \emph{double compare-and-swap (DCAS)}.  Similar to CAS,
DCAS($x_1, x_2, y_1, y_2, v_1, v_2$) compares $x_1$ to $y_1$, and $x_2$ to $y_2$.  If both
sets of comparisons return true, then DCAS sets $x_1 \gets v_1$, and $x_2 \gets v_2$.
Otherwise, it returns an error code saying the DCAS failed.

Another approach to ensuring atomicity is to use \emph{transactional memory}.  This
technique performs a sequence of memory operations atomically, all as part of a
transaction.  This allows a simple and uniform way of implementing many complex concurrent
data structures.


%%  \begin{thebibliography}
%%  \bibitem{FL98} M. Frigo and V. Luchangco, ``Computation-centric Memory
%%    Models'', \emph{Proceedings of the Tenth Annual ACM Symposium on
%%      Parallel Algorithms and Architectures'', 1998, pp. 240--249.
%% \end{thebibliography} 

\end{guestlecture}
\theend
