% Begin your document as any latex file
%\documentclass{article}
% Use package 6895
%\usepackage{6895}
%\usepackage{epsfig}

\newcommand{\claimlabel}[1] {\label{thm:#1-\lecturenum}}
\newcommand{\claimreftwo}[2] {Claims~\ref{thm:#1-\lecturenum}
and~\ref{thm:#2-\lecturenum}}

\newcommand{\xmpllabel}[1] {\label{thm:#1-\lecturenum}}
\newcommand{\xmplref}[1]{Example~\ref{thm:#1-\lecturenum}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Begin the lecture with a line like the following:
% This replaces the usual \begin{document} that a latex
% file begins with.
\begin{guestlecture}{13}{Implementation of Memory Consistency}
                        {Seth Gilbert, Xie Yong}
                        {Bradley C. Kuszmaul}
  
% Now summarize the lecture.  This should be like an abstract
% for the lecture, broken down into a few bullet points.
\begin{summary}
  \sumitem{Sequential Consistency} {We review the definition of
  sequential consistency.}
%%
  \sumitem{Uncached Consistency Algorithm} {We present a simple
  algorithm and prove that it guarantees sequential consistency.}
%%
  \sumitem{Overlapping Operations and Caching}{We discuss the difficulties
  introduced by allowing overlapping operations at a single processor,
  and by using caches.}
%%
  \sumitem{The MESI Protocol}{We present the MESI protocol, and
  algorithm that takes advantage of caching to provide an efficient,
  sequentially consistent memory.}
\end{summary}

\section{Memory Consistency}

We first review the definition of sequential consistency in
\secref{sequential}.  We then define a system model in
\secref{sysmodel}, and present an algorithm for implementing the usual
memory operations, \texttt{load} and \texttt{store}, in
\secref{uncached}.  We then prove that this algorithm guarantees
sequential consistency in \secref{uncachedproof}.

\subsection{Sequential Consistency}
\seclabel{sequential}

We review here the definition of sequential consistency, otherwise
known as strong consistency.

\begin{definition}
\deflabel{def-seq-con}
%
Assume that $G$ is a DAG representing a multithreaded program, and
that $\alpha$ is an execution (i.e., a single run) of $G$.  The
execution $\alpha$ is \emph{sequentially consistent} if and only if
there exists a topological sort (i.e., a total sort), \texttt{S}, of
$G$ such that for every load, the most recent (in the total order
\texttt{S}) store to that memory location wrote the value that the
load received.
\end{definition}
Note that there can be more than one topological sort of the DAG
demonstrating sequential consistency.

\subsection{System Model}
\seclabel{sysmodel}

\begin{figure}[b]
\centering \includegraphics[scale=0.9]{13/proc-network-memory.eps}
\caption{\small Model for processors to access memory through communication network.}
\label{fig:proc-network-memory}
\end{figure}

We consider a model which contains of a set of processors, a set of
memory modules and a communication network, as shown in
Figure~\ref{fig:proc-network-memory}.  We assume the following:

\begin{itemize}
\item The memory is organized into words.
\item Processors execute \texttt{load} and \texttt{store} instructions
on words.
\item Processors send messages to the memory using the network.
\item The memory modules perform \texttt{read} and \texttt{write}
operations on the memory.
\end{itemize}

\begin{example}
Let the memory module $M_i$ contain the set of all memory locations in
which the low-order two bits of the page number are $i$. On a system
with $2^{12}$ words per page, that means that $M_i$ contains any memory
address that when written in binary looks like:
$$
\texttt{XXXX XXXX XXXX XXXX XXab XXXX XXXX XXXX}
$$
where $i = 2 * a + b$, and $0 \leq a, b<2$. $M_i$, then, consists
of every address $X$ for which 
$$
(X \gg 12) ~~ \& ~~ 3 == i
$$
\end{example}

\subsection{Uncached Consistency Algorithm}
\seclabel{uncached}

It turns out that it is easy to implement sequential consistency, if
at each processor load and store operations do not overlap and
processors do not cache memory values.

For this section and \secref{uncachedproof}, then, we assume that each
processor can only perform a single load or store operation at any
given time.  That is, once a processor begins an operation, it must
wait until the operation completes before it can begin a new
processor.  We now present a simple algorithm in which there is no
caching:

\begin{itemize}
\item $\texttt{load}(x)_i$: When processor $i$ receives a
  \texttt{load} request for location $x$, it sends a message to the
  memory module containing the address $x$, asking for the current
  contents.  Processor $i$ then waits for the reply containing the
  value.

\item $\texttt{read}(x)_j$: When memory module $j$ receives a
  \texttt{read} request for location $x$, it reads the context of
  location $x$ and sends a reply.

\item $\texttt{store}(x, v)_i$: When processor $i$ receives a
  \texttt{store} request to place value $v$ in location $x$, it sends
  a message containing $x$ and $v$ to the memory module containing $x$
  and waits for a reply indicating that the operation is complete.

\item $\texttt{write}(x, v)_j$: When the memory module $j$ receives a
write request, the memory writes $v$ into location $x$ and sends a
reply indicating that the operation is complete.

\end{itemize}

\subsection{Proving Sequential Consistency}
\seclabel{uncachedproof}

In this section, we show that the \textsc{Uncached Consistency
Algorithm} implements sequential consistency.  To prove this, we need
to first construct a topological sort of the program DAG, and then
show that every \texttt{load} operations returns the most recently
stored value to the same location in the total order.

\begin{definition}
\deflabel{sort-S}
%%
We sort the load and store operations in the same order in which the
memory modules acutally process the corresponding messages. If the
times at which two operations are processed by the memory modules
overlap, then either order is allowed.

More formally, we define sort \texttt{S} as follows: given two
operations in the DAG, $i$ and $j$, we say that $i < j$ if the memory
module accessed by $i$ completes the operation associated with $i$
before the memory module accessed by $j$ completes the operation
associated with $j$.  If the operations overlap, we arbitrarily order
$i$ and $j$, for example, we let $i<j$ if the address associated with
$i$ is smaller than the address associated with $j$.
\end{definition}

\begin{claim}
\claimlabel{sort}
Sort \texttt{S} is a topological sort of the program DAG G where G=(V,E).
\end{claim}
\begin{proof}
To prove that Sort \texttt{S} is a topological sort of the DAG, $G$,
we need to show:
$$
%%
\forall~i \in V, j \in V, ~~ i \prec j \textrm{ (in $G$)}~~\implies~~i < j
  \textrm{ (in \texttt{S})}\ .
%%
$$
If $i \prec j$, then $i$ sends a message to the memory, and waits for
the reply before allowing any of its successors to run. The memory
acutally processes the operation before sending the reply, and
therefore $j$ cannot start execution until the memory operation for
$i$ has finished. The memory operation for $j$ cannot start until $j$
starts. Therefore, the memory operations for $i$ finish before the
memory operation $j$ starts, and $i < j$ in the sort \texttt{S}. Hence
the sort \texttt{S} is a topological sort of the program DAG.
\end{proof}

\begin{claim}
\claimlabel{loadstore}
Every load observes the most recent store to the same location in the
total order.
\end{claim}
\begin{proof}
Consider a particular location $X$. The total order of all memory
operations induces a total ordering of the operations on memory
location $X$. Hence if $i$ and $j$ are two operations on the same
memory location, where $i$ occurs before $j$, then $i < j$ in the
total order. For each load, the memory returns the value most recently
stored value, and hence every load observes the most recent store to
that location.
\end{proof}

\begin{theorem}
The \textsc{Uncached Consistency Algorithm} implments sequential consistency
\end{theorem}
\begin{proof}
Combining \claimreftwo{sort}{loadstore}, we have shown that Sort
\texttt{S} is a topological sort of the program DAG, $G$ and every
load observes the most recent store to the same location in the total
order. Therefore, by defintion of sequential consistency, the
\textsc{Uncached Consistency Algorithm} implements sequential
consistency.
\end{proof}

\section{More Complicated Memory}

Although the \textsc{Uncached Consistency Algorithm} implements
sequential consistency, it has two problems:
\begin{itemize}
\item Operations at a single processor may not overlap.
\item Processors are not allowed to cache memory values.
\end{itemize}
In \secreftwo{overlap}{caching} we discuss why these two problems are
difficult to solve.  We then present the MESI algorithm in
\secref{mesi}, which takes advantage of caches to implement sequential
consistency more efficiently.

\subsection{Overlapping Operations}
\seclabel{overlap}
An overlapping operation occurs when a processor begins executing a
new operations prior to receiving a reply from an earlier operations.
Allowing overlapping operations lets the processor take advantage of
the parallelism in operations, which is illustrated in the following
example.

\begin{example}
\xmpllabel{nooverlap-cocache}
Assume there is no overlapping of operations.
\begin{verbatim}
    R1 = *X;
    R2 = *Y;
    R3 = R1 + R2;
    *z = R3;
\end{verbatim}
The critical path of this code fragment is:
\begin{itemize}
\item 3 round-trip delays through the network +
\item 2 read operations +
\item 1 write operation +
\item 1 add operation
\end{itemize}
If we can fetch X and Y in parallel, we can save one read and one
round trip delay, leading to a critical path of:
\begin{itemize}
\item 2 round-trip delays through the network +
\item 1 read operations +
\item 1 write operation +
\item 1 add operation
\end{itemize}
Notice that round-trip delays are significantly larger than the other
operations.
\end{example}

It is very tough, however, for hardware to detect whether the
algorithm allows such parallelism.  As a result, we cannot perform
overlapping operations, since the algorithm might rely on sequential
consistency. The following example illustrates the problem with
overlapping.

\begin{example}
Initially all memory locations are 0.
$$
\begin{array}{cc|ccl}
\\
P_0 & ~ & ~ & P_1 \\
&&&&\\
*X = 1 &&&& \\
*Y = 1 &&&& \\
&&& R_1 = *Y  & \textrm{// gets 1} \\
&&& R_2 = *X  & \textrm{// might get 0}
\end{array}
$$
\end{example}

Assume that the operations are allowed to overlap at processor $P_0$.
It is possible, then, that the contents of $R_2$ could be 0 at the end
of the execution of this code fragment.  This can happen when the
message for the second operation, which sets $*Y = 1$ arrives at its
memory module before the message for the first operation, which sets
$*X = 1$ arrives at its memory module.  In this case, $R_1$ can return
the value 1, while $R_2$ returns the value 0.  (At some later point,
the first operation will complete and $*X = 1$.  This may, however, be
too late.)  Notice that sequential consisteny has been violated: the
execution DAG orders the writing of $*X$ earlier than the writing of
$*Y$, while the memory modules order the operations in the opposite
order.

To solve this problem, we can try to fix the network. We can keep the
packets in-order between a processor and a memory module (pair-wise
First-In-First-Out).  This does not fix the problem, however.  If all
the packets are delivered in the order in which they are sent, then we
can avoid this problem.  This solution, though, is not scalable.

\subsection{Caching}
\seclabel{caching}

We now consider the possibility of caching memory locations.  Consider
again \xmplref{nooverlap-cocache}.  If we allow caching, the critical
path is reduced to:
\begin{itemize}
\item 1 read +
\item 1 write +
\item 1 add
\end{itemize}
in the case where all the values are cached.  Caching, however,
introduces a new problem.  Memory may be modified by another
processor, while the local cache is not updated, as shown in the
following example.

\begin{example}
\xmpllabel{cache}
$$
\begin{array}{cc|ccl}
\\
P_0 &&& P_1 & \\
&&&&\\
&&& R_1 = *X & \textrm{// gets 0, and stores it in the cache on $P_1$} \\
*X = 1 &&&& \\
*Y = 1 &&&& \\
&&& R_2 = *Y & \textrm{// gets 1, strong consistency implies *X=1} \\
&&& R_1 = *X & \textrm{// still sees 0 because it is in the cache}
\end{array}
$$
%%
The local cache of $*X$ on $P_1$ is not updated when $*X$ is updated
to 1, so processor $P_1$ cannot see the changes of $*X$ in the memory.
\end{example}

\subsection{MESI}
\seclabel{mesi}

\begin{figure}
  \centering
  \hrule
  \FF
  \input{13/cache.pstex_t}
  \FF
  \hrule
  \caption{Diagram of a system with caches.  The circles on the left
  represent processors, the circles on the right represent memory
  modules.  The squares represent caches, and the oval represents a
  bus.}
  \figlabel{cache}
\end{figure}

The MESI protocol takes advantage of caches to implement a
sequentially consistent memory more efficiently.  Our system, as
before, consists of processors, memory modules, and a network.  Each
processor, however, has a cache associated with it.  (See
\figref{cache}.)  We also assume that the network is implemented by a
bus.  This means that every processor sees every message that is sent
on the network.  This allows processor to silently update their state
in response to ongoing operations.  Note that this also implies that
only one processor can broadcast at a time, so a separate protocol is
needed to handle contention.  As a result, this does not scale all
that well.  However, this is the basis for more complicated algorithms
that do scale.

The caches are (typically) organized into lines, each containing
several words of data.  As a result, if the algorithm chooses to cache
location 100, it also fetches (and caches) locations 101, 102, and
103.  This is useful, since many programs exhibit \emph{spatial
locality}: often, when a program performs an operation on a given
location in the memory, it is quite likely to perform another
operation on a nearby location soon afterwards.  There are two types
of caches.
\begin{itemize}
  \item \emph{Write-through caches} immediately notify the main memory
  of every \texttt{store} operation.  This ensures that the value in
  the cache is always the same as the value in the main memory.  (This
  leads to simple memory algorithms.)

  \item \emph{Write-back caches} do not always immediately notify the
  main memory of a \texttt{store} operation.  Sometimes, the cache
  maintains a new value, unknown to the main memory.  Eventually, the
  value will get ``written-back'' to the main memory, but this may
  take a long time.  This is more efficient, since it saves network
  bandwidth when the value does not really need to be written back.
\end{itemize}

\begin{figure}[t]
  \hrule
  \centering
  \FF
  \includegraphics[width=4in]{13/mesi.eps}
  \FF
  \hrule
  \caption{State diagram for the MESI protocol.  Circles represent
  states, arrow represent transitions.  Arrow are labeled
  \texttt{input / response}.  The \texttt{input} is either a processor
  action (i.e., \texttt{load} or \texttt{store}) or a bus action
  (i.e., \texttt{bus-read} or \texttt{bus-read-X}).  The responses are
  bus actions (i.e., \texttt{bus-read}, \texttt{bus-read-X}, or
  \texttt{write-back}).}
  \figlabel{mesi}
\end{figure}

The MESI protocol assumes that the system contains write-back caches.
It maintains two extra bits of data for every line in the cache to
determine the state of that line of the cache.  \figref{mesi} presents
the state transition protocol enforced by the algorithm.  There are
four possible states:
\begin{itemize}
  \item \emph{[M]odified}: No other cache in the system contains this
  line in its cache in the M, E, or S state.  The cache contains a new
  value that needs to be written-back to main memory.

  \item \emph{[E]xclusive}: No other cache in the system contains this
  line in its cache in the M, E, or S state.  The cache contains the same
  value as main memory.

  \item \emph{[S]hared}: No other cache in the system contains this
  line in its cache in the M or E state.

  \item \emph{[I]nvalid}: The cache line contains no data.
\end{itemize}
%%
\renewcommand{\thefootnote}{\fnsymbol{footnote}}
\footnotetext[1]{The response \texttt{bus-read$^*$} indicates that the
bus action resulted in an exclusive read.}
\renewcommand{\thefootnote}{\arabic{footnote}}
%%
A \texttt{load} operation can take place if the appropriate cache line
is in the M, E, or S state.  That is, a processor can read the data
whenever it is available in the cache.  A \texttt{store} operation can
take only only if the appropriate cache line is in the E or S state.
That is, a processor can only modify the memory when it it is the only
cache maintaining a copy of the data.  The operations are implemented
as follows:
\begin{itemize}
\item $\texttt{load}(x)_i$: If processor $i$ does not have the
  location $x$ cached in the M, E, or S state, it sends a message on
  the bus requesting the location $x$ in the S state.  (This is
  notated in \figref{mesi} as a \emph{bus-read}.)  Since the message
  is broadcast on the bus, all other processors see this message.  If
  some processor, $j$, sees this message and discovers that it has
  cached $x$ in the M state, then it writes $x$ back to memory and
  sets the state of $x$ to S.  (This is notated in \figref{mesi} as
  \emph{write-back}.)  If processor $i$ sees the value written back to
  memory, it takes the value written and stores it in its own cache in
  the S state.  (This is referred to as ``snooping'' the bus.)  If no
  processor writes back a value to memory, then the memory module
  sends a message back to processor $i$ indicating the current value
  of $x$, Processor $i$ then caches the value of $x$ and sets the
  state to S.

\item $\texttt{store}(x,v)_i$: If processor $i$ has location $x$ in
  the E state, it sets the state to M and modifies the cached copy,
  setting it to $v$.  If the state of $x$ is already M, then again $i$
  can simply modify the cached copy.  If $x$ is in the S or I state,
  then $i$ broadcasts a message on the bus requesting exclusive access
  to $x$.  (This is notated in \figref{mesi} as a \emph{bus-read-X}.)
  Every processor that has $x$ in the S or E state sets the state of
  the cache line to I.  If some processor $j$ has $x$ in the M state,
  then $j$ writes $x$ back to memory.  If no processor writes the
  value back to memory, then the main memory broadcasts the value of
  $x$ on the bus.  Either way, processor $i$ sees the value of $x$ on
  the bus and updates its cache, setting the state to M.  Processor
  $i$ can then modify the value to $v$ directly in the cache.
\end{itemize}
Further details of the MESI protocol can be found in \figref{mesi}.
Consider the following example, where $*X$ and $*Y$ are initially
zero:
\begin{example}
\xmpllabel{cache-mesi}
$$
\begin{array}{lcc|ccl}
\\
& P_0 &&& P_1 & \\
&&&&\\
(1)& &&& R_1 = *Y &\\
(2) & *X = 1 &&&& \\
(3) & *Y = 1 &&&& \\
(4) & &&& R_1 = *Y & \\
\end{array}
$$
\end{example}
In line~1, $P_1$ sets $R_1$ to zero, and sets the cache-line
containing $Y$ to S or E.  In line~2, processor $P_0$ updates the
value of $X$, and as a result the cache-line containing $X$ is set to
the M state.  In line~3, processor $P_0$ similarly wants to update the
value of $Y$.  However, $P_0$ first has to obtain exclusive access to
that cache-line.  Processor $P_0$ therefore broadcasts on the bus a
message requesting exclusive access to $Y$.  When processor $P_1$ sees
this request, it broadcasts its value of $Y$, and sets the state to I.
Processor $P_0$ records this value of $Y$ and sets the state to M.  It
then modifies the value directly in the cache.  In line~4, processor
$P_1$ wants to read $Y$.  As a result, $P_1$ broadcasts a request for
shared access to $Y$.  Processor $P_0$ sees this request, and writes
the value of $Y$ back to main memory, setting its state locally to S.
Processor $P_1$ snoops the value, and records the value of $Y$ in its
cache, setting it to state S.
  
\end{guestlecture}
\theend
Aura
Rayuela
Julio Cortazar

Gabriel Garcia Marquez