% Begin your document as any latex file
%\documentclass{article}
% Use package 6895
%\usepackage{6895}
%\usepackage{epsfig}
\begin{lecture}{10}{Cilk Implementation}{Kunal Agrawal and Xie Yong}

\begin{summary}

  \sumitem{Compiling Cilk}{This section gives some of the implementation details and
  clever tricks used in compilation of Cilk.}
    \sumitem{Deque Protocols} {Description of the TH and THE protocols Cilk uses for
  managing the deques.} 
\end{summary}


\section{Compiling Cilk}

In this section, we will see some implementation details of conversion of
Cilk source to C post-source. We will subsequently refer to this conversion  as compiling
the Cilk program. We will also see the reasons behind choosing to implement the 
conversion in this particular way.  

\subsubsection*{The Cilk compilation path}

The Cilk compilation path is shown in \figref{cilkCompPath}. The Cilk source is run
through the \defn{cilk2c} translator which converts the Cilk source to C 
post-source. This C post-source is compiled using \texttt{gcc} and the object code is
linked with the \defn{Cilk run-time system} to produce the native binary code. 

The software components that needed to be built to implement the Cilk system were
\texttt{cilk2c} and \texttt{Cilk run-time system}. The \texttt{cilk2c} translator
translates the Cilk commands like \texttt{spawn} and \texttt{sync} to C instructions that
call the run-time system routines and do the parallel bookkeeping. The translation process
is simple, because it only needs to transform Cilk keywords, leaving the C instructions as they
are. In this section, we shall discuss some implementation details of the \texttt{cilk2c}
translator. Some of the mechanisms used in the \texttt{Cilk run-time system} will be
discussed in the next section.


\begin{figure}[h]
\begin{center}
\psfig{figure=10/cilkCompPath.eps,width=4in}
\end{center}
\vspace{-6pt}
\caption{\small The compilation path for Cilk programs.}
\figlabel{cilkCompPath}
\end{figure}

\subsubsection*{Work-First Principle}

Cilk's greedy scheduler operates on the ``work-first'' principle: it is better to amortize
overheads against the critical path than against work. Recall that for a job with
$T_1$ work and $T_{\infty}$ critical path length, the Graham Brent greedy scheduler
achieves the following bound on time taken to complete the job on $P$ processors:

\begin{equation}
T_P \le \frac{T_1}{P} + T_{\infty}\nonumber \ .
\inequlabel{ineq:1}
\end{equation}
Recall also that this bound is within a factor of 2 of optimal. Thus, a greedy scheduler
with comparable performance should exhibit a running time something like :

\begin{equation}
T_P \le C_1\frac{T_1}{P} + C_{\infty}T_{\infty} \ .
\inequlabel{ineq:2}
\end{equation}
Where $C_1$ and $C_{\infty}$ are constants due to the overhead in the
system. For parallel programs for which we can expect linear
speedups, we have $P \ll T_1/T_{\infty}$. Thus, the first term in \inequref{ineq:1}
is 
much larger than the second term. Consequently, in order to optimize the performance of
the scheduler, we should try to move the overhead from $C_1$ to $C_{\infty}$.

%\begin{figure}
%\centering
%\epsfig{figure=cilkCompPath.eps, bbllx=10, bblly=500, bburx=400, bbury=750, width=\textwidth}
%\caption{\small The compilation path for cilk programs.}
%\figlabel{cilkCompPath}
%\end{figure}

\subsubsection*{Shadow Stack}

Cilk maintains a
\defn{shadow stack} made up of \defn{frames} each of which holds all the live local
variables for a single procedure. Since \texttt{cilk2c} has no control over the C stack,
the code to store the local variables on the stack may duplicate some work done by the C
run-time system. The overhead incurred due to this additional storage is small.
In addition to storing the local variables, each frame has two extra locations
\defn{entry} and \defn{join}. The \defn{entry} location holds the point at which the
procedure should start executing when it is resumed in case it gets stolen and the
\defn{join} location holds the number of currently spawned children of the procedure. 

\subsubsection*{Cilk's Compiler Strategy}

The \texttt{cilk2c} converter generates two clones of each function: a \defn{fast clone}
and a \defn{slow clone}. The \defn{fast clone} is the serial common case code and it is
almost the same as what a C function looks like. The \defn{slow clone} is the clone
with the parallel book-keeping.\footnote{The slow clone is only 20-30 \%  slower
  than the fast clone. Its called the slow clone just to contrast it with the fast
  clone.}

Whenever a Cilk procedure is spawned, the fast clone is invoked after saving local live
variables on the shadow stack. If a thread
is stolen however, the slow clone is resumed at the other processor. Thus, a slow clone of a
procedure is only executed if and when this procedure is stolen from one processor to
another. which means that most of the overhead of executing the slow clone goes in the
$C_{\infty}$ term in \inequref{ineq:2}.

As can be seen in \figref{dequeView} all the procedures on a processor's deque are
fast clones, except possibly the procedure on the top of the deque. Moreover, a procedure can have more than one outstanding child only if it has been stolen from
some other processor in which case, it would be a slow clone. Thus, fast clones
can have at most one outstanding child at any time, and this child is below them on the
processor's deque. Thus, when a fast clone is being executed, it has no outstanding
children. 

\begin{figure}[h]
\begin{center}
\psfig{figure=10/dequeView.eps,width=1in}
\end{center}
\vspace{-6pt}
\caption{\small Shadow Stack view of any processor}
\figlabel{dequeView}
\end{figure}

\subsubsection*{Fibonacci Example}

We can see how \texttt{cilk2c} compiles programs by considering the following Cilk program
for computing the Fibonacci numbers: 

\begin{verbatim}
1: cilk int proc{fib}(int n) { 
2:   if (n<2) return (n);
3:   else {
4:     int x,y;
5:     x = spawn fib(n-1);
6:     y = spawn fib(n-2);
7:     sync;
8:     return(x+y);
9:   }
10:}
\end{verbatim}
This procedure is subsequently referred to as \texttt{fib} program:
 
\subsubsection*{Compiling \texttt{spawn} in the fast clone}

\figref{fastspawn} shows the compilation of \texttt{spawn} in line 5 of the \texttt{fib} program. In the
fast clone, whenever a procedure spawns a child, all the live local variables of 
the parent are stored on a shadow stack. The procedure also saves the appropriate \texttt{entry}
location, in this
case entry number 1 on the shadow stack and increments the \texttt{join} counter.

After the \texttt{return} from the spawned function, a check is made to see if the parent is
still on this processor's deque. If it is, then the parent executes just as it would in
C. If the parent has been stolen, the returned value is updated in the shadow
stack and the \texttt{join} counter is decremented by 1. If the value in the \texttt{join} counter
is now $0$ and the parent is stalled on a \texttt{sync}, then this child was the last of
it's siblings to complete in which case, this processor begins executing the parent. If
this child has executing siblings or the parent is not stalled, then this processor's control
goes to the scheduler, and it starts work-stealing.

\begin{figure}[h]
\begin{center}
\psfig{figure=10/fastspawn.eps,width=4in}
\end{center}
\vspace{-6pt}
\caption{\small The compilation of \texttt{spawn} in the fast clone.}
\figlabel{fastspawn}
\end{figure}


%\begin{figure}
%\centering \includegraphics[scale=0.5]{fastspawn.eps}
%\caption{\small The compilation for spawn in the fast clone.}
%\label{cilkCompPath}
%\end{figure}

%\begin{figure}
%\begin{center}
%\epsfig{figure=fastspawn.eps,bbllx=19,bblly=507,bburx=404,bbury=742}
%\end{center}
%\vspace{-6pt}
%\caption{\small The compilation of spawn in the fast clone.}
%\figlabel{fastspawn}
%\end{figure}

\subsubsection*{Compiling \texttt{sync} in the fast clone}

As shown in \figref{fastsync}, in the fast clone, \texttt{sync} statement compiles to a
\texttt{no-op}. This is because a fast clone has no outstanding children when it is executing.

\begin{figure}[h]
\begin{center}
\psfig{figure=10/fastsync.eps,width=4in}
\end{center}
\vspace{-6pt}
\caption{\small The compilation of \texttt{sync} in the fast clone.}
\figlabel{fastsync}
\end{figure}


\subsubsection*{Compiling \texttt{spawn} in a slow clone}
\figref{slowspawn} shows the compilation of the \texttt{spawn} statement on line 5 in the slow
clone. The slow clone of a procedure is executed whenever it is stolen from a
processor and 
resumed on another processor. Thus, in a slow clone, we need entry points to resume the
thread after each \texttt{spawn}. At the beginning of the procedure, the slow clone reads
the \texttt{entry} location and a \texttt{switch} statement dispatches control to the
appropriate entry point as shown in \figref{slowspawn}. The live variables are then restored from the shadow stack and the execution of the
program continues. 

As shown in \figref{slowspawn}, the segment of the
program that restores the local variables is enclosed in a false conditional. This construct
prevents the program from restoring the variables when it reaches this point during the
normal sequential execution, that is when it is not being resumed after a steal. Thus, this
overhead of restoring variables is only incurred when this procedure has been stolen and
can be amortized against the $T_{\infty}$ term in \inequref{ineq:1}. 

\begin{figure}[h]
\begin{center}
\psfig{figure=10/slowspawn.eps,width=4in}
\end{center}
\vspace{-6pt}
\caption{\small The compilation of \texttt{spawn} in the slow clone.}
\figlabel{slowspawn}
\end{figure}

\subsubsection*{Compiling \texttt{sync} in the slow clone}

When the slow clone of a procedure reaches a \texttt{sync} point, it checks whether it has any
outstanding children by examining the \texttt{join} counter in its shadow stack. If
this counter is 0, then there are no outstanding children and the procedure continues
executing. If there are outstanding children, then the processor starts
work-stealing. \figref{slowsync} shows the example for the \texttt{fib} program.

\begin{figure}[h]
\begin{center}
\psfig{figure=10/slowsync.eps,width=4in}
\end{center}
\vspace{-6pt}
\caption{\small The compilation of \texttt{sync} in the slow clone.}
\figlabel{slowsync}
\end{figure}

\subsubsection*{Breakdown of Work Overhead}

The efficiency of Cilk's compiling strategy can be tested by using the Fibonacci number
calculation benchmark. The \figref{overhead} shows the performance on various
single-processors machines. The breakdown of time spent on doing the various tasks is also
shown in the figure. 

According to these benchmarks, the Cilk program is about 6 times
worse than the equivalent C program in the worst case.
This overhead might sound large, but the \texttt{fib} program has very little
computation. Every procedure just does two decrements and one addition. On the other hand it
has many spawns. In general, most programs are likely to have much more computation
than \texttt{fib} does and in that case the spawn overhead will be amortized against the
computation, that is, for most programs, we expect that 
\begin{equation}
Number~of~spawns \ll Number~of~C~function~calls \ll
Number~of~total~instructions~executed. \nonumber  
\end{equation}

The fact that \textit{fib} is only 6 times slower in Cilk than in C is actually encouraging.
Most parallel programs are actually as fast in Cilk as they are in C. In
practice, for most reasonable programs, the constant $C_1$ in \inequref{ineq:1} is close to
1.  

\begin{figure}
\centering \includegraphics[scale=0.65]{10/overhead.eps}
\caption{\small The breakdown of work overhead, benchmarked using $fib$ on one processor.}
\figlabel{overhead}
\end{figure}


\section{Deque Protocols}

In Cilk, each processor, called a \defn{worker}, maintains a \defn{ready deque}
(doubly-ended queue) of ready Cilk procedures. Each deque has two ends, a \defn{head} and
a \defn{tail}, from which procedures can be added or removed. The worker treats the deque
as a stack, pushing and popping procedures from the \defn{tail} of the deque. If a worker
runs out of work, it becomes a \defn{thief} and tries to steal work at the head of the
ready deque of a \defn{victim} processor. Work-stealing is implemented through
shared-memory, and the thief and victim operate directly on the victim's ready deque. The
main issue is how to resolve the potential race condition that may arise because of the
concurrent accesses while at the same time, achieving good efficiency.  

The protocol that Cilk adopts for managing deque make uses of three atomic shared
variables \textsf{T}, \textsf{H}, and \textsf{E}, and it is called the \textsf{THE}
protocol. We first present a simplified protocol that uses only two shared variables
\textsf{T} and \textsf{H} designating the tail and the head of the deque,
respectively. Later, we extend the protocol with a third variable \textsf{E} that allows
exceptions to be signaled to a worker. The exception mechanism is used to implement Cilk's \texttt{abort}
statement.

\subsection{TH Protocol}

The \textsf{TH} protocol is an almost non-blocking protocol for managing deque, and the
pseudocode is shown in \figref{TH}. The main idea is to minimize work overhead by moving
the costs from the worker to the thief. To arbitrate among different thieves attempting to
steal from the same victim, a hardware lock is used, and the overhead caused by the lock
can be amortized against the critical path.  

\begin{figure}
\centering
\begin{minipage}{.45\textwidth}
\begin{verbatim}
1: push() {
2:   T++;
3: }

4: pop() {
5:    T--;
6:    if (H > T) {
7:        T++;
8:        lock(L);
9:        T--;
10:       if (H > T) {
11:            T++;
12:            unlock(L);
13:            return FAILURE;
14:        }
15:        unlock(L);
16:    }
17:    return SUCCESS;
18:}

\end{verbatim}
\centerline{\textbf{(a)}}
\end{minipage}\hspace{.05\textwidth}
\begin{minipage}{.4\textwidth}
\begin{verbatim}
1: steal() {
2:    lock(L);
3:    H++;
4:    if (H > T) {
5:        H--;
6:        unlock(L);
7:        return FAILURE;
8:    }
9:    unlock(L);
10:   return SUCCESS;
11: }
\end{verbatim}
\centerline{\textbf{(b)}}
\end{minipage}
\caption{\small \textbf{(a)} Pseudocode of the actions performed by the worker/victim in
  the \textsf{TH} protocol.  \textbf{(b)} Pseudocode of the actions performed by the thief
  in the \textsf{TH} protocol.} 
\figlabel{TH}
\end{figure}

The code assumes that the deque is implemented as an array of frames. The head and tail of the
deque are determined by two indices \textsf{T} and \textsf{H}. The index \textsf{T} points
to the first unused element in the array, and \textsf{H} points to the first frame on the
deque. Indices grow from the head towards the tail, and most of the time, $\textsf{T} \geq
\textsf{H}$. The worker pushes and pops frames by altering \textsf{T}, while the thief
only increments \textsf{H} and does not alter \textsf{T}. The lock \textsf{L} ensures that
only one thief can steal from the deque at a time. 

The \textsf{push} operation is always safe, because it does not involve any interaction
between a thief and its victim. For the \textsf{pop} operation, there are three cases,
shown in \figref{deque}. When there are many thieves, only one of them will succeed in
stealing the frame because of the locking mechanism. Thus we only need to consider the
case of one thief.  

\begin{figure}
\centering \includegraphics[scale=0.8]{10/deque.eps}
\caption{\small The three cases of the ready deque in the TH protocol. A shaded block indicates the presence of a frame at a certain position in the deque. The head and the tail are marked by \textsf{T} and \text{H}.}
\figlabel{deque}
\end{figure}

\begin{enumerate}
\item  Both the thief and the victim attempt to obtain different frames from the deque
  concurrently. In this case both are successful, and they do not interfere.
\item The deque contains only one frame. Either victim and thief will get the frame if the
  other is not making an attempt to obtain it. If both victim and thief try to get the
  frame, the protocol guarantees that at least one of them discovers that $\textsf{T} >
  \textsf{H}$. If the thief discovers that $\textsf{T} > \textsf{H}$, it restores
  \textsf{H} to its original value and retreats. If the victim discovers $\textsf{T} >
  \textsf{H}$, it will restart the protocol after acquiring the lock \textsf{L}, which
  guarantees that it will get the frame without interference from any thief unless a
  thief has already stolen the frame.
\item The deque is empty. A thief always fails to steal, and the victim also fails to pop
  the  frame. The control passes to the Cilk runtime system. 
\end{enumerate} 

The \textsf{TH} protocol contributes little to the work overhead. Pushing only involves
updating \textsf{T}. Successfully popping a frame involves only 6 operations --- $2$ memory
loads, $1$ memory store, $1$ decrement, $1$ comparison, and $1$ (predictable) conditional
branch. In the case where both thief and victim simultaneously try to grab  the same
frame, the cost incurred by the lock can be considered as part of the critical-path
overhead and does not influence the work overhead. 

\subsection{THE Protocol}

The \textsf{THE} protocol extends the \textsf{TH} protocol to support signaling of
exceptions to a worker. The index \textsf{H} in the \textsf{TH} protocol plays two roles:
it marks the head of the deque, and it marks the point that the worker cannot cross when it
pops. In the \textsf{THE} protocol, the two roles of \textsf{H} are separated into two
variables: \textsf{H}, which now only marks the head of the deque, and \textsf{E}, which
marks the point that the victim cannot cross. \figref{THE} shows the pseudocode of
the \textsf{THE} protocol. 

\begin{figure}
\centering
\begin{minipage}{.45\textwidth}
\begin{verbatim}
1: push() {
2:   T++;
3: }

4: pop() {
5:   T--;
6:   if (E > T) {
7:      T++;
8:      lock(L);
9:      T--;
10:     if (H > T) {
11:       T++;
12:       unlock(L);
13:       return FAILURE;
14:     }
15      unlock(L);
16:   }
17:   return SUCCESS;
18: }
\end{verbatim}
\centerline{\textbf{(a)}}
\end{minipage}\hspace{.05\textwidth}
\begin{minipage}{.4\textwidth}
\begin{verbatim}
1: steal() {
2:   lock(L);
3:   E++;
4:   H++
5:    if (H > T) {
6:     E--;
7:     H++;
8:     unlock(L);
9:     return FAILURE;
10:  }
11:  unlock(L);
11:  return SUCCESS;
12: }
\end{verbatim}
\centerline{\textbf{(b)}}
\end{minipage}
\caption{\small \textbf{(a)} Pseudocode of the actions performed by the worker/victim in
  the \textsf{THE} protocol.  \textbf{(b)} Pseudocode of the actions performed by the
  thief in the \textsf{THE} protocol.} 
\figlabel{THE}
\end{figure}

The variable \textsf{E} is used to signal the worker that an exception has occurred. For
example setting $\textsf{E} = \infty$ causes the worker to discover the exception at
its next pop. At this point, it returns the control to the run time system, which can
then handle the exception appropriately. Different high values of E are used for different
exceptions. This mechanism is used to implement \textsf{abort} in Cilk. The limitation
of this approach is that the worker is notified of an exception only after it
returns from a \texttt{spawn} for the first time after the exception is signaled. Thus if a procedure
does a lot of computation before returning, it might continue to compute for some time
before discovering the \texttt{abort}. Although no bounds are placed on how long it 
takes for a sub-computation to stop after the exception is signaled, the exception
mechanism seems to work well in practice. 

%%  \begin{thebibliography}
%%  \bibitem{FL98} M. Frigo and V. Luchangco, ``Computation-centric Memory
%%    Models'', \emph{Proceedings of the Tenth Annual ACM Symposium on
%%      Parallel Algorithms and Architectures'', 1998, pp. 240--249.
%% \end{thebibliography} 

%\bibliographystyle{plain}
%\bibliography{cilk}

\end{lecture}
\theend













