\documentclass[11pt]{article}
\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{epsfig}
\usepackage{graphicx}

\newcommand{\eps}{\varepsilon}


\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf 6.851: Advanced Data Structures } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}

% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex
%\renewcommand{\baselinestretch}{1.25}

\begin{document}

\lecture{12 --- March 16, 2010}{Spring 2010}
    {Prof.\ Erik Demaine}{Haitao Mao}

\section{Overview}

In the last lecture we covered the round elimination technique and lower bounds on the static predecessor problem.

In this lecture we cover the signature sort algorithm for sorting large integers in linear time.

\section{Introduction}
Thorup \cite{sort} showed that if we can sort $n$ $w$-bit integers in $O(nS(n,w))$, then we have a priority queue that can support the insertion, deletion, and find minimum operations in $O(S(n,w))$. To get a constant time priority queue, we need linear time sorting, but whether we can get this is still an open problem. Following is a list of results outlining the current progress on this problem. 

\begin{itemize}
\item Comparison model: $O(n\lg n)$

\item Counting sort: $O(n+2^w)$

\item Radix sort: $O(n \cdot \frac{w}{\lg{n}})$ 

\item van Emde Boas: $O(n\lg{w})$, improved to $O(n \lg
  \frac{w}{\lg{n}})$ (see \cite{kr}).

\item Signature sort: linear when $w=\Omega(\lg^{2+\eps}n)$ (see \cite{ahnr}).

\item Han \cite{h}: $O(n\lg\lg{n})$ deterministic, AC$^0$ RAM.

\item Han and Thorup: $O(n\sqrt{\lg\lg{n}})$
  randomized, improved to $O(n \sqrt{\lg\frac{w}{\lg
  n}})$ (see \cite{ht} and \cite{kr})).
\end{itemize}

Today, we will focus entirely on the details of the signature sort. This algorithm works whenever $w=\Omega(\log^{2+\eps}n)$. Radix sort, which we should already know, works for smaller values of $w$, namely when $w=O(\log n)$. For all other values of $w$ and $n$, it is open whether we can sort in linear time. We previously covered the van Emde Boas tree, which allows for $O(n \log \log n)$ sorting whenever $w=\log^{O(1)} n$. The best we have done in the general case is a randomized algorithm in $O(n \sqrt{\log\frac{w}{\log n}})$ time by Han, Thorup, Kirkpatrick, and Reisch.

\section{Sorting for $w=\Omega(\log^{2+\eps}n)$}

The signature sort was developed in 1998 by Andersson, Hagerup, Nilsson, and Raman \cite{ahnr}. It sorts $n$ $w$-bit integers in $O(n)$ time when $w=\Omega(\log^{2+\eps}n)$ for some $\eps>0$. This is a pretty complicated sort, so we will build the algorithm from the ground up. First, we give an algorithm for sorting bitonic sequences using methods from parallel computing. Second, we show how to merge two words of $k \le \log n \log \log n$ elements in $O(\log k)$ time. Third, using this merge algorithm, we create a variant of mergesort called $\emph{packed sorting}$, which sorts $n$ $b$-bit integers in $O(n)$ time when $w \ge 2(b+1) \log n \log\log n$. Fourth, we use our packed sorting algorithm to build signature sort. Five.

\subsection{Bitonic Sequences}
A \emph{bitonic sequence} is a sequence such that when examined cyclically, has only one local minimum and one local maximum, so it can be broken into a monotically increasing sequence and a monotonically decreasing sequence. 

\begin{center}
  \includegraphics[scale=0.25]{diagram4.png} \\
{\small Cyclic shifts preserve the binoticity of a sequence.}
\end{center}

To sort a bitonic sequence btcseq, we run the following algorithm. Assume $n=\text{len(btcseq)}$ is even.
\begin{verbatim}
btcsort(btcseq):
  for i from 0 to n/2-1:
    if btcseq[i]>btcseq[i+n/2]:
   	  swap(btcseq[i], btcseq[i+n/2])
  btcsort(btcseq[0:n/2-1])
  btcsort(btcseq[n/2:n-1])
\end{verbatim}
For more information about sorting bitonic sequences, including a proof
of the correctness of this algorithm, see \cite[Section 27.3]{clrs}.

\subsection{Logarithmic Merge Operation}
The next step is to merge two sorted words, each containing $k$ $b$-bit elements.  First, we concatenate the first word with the reverse of the second word, getting a bitonic sequence. To efficiently reverse a word, we mask out the leftmost $\frac{k}{2}$ elements and shift them right by $\frac{k}{2} b$, then mask out the rightmost $\frac{k}{2}$ elements and shift them left by $\frac{k}{2}b$. Taking the OR of the two resulting words leaves us with the original word with the left and right halves swapped. We can now recurse on the left and right halves of the word, giving us the recursion $T(n) = T(\frac{k}{2}) + O(1)$, so the whole algorithm takes $T(k) = O(\log k)$ time. The two words may now be concatenated by shifting the first word left by $kb$ and taking its OR with the second word. The key here is to perform each level of the recursion in parallel, so that each level takes the same amount of time. 

\begin{center}  
\includegraphics[scale=0.18]{diagram5.png}\\
{\small The first two steps in the recursion for reversing a list.}
\end{center}

All that remains is to run the bitonic sorting algorithm on the elements in our new word.  To do so, we must divide the elements in two halves and swap corresponding pairs of elements which are out of order. Then we can recurse on the first and second halves in parallel, once again giving us the recursion $T(n) = 2T(\frac{k}{2}) + O(1) \Rightarrow T(k) = O(\log k)$ time.  Thus we need a constant-time operation which will perform the desired swapping.

\begin{center}
  \includegraphics[scale=0.18]{diagram6.png} \\
{\small Parallel swap operation in bitonic sorting.}
\end{center}
Assume we have an extra 0 bit before each element packed into the word. We use this spare bit to help bitmask our word. We will mask the left half of the elements and set this extra bit to 1 for each element, then mask the right half of the elements and shift them left by $\frac{k}{2}b$. After we subtract the second word from the first, a 1 will appear in the extra bit iff the element in the corresponding position of the left half is greater than the element in the right half.  Thus we can mask the extra bits, shift the word right by $b-1$ bits, and subtract it from itself, resulting in a word which will mask all the elements of the
right half which belong in the left half and vice versa. From this, we use bit shifts, OR, and negation to get our sorted list. See the diagram for clarfication; the process is fairly straightforward.

Using these bit tricks, we end up with our desired constant time all-swap operation. Since the bitonic sorting algorithm has a recursion depth of $O(\log k)$, our merge operation ends up with that time complexity.

\subsection{Packed Sorting} \label{sec:packed}

Packed sorting \cite{ah} sorts $n$ $b$-bits integers in $O(n)$ time for a word size of $w \ge 2(b+1) \log{n}\log\log{n}$. This bound for $w$ allows us to pack $k = \log{n} \log\log{n}$ elements into one word, leaving a zero bit in front of each integer, and $\frac{w}{2}$ zero bits at the beginning of the word.

\begin{center}
  \includegraphics[scale=0.2]{diagram3.png} \\
{\small Structure for packing $b$-bit integers into a $w$-bit word.}
\end{center}

To sort these elements using packed sorting, we build up merge operations as follows:
\begin{enumerate}
\item Merge two sorted words in $O(\lg{k})$ time, as shown in the previous section.

\item Merge sort on $k$ elements with (1) as the merge operation. This sort has the recursion $T(k)=2T(\frac{k}{2})+O(\lg{k}) \Rightarrow T(k) = O(k)$.

\item Merge two sorted lists of $r$ sorted words into one sort list of $2r$ sorted words. This is done in the same manner as in a standard merge sort, except we use (1) to speed up the operation. We merge the lists one word at a time using (1). The small values are small overall, so we output them; the large values we keep in the list. Thus, the operation takes $O(r\log{k})$ overall.

\item Merge sort all of the words with (3) as the merge operation and (2) as the base case. This sort has the recursion $T(n)=2T(\frac{n}{2})+O(\frac{n}{k}\log{k})$ and the base case $T(k)=O(k)$. Our recursion depth is $O(\log n)$, and each level takes $O(\frac{n}{k}\lg{k}) = O(\frac{n}{\lg n})$ time, so they contribute a cost of $O(n)$. There are also $\frac{n}{k}$ base cases, each taking $O(k)$ time, giving a total runtime of $O(n)$, as desired.
\end{enumerate}


\subsection{Signature Sort Algorithm}

We use our packed sorting algorithm to build our seven step signature sort. We assume that $w \ge \log^{2+\eps} n \log \log n$. 
\begin{enumerate}
\item Break each integer into $\lg^{\eps}n$ equal-size chunks. (Note the distinguishment from a fusion tree, which has chunks of size $\lg^{\eps}n$)

\item Replace each chunk by a $O(\lg{n})$-bit hash (static perfect hashing is fine). By doing this, we end up with $n$ $O(\lg^{1+\eps}n)$-bit \emph{signatures}. One way we can hash is to multiply by some random value $x$, and then mask out the hash keys. This will allow us to hash in linear time. Now, our hash does not preserve order, but the important thing is that it does preserve identity.

\item Sort the signatures in linear time with packed sorting, shown above.

\item Now we want to rescue the identities of the signatures. Build a compressed trie over the signatures, so that an inorder traversal of the trie gives us our signatures in sorted order. 


\begin{center}
  \includegraphics[scale=0.18]{diagram1.png} \\
{\small A trie for storing signatures. Edge represent hash values of chunks; leaves represent possible signature values.}
\end{center}


To do this in linear time, we add the signatures in order from left to right. Since we are in the word RAM, we can compute the LCP with $(i-1)^{\text{st}}$ signature by taking the most significant 1 bit of the XOR. Then, we walk up the tree to the appropriate node, and charge the walk to the decrease in the rightmost path length of the trie. The creation of the new branch is constant time, so we get linear time overall. This process is similar to the creation of a Cartesian tree.

\item Recursively sort the edges of each node in the trie based on their actual values. This is a recursion on (node ID, actual chunk, edge index), which takes up $(O(\log{n}), O(\frac{w}{\log^{\eps} n}), O(\log{n}))$ space. The edge indices are in there to keep track of the permutation. After a constant $\frac1{\eps}+1$ levels of recursion, we will have $b=O(\log n + \frac{w}{\log^{1+\eps} n}) = O(\frac{w}{\log n \log \log n})$, so we can use packed sorting as the base case of the recursion.

\item Permute children of each node.

\item Do an inorder traversal of the trie to get the desired sorted list from the leaves.
\end{enumerate}

\subsection{We have our signature sort!}

\bibliographystyle{alpha}
\begin{thebibliography}{77}

\bibitem{ah}
Susanne Albers, Torben Hagerup:
\emph{Improved Parallel Integer Sorting without Concurrent Writing},
Inf.~Comput. 136(1): 25-51, 1997.

\bibitem{ahnr}
A. Andersson, T. Hagerup, S. Nilsson, R. Raman,
\emph{Sorting in Linear Time?},
J. Comput. Syst. Sci. 57(1): 74-93, 1998.

\bibitem{clrs}
T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein:
\emph{Introduction to Algorithms}, Second Edition,
The MIT Press and McGraw-Hill Book Company 2001.

\bibitem{h}
Y. Han:
\emph{Deterministic Sorting in $O(n\log\log{n})$ Time and Linear Space},
J. Algorithms 50(1): 96-105, 2004.

\bibitem{ht}
Y. Han, M. Thorup:
\emph{Integer Sorting in $O(n\sqrt{\log\log{n}})$
  Expected Time and Linear Space},
FOCS 2002: 135-144.

\bibitem{kr}
D.G. Kirkpatrick, S. Reisch:
\emph{Upper Bounds for Sorting Integers on Random Access Machines},
Theoretical Computer Science 28: 263-276 (1984).

\bibitem{sort} M. Thorup: \emph{Equivalence between Priority Queues and
  Sorting}. FOCS 2002: 125-134 (2002).


\end{thebibliography}

\end{document}
