\documentclass[11pt]{article}
\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{epsfig}
\usepackage{psfig}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf 6.851: Advanced Data Structures } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}

% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex
%\renewcommand{\baselinestretch}{1.25}

\begin{document}

\lecture{19 --- April 15, 2010}{Spring 2010}{Prof.\ Erik Demaine}{Ye Wang}

\section{Overview}
This lecture is an interlude on time travel in data structures.  We'll consider two kinds of
time-travel (temporal) data structures, based on the two leading theories of time travel:
\begin{enumerate}
\item Branching-universe model a.k.a. persistence:
   You can change the past, but then you enter a different branch
   of the universe, and never to return to the old one.
   In some cases, though, branches might merge ("confluence").
\item Round-trip model a.k.a. retroactivity:
   You can change one thing in the past and then teleport back
   to the present and see what changed.
\end{enumerate}

In data structures, persistence is much easier to achieve than retroactivity:
in most cases, persistence can be done with tiny overhead,
whereas retroactivity can require linear overhead.
This is strong computational evidence that the second model of
time travel is impossible, while the first model (and a model
where you can go back, but not forward, in time) is conceivable computationally.

These data structural paradigms are also useful for more
mundane applications like computational geometry, but
we won't discuss those here.

\section{Persistence}
\subsection{4 Levels of Persistence}
We begin by describing the levels of desired persistence. With data structure persistence,
we would like to keep all versions of the data structure available for updates and queries.
Each persistence level, however will vary based on where updates are allowed and how branches
and nodes are modified and created.

\begin{enumerate}
\item{Partial Persistence} --
In this persistence model, we may query any previous version of the data structure,
but we may only update the latest version. This implies a linear ordering among the versions.

\item{Full Persistence} --
In this model, both updates and queries are allowed on any version of the data structure.
The versions here form of a branching tree.

\item{Confluent Persistence} --
In this model, we use combinators to combine input of more than one previous versions to output a new single version.
Rather than a branching tree, combinations of versions induce a DAG(direct acyclic graph) structure on the version graph.

\item{Functional Persistence} --
This model takes its name from functional programming where objects are immutable.
The versions in this model are likewise immutable, so revisions do not alter the
existing nodes in the data structure, but create new ones instead. Okasaki discusses
these as well as other functional data structures in his book \cite{okasaki}.
\end{enumerate}

Each of the succeeding levels of persistence imply the preceding ones.
That is, Functional Persistence $\implies$ Confluent Persistence $\implies$ Full Persistence
$\implies$ Partial Persistence. Functional implies confluent because we simply use the combinators
to append a new combined version. Confluent implies full if we do not use combinators. Lastly, full
implies partial if we only update the most recent version.

\begin{figure}[h]
  \begin{center}
      \scalebox{.4}{\includegraphics[width=6in]{f1.pdf}}
  \end{center}
  \caption{\small partial persistence: versions are linearly ordered.}
  \begin{center}
    \scalebox{.4}{\includegraphics[width=6in]{f2.pdf}}
  \end{center}
  \caption{\small full persistence: versions form a branching tree.}
  \begin{center}
    \scalebox{.5}{\includegraphics[width=6in]{f3.pdf}}
  \end{center}
  \caption{\small full persistence: versions form a DAG.}
  \label{fig-label}
\end{figure}

\subsection{Partial Persistence}
This result is due to Driscoll, Sarnak, Sleator, and Tarjan \cite{dsst}.
We work within the pointer machine model and require $O(1)$ in-degree per node,
meaning that $\leq p = O(1)$ nodes point to any node. Each node stores some data
and a constant number of pointers to children, reverse pointers to parents, and
version modification data (in a ``modification box'').

To maintain partial persistence,
each node stores a reverse pointer to the parent node representing the most recent version
of the data structure. A modification can be thought of as the tuple $(time, field, value)$,
consisting of the time of the modification, the field being modified, and the new value. We
allow $\leq c+1$ modifications in a node, $c$ is the number of pointers per node, and is also $O(1)$.

\begin{figure}[h]
  \begin{center}
      \scalebox{.6}{\includegraphics[width=6in]{f8.pdf}}
  \end{center}
  \caption{\small nodes in the pointer machine.}
  \label{fig-label}
\end{figure}

To read files and check for nodes take $O(1)$ time.
An update on a field at some time $t$ can come across two cases:
\begin{itemize}
\item{The node has space} --
We can simply add the modification $(t, field, value)$ to the modification box.
All subsequent accesses of this modified node will check the modification box to
override any initial data stored in the node.
\item{The node is full} -- We make a copy of the node, but using only the latest
values. That is, we overwrite one of the node’s fields with the value that was
stored in the modification box, and make the modification box of the new node empty.
We propagate this change up to node's ancestors as follows: each ancestor makes a
modification to change its child pointer to the newly created node. If that ancestor's
modification box happens to be full, then we copy that node and propagate up. These changes
propagate until we stop at the root.
\end{itemize}

Using $\Phi = $ the number of full nodes in the latest version. Since there are at most $c$ pointers to point,
there's at least one update spot in the modification box. Hence $\Phi$ can go down by $1$.
So we can prove constant time amortized updates. Further study by Brodal \cite{brodal} has shown it to also be $O(1)$ in the worst case.

\subsection{Full Persistence}
This result is also due to \cite{dsst}. We again assume a pointer machine with $\leq p$ incoming
pointers per node. We can linearize the tree of versions via the in-order traversal marking
the begin and end time of each version.

\begin{figure}[h]
  \begin{center}
        \scalebox{.6}{\includegraphics{f4.pdf}}
  \end{center}
  \caption{\small in-order traversal}
  \label{fig-label}
\end{figure}

We can store the begin and end times in an order-maintenance data structure by Deitz and Sleator \cite{dietz}, discussed in lecture 17.The
data structure can do the following two operations in $O(1)$ time
\begin{itemize}
\item insert time before or after a specified time
\item compare if time $s$ precede time $t$. In our case, answer whether version $v$ is an ancestor of version $v'$.
\end{itemize}
By the order-maintenance data structure, we can tell which modifications apply to the desired version.
For each node, we store $p$ back pointers, and allow up to $2(p+c+1)$ modifications. As defined above, $c$ is the pointers per node.

When a node is full, we split it into two nodes. But each is roughly half full (like B-tree node), then
recursively update pointers and back pointers to this node. So even if
all its $c$ pointers move and all $p$ back pointers move, the half full node - with $(p+c+1)$ modifications - is still not full.

Again define $\Phi$ as the number of full nodes, after each split, it decreases at least one. We get $O(1)$ amortized cost. A related open question is

{\bf OPEN:} Can we support $O(1)$ worst-case full persistence?

Each data structure node is represented by a linked list of nodes,
and there's a second phase of the operation to update reverse pointers.
Deitz developed a fully persistent array that can be achieved in $O(\lg \lg n)\ \times$ overhead in the word RAM model \cite{dietz1}.

{\bf OPEN:} Is there a matching lower bound for both full and partial persistence? This question may have been solved by P$\breve{a}$trascu et al. (unpublished).

\subsection{Confluent Persistence}

Confluent persistence has been explored in functional data structures \cite{okasaki}.
Deques (double ended queues allowing stack and queue operations) with concatenation can be
done in constant time per operation (Kaplan, Okasaki, and Tarjan \cite{kot}). We can create implicity exponential
deques in polynomial time by recursively concatenating a node with itself. The general transformation due to Fiat and Kaplan \cite{fiat} is as follows:
\begin{itemize}
\item $d(v)$ = depth of node $v$ in version DAG
\item $e(v) = 1 + lg($ number of paths root to $v$)
\item overhead: $\lg($ number of updates $) + max_v(e(v))$
\item poor when $e(v) = 2^{u}$ where $u$ is the number of updates. This is still exponentially better than the non-persistent model.
\end{itemize}

\begin{figure}[h]
  \begin{center}
    \scalebox{.6}{\includegraphics{f5.pdf}}
  \end{center}
  \caption{\small An example of $e(v)$ being exponential to the number of updates.}
  \label{fig-label}
\end{figure}

One example is Tries with $O(1)$ fingers to support local navigation and subtree copy and delete \cite{dlp}.
\begin{center}
\begin{tabular}{|l|l|l|l|}
  \hline
   & \multicolumn{2}{|c|}{figure move} & modification\\
  \hline
  method & time& space & (time = space)\\
  \hline
  path copying & $\lg{\Delta}$ & $O(1)$ & depth\\
  \hline
  1. functional & $\lg{\Delta}$ & $\lg \Delta$ & $\lg \Delta$\\
  \hline
  1. confluent & $\lg{\lg{\Delta}}$ & $\lg{\lg{\Delta}}$ & $\lg{\Delta}$\\
  \hline
  2. functional & $\lg{\Delta}$ & $O(1)$ & $\lg{n}$\\
  \hline
  2. confluent & $\lg{\lg{\Delta}}$ & $O(1)$ & $\lg{n}$\\
  \hline
\end{tabular}
\end{center}

The 1. functional and confluent persistence data structures are cheap with local modifications. The 2. data structures are globally balanced.

{\bf OPEN:} Can we do better transformation with $O(1)$ fingers? separations? functional transformations?

{\bf OPEN:} When can you do better? Lists with split and concatenate? Trees? General pointer machine? Array with cut and paste? Special DAGs? Others?

\section{Retroactivity}
Retroactive data structures store a data structure under a sequence of operations. We would like to be able to go back in time, change an operation, and then observe the effects of that change in the current state of the data structure. The induced timeline is linear. Much of this work is due to Demaine, Iacono, and Langerman \cite{dil}.

The allowed operations are
\begin{itemize}
\item{Insert($t, x$)} -- Retroactively do operation $x$ at time $t$
\item{Delete($t$)} -- Retroactively undo operation at time $t$	
\item{Query($t, x$)} -- Do query $x$ at time $t$
\end{itemize}

We define partial retroactivity as allowing queries at the present time and full retroactivity as allowing queries at any time.	Some cases of partial retroactivity are easy to implement. If updates are commutative: $x \circ y = y \circ x$, then we can support retroactive insertion of operations at no additional asymptotic cost (implement $Insert(t,x)$ by executing $x$ at the present time). If updates, in addition to being commutative, are also invertible: $x \circ x^{-1} = NOP$, then we can support partial retroactivity at no additional asymptotic cost ($Insert(t,x)$ by executing $x$ at the present time and $Delete(t)$ by executing $x^{-1}$ at the present time where $x^{-1}$ is the inverse of the operation at time $t$).

\subsection{The Rollback Method}
There are a few general transformations that we can prove bounds for. One is the rollback method, in which we perform a retroactive operation at $r$ time units in the past. We can do this with a factor of $r$ overhead by keeping a log of all updates done to the DS such that every change can be reversed. The rollback method needs an $\Omega(r)$ lower bound. To see this, we examine a data structure that maintains two values, $X$ and $Y$, both initialized to 0. We can perform the following operations on our data structure:
\begin{itemize}
	\item{set$X(x)$} -- Sets $X \leftarrow x$
	\item{add$Y(\Delta)$} -- Sets $Y \leftarrow Y + \Delta$
	\item{mult$XY$()} -- Sets $Y \leftarrow X \cdot Y$
  \item{query()} -- Returns $Y$
\end{itemize}
Consider the following sequence of operations: add$Y(a_n)$, mult$XY$(), add$Y(a_{n-1})$, mult$XY$(), ...,  add$Y(a_0)$. This is Horner's rule for evaluating the polynomial $p(x) = \sum_{i=0}^n a_i x^i$. Now suppose we perform $Insert(t=0,$ set$X(x_0))$ to change the $x$ value of the evaluated polynomial $x_0$. Frandsena, Hansen, and Miltersen in 2001 showed that evaluating a polynomial of degree $n$ requires $\Omega(n)$ time over any field, independent of any preprocessing of the $a_i$s. This holds in the ``history-independent algebraic decision tree'' model, which implies the same result for the integer RAM and generalized real RAM models \cite{fhm}. This is somewhat disappointing result because it says that in the retroactive model, we can't do any persistence maintenance that's better than just going back in time, performing the new operation, and then re-executing all of the operations in our history past that point.

In the cell-probe model, Frandsena et al. also proved a lower bound of $\Omega(\sqrt{r/\lg r})$. They had a data structure that maintained $n$ words and supported arithmetic updates ($+/\cdot$). Computing a fast fourier transform takes $O(n \lg n)$ time, but changing one weights $w_i$ of the FFT needs $\Omega(\sqrt{n})$ time, from which we derive the $\Omega(\sqrt{r/\lg r})$ lower bound. An open question is whether the tightest cell-probe lower bound is $\Omega(\sqrt{r/\text{poly} \lg r})$.

\subsection{Priority Queues}
Let us turn our attention to partially retroactive priority queues. The defining features of priority queues is the delete-min() operation, which makes the set of operations on priority queues non-commutative. We can plot the status of our data structure in the plane. The $x$-axis represents time and $y$-axis represents key value. Every $insert(t,k)$ operation creates a horizontal ray that starts at point $(t,k)$ and shoots to the right. Every $delete$-$min()$ operation creates a vertical ray that starts at $(t,-\infty)$ and shoots upwards, stopping at the horizontal ray of the element it deletes. It turns the horizontal ray into a line segment with endspoints $(t,k)$ and $(d_k,k)$, where $d_k$ is the time of key $k$'s deletion. This creates nonintersecting upsidedown ``L'' shapes, where each L corresponds to an $insert$ and the $delete$-$min()$ that deletes it. Refer to Figure \ref{fig:bridges} for an animation.

Let $Q_0$ be the current state of our priority queue and $Q_t$ be its state at time $t$. We call time $t$ a \emph{bridge} if $Q_t \subseteq Q_0$.
There are four combinations of retroactive operations:

\begin{enumerate}
	\item{$Insert(t, ``insert(k)$'')} -- Insert key $k$ into $Q_t$. The resulting element we insert into $Q_0$ is the largest element that
																			 was deleted after time $t$. See Figure \ref{fig:insert}.
	\item{$Delete(t, ``delete$-$min()$'')} -- Undo the $delete$-$min$ at time $t$. This is identical to re-inserting the element that was being
																						deleted at the time of deletion (i.e. nullify the upwards $delete$-$min()$ arrow by inserting
																						the appropriate key right at that time). Thus, it is the same as the above case and we insert
																						into $Q_0$ the largest element that was deleted after time $t$.
	\item{$Insert(t, ``delete$-$min()$'')} -- Delete the minimum key at time $t$. The element we delete from $Q_0$ is the minimum value of $Q_{t'}$,
																						where $t'$ is the first bridge after $t$. This essentially pushes the bridge forward in time.
																						See Figure \ref{fig:delete}.
	\item{$Delete(t, ``insert(k)$'')} -- Undo the insertion of key $k$ at time $t$. If $k \in Q_0$ we remove it from there. If not, then we again
																			 delete the minimum value of $Q_{t'}$ where $t'$ is the first bridge after $t$. The idea is that since
																			 $k \notin Q_0$, it didn't make it to its next bridge. Therefore, removing the insertion of that number
																			 will cascade up the deletes before that bridge, so the minimum element from the that bridge will get
																			 removed by the last cascaded delete.
\end{enumerate}

\begin{figure*}[h]
  \centering
  \leavevmode
  \scalebox{1}{\includegraphics{f7.pdf}}
  \caption{\small Priority queues. }
  \label{fig:insert}
\end{figure*}

We can perform all of these operations in $O(\lg m)$ worst-case, where $m$ is the total number of updates, present or retroactive, performed on the priority queue. We first store a balanced BST insertions keyed on insertion time. We augment each node of this tree with the max key $k' \notin Q_0$ over every node's subtree. This lets us find the maximum key among all elements inserted after a time $t'$ but not in $Q_0$ in $O(\lg m)$ time. We can also find the minimum key among all elements inserted before a time $t'$ and still in $Q_0$ in the same runtime if we maintain in every node the minimum of all keys in its subtree still in $Q_0$. These are useful for operations 1 and 2, and 3 and 4, respectively.\\
\\
To find the last bridge before $t$ or the first bridge after $t$, we maintain a list of updates which we store in a modified $(a,b)$-tree developed by Fleischer \cite{fleischer}. If we assign a weight of 0 to $insert(k)$ for $k \in Q_0$, +1 to $insert(k)$ for $k \notin Q_0$, and -1 for $delete$-$min()$, every bridge in the tree corresponds to a prefix sum of 0. This allows us to find bridges in $O(\lg m)$ time, which we use for every operation.

\subsection{Other Data Structures}
\begin{figure*}[h]
  \centering
  \scalebox{0.8}{\includegraphics{f6.pdf}}
  \caption{\small Successor. }
  \label{fig:insert}
\end{figure*}

\begin{itemize}
\item queue: partial retroactivity is $O(1)$, and full retroactivity is $O(\lg{m})$.
\item deque: full retroactivity is $O(\lg{n})$.
\item union-find (incremented connectivity): full retroactivity is $O(\lg{m})$.
\item priority queue: partial retroactivity is $O(1)$, and full retroactivity is $O(\lg{m})$.
\item successor: $O(\lg{m})$ for partial retroactivity. The easy approach to full retroactivity takes $O(\lg ^2 {m})$. Giora and Kplan \cite{gk} can achieve $O(\lg{m}$ time for full retroactivity. Find the successor is is equivalent to optimal dynamic vertical ray shooting among horizontal line segments (see the figure below).
\end{itemize}

{\bf{OPEN:}} Can we solve optimal dynamic vertical ray shooting among general (not necessary horizontal) line segments?

\bibliography{mybib}
%\bibliographystyle{alpha}

\begin{thebibliography}{77}

\bibitem{fks}
M. Fredman, J. Koml\'{o}s, E. Szemer\'{e}di,
\emph{Storing a Sparse Table with $O(1)$ Worst Case Access Time},
Journal of the ACM, 31(3):538-544, 1984.

\bibitem{brodal}
Gerth Stølting Brodal: \emph{Partially Persistent Data Structures of Bounded Degree with Constant Update Time}. Nord. J. Comput. 3(3): 238-255 (1996)

\bibitem{dil}
Erik D. Demaine, John Iacono, Stefan Langerman: \emph{Retroactive data structures}. SODA 2004: 281-290

\bibitem{dlp}
Erik D. Demaine, Stefan Langerman, and Eric Price: \emph{Confluently Persistent Tries for Efficient Version Control}. Algorithmica (2008).

\bibitem{dietz}
Paul F. Dietz, Daniel Dominic Sleator: \emph{Two Algorithms for Maintaining Order in a List} STOC 1987: 365-372

\bibitem{dietz1}
Paul F. Dietz: \emph{Fully Persistent Arrays (Extended Array)}. WADS 1989: 67-74

\bibitem{dsst}
James R. Driscoll, Neil Sarnak, Daniel Dominic Sleator, Robert Endre Tarjan: \emph{Making Data Structures Persistent}. J. Comput. Syst. Sci. 38(1): 86-124 (1989)

\bibitem{gk}
Yoav Giora, Haim Kaplan: \emph{Optimal dynamic vertical ray shooting in rectilinear planar subdivisions} ACM Transactions on Algorithms 5(3) (2009)

\bibitem{kot}
Haim Kaplan, Chris Okasaki, Robert Endre Tarjan: \emph{Simple Confluently Persistent Catenable Lists}. SIAM J. Comput. 30(3): 965-977 (2000)

\bibitem{fiat}
Amos Fiat, Haim Kaplan: \emph{Making data structures confluently persistent}. J. Algorithms 48(1): 16-58 (2003)

\bibitem{okasaki}
Chris Okasaki: \emph{Purely Functional Data Structures}. New York: Cambridge University Press, 2003.

\bibitem{fhm}
Gudmund Skovbjerg Frandsen, Gudmund Skovbjerg Frandsen, Peter Bro Miltersen: \emph{Lower Bounds for Dynamic Algebraic Problems}. Information and Computation 171(2): 333-349 (2001)

\bibitem{fleischer}
Rudolf Fleischer: \emph{A Simple Balanced Search Tree with $O(1)$ Worst-Case Update Time}. Int. J. Found. Comput. Sci. 7(2): 137-150 (1996)

\end{thebibliography}

\end{document}
