\documentclass[11pt]{article} \usepackage{latexsym} \usepackage{amsmath} \usepackage{amssymb} \usepackage{amsthm} \usepackage{epsfig} \usepackage{psfig} \usepackage{stmaryrd} \def\polylg{\operatorname{polylg}} \def\adj{\operatorname{adj}} \def\symb{\operatorname{symb}} \def\rand{\operatorname{rand}} \newcommand{\handout}[5]{ \noindent \begin{center} \framebox{ \vbox{ \hbox to 5.78in { {\bf 6.897: Advanced Data Structures } \hfill #2 } \vspace{4mm} \hbox to 5.78in { {\Large \hfill #5 \hfill} } \vspace{2mm} \hbox to 5.78in { {\em #3 \hfill #4} } } } \end{center} \vspace*{4mm} } \newenvironment{itemize*}% {\vspace{-2ex} \begin{itemize} % \setlength{\itemsep}{-1ex} \setlength{\parsep}{0pt}}% {\end{itemize}} \newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}} \newtheorem{theorem}{Theorem} \newtheorem{corollary}[theorem]{Corollary} \newtheorem{lemma}[theorem]{Lemma} \newtheorem{observation}[theorem]{Observation} \newtheorem{proposition}[theorem]{Proposition} \newtheorem{definition}[theorem]{Definition} \newtheorem{claim}[theorem]{Claim} \newtheorem{fact}[theorem]{Fact} \newtheorem{assumption}[theorem]{Assumption} % 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988. \topmargin 0pt \advance \topmargin by -\headheight \advance \topmargin by -\headsep \textheight 8.9in \oddsidemargin 0pt \evensidemargin \oddsidemargin \marginparwidth 0.5in \textwidth 6.5in \parindent 0in \parskip 1.5ex %\renewcommand{\baselinestretch}{1.25} \begin{document} \lecture{8 --- March 1, 2005}{Spring 2005}{Prof.\ Erik Demaine}{Vincent Yeung} \section{Overview} In this lecture we discuss two problems for dynamic directed graphs: {\bf transitive closure} and {\bf all-pairs shortest paths}. We will present various results and open problems for each. Then, we will focus on one particular result due to Sankowski \cite{sankowski} for transitive closure, namely that of $O(n^2)$ updates and $O(1)$ queries, both worst-case times. \section{Dynamic directed graphs} Instead of the standard insert and delete operations for edges, we would like our data structures to support more powerful {\em bulk update} operations. These refer to the insertion/deletion of a vertex and an arbitrary set of incident edges. \subsection{Transitive closure} The dynamic transitive closure problem is essentially that of dynamic reachability. We would like to answer queries of the form, ``Is there a path from $v$ to $w$?'' We will now present various results for dynamic transitive closure: \subsubsection{Results} \begin{itemize} \item bulk update: $O(n^2)$ amortized\\ query: $O(1)$ worst case \cite{demetrescu1} \item bulk update: $O(n^2)$ worst-case, but randomized \\ query: $O(1)$ worst case \cite{sankowski} {\bf (what we will show today)} \item {\bf OPEN}: Can we achieve bulk update in $o(n^2)$ worst case? \end{itemize} Note that, in some sense, the $O(n^2)$ result is optimal if we approach this problem by storing the transitive closure matrix {\em explicitly} (we don't necessarily have to do this, however, to answer queries). This is because in the worst case, an update can modify $\Omega(n^2)$ edges in the transitive closure. Next are two results for which the product of the bulk update and query times is $O(mn)$, where $m$ is the number of edges. The first of the two results allows for some amount of trade-off between the two operations. \begin{itemize} \item bulk update: $O(m \sqrt{n}\ t)$ amortized\\ query: $O(\sqrt{n}/t)$ worst case, where $t=O(\sqrt{n})$ \cite{roditty1} \item bulk update: $O(m+n \lg n)$ amortized\\ query: $O(n)$ worst case \cite{roditty2} \item {\bf OPEN}: Can we achieve full trade-off with the product of update and query times equal to $O(mn)$ or $O(n^2)$? \end{itemize} Finally, we give two results for special cases of the problem: \begin{itemize} \item {\bf (acyclic graphs)}\\ update: $O(n^{1.575}\ t)$\\ query: $O(n^{0.575}/t)$, $t=O(n^{0.325})$ \cite{demetrescu1} \item {\bf (decremental transitive closure)}\\ update: $O(n)$ amortized\\ query: $O(1)$ worst case \cite{demetrescu1} \end{itemize} \subsection{All-pairs shortest paths} Here we present various results for the all-pairs shortest paths problem (dynamic shortest paths), which answers the question of, ``What is the weight of the shortest path from $v$ to $w$?'' \subsubsection{Results} \begin{itemize} \item bulk update: $O(n^2(\lg n + \lg^2 {(1+m/n)}))$ amortized\\ query: $O(1)$ worst case \cite{thorup1} \item {\bf OPEN}: Is it possiblie to achieve $O(n^2)$ or $o(n^2)$ update, the latter even for just undirected graphs? \item update: $O(n^{2.75})$ worst case\\ query: $O(1)$ worst case \cite{thorup2} \item {\bf (unweighted graph)}\\ update: $O(m\sqrt{n}\ \polylg n)$ amortized\\ query: $O(n^{3/4})$ worst case \cite{roditty3} \item {\bf (unweighted, undirected, and (1+$\epsilon$)-approximate)}\\ update: $O(\sqrt{m}\ nt)$ amortized\\ query: $O(\sqrt{m}/t)$ worst case, $t=O(\sqrt{m})$ \cite{roditty4} \end{itemize} \section{$O(n^2)$ worst case dynamic transitive closure} As promised, we now discuss the result in \cite{sankowski}. The key idea is to transform the problem at hand (transitive closure) into one ({\bf disjoint cycle cover}) that is solvable via linear algebra, with the use of various matrix properties. \paragraph{Preliminaries:} We will show the connection between transitive closure and disjoint cycle cover. To do so, we observe that each of the following is equivalent: \begin{itemize} \item[-] There is a path from $i$ to $j$ in the graph. \item[$\Leftrightarrow$] If after adding the edge $(j,i)$, there is a cycle through $(j,i)$. \item[$\Leftrightarrow$] If after removing edges $(*,i)$ (all incoming edges to $i$) and $(j,*)$ (all outgoing edges from $j$), and adding the edge $(j,i)$, there is a cycle through $(j,i)$ (or through $i$, or $j$ -- these are equivalent conditions). \item[$\Leftrightarrow$] If after adding loops $(k,k)$ for all vertices $k$, and removing edges $(*,i)$ and $(j,*)$ (note that this removes $(i,i)$ and $(j,j)$), there is a disjoint set of cycles covering all vertices. This is the {\em disjoint cycle cover} problem. (Here, all nodes other than $i$ and $j$ can be covered by their self-loops, so again we are looking to see if there is a cycle through $j$.) \item[$\Leftrightarrow$] If after the same modifications as immediately above, there is a permutation $\pi$ on the vertices such that $(i,\pi (i))$ is an edge $\forall i$. \end{itemize} \subsection{Connection to linear algebra} We use the standard definition of a directed graph's {\bf adjacency matrix} $A$: \begin{displaymath} A_{ij} = \left\{ \begin{array}{ll} 1 & \textrm{if $(i,j)$ is an edge}\\ 0 & \textrm{otherwise} \end{array} \right. \end{displaymath} Now we need to make the transformations discussed earlier to the graph. We begin by adding $I$ to $A$ to put in self-loops $(k,k)$ on each vertex (we are assuming the graph is initially loop-free). Next, we introduce the {\em zap} operation, represented as $\lightning$, and define $Z=(A+I) \lightning (j,i)$ as follows: \begin{enumerate} \item Zero out column $i$ and row $j$ in $(A+I)$; this removes the edges $(*,i)$ and $(j,*)$. \item Set the element in the $(j,i)$ position to 1; this adds the edge $(j,i)$. \end{enumerate} At this point, we are ready to start using linear algebra. First, we employ the following definition of the determinant: $$\det Z = \sum_{\pi} {\Big[ \textrm{sign}(\pi) \underbrace {\prod_{i} {Z_{i \pi(i)}}}_{\substack{\textrm{0 $\Leftrightarrow$ any edge} \\ \textrm{$(i,\pi(i))$ is missing}}} \Big]} $$ If there is a disjoint cycle cover, then at least one of the terms (products) in this sum must be non-zero, so the determinant should (hopefully) be nonzero, but unfortunately, the sign rules for computing determinants may potentially cancel out non-zero terms. Before we resolve this issue, though, let us first extend the determinant idea a bit further. One can find that the {\em adjugate} (known as the {\em adjoint} in some circles) of a square matrix $B$ contains as its entries the determinants of $B$'s $n^2$ ``zapped'' matrices (one for each possible $(j,i)$). Applying this on our matrix $(A+I)$, we have: $$(\adj {(A+I)})_{ij} = \det{((A+I) \lightning (j,i))}$$ In other words, $\adj {(A+I)}$ contains all the terms we need to find the transitive closure (i.e. to answer reachability queries for any pair $(i,j)$) -- if we resolve the cancellation issue above. \subsection{Avoiding cancellation} Conceptually, we want a {\bf symbolic matrix}, which we define as: \begin{displaymath} (\symb Z)_{ij} = \left\{ \begin{array}{ll} a_{ij} & \textrm{if $A_{i,j} \neq 0$} \\ 0 & \textrm{otherwise} \end{array} \right. \end{displaymath} In other words, every nonzero entry is represented by a unique variable. Clearly, the determinant of $\symb Z$ is a polynomial of degree $\leq n$ in at most $n^2$ variables ($a_{ij}$, etc.). The symbolic determinant of this matrix, stored in $(\adj \symb (A+I))_{ij}$ is equal to a zero polynomial iff there is no edge from $i$ to $j$. This is because all variable are distinct, so permutation terms cannot cancel out. But calculating the symbolic determinant takes too long. Instead, we settle for a {\bf random matrix}, which we define as: \begin{displaymath} (\rand Z)_{ij} = \left\{ \begin{array}{ll} \substack {\textrm{independent random number chosen} \\ {\textrm{uniformly between 0 and $p-1$}}} & \textrm{if $A_{i,j} \neq 0$} \\ 0 & \textrm{otherwise} \end{array} \right. \end{displaymath} So what we want to do is replace the ones in $(A+I)$ with random numbers ($p$ is a large prime number). Then, if the determinant of the symbolic matrix is zero, the determinant of the random matrix will also be zero. But what about when the polynomial is not zero? The following lemma can be used to calculate the probability that the determinant of the random matrix will be zero in this case (which will make our data structure answer queries incorrectly). \begin{lemma}[Zippel-Schwartz lemma \cite{zippel,schwartz}] $$\Pr {\{ \textrm{nonzero polynomial of degree $d$ evaluated at uniform random inputs} \equiv 0\pmod{p} \}}\leq d/p $$ \end{lemma} So, in the case that $\det \symb Z \neq 0$, we have $\Pr{\{\det \rand Z=0\}} \leq n/p \leq \frac{1}{n^{c-1}}$, if we choose a prime $p \geq n^c$. We can extend this to all $n^2$ $Z$ matrices that our data structure needs to handle, and we get that: $$\Pr {\{\adj \rand {(A+I)} \textrm{\ has more zeros than\ } \adj \symb {(A+I)}\}} \leq \frac{1}{n^{c-3}} \textrm{\ \ (by union bound)}$$ It follows that if the data structure runs for polynomial time, we can make the probability of error $\leq \frac{1}{n^{c'}}$ for any $c' > 0$. \paragraph{Rebuilding:} If the data structure runs for a long time, we need to reinitialize the matrix with other random values every $n^{O(1)}$ operations. The bound remains $O(n^2)$ amortized, and can be made $O(n^2)$ worst-case by deamortizing global rebuilding as in problem 3. \subsection{Dynamic matrix inverse} Now comes the algorithmic part of this discussion. How do we maintain the matrix $\adj \rand (A+I)$ in $O(n^2)$ worst case time for bulk updates and $O(1)$ worst case time for queries? The solution is to use the following relationship between the inverse and adjugate matrices: $$ B^{-1} = \frac {\adj B} {\det B} \textrm{, when it exists } (\det B \ne 0) $$ Thus, if during bulk udpates we can maintain the inverse and determinant of $\rand (A+I)$ in $O(n^2)$ time, we can answer queries in constant time. This is what we will do. \paragraph{Approach:} When a bulk update to the vertex $i$ occurs, row $i$ and column $i$ of $A$ (really $\rand (A+I)$, but we will just say $A$ for the rest of the analysis for simplicity) may be modified. Here, we will only consider modifications to column $i$. We represent this modification by the $\Delta$ vector: $A'_{ji} \leftarrow A_{ji} + \Delta_j,\ \forall j$. We also define the row vector $e_i^T = \left( \begin{array}{ccccccc} 0 & \ldots & 0& \underbrace{1}_{i^{th}\ column}&0 & \ldots & 0 \end{array} \right) $. Then, $$ \Delta e_i^T = \left( \begin{array}{c} \Delta_1 \\ \vdots \\ \Delta_n \end{array} \right) \left( \begin{array}{ccccccc} 0 & \ldots &0& 1 &0 & \ldots & 0 \end{array} \right) = \left( \begin{array}{ccccccc} 0 & \ldots & 0&\Delta_1 &0 & \ldots & 0 \\ & & &\vdots& & & \\ 0 & \ldots &0& \Delta_n &0 & \ldots & 0 \end{array} \right) $$ Thus, the new matrix is $A'=A+\Delta e_i^T$. The key idea is to write $A'=AB$, where $B=I+A^{-1}\Delta e_i^T = I+(A^{-1}\Delta) e_i^T = I+be_i^T$. Here $b = A^{-1} \Delta$ is a column vector $ \left( \begin{array}{c} b_1 \\ \vdots \\ b_n \end{array} \right) $ that is a linear combination of the columns of $A^{-1}$ and takes $O(n^2)$ time to compute. Hence, we can write $B$ in the form: $$ B= \left( \begin{array}{ccccccc} 1 & & & b_1 & && \\ & \ddots & & b_2 & & & \\ & & 1 & \vdots & & & \\ & & & 1+b_i & & & \\ & & & \vdots & 1 & & \\ & & & b_{n-1}& & \ddots & \\ & & & b_n & & & 1 \end{array} \right) $$ We know $A'^{-1} = B^{-1}A^{-1}$, so we need to find $B^{-1}$, which we can do with $\det B$ and $\adj B$, both of which can be computed in $O(n^2)$ with the following formulas: \begin{eqnarray*} &&B^{-1}= \left( \begin{array}{ccccccc} 1 & & & -\frac{b_1}{1+b_i} & & & \\ & \ddots & & -\frac{b_2}{1+b_i} & & & \\ & & 1 & \vdots & & & \\ & & & \frac{1}{1+b_i} & & & \\ & & & \vdots & 1 & & \\ & & & -\frac{b_{n-1}}{1+b_i} & &\ddots & \\ & & & -\frac{b_n}{1+b_i} & & & 1 \end{array} \right) \\ &&\det B = 1+b_i \end{eqnarray*} We also see that $B^{-1}$ has $O(n)$ nonzeros. This is good because it means we can get $A'^{-1} = B^{-1} A^{-1}$ in $O(n^2)$ time (as opposed to the standard $O(n^3)$ time needed for matrix multiplication with the obvious algorithm). The new determinant can also be quickly computed since $\det{A'} = \det B\det A$. There's a caveat, though. What if $A$ is singular? We can show that this happens with very low probability (indeed, as low as we want to make it) because remember in the analysis $A$ was really $\rand (A+I)$, and from the probability analysis in the previous section, we know that if $\det \symb (A+I) \neq 0$, then $\det \rand (A+I) \neq 0$ with high probability, and we know that $\det \symb (A+I)$ is never zero. %\bibliography{mybib} \bibliographystyle{alpha} \begin{thebibliography}{77} \bibitem{demetrescu1} Camil Demetrescu, Giuseppe F. Italiano. \emph{Fully Dynamic Transitive Closure: Breaking Through the O($n^2$) Barrier}. FOCS 2000: 381-389 \bibitem{roditty2} Liam Roditty, Uri Zwick. \emph{A fully dynamic reachability algorithm for directed graphs with an almost linear update time}. STOC 2004: 184-191 \bibitem{roditty1} Liam Roditty, Uri Zwick. \emph{Improved Dynamic Reachability Algorithms for Directed Graphs}. FOCS 2002: 679- \bibitem{roditty3} Liam Roditty, Uri Zwick. \emph{On Dynamic Shortest Paths Problems}. ESA 2004: 580-591 \bibitem{roditty4} Liam Roditty, Uri Zwick. \emph{Dynamic Approximate All-Pairs Shortest Paths in Undirected Graphs}. FOCS 2004: 499-508 \bibitem{sankowski} Piotr Sankowski. \emph{Dynamic Transitive Closure via Dynamic Matrix Inverse}. FOCS 2004: 509-517. \bibitem{schwartz} Jacob T. Schwartz. \emph{Fast Probabilistic Algorithms for Verification of Polynomial Identities. J} ACM 27(4): 701-717 (1980) \bibitem{thorup1} Mikkel Thorup. \emph{Fully-Dynamic All-Pairs Shortest Paths: Faster and Allowing Negative Cycles}. SWAT 2004: 384-396 \bibitem{thorup2} Mikkel Thorup. \emph{Worst-case update times for fully-dynamic all-pairs shortest paths}. STOC 2005 \bibitem{zippel} Richard Zippel. \emph{Probabilistic algorithms for sparse polynomials}. EUROSAM 1979: 216-226 \end{thebibliography} \end{document}