%%%%%%%%%%% Define some micros %%%%%%%%%%%%%%%%
\newcommand{\popt}{\mbox{$P_{opt}$} }
\newcommand{\opone}{\mbox{$O_{1}$} }
\newcommand{\optwo}{\mbox{$O_{2}$} }
\newcommand{\opn}{\mbox{$O_{n}$} }
\newcommand{\opx}{\mbox{$O_{x}$} }

\newtheorem{intuition}{Intuition and Key Fact}


\documentclass[12pt]{article}
\usepackage{6895}
\usepackage{psfig}
\usepackage{epsfig}

\bibliographystyle{plain}

\setlength{\parskip}{0.3cm} \setlength{\parindent}{0in}

\begin{guestlecture}{22}{Pick a Winner}{He Yuxiong and Sajindra
Jayasena}{Tom Leighton}

\begin{summary}
\sumitem{The Pick-a-Winner game}{The section describes the rules
of the pick-a-winner game and shows two simple observations.}
\sumitem{Strategies to maximize the profit}{This section describes
the optimal strategy, analyzes its performance, shows the key
intuitions and sketches the proof of the performance guarantee
theorem.} \sumitem{Pick-a-Winner - K player Game }{ This section
looks at how the game and strategy vary with multiple players}
\sumitem{Applications }{This section looks at some applications
that can be modelled using Pick-a-Winner single player and
multi-player games}

\end{summary}

In the lecture, we consider a class of problems in which a
decision-maker is required to select from among numerous
dividend-paying options. The goal of the decision-maker is to
select an option (or a set of options) that will have the best
future return. The goal of the lecture is to show how to use
probabilistic method to get the performance guarantee without
making probabilistic assumptions.

\section{The Pick-a-Winner game}

The section describes the rules of the pick-a-winner game and
shows two simple observations.

Pick-a-winner game has one player.  The player is required to
select one from among n dividend-paying options.  The selection
can be made on any day within d days.  Once the player makes the
decision, he is not allowed to change it.  The profit of the
player is equal to the dividends paid by the option after it has
been selected.  The goal of the game is to maximize the profit.

In the online version of this game, the player only knows the past
dividend payments but he does not know the future payment
schedule. In addition, the past and the future schedule may not be
correlated. In fact, the future is decided by the adversary who
knows the player's strategy and seeks to minimize the profit of
the player. The question is how much money the player can make.

We can make two simple observations based on the rules of the
game.
\begin{observation}
The profit for any deterministic player is 0, even if $\popt=\$d$.
\end{observation}
\begin{proof}
Before the player makes any decision, the adversary pays a
dividend for every option on every day. After the player makes a
selection, the dividends are stopped for the selected option.  The
adversary can still pay a dividend for all the other options on
every day. So even if $\popt=\$d$, the profit of the deterministic
player is 0.
\end{proof}
\begin{observation}
By choosing a random option at the start, we can get an expected
profit \popt/n.
\end{observation}
\begin{proof}
Since the adversary only knows our strategy but not the random
number, we have a $1/n$ chance of selecting the optimal option.
\end{proof}

\section{Strategies to maximize the profit}
This section describes an optimal strategy, analyzes its
performance, shows the key intuitions and sketches the proof of
the performance guarantee theorem.

An \textbf{Optimal Strategy} is described below:
\\*
Player selects target profit $P$ and confidence level $1-q$.  We
label the options as $\opone, \optwo \ldots \opn$.  Each day, the
player checks the status of each option to see which one has a
dividend.  For each $x$ $(x=1\ldots n)$, if \opx has a dividend on
the day, the player flips a coin to decide whether or not to make
a selection to \opx.  The player will make a selection on \opx
with probability $2^{(qi/P)}/(Pn^2)$, where $i$ is the total
number of dividend \opx has generated so far.  If the selection
has been made, the player stops flipping coins and waits for the
job to be completed.

There are two important intuitions and facts regarding to the
optimal strategy.  They are also the main supports for
\thmref{opt-strategy}
\begin{intuition}
The selection probabilities start small and grow quickly.  The
purpose of the slow start is to reduce the possibility of
misleading by adversary for the false start.  The objective of
fast growth is to ensure that we make some selections before the
end of the game with high probability.
\end{intuition}
\begin{intuition}
The growth must be slow enough to ensure that whichever option is
selected, the posteriori probability is high such that the option
was selected at least P payments before the final payment.
\end{intuition}

\begin{theorem}
\thmlabel{opt-strategy} If \popt is known and $\popt > 3P\log
n/q$, the strategy will return a profit $\geq P$ with probability
$> 1-q$
\end{theorem}

\begin{proof-sketch}
We will only outline the proof in the class.  The proof was based
on the two intuitions and following the procedures given in
\cite{1}.

Let $S$ denote a sample space of coin flips where there will be
one coin flip for each pair $(x,y)$ for which \opx has its
dividend at day $y$.  If \opx has its i'th dividend at day $y$,
the probability of getting head is $2^{(qi/P)}/Pn^2$.  In
particular, the player will schedule his job on \opx, at step $y$
if the $(x,y)$ coin is head and if the $(x', y')$ coin is tail for
all $(x', y')$ such that $x'<x$ or $y'=y$ and $y'<y$.  Let
$S_{win}$ denote the subspace of $S$ consisting of sample points
for which the player gets its targeted profit $P$ or higher. In
order to show the $Pr(S_{win})$, a second space $S'$ is considered
for which it is easy to show that $Pr(S')$ is large and that
$Pr(S_{win})$ is nearly as large as $Pr(S')$. $S'$ is defined to
comprise of the sample points for which there is at least one Head
and for which the first $d$ flips for each $O_i$ resulted in
Tails. There are two steps involved.

\begin{enumerate}
\item Prove $Pr(S') \geq (1-1/n)$
\\*
Because of the slow start and the fast growth, this step is to
show that the probability of misleading by adversary at the
beginning and the probability of not making any choice at the end
are small. The probability of making a selection among the $Pn$
flips for which $i\leq P$ is at most
\begin{eqnarray*}
P n(\frac{2^q}{pn^2})=O(\frac{1}{n})
\end{eqnarray*}
The probability of making no selection at the end of the game is
at most
\begin{eqnarray*}
Pr[no~selection] & \leq & \prod_{i=1}^{\popt}[1-(\frac{2^{qi/P}}{Pn_2})] \\
& < & [1-(\frac{2^{q(\popt-P)/P}}{Pn_2})]^{P}  \\
& < & [1-(\frac{n2^{-q}}{P})] \\
& < & e^{-\frac{n}{2}}
\end{eqnarray*}
So $Pr(S') \geq 1-O(1/n)$

\item Prove $Pr(S_{win})\geq(1-O(P\log n/\popt))Pr(S')$
\\*
It is the step to prove that $S_{win}$ is nearly as large as $S'$.
It is the proof step from Intuition 2 and shows that the
probability of making selection at least $P$ payments before the
final payment is almost as likely as making a selection before the
final payment.

The proof of this step is to construct an injection from $S'$ to
$S_{win}$.  Please refer to lemma 2.2 in \cite{1} for the details.
\end{enumerate}

Combining the results of step 1 and step 2,   we are able to prove
the theorem.  If \popt is known, choose target $P=q\popt/(3 \log
n)$ and confidence parameter $1-q$ as desired, profit $P$ will be
obtained with probability $1-q$.
\end{proof-sketch}

\thmref{opt-strategy} shows the performance guarantee when \popt
is known. When \popt is unknown, we need to make a guess of it and
choose target $P=q2^R/(3\log n)$ where $R$ is uniformly selected
from $[1, \log d]$. $2^R$ will approximate \popt value among $2,
2^2 \ldots d$ with probability $1/\log d$.  So profit
$\popt/(c\log n)$ will be obtained with probability $1/(c \log d)$
where $c$ is a fixed constant. Combining the result with that in
\thmref{opt-strategy}, we know the expected return is
$c\popt/(\log n \log d)$ without the knowledge of \popt.

From the theorem and analysis in the section, we know that the
past can be combined at random to provide a very good prediction
of future performance even if the past does not correlate with the
future.
\newpage
\section{Pick-a-Winner Game - with Variations}
Now let us look at a variation of the previous game. The game has
the following conditions and rules. \begin{closeitemize} \item
\emph{K} players are playing the game. \item Each player is
allowed to select one option. \item There are \emph{N} options
that the players can select. \item There are \emph{D} days that a
game can be played. \item A particular dividend of an option is
paid only to the highest priority player holding the particular
option. \item If a player needs to change the option ,it will cost
1 $\$$ per change.
\end{closeitemize}
let $P_i$ denote the profit of the $i^{th}$ ranked player.
\\
Let us look at an example with 3 players and consider the
following observations.
\begin{closeitemize}
\item Player \emph{1} selects \emph{D} on day \emph{2} , swaps to
\emph{E} on day \emph{5}.\item Player \emph{2} selects \emph{D} on
day \emph{1}, swaps to \emph{C} on day \emph{3}. \item Player 3
selects \emph{E} on day \emph{2}. No swaps.
\end{closeitemize}
The outcome would be as follows:
\begin{figure}[h]
\begin{center}
\includegraphics[scale=.6]{game2.eps}
\caption{Game Results  }
\end{center}
\end{figure}
\\After playing the game, now let us try to answer the following
questions. \\
\\\textbf{Q.Can Swapping Help ?}
\\Our own intuitive answer would be \textbf{NO}. Since the the future
outcome cannot be predicted , the future of one option seems to be
indistinguishable from the future of another.
\\But further analysis shows that the answer is \textbf{YES}.The basic idea is that once the
past is seen , the randomize strategy may dictate change.
\\
The advantages obtained by swapping  are : \begin{closeitemize}
\item Increased profits gained , where the increase in the new
income surpassed the swapping cost, \item Increased probability of
success, \item Knowledge of $P_{OPT}$ is no longer needed.
\end{closeitemize}
\textbf{Key Idea}
\\One of the key ideas that could be gained is:\\
Break the time into epochs , and run independent trials in each
epochs. It could be observed that with a high probability, most
trials would be successful.

\textbf{Q. Is Coordination needed among players to maximize one's
profits? }
\\
The answer is \textbf{YES}. But very little coordination is
needed.
\\The following items need to be considered:
\begin{closeitemize}
\item As long as no player is allowed to be too greedy, and each
player can select in isolation, \item Other players are considered
to be part of the adversary, \item At worst, the $j^{th}$ ranked
player can choose  the $j^{th}$ ranked option.
\end{closeitemize}

\section{Results in Summary}
In Figure 2 , summarized results of the single and multiple player
Pick-a-Winner games are given.
\begin{figure}[h]
\begin{center}
\includegraphics[scale=1.0]{results.eps}
\caption{Game Results in summary  }
\end{center}
\end{figure}
\section{Applications}
In this section we will be looking at some real life examples
where the strategy of Pick-a-Winner game  can be applied.
\subsection{Scheduling a single job on NOWs}
The strategy applied in single player Pick-a-Winner game can be
applied in solving this problem.
\begin{figure}[h]
\begin{center}
\includegraphics[scale=1.0]{workstations.eps}
\caption{which workstation to select? }
\end{center}
\end{figure}
The problem formulation is as follows:
\begin{closeitemize}
 \item Bob has a job requiring  P cycles that he wants to run
overnight, \item Bob picks a workstation for his job and assigns
the job a background priority, \item The job gets done if the
chosen workstation has at least  P available cycles during the
night.
\end{closeitemize}
The Question to answer is : How should Bob choose a Workstation?
\\\textbf{Solution:}
\\ We will try to model the answer from the single player
Pick-a-Winner game with the following mapping.
\begin{closeitemize} \item Workstations $\rightarrow$ Options
\item Cycles $\rightarrow$ Days ( overnight - d days) \item
Availability $\rightarrow$ Dividends paid \item Target profit
$\rightarrow$ P cycles
\end{closeitemize}
The goal is to maximize probability of success,  \emph{\textbf{1 -
q}}
\begin{figure}[h]
\begin{center}
\includegraphics[scale=0.7]{application1.eps}
\caption{Availability of Workstations  }
\end{center}
\end{figure}
\\ In the figure 3, A denotes availability.
\\\textbf{Results:}\\
If some work stations will be available for at least
\emph{\textbf{3 P $\log$ n}} steps, Bob can get\emph{\textbf{ c
$\log$ n}} jobs of duration \emph{\textbf{P}} done with a
probability of\emph{\textbf{ 1 - o(1/n)}}.
\\ The following pointers should be considered:
\begin{closeitemize}
\item At most 1 work station is used by Bob at any time, \item Bob
can kill a job in progress and restart the job elsewhere, \item
The high probability of success holds for all patterns of
workstations availability, \item If Bob can use \textbf{O(1)}
workstations at a time, then he can attain 99 $\%$ efficiency.
\end{closeitemize}
\subsection{Scheduling multiple Jobs on NOWs}
Now we will look at scheduling multiple jobs. If there are
\emph{\textbf{K}} people , each with background jobs to run ,we
can use the strategy adopted in \emph{\textbf{ K-player
Pick-a-Winner}} game.

\textbf{Coordination:}
\\Each person receives a priority , but otherwise works
independently.Competing demands for service would also be resolved
by priority.
\\Now Let us look at the results.\\
\textbf{Results:}
\\
If the $j^{th}$ best workstation will be
available for at least $T_j$  steps, then the $j^{th}$ ranked
person will get c$\centerdot$$T_j$ cycles with a high probability.
\\Let us look at another game.
\subsubsection{\textbf{Application: Video Servers}} Let's formulate the
problem the following way.
\begin{closeitemize}
\item During the day,\emph{d} customers request movies. \item At
most \emph{k} out of \emph{n} movies will be shown at a
pre-arranged time in each evening. \item Each request is
immediately accepted or rejected.\item If a request is
accepted,the movie must be shown. Rejected customers are lost.
\end{closeitemize}

\textbf{Goal:}
\\
The goal is to maximize the accepted requests thus maximizing
profits.\\ Before looking at the game based solution, let's look
at a previous solution proposed by Aggarawal, Garay and Herzberg
\cite{2}.
\\ According to their solution:
\begin{closeitemize}
\item In the worst case, \emph{\textbf{k}} requests would be
accepted . \item \emph{\textbf{c $P_{OPT}$ / $\log$ n}} requests
would be accepted with some lookahead.
\end{closeitemize}
Now let's look at the solution that was obtained from the K-player
Pick-a-Winner game. Following are the mappings.
\begin{closeitemize}
\item Movies $\rightarrow$ Options \item Customers $\rightarrow$
Days  \item Requests $\rightarrow$ Dividends Paid \item Target
Profit $\rightarrow$ Number of Accepted Requests.
\end{closeitemize}
\textbf{Strategy:} The strategy is to guess the number of requests
$R_j$ for $j^{th}$ most popular movie and set target profit for
$j^{th}$ movie to be \emph{\textbf{c $R_j$/ $\log$ n }}.
\\
\textbf{Results:} The expected number of accepted requests would
be =
\\ \emph{\textbf{c$\centerdot$$P_{OPT}$ / ( $\log$ n / $\log$ d )}}
\\ The following diagram denotes the requests and movies in the
same analogy of playing the K-player game.
\begin{figure}[h]
\begin{center}
\includegraphics[scale=1.0]{application-videoserver.eps}
\caption{Movies Vs. Requests  }
\end{center}
\end{figure}

\subsubsection{\textbf{Application: On-line Set Cover}}
\textbf{Problem:}
\\Let's formulate the problem in the following manner.
\begin{closeitemize}
\item Given \textbf{\emph{n}} Sets $S_1, S_2 ... ,S_n$. \item
\emph{\textbf{d}} items $V_1,V_2,...., V_d$ arrive one per each
step. \item As each item arrives , we are able to distinguish the
set(s) the item belong to.
\end{closeitemize}
\textbf{Goal: }
\\
The goal is to pick \textbf{k} sets to cover as many items as
possible. The rewarding mechanism is as
follows.\begin{closeitemize} \item Credit is given only once for
each item, even if it is contained in several sets. \item Credit
for an item is given only if the set was selected before or during
the step when the item arrives.
\end{closeitemize}
Let's look at an example where this strategy can be applied.

\textbf{Example: }
\\
A problem associated with Video server contention and
availability,  where each customer specifies a collection of
movies , any one of which is fine.

\textbf{Solution:}
\\
The solution is the same as for Video servers except that each
customer can specify several movies.
\\
The following mapping maps the problem entities to the set cover
problem.
\begin{closeitemize}
\item Items $\rightarrow$ Customers \item Sets $\rightarrow$
Movies.
\end{closeitemize}

The following results could be obtained.\\
 \textbf{Results:}
\\
Total expected credit would be
\begin{eqnarray*}
c P_{OPT} / ( \log n \log d)
\end{eqnarray*}
 The following diagram denotes the representation of the problem
where the request for each movie is denoted by a set $S_1$ to
$S_n$ and customers  are represented through the numbering
 from $V_1$ to $V_d$.

In Figure 6, $\in$ denotes each customer's request for a
particular movie.
\begin{figure}[h]
\begin{center}
\includegraphics[scale=0.7]{setcover.eps}
\caption{Movies Vs. Requests  }
\end{center}
\end{figure}
\newpage
\bibliography{scribe-tom leighton-final}
\end{guestlecture}
\theend
