---
title: "Scheduling Problem"
format:
  html:
    toc: true
    number-sections: true
    code-line-numbers: true
---
[Pre-class notes](hand_written_notes/GreedyPre.pdf)

[Post-class notes](hand_written_notes/GreedyPost1a.pdf)

[Pre-class notes Part 2](hand_written_notes/GreedyPart2Pre.pdf)

[Post-class notes Part 2](hand_written_notes/GreedyPart2Post.pdf)


## Learning Goals

- Describe typical properties of greedy algorithms, as well as related terms like "objective function," "optimization problem," and "completion time," "scoring function," "optimal order," "greedy order"
- Create counterexample to a greedy ordering [G1]
- Prove optimality of a greedy algorithm using an exchange argument [G2]


## Introduction to Greedy Algorithms

There is not an agreed upon definition of greedy algorithms, and many algorithms that at first seem different from each other are all classified as greedy. Over the course of the semester we will see several examples, and this will hopefully give a better sense of what "greedy" means. However, I'll give you an informal definition 

::: {#def-greedy}
(informal definition) A greedy algorithm sequentially constructs a solution through a series of myopic/short-sighted/local/not-global/not-thinking about-the-future decisions
:::

Greedy algorithms often have the follow properties:

- Easy to create (non-optimal) greedy algorithms
- Runtime is relatively easy to analyze
- Hard to find an optimal/always correct greedy algorithm
- Hard to prove correctness 

### Order Problem

Greedy algorithms are frequently used to design algorithms for ordering problems, which are a subclass of optimization problems (ordering problems are a broad class of problems that are very general - we will look at a specific ordering problem shortly).

:::{#def-opt}
An *optimization problem*  is a problem where you would like your solution to be better (according to some pre-decided metric) that any other possible/feasible solution.
:::

**Generic Ordering Problem**

- Input: $n$ items.
- Output: Ordering $\sigma$ of the $n$ items that minimizes (or maximizes) $A(\sigma).$



:::{#def-obj}
We call $A(\sigma)$, the function that we want our output to maximize or minimize, the *objective function*
:::



For example, the following figure shows the possible orders of 3 items, and made-up $A$-values for each ordering.

![Example of A values for orderings of 3 items](quarto_note_pdfs/Order_A.png){#fig-Aex width=80%}

If our goal is to minimize $A$, then for the example of @fig-Aex, the output of this ordering problem should be $(3,1,2)$, but if our goal was to maximize $A$, then the output of this ordering problem should be $(2,1,3)$.

Here are two approaches for designing an algorithm to solve the Ordering Problem:

- **Brute Force:** 
  * Evaluate $A(\sigma)$ for every possible ordering of the $n$ items. Keep track of the minimium (maximum) value found and the corresponding ordering, and then return that ordering.
  * Takes $\Omega(n!)$ time ($\Omega$ is asymptotic lower bound)
- **Greedy**
  * Design a simple scoring function $f$. A *scoring function* is a function that maps each item to a score. Note that while the objective function maps orderings to numbers, the scoring function maps items to numbers.
  * Evaluate the score of each item
  * Return $\sigma$ that puts the items in decreasing (increasing) order by score
  * Takes $\Omega(n\log n)$ time.

## Scheduling Problem

### Description

The scheduling problem is a type of ordering problem.

For the scheduling problem, we imagine that we have a series of tasks that we want to accomplish, but we can only do one task at a time, and must complete that task before moving onto the next. Each task takes some amount of time (tasks can have the same or different times), and each task has a weight, which tells us the importance of that task. We would like to figure out how to schedule tasks to prioritize important, short tasks.
 
**Input:** $n$ tasks. Information about the tasks is given in the form of two length-$n$ arrays, one labeled $t$, and one labeled $w$. The $i^\textrm{th}$ element of array $t$ is the time required to complete the $i^\textrm{th}$ task, which we call $t_i$, and the $i^\textrm{th}$ element of array $w$ is the weight (importance) of the $i^\textrm{th}$ task, which we call $w_i$, where larger weight means more important. For example, we might have the input

task | 1 | 2 | 3|
-----|---|---|---|
time | 3 | 4 | 2|
weight| 5| 1 | 2|

so in this case, $t_1=3$ and $w_3=2$

**Output:** Ordering of tasks $\sigma= (\sigma_1,\sigma_2,\dots,\sigma_n)$ that minimizes
$$
A(\sigma)=\sum_{i=1}^nw_iC_i(\sigma), 
$${#eq-objective}
where $C_i(\sigma)$ is the *completion time* of task $i$ with ordering $\sigma$, described below. Note that @eq-objective is the *objective function* for this scheduling problem

Assuming we start our first task at time $0$, then run our sequence of tasks, $C_i(\sigma)$ is the time at which the task $i$ is completed when the tasks are done in the order given by $\sigma$. For an example, suppose we have
$$
t=(3,4,2)
$$
and consider the ordering $\sigma=(3,1,2)$, so we do the third task first, then the first, and then the second. This is shown graphically in @fig-comp:

 ![Graphical representation of how jobs are ordered in time with ordering (3,2,1). Each job must be completed before the next can start.](quarto_note_pdfs/completion1.png){#fig-comp width=50%}


So $C_3(3,1,2)=2$ because we do it first, it takes time $2$, so it is completed at time $2$. Next we do task $1$, but it only gets started at time $2$ after completing task $3$, and runs until time $5$, so $C_1(3,1,2)=5.$ Now that task $1$ is completed, we can finally do task $2$ which takes another $4$ time units, and so we complete it at time $9$, so $C_2(3,1,2)=9$. (When it is clear which ordering $\sigma$ we are talking about, we will sometimes write $C_i$ instead of $C_i(\sigma)$.)


**Application:** Scheduling tasks on a single CPU (central processing unit) of a computer. Or scheduling your tasks...in the case that you must complete each task before moving on to the next.

::: {.callout-tip appearance="simple"}

### ABCD Question

If $t=(3,4,2)$ (the same example as above), what is $C_3(2,1,3)$

A) 2
A) 3
A) 7
A) 9

:::

::: {.callout-tip appearance="simple"}
### Group Exercises

1. Given the following input:

task | 1 | 2 | 
-----|---|---|
time | 3 | 4 |
weight| 5| 1 |

What is the best order? (Use a brute force approach, and calculate $A(\sigma)$ for all possible orderings.)

Hint: first calculate $C_1$ and $C_2$ for each ordering


2. What is the objective function $A(\sigma)=\sum_{i=1}^nw_iC_i(\sigma),$ optimizing? (What time/weight jobs is it prioritizing?)
:::

## Creating Scoring Functions and Counterexamples

Creating a greedy algorithm is easy! All you need to do is come up with a scoring function that you think might optimize the objective function. This scoring function should be a function of the properties of each item. For our scheduling algorithm, this means the scoring function can depend on $t_i$ and $w_i.$

For example, we can create the scoring function
$$
f(i)=t_i-6 \qquad \textrm{(increasing)}
$${#eq-score1}
where we will order items from smallest time to largest time. This greedy algorithm will choose the ordering that puts the shortest jobs first, and the longest jobs last.

Done! We've designed a greedy algorithm!

Problem? It might not be a good algorithm. The *greedy ordering*, the ordering created by $f$, might not be the *optimal ordering*, the one with the optimal $A$ value.

In any case, it might not be good, but we can still practice creating a greedy algorithm using @eq-score1. We first need to evaluate the score for each item (we'll use our example input from the group exercise above):

task | 1 | 2 | 
-----|---|---|
time | 3 | 4 |
weight| 5| 1 |
f | -3 | -2|

Since we have decided to order jobs in increasing $f$-value, our greedy ordering is $(1,2)$. In fact, when we did the brute force approach, we also found $(1,2)$ was the optimal ordering. This is great! Our greedy algorithm found the best possible ordering!!

::: {.callout-important}

The scores, the $f$-values, of items mean nothing on their own. They can be negative, or 0, or positive, and it doesn't tell you anything. Scores only have meaning when compared to the scores of other items.

:::

### Finding Counterexamples

Our greedy ordering was correct for the example above, but....will this greedy ordering always be correct?? To show that a scoring function will not always give us the optimal ordering, we can try to find a counterexample.

Tips for finding a counterexample:

- Think extreme
- Create tasks that are close in score but very different otherwise

::: {.callout-tip}

## Example

[G1]

We will find a counter example to $f(i)=t_i$ (ordered in increasing $f$-value).

We design the following set of jobs, that have very close times, but very different weights. In particular, we set the job that takes slightly longer to be much much more important.


task | 1 | 2 | 
-----|---|---|
time | 1 | 2 |
weight| 1| 100 |
f | 1 | 2|

So in this case, our greedy ordering is again $(1,2)$.

However, if you do a brute force approach, you will find that actually, the optimal ordering for this input to the scheduling problem is $(2,1)$.
:::


::: {.callout-tip appearance="simple"}
### Group Exercises

[G1]

Create counterexamples for the following scoring functions:

A) $f(i)=w_i$ (decreasing)
A) $f(i)=w_i-t_i$ (decreasing)

:::

### Design Approach

Here is a general approach for creating a good greedy algorithm:

1. Brainstorm several reasonable scoring functions. For example:
$$
f(i)=w_i-t_i \textrm{ (decreasing)}\qquad f(i)=w_i/t_i \textrm{ (decreasing)}\qquad f(i)=w_i^2-2t_i \textrm{ (decreasing)}
$$
These are "reasonable" because they all tend to prioritize jobs that are important and short, which is our goal. 
1. Test the scoring functions on examples, see how they do, and see if you can create counterexamples showing a scoring function is not optimal. (In some cases, you might also be satisfied with an algorithm that is not always optimal, but is close to optimal. For example, Prof. Das in her research often tries to show that an algorithm never does worse than 90% of the optimal value of the objective function.) These types of algorithms that are not always optimal or that you can't prove are optimal, are called **heuristics**.
1. If you can't find any counterexamples, you can try to prove that your greedy algorithm is correct.

## Proving Correctness of Greedy Scheduling {#sec-proofgreedy1}

::: {@thm-schedule}
Ordering jobs by decreasing value of $f(i)=w_i/t_i$ is optimal for minimizing $A(\sigma)=\sum_iw_iC_i(\sigma)$.
:::

[G2]

::: {.proof}
[We will use an "Exchange Argument"]

Without loss of generality, relabel the tasks in decreasing score so that task $1$ has the highest score, then task $2$, etc, so that
$$w_1/t_1>w_2/t_2>w_3/t_3\dots w_n/t_n.$$
With this relabeling, the greedy ordering is
$$\sigma=(1,2,3,\dots,n).$$

Suppose you have an ordering $\sigma^*$ that is the optimal ordering.
Since $\sigma^*\neq \sigma$, there must be tasks $b,y\in \{1,2,\dots,n\}$ where $b<y$, but $y$ appears immediately before $b$ in $\sigma^*$. In other words, $b$ and $y$ are in the wrong order, and right next to each other somewhere in $\sigma^*$, so $\sigma^*$ looks like:
$$\sigma^*=(\dots, y, b, \dots).$$
For example, if $\sigma=(1,2,3)$, and $\sigma^*=(3,1,2)$, then $3$ and $2$ are right next to each other but out of order, so we set $y=3$ and $b=1$. 

Now let ${\sigma^*}'$ be an ordering that is exactly the same as $\sigma^*$ *except* that $b$ and $y$ are exchanges (i.e. swapped). Thus ${\sigma^*}'$
looks like 
$${\sigma^*}'=(\dots, b, y, \dots)$$
where the $\dots$ are the same sequence as in $\sigma^*$. 

Next we analyze $A(\sigma^{*})-A({\sigma^*}')$.

::: {.callout-tip appearance="simple"}
### Group Exercise

Show $A(\sigma^{*})\geq A({\sigma^*}')$.

:::

[See hand written notes for analysis.]

Now if ${\sigma^*}'$ is not the greedy ordering, we can repeat this process, by exchanging two more elements that are out of order to get an ordering ${\sigma^*}''$. By the same argument, 
$A({\sigma^*}')\geq A({\sigma^*}'')$. Continuing in this way, we will eventually swap enough elements that put all of the elements of the schedule in order, and we produce $\sigma$. This is essentially be cause we are doing a bubble sort process. At each step, with each swap, we never increase $A$, and we might decrease $A:$
$$
A(\sigma^{*})\geq A({\sigma^*}')\geq A({\sigma^*}'')\dots\geq A(\sigma).
$$
Thus we have shown that no matter what schedule we start with, it has an $A$-value the same or larger than our greedy schedule. Thus the greedy schedule must be optimal.
:::

## Big Picture: Exchange Argument Structure

Exchange arguments are commonly (although not always) used to prove the correctness of greedy algorithms. Here is the general structure of an exchange algorithm proof.

**Exchange Argument**

1. Let $\sigma$ be the strategy that your greedy algorithm gives you, the one that you are trying to prove is optimal.
2. Consider some other strategy $\sigma^*$.
3. But then at some point $\sigma^*$ must differ from $\sigma$. Create a new sequence ${\sigma^*}'$ by exchanging two elements (or doing some other small change) of $\sigma^*$ to make it more like the greedy algorithm $\sigma$.
4. Show that ${\sigma^*}'$ is better than $\sigma^*$ (or at least no worse). 
5. Show that if one continues to make exchanges in the same way, one will eventually reach $\sigma$. Since at each stage, the $A$ value is better (or at least no worse), $\sigma$ must be optimal.

## Continuing to Analyze our Greedy Scheduling Algorithm

### Runtime of our greedy algorithm

::: {.callout-tip appearance="simple"}

### ABCD Question

What is the runtime of our greedy scheduling algorithm?

A) $O(1)$   
A) $O(n)$   
A) $O(n\log n)$   
A) $O(n^2)$   

:::



### How to Derive Scoring Function

It is also sometimes possible to derive the scoring function without having to guess it. To do this, you would write an exchange proof, but replace $w_i/t_i$ with the function $f_i$. Then you see what you need to set $f_i$ to in order to get a contradiction. $f_i=w_i/t_i$