Divide and Conquer Introduction

1 Learning Goals

Describe Divide and Conquer Structure [DC2]
Create recurrence relation for runtime of Divide and Conquer Algorithms [DC1]
Use tree formula to solve recurrence relations [DC1]

2 Divide and Conquer

2.1 Structure

A divide and conquer algorithm is a recursive algorithm that divides the original problem into equal sized parts, recursively solves each part, and then combines the solutions to the parts to get a solution to the whole.

In fact, most of you are familiar with a divide and conquer algorithm, MergeSort:

Mergesort(A)
# Input: Integer array A of size n
# Output: Sorted array

# Base case:
if A==1:
    return A

# Preprocessing
# None for MergeSort

# Divide and Conquer
A1=MergeSort(A[1:n/2])
A2=MergeSort(A[n/2+1:n])

#Combine
p1,p2=1
for i=1 to n:
    if A1[p1]<A2[p2]:
        A1[i]=A1[p1]
        p1++
    else:
        A[i]=A2[p2]
        p2++

It is not important at this point that you remember the details of MergeSort. Instead, let’s look at the general structure of the algorithm.

[DC2]

Base case: (lines 5-7) To figure out when it is necessary to use the base case, you need to think about when is the input too small to divide any further?
Divide and Conquer: (lines 12-14) The heart of the divide and conquer algorithm. The input is divided into equal sized parts. (In this case, the array is divided into two halves, but in some cases, the input is divided into thirds or fourths, etc.) Then the algorithm is recursively run on each part.
Combine: (lines 16-24) In this step, you need to think about what additional work you need to do to combine the outputs of the recursive call (which you assume by induction correctly solves the problem on each part) in order to solve the entire problem. This step is usually the most difficult part of the algorithm to code and prove correctness of.
Preprocessing: (lines 9-10) In MergeSort, no additional steps need to be taken before the input is divided, but in some algorithms, a preprocessing step is needed.

Note that not all recursive algorithms are divide and conquer algorithms, for example, the recursive search algorithm below is not divide and conquer because the size of the subarray in the recursive call in line 15 is just 1 less than the original input, not a half, or a third (etc) of the size.

Search(A,t)
# Input: Array A, target t
# Output: Position of t in A, or 0 if not present

# Base case:
if A==1 and A[1]=t:
    return 1
else:
    return 0

# Recursion
if A[end]=t:
    return end
else
    return Search(A[1:end-1],t)

2.2 Analyzing the Runtime

2.2.1 Creating a Recurrence Relation

[DC1]

To analyze the runtime, you first need to determine the variable that determines the inputs size. In the case of MergeSort, it is \(n\), the size of the input array. Then our goal is to describe a function \(T(n)\), which is the runtime of the algorithm on an array on size \(n\).

We first describe \(T(n)\) using a recurrence relation, and then we solve the recurrence relation. To create the recurrence relation, we separately analyze the base case and the inductive step, and use big-Oh, notation, or \(T\), to describe the runtime.

Base case: (lines 5-7) Runs in time \(O(1)\)
Divide and Conquer: (lines 12-14) When we have a recursive call, we use \(T\) to describe the runtime. In this case, the input size to the recursive call is \(n/2\), so the runtime from these lines is \(T(n/2)+T(n/2)=2T(n/2)\). This is acceptable as long as the input to \(T\) is less than the original input size \(n\), because we are creating a recurrence relation.
Combine: (lines 16-24) Line 17 takes \(O(1)\) time. Then working from the inside of the for loop outwards, we see that inside the for loop we do \(O(1)\) work each iteration. The for loop iterates \(O(n)\) time, giving us \(O(n)\) total work. All together the combine step takes \(O(1)+O(n)=O(n)\) time.

Putting the base case and the recursive runtimes together, we have \[ T(n)= \begin{cases} O(1)&\textrm{if }n= 1\\ 2T(n/2)+O(n)&\textrm{if }n> 1\\ \end{cases} \]

Important

When analyzing the runtime of the recursive part of the algorithm, any non-recursive work should be described with big-Oh notation, and can be combined into a single big-Oh term using standard big-Oh rules. The terms with \(T\) in them should then be added to the single big-Oh term. If you have multiple \(T\) terms with the same input, you can combine them. For example, \(T(n/3)+T(n/3)=2T(n/3).\) However, you can not combine multiple \(T\) terms that have different inputs. For example, \(T(n/2)+T(n/3)\) can not be combined.

Additionally, since \(T\) might not be a linear function, you can not distribute the factor outside the \(T\) inside the \(T.\) In other words, you can NOT do \(2(T/3)=T(2/3)\) (NO!!) You must keep the \(2\) out front.

But for big-Oh, you can distribute the factor that is outside the big-Oh. So for example, \(nO(n)=O(n^2)\). (OK!!)

2.2.2 Solving a Recurrence Relation

Now that we have created a recurrence relation, we want to solve it to get a single big-Oh term that bounds \(T(n)\). There are two methods for doing this: the tree method/formula (aka master method) or the expand and hope method. (For a refresher on the expand and hope method, see this review video.) In this class we will be focusing on the tree method:

Tree Formula/Tree Method

If a recurrence relation takes the following form: \[\begin{align} T(n)= \begin{cases} O(1) \textrm{ if }n\leq c\\ aT(n/b)+O(n^d) \textrm{ else } \end{cases} \end{align}\] for constants \(a\), \(b\), \(c\), and \(d\), read off the values for \(a\), \(b\), and \(d\) from your recurrence relation, and use the following expressions to determine the big-Oh behavior of \(T\): \[\begin{align}\label{eq:tree} T(n)&=\begin{cases} O(n^d\log_b n )&\textrm{if } a=b^d\\ O(n^d)&\textrm{if } a<b^d\\ O(n^{\log_ba})&\textrm{if } a>b^d \end{cases} \end{align}\]

For MergeSort, we found: \[ T(n)= \begin{cases}& O(1) &\textrm{ if }n\leq 1 \\ &2T(n/2)+O(n) &\textrm{ else} \end{cases} \] so in this case \(a=2\), \(b=2\), \(c=1\) and \(d=1\), so \(a=b^d\) and the runtime is \(O(n\log_2 n)=O(n\log n)\).

ABCD Question

[DC1]

We add two additional lines at the end of MergeSort to get WackyMergeSort:

A1=MergeSort(A[1:n/2])
A1=MergeSort(A[1:n/2])

What is the runtime of WackyMergeSort

\(O(n)\)
\(O(n\log n)\)
\(O(n^2)\)
\(O(n^4)\)