Divide and Conquer Introduction

In-class notes

1 Learning Goals

  • Describe Divide and Conquer Structure
  • Describe methods for proving correctness and analyzing time complexity (runtime) of divide and conquer algorithms

2 Divide and Conquer

2.1 Structure

A divide and conquer algorithm is a recursive algorithm that divides the original problem into equal sized parts, recursively solves each part, and then combines the solutions to the parts to get a solution to the whole.

In fact, most of you are familiar with a divide and conquer algorithm, MergeSort:

Mergesort(A)
# Input: Integer array A of size n
# Output: Sorted array

# Base case:
if A==1:
    return A

# Preprocessing
# None for MergeSort

# Divide and Conquer
A1=MergeSort(A[1:n/2])
A2=MergeSort(A[n/2+1:n])

#Combine
p1,p2=1
for i=1 to n:
    if A1[p1]<A2[p2]:
        A1[i]=A1[p1]
        p1++
    else:
        A[i]=A2[p2]
        p2++

It is not important at this point that you remember the details of MergeSort. Instead, let’s look at the general structure of the algorithm.

  • Base case: (lines 5-7) there is a non-recursive base case that kicks in when the input is small enough.
  • Divide and Conquer: (lines 12-14) The heart of the divide and conquer algorithm. The input is divided into equal sized parts. (In this case, the array is divided into two halves, but in some cases, the input is divided into thirds or fourths, etc.) Then the algorithm is recursively run on each part.
  • Combine: (lines 16-24) The outputs of the divide and conquer step need to be combined to create the solution for the entire input. This step is usually the most difficult part of the algorithm to code and prove correctness of.
  • Preprocessing: (lines 9-10) In MergeSort, no additional steps need to be taken before the input is divided, but in some algorithms, a preprocessing step is needed.

Note that not all recursive algorithms are divide and conquer algorithms, for example, the recursive search algorithm below is not divide and conquer because the size of the subarray in the recursive call in line 15 is just 1 less than the original input, not a half, or a third (etc) of the size.

Search(A,t)
# Input: Array A, target t
# Output: Position of t in A, or 0 if not present

# Base case:
if A==1 and A[1]=t:
    return 1
else:
    return 0

# Recursion
if A[end]=t:
    return end
else
    return Search(A[1:end-1],t)

2.2 Proving Correctness

A divide and conquer algorithm can be proven correct using strong induction. We need strong induction because the size of the input in the recursive call is potentially much less than the size of the input to the original problem. For the Search algorithm above, because the input to the recursive call is an array with size just one less than the size of the original array, regular induction could be used.

If you need a refresher on how to use strong induction to prove the correctness of a recursive algorithm, check out this review video.

2.3 Analyzing the Runtime

To analyze the runtime, you should create a recurrence relation of the runtime, and then solve using the tree formula (aka master method) or the expand and hope method. (For a refresher on the expand and hope method, see this review video.)

For MergeSort, if \(T(n)\) is the runtime for an array of size \(n\), the recurrence relation is: \[ T(n)= \begin{cases}& O(1) &\textrm{ if }n\leq 1 \\ &2T(n/2)+O(n) &\textrm{ else} \end{cases} \tag{1}\]

where the base case runtime applies if the input array size is at most 1, and the runtime in the general case involves the runtime on an input of size \(n/2\), since we do recursive calls on inputs of size \(n/2\).

Once you have created a recurrence relation for the runtime like Equation 1, you still need to solve it to get an expression for the runtime that does not have \(T\) on the right side of the equation.