Design of Algorithms - Homework III (Solutions)

K. Subramani LCSEE, West Virginia University, Morgantown, WV {ksmani@csee.wvu.edu}

1 Problems

1. Professor Krustowski claims to have discovered a new sorting algorithm. Given an array A of n numbers, his algorithm breaks the array into 3 equal parts of size \frac{n}{3}, viz., the first third, the middle third and the bottom third. It then recursively sorts the first two-thirds of the array, the bottom two-thirds of the array and finally the first two-thirds of the array again. Using mathematical induction, prove that the Professor has indeed discovered a correct sorting algorithm. You may assume the following: The input size n is always some multiple of 3; additionally, the algorithm sorts by brute-force, when n is exactly 3. Formulate a recurrence relation to describe the complexity of Professor Krustowski’s algorithm and obtain tight asymptotic bounds.

Solution: As per the specifications, the algorithm works correctly, when n ≤ 3.

Assume that the algorithm works correctly, as long as 1 ≤ n ≤ k, where k > 3.

Now consider the case where the array A contains k + 1 elements.

We divide the array A[1··k+1] (conceptually) into the following regions: L_{1} : A[1··\frac{k+1}{3}], L_{2} : A[\frac{k+1}{3} +1··\frac{2}{3} ·(k+1)] and L_{3} : A[\frac{2}{3} · (k + 1) + 1 · ·k + 1]. Thus, the first recursive invocation is called on L_{1} ∪ L_{2}, the second recursive invocation is called on L_{2} ∪ L_{3}, and the third recursive invocation is called on L_{1} ∪ L_{2} again.

By the inductive hypothesis, the first recursive invocation will return a correctly sorted array, since the size of the input array is at most k. (Note that \frac{2}{3} ·(k +1) ≤ k as long as k ≥ 2.) It follows that after the first recursive invocation, each element in L_{2} is at least as large as every element in L_{1}. Arguing similarly, we observe that the second recursive invocation correctly sorts the set L_{2} ∪ L_{3}; further every element in L_{3} is at least as large as each element in L_{2}. From the correctness of the first recursive invocation, we can therefore conclude that each element in L_{3} is at least as large as every element in L_{1} ∪ L_{2}; further as per the inductive hypothesis, all the elements in L_{3} have been correctly sorted. The third recursive invocation completes the sorting procedure, since by the inductive hypothesis, it correctly sorts L_{1} ∪ L_{2} and thus the set L_{1} ∪ L_{2} ∪ L_{3} is correctly sorted.

Let T (n) denote the running time of Professor Krustowski’s algorithm. We have,

T (n) = 3, n ≤ 3
= 3 · T (\frac{2}{3} · n), otherwise

The recurrence relation fits the template of the Master Theorem, with a = 3, b = \frac{3}{2} and f(n) = 0.

It is easy to see that f(n) = O(n^{(log_{1.5} 3)−ϵ}) for some ϵ > 0, so T (n) ∈ Θ(n^{2.7}) by the master theorem (case 1).

□

2. Assume that you are given a chain of matrices ⟨A_{1} A_{2} A_{3} A_{4}⟩, with dimensions 2 × 5, 5 × 4, 4 × 2, and 2 × 4, respectively. Compute the optimal number of multiplications required to calculate the chain product and also indicate what the optimal order of multiplication should be using parentheses.

Solution: Let m[i, j] denote the optimal number of multiplications to multiply the chain ⟨A_{i}, A_{i+1}, …, A_{j}⟩, where matrix A_{i} has dimensions d_{i−1} × d_{i}. As per the discussion in class, we know that

m[i, j] = 0, if j ≤ i
= min_{k:i≤k<j} m[i, k] + m[k + 1, j] + d_{i−1} · d_{k} · d_{j}

Computing M = [m[i, j]], i = 1, 2, 3, 4; j = i, i + 1, …, 4, in bottom-up fashion, we get,

M = [ \begin{matrix}0 & 40 & 56 & 72 \\ 0 & 0 & 40 & 80 \\ 0 & 0 & 0 & 32 \\ 0 & 0 & 0 & 0 \end{matrix} ]

By the above table, the optimal number of multiplications to multiply the given chain is 72; by recording the split values, we see that the optimal order of multiplication is (((A_{1} · A_{2}) · A_{3}) · A_{4}).

□

3. A hiker has a choice of n objects {o_{1}, o_{2}, …, o_{n}} to fill a knapsack of capacity W. Object o_{i} has benefit p_{i} and weight w_{i}. A subset of objects is said to be feasible if the combined weight of the objects in the subset is at most W. The hiker’s goal is to select a feasible subset of objects such that the benefit to him is maximized (benefits are additive). Note that an object cannot be selected fractionally; it is either selected or not. Design a dynamic program to help the hiker.

Solution: We first formulate the problem as an integer program. Let x_{i} denote a variable, which is 1 if the hiker selects object o_{i} and 0, otherwise.

Accordingly, the benefit to the hiker is: ∑_{i=1}^{n} p_{i} · x_{i} and the cumulative weight of the objects in the knapsack is: ∑_{i=1}^{n} w_{i} · x_{i}. Since, the capacity constraint of the knapsack cannot be violated, we must have ∑_{i=1}^{n} w_{i} · x_{i} ≤ W.

Thus the integer program is:

max ∑_{i=1}^{n} p_{i} · x_{i}

∑_{i=1}^{n} w_{i} · x_{i} ≤ W.

Let S_{i} denote that subset of the set of objects, that includes the first i objects only. In other words, S_{0} = ∅, S_{1} = {o_{1}}, S_{5} = {o_{1}, o_{2}, …, o_{5}} and so on.

Define m[i, w] to be the maximum benefit that can be reaped using only the objects in S_{i} and a knapsack of capacity w. The following recurrences follow naturally:

m[0, w] = 0
m[i, 0] = 0
m[i, w] = m[i − 1, w], if w_{i} > w
m[i, w] = max{m[i − 1, w], m[i − 1, w − w_{i} + p_{i}}, if w_{i} ≤ w

The first equality states that if there are no objects at all to choose from, then regardless of the capacity of the knapsack, the benefit reaped is 0. Likewise, the second equality states that if the knapsack has capacity 0, it does not matter how many objects you select; the accrued benefit is 0. The third equality states that if the o_{i} has weight exceeding the capacity of the knapsack, then we might as well focus on the first (i − 1) objects. If w_{i} ≤ w, then we have two choices, viz., we could exclude object o_{i}, in which case the benefit accrued is m[i − 1, w] or we could include object o_{i}, in which case the benefit accrued is m[i − 1, w − w_{i}] + p_{i}.

As with the other dynamic programs that we studied, we can construct a table for m[i, j] and compute m[n, W], which is the entry that we are interested in. Since each entry can be computed in O(1) time, and there are (n + 1) · W entries in the table, we can implement the dynamic program in O(n · W) time. □

4. Let T denote a binary search tree. Show that

(a) If node a in T has two children, then its successor has no left child and its predecessor has no right child.

(b) If the keys in T are distinct and x is a leaf node and y is x’s parent, then y · key is either the smallest key in T larger than x · key, or the largest key in T smaller than x · key.

Solution:

(a) Since a has two children, its right subtree is non-empty. Its successor therefore is the leftmost node in the right subtree. But by definition, the leftmost node has no left child. A similar argument shows that the predecessor of a has no right child.

(b) Since the keys in T are distinct, either x · key < y · key, or y · key < x · key.

Assume that x · key < y · key; it follows that x is the left child of y. Furthermore, assume the existence of a node z in T, such that x · key < z · key < y · key. Let r denote the root of T. Clearly, z must be in the same subtree of r (left or right) as y. (Why?) Then observe, that z must be an ancestor of y, because it cannot be part of the left subtree of y (x is the only node in the left subtree), and at the same time, z · key < y · key, i.e., z cannot be part of the right subtree of y either. Finally note that y cannot be part of the left subtree of z, since y · key > z · key and y cannot be part of the right subtree of z, since that would imply that x · key > y · key, contradicting our initial assumption. In other words, such a z cannot exist. We conclude that y · key is the smallest key in T, that is larger than x · key. A similar argument applies to the case in which y · key < x · key.

□

5. An AVL tree is a binary search tree that is height balanced: for each node x, the heights of the left and right subtrees of x differ by at most 1. Prove that an AVL tree with n nodes has height O(log n). Solution: We need the following lemma.

Lemma 1.1 In an AVL tree of height h, there are at least F_{h} nodes, where F_{h} is the h^{th} Fibonacci number.

Proof: The base h = 0 is easy: a tree of height 0 has either 1 or 0 nodes, and F_{0} = 0. Now assume that the claim is true for all AVL trees of height h. Let T be an AVL tree of height h + 1. Let T_{l} and T_{r} be the left and right subtrees of the root of T, respectively. The AVL condition implies that T_{l} and T_{r} are both AVL trees. Since T has height h + 1, at least one of T_{l}, T_{r} has height h and neither has a height greater than h. Thus, using the AVL condition, either both have height h, or one has height h and the other has height h − 1. By induction, the number of nodes in T is at least

F_{h} + F_{h−1} + 1,

which is at least F_{h+1}. □ Now we use a standard fact about the Fibonacci numbers:

F_{i+2} ≥ ϕ^{i} for i ≥ 0

where ϕ(≈ 1.61803399) is the golden ratio (see Section 4-5 of [CLRS09]). Combining Lemma 1.1 and this fact, we see that an AVL tree of height h ≥ 2 has at least ϕ^{h−2} many nodes. This implies that an AVL tree with n nodes has height at most

⌊\frac{log n}{log ϕ} + 2⌋,

which is O(log n). □

References

[CLRS09] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. Introduction to Algorithms. MIT Press, 3^{rd} edition, 2009.