Analysis of Algorithms - Final (Solutions)

K. Subramani LCSEE, West Virginia University, Morgantown, WV {ksmani@csee.wvu.edu}

1 Problems

1. Induction and Recurrences:

(a) Professor Rabinowitz claims that the following property is true of all positive integers n: Either n is a power of 2, or there is some number between n and 2 · n, which is a power of 2. Prove that the Professor is correct.

(b) Solve the following recurrence (exactly or asymptotically):

T (n) = 1, if n = 1
= 2 · T (n − 1) + 1, if n ≥ 2.

Solution:

(a) Let P (n) denote the proposition that either n is a power of 2, or there exists some number between n and 2 · n, which is a power of 2. Observe that P (1) is true by inspection. Now assume that P (n) is true for some fixed k, where 2 ≤ n ≤ k, and consider n = k + 1. If k + 1 is a power of 2, we are done. Assuming it is not, we consider the following cases, which are exhaustive and mutually exclusive.

(i) k is a power of 2 − Note that 2 · k is also a power of 2 and further that (k + 1) ≤ 2 · k ≤ 2 · (k + 1), for k ≥ 1. It follows that there is a power of 2, between (k + 1) and 2 · (k + 1) and hence P (k + 1) is true.

(ii) k is not a power of 2 - By the inductive hypothesis, there must be a power of 2 between k and 2 · k. Let us call this number l; clearly, (k + 1) < l < 2 · k < 2 · (k + 1), since we have assumed that k + 1 is not a power of 2. It follows that P (k + 1) is true.

We thus seee that P (k) → P (k + 1) and applying the first principle of mathematical induction, we can conclude that the Professor is correct.

(b) (i) Exact solution - Applying the definition, we see that T (1) =, T (2) = 3, T (3) = 7 and thus it seems reasonable to conjecture that T (n) = 2^{n} − 1. We now proceed to prove the correctness of this conjecture using induction. Observe that the basis is true, since T (1) = 1 = 2^{1} − 1. Assume that T (k) = 2^{k} − 1. Now,

T (k + 1) = 2 · T (k) + 1, by definition
= 2 · (2^{k} − 1) + 1, by the inductive hypothesis
= 2^{k+1} − 1

Applying the first principle of mathematical induction, we can conclude that our conjecture is correct. (ii) We can also solve this problem using the Master Theorem. Substitute n = log_{2} k, where we assume without

loss of generality that k is a power of 2. It follows that n = 2^{k} and that (n − 1) = log_{2} \frac{k}{2} . We can then rewrite the given recurrence as:

T (log_{2} k) = 1, if log_{2} k = 1
= 2 · T (log_{2} \frac{k}{2} ) + 1, if log_{2} k ≥ 2

Substituting G(k) = T (log_{2} k), we get,

G(k) = 1, if k = 2
= 2 · G(\frac{k}{2} ) + 1, if k ≥ 4

The recurrence is now in the form required by the Master theorem and applying the theorem, we conclude that G(k) ∈ Θ(k), from which it follows that T (n) ∈ Θ(2^{n}).

□

2. Counterexamples: In class, we showed that the fractional knapsack problem can be solved efficiently using a greedy approach. Now consider the 0/1 knapsack problem, in which the objective (maximizing profit) and constraint (respecting knapsack capacity) are identical to the fractional knapsack problem; however, an object is either completely selected or completely discarded, i.e., you cannot select a fraction of an object. Does the greedy strategy discussed in class result in an optimal solution for the 0/1 knapsack problem? If so, provide a proof of the same; if not, provide a numeric counterexample.

Solution: Consider three objects o =< o_{1}, o_{2}, o_{3} >, with respective profit values p =< 50, 60, 140 >, respective weights w =< 5, 10, 20 > and knapsack capacity M = 30. The greedy approach that succeeded in the fractional case will choose objects o_{1} and o_{3} (with profit 190), whereas the optimal solution is to choose o_{2} and o_{3} (with profit 200). It follows that the greedy approach is not optimal for the 0/1 knapsack problem. □

3. Graph Theory: Let G =< V, E > denote an undirected, unweighted graph. The square of G is the graph G^{2} =< V, E^{2} > defined as follows: G^{2} has the same vertex set as G. (v_{i}, v_{j}) is an edge in G^{2}, if and only if, either (v_{i}, v_{j}) is an edge in G or (inclusively) there exists a vertex v_{k}, such that (v_{i}, v_{k}) and (v_{k}, v_{j}) are edges in G. Design an efficient algorithm to compute G^{2}, given that G is stored using an adjacency matrix.

Solution: Consider Algorithm (1.1) which computes the adjacency matrix M_{2} of G^{2}.

Function Find-Square-Graph (M = G) 1: We assume that G is given to us an adjacency matrix M of dimensions n×n, where n represents the number of vertices

in G. 2: {We first initialize the M_{2} matrix.} 3: for (i = 1 to n) do 4: for (j = 1 to n) do 5: M_{2}[i][j] = M[i][j]. 6: end for 7: end for 8: for (i = 1 to n) do 9: for (j = 1 to n) do 10: for (k = 1 to n) do 11: if (M[i][k] = 1 and M[k][j] = 1) then 12: M_{2}[i][j] = 1. 13: end if 14: end for 15: end for 16: end for

Algorithm 1.1: Computing the Square of a graph

Algorithm (1.1) clearly runs in time O(n^{3}). □

4. Sorting and Searching: Explain briefly how Randomized Quickselect selects the k^{th} smallest element of an n-element integer array, using O(n) comparisons, in the expected case.

Solution: Let A[1 · ·n] denote the array; without loss of generality, assume that the elements of A are distinct. Recall that the Randomized Quickselect algorithm chooses a random element as the pivot and partitions the elements of A about this element. Let j be the index at which the pivot element is placed, after the partitioning. If j = k, the algorithm returns A[j]. If k < j, then the algorithm recurses on a smaller array, by finding the k^{th} smallest element in A[1 · ·j]. Finally, if k > j, the algorithm recurses by finding the (k − j)^{th} smallest element in A[(j + 1) · ·n].

Let t(n) denote the number of comparisons performed by the Randomized Quickselect algorithm on an array of n elements, in the worst case. Likewise, let b(n) denote the worst-case number of comparisons in a single partitioning step and let g(n) denote the worst-case number of recursive invocations that need to be made, before the size of the array drops to \frac{3·n}{4}. It follows that

t(n) ≤ g(n) · b(n) + t(\frac{3 · n}{4})  (1)

In a partitioning step involving n elements, at most n comparisons are made and hence b(n) ≤ n. Taking expectations on both sides of Equation (1), we get,

E[t(n)] ≤ E[g(n)] · n + E[t(\frac{3 · n}{4})]

If the pivot element is chosen uniformly and at random, with probability at least one half, an element whose rank is between \frac{1}{4}n and \frac{3}{4}n is chosen; any such element immediately results in the size of the array decreasing to at most \frac{3}{4}n and thus E[g(n)] = 2. Setting T (n) = E[t(n)], we get,

T (n) ≤ 2 · n + T (\frac{3 · n}{4})  (2)

Equation (2) can be solved using the Master Theorem to get T (n) ∈ Θ(n); we have thus shown that the Randomized Quickselect algorithm performs O(n) comparisons in the expected case.

□

5. Greedy Strategy: The problem of Coin changing is concerned with making change for a specified coin value using the fewest number of coins, with respect to the given coin denominations. Assume that your coin denominations are quarters (25 cents), dimes (10 cents), nickels (5 cents) and pennies (1 cent) and that you have an infinite supply of the same. For instance, given a coin value of 31 cents, the optimal change value is 3 coins, viz., 1 quarter, 1 nickel and 1 penny. (Any other breakup will result in more than 3 coins!) Design a greedy algorithm for the Coin changing problem and argue its correctness.

Solution:

Algorithm (1.2) represents our greedy strategy.

Function Opt-Coin-Change (n) 1: We assume that n represents the coin value that needs to be converted into quarters, dimes, nickels and pennies. 2: c_{q} = ⌊\frac{n}{25}⌋; n = n − c_{q} · 25. 3: c_{d} = ⌊\frac{n}{10}⌋; n = n − c_{d} · 10. 4: c_{n} = ⌊\frac{n}{5}⌋; c_{p} = n − c_{n} · 5. 5: return(c_{d} + c_{d} + c_{n} + c_{p})

Algorithm 1.2: Optimal Coin changing

In effect, Algorithm (1.2) converts as much of n into quarters. This is followed by converting as much of what is left over into dimes. Likewise, as much of the remainder that can be converted into nickels is so converted, with the remaining value represented by pennies. We need to argue that this strategy converts n into change, using the fewest number of coins.

Let c =< c_{q}, c_{d}, c_{n}, c_{p} > represent the greedy solution and let c′ =< c′_{q}, c′_{d}, c′_{n}, c′_{p} > represent the output of a supposedly optimal algorithm (called A), such that c′_{q} + c′_{d} + c′_{n} + c′_{p} < c_{q} + c_{d} + c_{n} + c_{p}.

We make the following crucial observation. Any optimal solution to the Coin changing problem must satisfy the following criteria (Why?):

(i) The number of pennies is at most 4,

(ii) The number of nickels is at most 1, and

(iii) The number of dimes is at most 2.

We refer to the above criteria as the optimality conditions; it is clear that the greedy strategy satisfies the optimality conditions.

Observe that c′_{q} ≤ c_{q}, since c_{q} = ⌊\frac{n}{25}⌋ represents the maximum amount that can be converted into quarters. If c′_{q} < c_{q}, we can convert (c_{q} − c′_{q}) · 25 worth of coin value into quarters; doing so does not increase c′_{d}, c′_{n} or c′_{p}. To see this, note that each quarter that is missing in c′_{q} has been distributed among its dimes and nickels. Further as per the optimality conditions, a quarter can only be represented as 2 dimes and a nickel; replacing these three coins by a single coin clearly decreases the number of coins in A’s solution, thereby contradicting its optimality. In other words, if A is optimal, c_{q} = c′_{q}. Likewise, given that c_{q} = c′_{q}, we must have c′_{d} ≤ c_{d}. If c′_{d} < c_{d}, then each dime that is missing in c′_{d} is distributed among the nickels of A’s output; however, since a single dime corresponds to two nickels, we can always increase c′_{d} to c_{d} while decreasing c′_{n} and not increasing c′_{p}. Thus, if A is to be optimal, we must have c_{d} = c′_{d}. An identical argument establishes that c_{n} must equal c′_{n}, from which it follows that c_{p} is equal to c′_{p}. In other words, the hypothesis that the number of coins output by A is less than the number of coins output by the greedy algorithm is false. We can therefore conclude that the greedy algorithm is optimal.

□