CMPS 144
Recursion: Calculating Asymptotic Running Time

To derive the asymptotic running time (e.g., O(n), O(n×log n)) of an algorithm, we identify a "basic operation" that, when the algorithm is executed, will be applied at least as often as any other. We then figure out how many times that operation will be applied as a function of the size of the input data.

For an algorithm that involves loops, this translates into figuring out how many loop iterations will occur. A different approach is needed for a recursive algorithm, however, because the mechanism by which the basic operation gets applied over and over again is not a loop but rather the descent and ascent of recursion.

The standard approach for calculating a recursive algorithm's asymptotic running time is to characterize it using a recurrence relation and then to solve that relation. To illustrate, we show some examples.

Example #1: Recursive Array Sum

/* Returns the sum of the elements in the specified ** array segment (namely, a[low..high)). ** pre: 0 ≤ low ≤ high ≤ a.length */ public static int segSum(int[] a, int low, int high) { if (low == high) { return 0; } // empty segment else // The sum of a nonempty segment is its // first element plus the sum of the rest. { return a[low] + segSum(a, low+1, high); } }

Examining the Java method to the right, we intuit that a reasonable measure of input size is the length of the segment whose elements are to be added together. By convention, we use n to refer to that size. In terms of the method's formal parameters, n = high − low.

The two branches of the method's if-else statement correspond to the base and recursive cases, respectively. The base case applies when n = 0; the recursive case when n > 0.

If the base applies, the work carried out by the method includes testing the condition low == high and, upon finding it to be true, returning zero. Each of these is a "basic step" (as Gries calls them) that takes constant time. We conclude that, when the base case applies, the method runs in constant time. Let's call this constant b.

If the recursive case applies, the work carried out by the method includes testing the condition low == high and, upon finding it to be false, computing low+1 in preparation for passing that value as a parameter, making a recursive call to itself, adding the returned result of that call to a[low], and, finally, returning the result. Each of these computations takes constant time. Let's call their total c. But in estimating how much time all this takes, we must account for the fact that "this" instance of the method must "wait", after it calls itself recursively, for "that" instance of the method to complete its execution. And how long will that be?

Well, let's "define" T(n), for all n≥0, to be the amount of "time" that our method needs to compute the sum of the elements in an array segment of length n.

In the base case, when n is zero, we reasoned above that that time would be the constant b. As for the recursive case, we reasoned above that it would be the constant c plus the time for the "recursive instance" of the method to complete its work. But that recursive instance of the method computes the sum of an array segment of length n−1. (If high-low = n, then high - (low+1) = n-1.) Thus, it should take time T(n-1)!

Summarizing, our reasoning has led us to characterize the running time of our method using this recurrence relation:

T(n) = {	b	if n=0	(1)
	T(n−1) + c	otherwise (i.e., n>0)	(2)

T(n) = < (2) applies > T(n-1) + c = < (2) applies > (T(n-2) + c) + c = < algebra > T(n-2) + 2c = < (2) applies > (T(n-3) + c) + 2c = < algebra > T(n-3) + 3c = ... ... = < recognition of pattern > T(n-k) + k·c = < choosing k to be n > T(n-n) + n·c = < algebra > T(0) + cn = < (1) applies > b + cn

Of course, this description of T is itself recursive, as its "right-hand side" refers to T. We would like to obtain a description of T whose right-hand side does not refer to itself, but rather is expressed in terms of standard arithmetic operators such as addition, multiplication, and exponentiation. Obtaining such a description from a recurrence relation is called "solving" it. One way to solve a recurrence relation is to apply the repeated substitution approach. More sophisticated and powerful techniques are available, but this one works on most recurrence relations that describe algorithm running times. To the right, we show how repeated substitution can be used to solve T. We begin by assuming that n is "large" (i.e. so that even after several applications of recursive case (2), the base case (1) will not be reached).

After applying (2) (followed by algebraic simplification) several times, a clear pattern emerges suggesting that the equation T(n) = T(n−k) + k·c holds. Indeed, it does, for all k in the range [0..n], which can be proved by mathematical induction.

At that point, we chose k to be n so as to arrive at T(n) = T(n−n) + n·c. Why did we make that choice? So that T's argument would become zero, which then makes it possible to apply (1), the base case, so as to make T "disappear"! And what we are left with is T(n) = b + cn. What this tells us is that our method runs in linear time, which, using Big-O notation, is O(n).

To generalize from this example, the repeated substitution method works like this: You repeatedly apply the recursive case of the recurrence relation until a pattern becomes evident, at which point you plug into that pattern a value for the "pattern variable" (in this example, k) that makes the base case of the recurrence applicable. Applying that base case yields an expression that no longer refers to the recurrence relation by name.

Example #2: Binary Search

/* Returns k, where a[0..k) < x ≤ a[k..a.length). ** That is, it returns the lowest-numbered location in ** a[] containing a value that is ≥ the search key x ** (or a.length if x > all array elements). ** ** pre: Elements in a[] are in ascending order ∧ ** 0≤low≤high≤a.length ∧ ** a[0..low) < x ≤ a[high..a.length) */ public static int binSearch(int[] a, int low, int high, int x) { if (low == high) { return high; } else { int mid = (low + high) / 2; if (a[mid] < x) { return binSearch(a, mid+1, high, x); } else // a[mid] ≥ x { return binSearch(a, low, mid, x); } } }

As in Example #1, the appropriate measure of input size n is the length of the array segment that is provided via the method's parameters, which is high−low.

In the base case, which applies when n = 0, the amount of work done by the method is a constant. Let's call it b. In each of the two recursive cases, the amount of work done is some constant, call it c, plus whatever work is done by the recursively called instance of the method. Notice that, in the second (respectively, third) branch of the if-else statement, the array segment described by the first three parameters in the recursive call has length high − (mid+1) (resp., mid−low). Each of these lengths is at most n/2 (i.e., (high−low)/2)) due to the fact that mid is halfway between low and high.

As in Example #1, let's say that T(n) is the amount of work (or time) spent by this method to search an array segment of length n. The reasoning in the paragraph above leads us to this recurrence:

T(n) = {	b	if n=0	(1)
	T(n/2) + c	otherwise (i.e., n>0)	(2)

T(n) = < (2) applies > T(n/2) + c = < (2) applies > (T(n/4) + c) + c = < algebra > T(n/4) + 2c = < (2) applies > (T(n/8) + c) + 2c = < algebra > T(n/8) + 3c = ... ... = < recognition of pattern > T(n/2k) + k·c = < choosing k to be log2n + 1 > T(n/2n) + (log2n + 1)·c = < algebra > T(0) + c·log2n + c = < (1) applies > b + c·log2n + c

On the right, we apply the repeated substitution approach to solve T. The pattern that emerges after several applications of the recursive case (2) is a little more difficult to spot here than it was in Example #1, but if you study recursive algorithms the "split in half" pattern will become familiar to you.

The result of the analysis is that T(n) is proportional to the logarithm (to the base 2) of n. In big-O notation, we say that binary search runs in O(log n) time. (We can omit the subscript on log because for any integers p and q greater than one, log_p n and log_q n differ by a constant factor.) This is a very slowly growing function, which indicates that our method will work quickly even if given a large array in which to search.

Example #3: Selection Sort

/* Rearranges the elements in the specified ** array segment to put them into ascending order. ** pre: 0 ≤ low ≤ high ≤ a.length */ public static int selectSort(int[] a, int low, int high) { if (low == high) { } // an empty segment is already in order else { int k = locOfMin(a, low, high); swap(a, low, k); selectSort(a, low+1, high); } }

Analyis yields this recurrence relation:

T(n) = {	b	if n=0	(1)
	T(n−1) + cn + d	otherwise (i.e., n>0)	(2)

The base case should be clear, as the only work carried out by the method when given an empty array segment is to compare low and high for the purpose of ascertaining that the segment is empty. The T(n−1) term in the recursive case comes from the recursive call, as it leads to the method recursively sorting an array segment of length n−1. The cn+d comes from the call to swap() (which takes constant time) and the call to locOfMin() (which takes linear time).

Solving it (see below), we end up with a sum whose dominant term is a constant multiplied by n². Thus, we conclude that the method's asymptotic running time is O(n²), which was to be expected (as we have already seen that the loop-based implementation of Selection Sort has the same asymptotic running time).

T(n) = < (2) applies > T(n-1) + cn + d = < (2) applies > (T(n-2) + c(n-1) + d) + cn + d = < algebra > T(n-2) + c(n + (n-1)) + 2d = < (2) applies > (T(n-3) + c(n-2) + d) + c(n + (n-1)) + 2d = < algebra > T(n-3) + c(n + (n-1) + (n-2)) + 3d = .... .... = < recognition of pattern > T(n-k) + c(n + (n-1) + (n-2) + ... + (n-k+1)) + kd = < choosing k to be n > T(n-n) + c(n + (n-1) + (n-2) + ... + (n-n+1)) + nd = < algebra > T(0) + c(n + (n-1) + (n-2) + ... + 1) + dn = < 1 + 2 + ... + n = (n2 + n)/2 > T(0) + c·(n2+n)/2 + dn = < (1) applies > b + c·(n2+n)/2 + dn = < algebra > cn2/2 + cn/2 + dn + b