CMPS 144  (Computer Science 2)
QuickSort: A Recursive Sorting Algorithm

Background

C.A.R. (Tony) Hoare, one of the world's most prominent computer scientists, discovered the QuickSort algorithm around 1960. Initially, he was unable to express it in a way that he found satisfying. After learning about the programming language Algol-60, which appeared at about the same time and which supported the mechanism of recursion (allowing a subprogram to call itself), Hoare quickly realized that his algorithm could be expressed quite simply and elegantly in a recursive fashion. Although QuickSort's running time, in the worst case, is bad (technically, O(n2)), its running time in the "average" case is very good (O(n lg(n))). Indeed, it is still considered to be, in practice, the fastest known general purpose sorting algorithm.

Development of the Algorithm

Suppose we have an array a[] of int's that is to be sorted. (That is, we are to rearrange the elements of a[] so that they occur in ascending order.) Let's say that we choose an int value z, which we will call the pivot. We call it the pivot because our first step is to partition a[] with respect to z. That is, we rearrange the elements of a[] so that any element less than z occurs to the left of (i.e., at a lower-numbered location) than any element equal to or greater than z. This is illustrated by

        0                       k                        n
       +-----------------------+------------------------+
     a |          < z          |          >= z          |          (1)
       +-----------------------+------------------------+  

That is, the result of partitioning a[] with respect to the pivot z is to establish, for some k, that every element in a[0..k) (the "left" segment) is less than z and every element in a[k..n) (the "right" segment) is not less than z. (Our choice to place elements equal to z on the right is arbitrary; we could just as well have placed them on the left, or even allowed them to occur on either side.)

Notice that the problem of partitioning a[] in this way is nothing more than the 2-color Dutch National Flag problem, as discussed in class. (We classify any value less than z as RED and any value not less than z as BLUE.)

Having partitioned a[] as indicated, to obtain a sorted version of a[] it suffices to sort its RED segment a[0..k) and, independently, its BLUE segment a[k..n). (If you doubt this statement, consider that, in order for it to be false, it would have to be that, to sort a[], some element from segment a[0..k) would have to be moved into segment a[k..n), and vice versa. But then we would have a sorted array in which some value not less than z occurred to the left of some value less than z. This is absurd.)

Summarizing, what we have so far is this sequence of steps for sorting an array a[0..n):

  1. Partition a[], thereby obtaining the value k, which indicates the location of the boundary between the resulting RED and BLUE segments.
  2. Sort a[0..k) (the RED segment) (recursively)
  3. Sort a[k..n) (the BLUE segment) (recursively)

Of course, the above addresses only the recursive case. A reasonable base case would be n < 2, because no action needs to be taken to sort an array of length less than two.

This leads us to the following Java method, which assumes the existence of a method partition() having the following specification:

/* pre:  none
*  post: Let N = a.length and let the value returned be p.  Then 0<=p<=N
*        and the elements of a[] have been rearranged so that no element
*        in a[0..p) is larger than any element in a[p..N).
*/
static int partition( int[] a ) { ... }


public void quickSort( int[] a ) {

   if (a.length < 2) { }
   else {
      int k = partition(a);
      quickSort( a[0..k) );
      quickSort( a[k..a.length) );
   }
}

Unfortunately, Java does not allow us to refer to array segments using notation such as a[0..k); thus, the above is syntactically incorrect. What we do to fix it is to give the method three parameters, one for the array and the other two to indicate the locations of the left and right boundaries of the segment that the method is supposed to sort. (For the same reason, we modify partition in the same way.)

We get the following refinement:

/* pre:  0 <= low <= high <= a.length
   post: a[low..high) has been sorted (into ascending order)
*/
public void quickSort( int[] a, int low, int high ) {

   if (high - low < 2) { }
   else {
      int k = partition( a, low, high );
      quickSort( a, low, k );
      quickSort( a, k, high );
   }
} 

There is, unfortunately, a flaw in the above. In order to be certain that this method terminates, we must ensure that each time it calls itself, the problem instance to be solved by that call is smaller than the "current" problem instance. (Because the current problem instance has size high - low, we must ensure that the problem instances to be solved via the recursive calls are less than that.) But, so far as we have described it, the partition() method is under no obligation to satisfy this. That is, the partition() method may very well rearrange the array segment so that the resulting RED segment is the entire segment and the resulting BLUE segment is empty, or vice versa.

To fix this, we strengthen the postcondition of partition() to require that it partitions a[low..high) into three segments: a[low..k) (RED segment), a[k..k] (of length one), and a[k+1..high)] (BLUE segment) such that all the values in the RED segment are less than a[k] and all those in the BLUE segment are not less than a[k]. This guarantees that the sizes of the resulting RED and BLUE segments are both less than high - low. Expressed as a picture:

   low                    k                         high
  +----------------------+-+-----------------------+
a |        < a[k]        | |        >= a[k]        |          (2)
  +----------------------+-+-----------------------+  

Assuming that partition() has this behavior, the element it places at location k will be in its "rightful" place; thus, it need not be included as part of the array segment to be sorted by either of the recursive calls. It follows that, to correct the quickSort() method above, we need only replace the second parameter in the second recursive call, k, by k+1:

/* pre:  0 <= low <= high <= a.length
   post: a[low..high) has been sorted (into ascending order)
*/
public void quickSort( int[] a, int low, int high ) {

   if (high - low < 2) { }
   else {
      int k = partition( a, low, high );
      quickSort( a, low, k );
      quickSort( a, k+1, high );
   }
} 

It remains to give the full details of the partition() method. As noted earlier, partitioning amounts to little more than the 2-color Dutch National Flag problem. However, recall that we recognized the need to strengthen its postcondition to correspond to the picture shown above.

For the moment, let's forget the need to strengthen the postcondition; instead, we simply translate the 2-color Dutch National Flag program covered in class (and presented in the "Loop Invariants" document available on the course web page), taking into account that RED translates into "is less than the pivot" and BLUE translates into "is not less than the pivot":

/* pre:  a[low..high) is nonempty (i.e., low < high)
*  post: Let value returned be p.  Then low <= p < high and
*        the elements of a[low..high) have been rearranged
*        so that every element in a[low..p) is less than every
*        element in a[p..high).
**/
public static int partition( int[] a, int low, int high ) {

   int k = low, m = low;
   int pivot = ?;

   /* loop invariant:
   *     low <= k <= m <= high  and
   *     all elements in a[low..k) are RED (i.e., < pivot) and
   *     all elements in a[k..m) are BLUE (i.e., >= pivot)
   */
   while (m != high)  {
      if (a[m] >= pivot)     // a[m] is BLUE
         { }
      else  {                // a[m] is RED
         swap(a,k,m);
         k = k+1;
      }
      m = m+1;
   }
   return k;
}

The above method has the following postcondition (and returns the value k to its caller):

   low                     k                        high
  +-----------------------+------------------------+
a |        < pivot        |        >= pivot        |        (3)
  +-----------------------+------------------------+ 

Recall that what we really want is a method whose postcondition is slightly stronger, as depicted above in (2), which we have repeated below as (4). How can we modify a method that produces the postcondition depicted in (3) into one that produces the postcondition depicted in (4)?

   low                    k                         high
  +----------------------+-+-----------------------+
a |        < a[k]        | |        >= a[k]        |          (4)
  +----------------------+-+-----------------------+  

Suppose we could work things out so that the pivot value ends up at location k. Then we'd have what we want! This requires, of course, that the pivot value be chosen to be one of the values in the array. (It doesn't really matter which one. We could choose a[low], for example, or even a[i] for some pseudo-randomly generated i in the range low..high-1.)

So, let's say that we've chosen a pivot value from the array and we've executed the loop that establishes the condition depicted by (3) above. How do we then get the pivot value into its proper place? One way would be to search for the pivot (which must occur somewhere in a[k..high)) and, upon finding it, to swap it with the value in location k. That would establish (4), as desired! Then, to complete the method, it would suffice to return the value k to the caller.

If we're a little more clever, however, we can avoid doing the post-loop search for the pivot within a[k..high). What if we were to choose a[high-1] as the pivot and then we were to partition the segment a[low..high-1) (i.e., terminating the loop when m reached high-1 rather than letting it make one more iteration). Then the condition established upon termination of the loop would be this:

   low                   k                         high
  +---------------------+----------------------+--+
a |     < a[high-1]     |     >= a[high-1]     |  |       (5)
  +---------------------+----------------------+--+ 

To achieve the desired postcondition (i.e., (4)), it would suffice to swap a[high-1] with a[k]! The correct value for the method to return would then be k. Here is the finished method:

/* pre:  a[low..high) is nonempty (i.e., low < high)
*  post: Let value returned be p.  Then low <= p < high and the
*        elements in a[low..high) have been rearranged so that
*        every element in a[low..p) is less than a[p] and
*        every element in a[p+1..high) is not less than a[p].
*/
public static int partition( int[] a, low, high ) {

   int k = low, m = low;

   /* loop invariant:
   *    low <= k <= m <= high and
   *    all elements in a[low..k) are RED (i.e., < a[high-1]) and
   *    all elements in a[k..m) are BLUE (i.e., >= a[high-1])
   */
   while (m != high-1)  {
      if (a[m] >= a[high-1])  // a[m] is BLUE
         { }
      else  {                 // a[m] is RED
         swap(a,k,m);
         k = k+1;
      }
      m = m+1;
   }
   /* all elements in a[low..k) are RED (i.e., < a[high-1]) and
   *  all elements in a[k..high-1) are BLUE (i.e., >= a[high-1])
   */

   swap(a, k, high-1);

   /* all elements in a[low..k) are RED (i.e., < a[k]) and
   *  all elements in a[k+1..high) are BLUE (i.e., >= a[k])
   */
   return k;
}

Note: Suppose that we were to restore the loop guard to m != high, as it was in our original formulation of partition(). Then the loop will iterate one extra time, with m having value high-1. The loop invariant tells us that, as the last iteration begins, we will have the following situation:

   low                   k                      m high
  +---------------------+----------------------+-+
a |     < a[high-1]     |     >= a[high-1]     | |       (6)
  +---------------------+----------------------+-+ 

Because m == high-1, it follows that a[m] >= a[high-1] is true, which means that the only effect of this loop iteration will be to increment m, after which (5) will hold, just as when the loop guard was m != high-1. Hence, it makes no difference which of the two loop guards we use, except that one results in a single "extra" loop iteration (but no "extra" swaps).