CMPS 134 (Computer Science 1)
Elementary Sorting Algorithms

In computer science, the verb sort means to rearrange a list of items so that they end up in either ascending (from least to greatest) or descending (from greatest to least) order. There are at least three well known elementary sorting algorithms. Here we will describe two of them. Our descriptions assume that the items are to be put into ascending order; the reader should have no difficulty in adjusting the algorithms so that the items are put into descending order.

Selection Sort

We are given a list of N items occupying positions in the range [0..N). We scan the entire list to find the largest item, and then we swap it with whatever item is at position N−1. Having placed the largest item where it belongs —at position N−1— it remains to sort the items in positions [0..N−1). So we scan those items to find the largest among them, and then we swap it with the item at position N−2. Having done that, it remains to sort the items in positions [0..N−2). So we find the largest item among those and swap it with whatever item is at position N−3. And so on and so forth.

Assuming that the list to be sorted is given in the form of an array a[], we can express the algorithm in pseudocode like this: 

int N = a.length;
for (int i = N-1; i > 0; i = i-1)
   k = location of largest value in a[0..i]
   swap values in locations i and k of a[]
}

Note that there is no need for the loop to iterate when i has value zero, because such an iteration would necessarily swap the element at location zero with itself, which has no effect. (This explains why the loop guard is i > 0 rather than i ≥ 0.)

Translating this into a Java method that sorts an array of int's (and that makes use of auxiliary methods for finding the maximum value in an array segment and for swapping two elements of an array, both of which were developed during class meetings), we get this: 

/* Sorts the given array into ascending order using the Selection Sort
** algorithm.
*/
public static void selectionSort(int[] a)
{
   final int N = a.length;
   for (int i = N-1; i > 0; i = i-1) 
   {
      int k = locOfMax(a, 0, i);  // find location of max value in a[0..i]
      swap(a,i,k);                // swap values in locations i and k of a[]
   }
}

/* Returns a location containing the maximum value in b[low..high].
** More precisely: Returns the smallest value k such that low ≤ k ≤ high
** and no value in b[low..high] is greater than b[k]. 
** pre: 0 <= low <= high < b.length
*/
public static int locOfMax(int[] b, int low, int high) {
   int locOfMaxSoFar = low;
   for (int j = low+1; j <= high; j = j+1) {
      if (b[j] > b[locOfMaxSoFar]) {
         locOfMaxSoFar = j;
      }
   }
   return locOfMaxSoFar;
}

/* Swaps the elements at locations m and n of array b[]
*/
public static void swap(int[] b, int m, int n) {
   int temp = b[n]; b[n] = b[m]; b[m] = temp;
}

Example: Suppose that the given array is 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 7 | -2| 9 | 3 | 6 | 4 |
+---+---+---+---+---+---+

Then successive iterations of selection sort will have these effects:

During the first iteration, the elements at locations 2 and 5 are swapped, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 7 | -2| 4 | 3 | 6 | 9 |
+---+---+---+---+---+---+

During the second iteration, the elements at locations 0 and 4 are swapped, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 6 | -2| 4 | 3 | 7 | 9 |
+---+---+---+---+---+---+

During the third iteration, the elements at locations 0 and 3 are swapped, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 3 | -2| 4 | 6 | 7 | 9 |
+---+---+---+---+---+---+

During the fourth iteration, the elements at locations 2 and 2 are swapped (which has no effect), resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 3 | -2| 4 | 6 | 7 | 9 |
+---+---+---+---+---+---+

During the fifth (and final) iteration, the elements at locations 0 and 1 are swapped, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| -2| 3 | 4 | 6 | 7 | 9 |
+---+---+---+---+---+---+

Analysis:

The loop in the selectionSort() method iterates with (the loop control) variable i assuming the values N−1, N−2, ..., 1, where N is the length of the array a[] being sorted. During each of those iterations, the method locOfMax() is called in order to find a location within array segment a[0..i] (which has length i+1) containing that segment's maximum value.

Examining the code in locOfMax(), we notice that the number of comparisons performed (which is equal to the number of loop iterations) is one less than the length of the array segment on which it operates. (During its loop iterations, the loop control variable j assumes the values low+1 through high, which is a range of high-low values, in order to find the maximum value in array segment b[low..high], which has length high-low+1.)

We conclude that, during each iteration of the loop in selectionSort(), the call to locOfMax() results in the latter performing i comparisons (equivalently, loop iterations) between array elements. The following table illustrates this.

Value of i in
selectionSort()		 # comparisons/iterations
performed
in locOfMax()
N−1N−1
N−2N−2
N−3N−3
..
..
22
11
Total:(N2 − N)/2

So how many comparisons (equivalently, loop iterations) does the locOfMax() method perform, in total, including all N-1 times it is called by selectionSort()? The answer is the sum of the numbers in the second column of the table. It is well known that, for any natural number N the sum 1 + 2 + 3 + ... + (N-2) + (N-1) has value (N-1)N/2, or (N2-N)/2, which, for large values of N, is close to N2/2.

We conclude that the selection sort algorithm takes time proportional to the square of the length of the array that it sorts. The standard way to express this idea is to use Big O notation. Here, we would say that selection sort runs in O(N2) time, meaning the rate of increase of its running times, when applied to arrays of increasing lengths, is no greater than the rate of increase of the function g(N) = N2.

To give you a sense of this growth rate, find a graph of the function y = x2 on the Internet. Notice that, for nonnegative values of x, as x increases not only does the function increase but its slope increases. Indeed, the slope of the function at each such value of x is 2x (as that is the derivative of the function, as you would have learned in Calculus I).

To get a feel for the increase in running time as the array grows in length, consider that it will take (approximately) four times as long to sort an array of length 2N as it will to sort an array of length N (because (2N)2 = 4N2). That is, doubling the length of the array quadruples the running time.

It turns out that there are more complicated sorting algorithms having running times proportional to N·(log N), which is a significant improvement. Such algorithms are studied in subsequent computer science courses.

Insertion Sort

We are given a list of N items occupying positions in the range [0..N). Assume that we have already rearranged the items in positions [0..i) so that they are in ascending order. Now we consider the item x at position i. Where should x be inserted, relative to the items in positions [0..i), so that x is in its rightful place among them? The answer is that x should be inserted at the first position, call it k, containing a value greater than x. (If there is no such position, x must be greater than the item at position i−1 and therefore it belongs where it is already, at position i.) Of course, if we insert x at position k, the items formerly occupying positions [k..i−1] must be made to occupy positions [k+1..i].

Assuming that the list to be sorted is given in the form of an array a[], we can express the algorithm in pseudocode like this: 

N = a.length;
for each i in the range 1..N-1 {
   x = a[i]  // store a[i] in a safe place
   k = smallest number in [0..i] such that either k==i or a[k] > x 
   shift elements in segment a[k..i-1] to a[k+1..i] (i.e., upward one place)
   a[k] = x 
}

Note that there is no reason to carry out an iteration with i=0 because nothing of significance would occur during it.

Translating this into a Java method that sorts an array of int's (and that makes use of of auxiliary methods for computing k and for shifting an array segment), we get this: 

/* Sorts the given array into ascending order using the Insertion Sort
** algorithm.
*/
public static void insertionSort(int[] a)
{
   final int N = a.length;
   for (int i = 1; i < N; i = i+1)
   {
      int x = a[i];                       // store a[i] before it gets overwritten 
      int k = locOfLeastGreater(a,0,i,x); // find location at which to insert x
      shiftUp(a,k,i);                     // shift a[k..i) one place "upward"
      a[k] = x;                           // store x in its rightful place
   }
}


/* Assuming that the elements in b[bottom..top) are in ascending order,
** returns the smallest value m in the range [bottom..top] such that either
** m = top or val < b[m].  In other words, it finds the lowest-numbered
** location in b[bottom..top) containing a value that is greater than the
** one given (or top if there are none).
** The linear search algorithm is employed, with searching going "downwards".
** Running time is proportional to top-bottom (the length of the array segment).
** precondition: 0 ≤ bottom ≤ top ≤ b.length &&
**               elements in b[bottom..top-1] are in ascending order
*/
public static int locOfLeastGreater(int[] b, int bottom, int top, int val) 
{
   int m = top;
   // loop invariant: 
   //   bottom ≤ m ≤ top && all elements in a[m..top) are > val
   while (m != bottom  &&  b[m-1] > val) {
      m = m-1;
   }
   return m;
}

/* This method performs the same service as the one above, but it employs
** the binary search algorithm.  Its running time is proportional to
** log2(top-bottom).
** pre: The elements in b[bottom..top) are in ascending order.
*/
public static int locOfLeastGreater2(int[] b, int bottom, int top, int val) 
{
   int low = bottom, high = top;
   // loop invariant: 
   //   bottom ≤ low ≤ high &&
   //   all elements in b[bottom..low) are ≤ val &&
   //   all elements in b[high..top) are > val &&
   while (low != high) {
      int mid = (low + high) / 2;
      if (val ≤ b[mid]) 
         { low = mid+1; }
      else  // val > b[mid]
         { high = mid; }
   }
   return low;
}

/* Shifts the elements in the segment b[bottom..top) "upwards" one
** place so that, afterwards, they occupy the segment b[bottom+1..top+1).
** precondition: 0 ≤ bottom ≤ top < b.length
*/
public static void shiftUp(int[] b, int bottom, int top) 
{ 
   for (int j = top; j != bottom; j = j-1) {
      b[j] = b[j-1];
   }
}

Example: Suppose that the given array is 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 13| 3 | 9 | -2| 6 | 4 |
+---+---+---+---+---+---+

Then successive iterations of insertion sort will have these effects:

During the first iteration, the element in segment [0..0] will be shifted upward one position and the value that occupied location 1 (3) will be placed into location 0, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 3 | 13| 9 | -2| 6 | 4 |
+---+---+---+---+---+---+

During the second iteration, the element in segment [1..1] will be shifted upward one position and the value that occupied location 2 (9) will be placed into location 1, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| 3 | 9 | 13| -2| 6 | 4 |
+---+---+---+---+---+---+
During the third iteration, the elements in segment [0..2] will be shifted upward one position and the value that occupied location 3 (-2) will be placed into location 0, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| -2| 3 | 9 | 13| 6 | 4 |
+---+---+---+---+---+---+
During the fourth iteration, the elements in segment [2..3] will be shifted upward one position and the value that occupied location 4 (6) will be placed into location 2, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| -2| 3 | 6 | 9 | 13| 4 |
+---+---+---+---+---+---+
During the fifth iteration, the elements in segment [2..4] will be shifted upward one position and the value that occupied location 5 (4) will be placed into location 2, resulting in: 

0   1   2   3   4   5
+---+---+---+---+---+---+
| -2| 3 | 4 | 6 | 9 | 13|
+---+---+---+---+---+---+

Analysis:

The loop in the insertionSort() method iterates with the loop control variable i assuming the values 1,2,3,...,N-1, where N is the length of the array a[] being sorted. During each of those iterations, the methods locOfLeastGreater() and shiftUp() are called in order, respectively, to find the location k into which the value at location i should be written and to shift the elements in the segment a[k..i) "upward" by one position. In both of these methods, the number of loop iterations is essentially i-k. In the worst case, k=0, which happens when a[i] is less than a[0]. In the best case, k=i, which happens when a[i] is found to be greater than a[i-1]. If we assume that the elements in a[] are more or less in random order to begin with, it would be reasonable to estimate that, on average, we'll have k = i/2, corresponding to the value of a[i] being smaller than about half of the elements in a[0..i-1].

Value of i in
insertionSort()		 # iterations performed in
locOfLeastGreater()
and
shiftUp()
Best Case		 # iterations performed in
locOfLeastGreater()
and
shiftUp()
Worst Case		 # iterations performed in
locOfLeastGreater()
and
shiftUp()
Average Case
1011/2
2022/2
3033/2
....
....
N-40N-4(N-4)/2
N-30N-3(N-3)/2
N-20N-2(N-2)/2
N-10N-1(N-1)/2

So how many loop iterations are performed by each of the locOfLeastGreater() and shiftUp() methods, in total, including all N-1 times they are called by insertionSort()? Summing up the numbers in the last three columns of the table, we find that, in the best case, zero, in the worst, approximately N2/2, and, in the average case, approximately N2/4.

What this tells us is that insertionSort() will, like selectionSort(), typically require time proportional to N2 to complete its task, which is to say that its asymptotic running time is O(N2). However, if the array to be sorted is "almost" in order to begin with, insertionSort() can take as little as time proportional to N. This would be the case, for example, if there were a constant c such that, in the given array, every element was at a location no farther than c locations away from where it ended up when sorting was completed.