Elementary Sorting Algorithms

In computer science, the verb **sort** means to rearrange a
list of items so that they end up in either **ascending**
(from least to greatest) or **descending** (from greatest to least)
order.
There are at least three well known elementary sorting algorithms.
Here we will describe two of them. Our descriptions assume that the
items are to be put into ascending order; the reader should have no
difficulty in adjusting the algorithms so that the items are put into
descending order.

We are given a list of N items occupying positions in the range [0..N-1]. We scan the entire list to find the largest item, and then we swap it with whatever item is at position N−1. Having placed the largest item where it belongs —at position N−1— it remains to sort the items in positions [0..N−2]. So we scan those items to find the largest among them, and then we swap it with the item at position N−2. Having done that, it remains to sort the items in positions [0..N−-3]. So we find the largest item among those and swap it with whatever item is at position N−3. And so on and so forth.

Assuming that the list to be sorted is given in the form of an array
`a[]`, we can express the algorithm in pseudocode like this:

int N = a.length; for (int i = N-1; i > 0; i = i-1) k = location of largest value in a[0..i] swap values in locations i and k of a[] } |

Note that there is no need for the loop to iterate when `i`
has value zero, because such an iteration would necessarily swap the
element at location zero with itself, which has no effect.
(This explains why the loop guard is `i > 0` rather than
`i ≥ 0`.)

Translating this into a Java method that sorts an array of `int`'s
(and that makes use of auxiliary methods for finding the maximum value in
an array segment and for swapping two elements of an array, both of which
were developed during class meetings), we get this:

/* Sorts the given array into ascending order using the Selection Sort ** algorithm. */ public static void selectionSort(int[] a) { final int N = a.length; for (int i = N-1; i > 0; i = i-1) { int k = locOfMax(a, 0, i); // find location of max value in a[0..i] swap(a,i,k); // swap values in locations i and k of a[] } } /* Returns a location containing the maximum value in b[low..high]. ** More precisely: Returns the smallest value k such that low ≤ k ≤ high ** and no value in b[low..high] is greater than b[k]. ** pre: 0 <= low <= high < b.length */ public static int locOfMax(int[] b, int low, int high) { int locOfMaxSoFar = low; for (int j = low+1; j <= high; j = j+1) { if (b[j] > b[locOfMinSoFar]) { locOfMaxSoFar = j; } } return locOfMaxSoFar; } /* Swaps the elements at locations m and n of array b[] */ public static void swap(int[] b, int m, int n) { int temp = b[n]; b[n] = b[m]; b[m] = temp; } |

**Example:**
Suppose that the given array is

0 1 2 3 4 5 +---+---+---+---+---+---+ | 7 | -2| 9 | 3 | 6 | 4 | +---+---+---+---+---+---+

Then successive iterations of selection sort will have these effects:

During the first iteration, the elements at locations 2 and 5 are swapped, resulting in:

0 1 2 3 4 5 +---+---+---+---+---+---+ | 7 | -2| 4 | 3 | 6 | 9 | +---+---+---+---+---+---+

During the second iteration, the elements at locations 0 and 4 are swapped, resulting in:

0 1 2 3 4 5 +---+---+---+---+---+---+ | 6 | -2| 4 | 3 | 7 | 9 | +---+---+---+---+---+---+

During the third iteration, the elements at locations 0 and 3 are swapped, resulting in:

0 1 2 3 4 5 +---+---+---+---+---+---+ | 3 | -2| 4 | 6 | 7 | 9 | +---+---+---+---+---+---+

During the fourth iteration, the elements at locations 2 and 2 are swapped (which has no effect), resulting in:

0 1 2 3 4 5 +---+---+---+---+---+---+ | 3 | -2| 4 | 6 | 7 | 9 | +---+---+---+---+---+---+

During the fifth (and final) iteration, the elements at locations 0 and 1 are swapped, resulting in:

0 1 2 3 4 5 +---+---+---+---+---+---+ | -2| 3 | 4 | 6 | 7 | 9 | +---+---+---+---+---+---+

**Analysis**:

The loop in the `selectionSort()` method iterates with
(the loop control) variable `i` assuming the values N−1,
N−2, ..., 1, where `N` is the length of the array
`a[]` being sorted.
During each of those iterations, the method `locOfMax()` is
called in order to find a location within array segment `a[0..i]`
(which has length `i+1`) containing that segment's maximum value.

Examining the code in `locOfMax()`, we notice that the number
of comparisons performed (which is equal to the number of loop iterations)
is one less than the length of the array segment on which it operates.
(During its loop iterations, the loop control variable `j` assumes
the values `low+1` through `high`, which is a range of
`high-low` values, in order to find the maximum value in
array segment `b[low..high]`, which has length `high-low+1`.)

We conclude that, during each iteration of the loop in
`selectionSort()`, the call to `locOfMax()` results in the
latter performing `i` comparisons (equivalently, loop iterations)
between array elements. The following table illustrates this.

Value of i inselectionSort() |
# comparisons/iterations performed in locOfMax() |
---|---|

N−1 | N−1 |

N−2 | N−2 |

N−3 | N−3 |

. | . |

. | . |

2 | 2 |

1 | 1 |

Total: | (N^{2} − N)/2 |

So how many comparisons (equivalently, loop iterations) does the
`locOfMax()` method perform, in total, including all
`N-1` times it is called by `selectionSort()`?
The answer is the sum of the numbers in the second column of the table.
It is well known that, for any natural number `N` the
sum `1 + 2 + 3 + ... + (N-2) + (N-1)` has value
`(N-1)N/2`, or `(N ^{2}-N)/2`, which,
for large values of

We conclude that the selection sort algorithm takes time proportional
to the square of the length of the array that it sorts.
The standard way to express this idea is to use
Big O notation.
Here, we would say that selection sort runs in **O(N ^{2})** time,
meaning the rate of increase of its running times, when applied to arrays of
increasing lengths, is no greater than the rate of increase of the
function g(N) = N

To give you a sense of this **growth rate**,
find a graph of the function y = x^{2} on the Internet.
Notice that, for nonnegative values of x, as x increases not only does
the function increase but its slope increases. Indeed, the slope of
the function at each such value of x is 2x (as that is the derivative
of the function, as you would have learned in Calculus I).

To get a feel for the increase in running time as the array grows in length,
consider that it will take (approximately) **four** times as long to sort
an array of length `2N` as it will to sort an array of length `N`
(because (2N)^{2} = 4N^{2}). That is, doubling the
length of the array quadruples the running time.

It turns out that there are more complicated sorting algorithms having
running times proportional to `N·(log N)`, which is a
significant improvement. Such algorithms are studied in subsequent
computer science courses.

We are given a list of N items occupying positions in the range [0..N−1].
Assume that we have already rearranged the items in positions
[0..i-1] so that they are in ascending order. Now we consider
the item *x* at position i. Where should *x* be inserted,
relative to the items in positions [0..i−1], so that *x*
is in its rightful place among them? The answer is that *x*
should be inserted at the first position, call it k,
containing a value greater than *x*. (If there is no such
position, *x* must be greater than the item at position
i−1 and therefore it belongs where it is already, at position i.)
Of course, if we insert *x* at position k, the items formerly
occupying positions [k..i−1] must be made to occupy positions
[k+1..i].

Assuming that the list to be sorted is given in the form of an array
`a[]`, we can express the algorithm in pseudocode like this:

N = a.length; for each i in the range 1..N-1 { x = a[i] // store a[i] in a safe place k = smallest number in [0..i] such that either k==i or a[k] > x shift elements in segment a[k..i-1] to a[k+1..i] (i.e., upward one place) a[k] = x } |

Note that there is no reason to carry out an iteration with i=0 because nothing of significance would occur during it.

Translating this into a Java method that sorts an array of `int`'s
(and that makes use of of auxiliary methods for computing k and for
shifting an array segment), we get this:

/* Sorts the given array into ascending order using the Insertion Sort ** algorithm. */ public static void insertionSort(int[] a) { final int N = a.length; for (int i = 1; i < N; i = i+1) { int x = a[i]; // store a[i] before it gets overwritten int k = locOfLeastGreater(a,0,i,x); // find location at which to insert x shiftUp(a,k,i); // shift a[k..i) one place "upward" a[k] = x; // store x in its rightful place } } /* Assuming that the elements in b[bottom..top) are in ascending order, ** returns the smallest value m in the range [bottom..top] such that either ** m = top or val < b[m]. In other words, it finds the lowest-numbered ** location in b[bottom..top) containing a value that is greater than the ** one given (or top if there are none). ** The linear search algorithm is employed, with searching going "downwards". ** Running time is proportional to top-bottom (the length of the array segment). ** precondition: 0 ≤ bottom ≤ top ≤ b.length && ** elements in b[bottom..top-1] are in ascending order */ public static int locOfLeastGreater(int[] b, int bottom, int top, int val) { int m = top; // loop invariant: // bottom ≤ m ≤ top && all elements in a[m..top) are > val while (m != bottom && b[m-1] > val) { m = m-1; } return m; } /* This method performs the same service as the one above, but it employs ** the binary search algorithm. Its running time is proportional to ** log |

**Example:** Suppose that the given array is

0 1 2 3 4 5 +---+---+---+---+---+---+ | 13| 3 | 9 | -2| 6 | 4 | +---+---+---+---+---+---+

Then successive iterations of insertion sort will have these effects:

During the first iteration, the element in segment [0..0] will be shifted upward one position and the value that occupied location 1 (3) will be placed into location 0, resulting in:

0 1 2 3 4 5 +---+---+---+---+---+---+ | 3 | 13| 9 | -2| 6 | 4 | +---+---+---+---+---+---+

During the second iteration, the element in segment [1..1] will be shifted upward one position and the value that occupied location 2 (9) will be placed into location 1, resulting in:

0 1 2 3 4 5 +---+---+---+---+---+---+ | 3 | 9 | 13| -2| 6 | 4 | +---+---+---+---+---+---+

0 1 2 3 4 5 +---+---+---+---+---+---+ | -2| 3 | 9 | 13| 6 | 4 | +---+---+---+---+---+---+

0 1 2 3 4 5 +---+---+---+---+---+---+ | -2| 3 | 6 | 9 | 13| 4 | +---+---+---+---+---+---+

0 1 2 3 4 5 +---+---+---+---+---+---+ | -2| 3 | 4 | 6 | 9 | 13| +---+---+---+---+---+---+

**Analysis**:

The loop in the `insertionSort()` method iterates with the
loop control variable `i` assuming the values 1,2,3,...,N-1,
where `N` is the length of the array `a[]` being sorted.
During each of those iterations, the methods `locOfLeastGreater()`
and `shiftUp()` are called in order, respectively, to find the
location `k` into which the value at location `i` should
be written and to shift the elements in the segment `a[k..i)`
"upward" by one position. In both of these methods, the number of
loop iterations is essentially `i-k`. In the worst case,
`k=0`, which happens when `a[i]` is less than
`a[0]`. In the best case, `k=i`, which happens when
`a[i]` is found to be greater than `a[i-1]`.
If we assume that the elements in `a[]` are more or less in
random order to begin with, it would be reasonable to estimate that,
on average, we'll have `k = i/2`, corresponding to the
value of `a[i]` being smaller than about half of the elements
in `a[0..i-1]`.

Value of i ininsertionSort() |
# iterations performed inlocOfLeastGreater()and shiftUp()Best Case |
# iterations performed inlocOfLeastGreater()and shiftUp()Worst Case |
# iterations performed inlocOfLeastGreater()and shiftUp()Average Case |
---|---|---|---|

1 | 0 | 1 | 1/2 |

2 | 0 | 2 | 2/2 |

3 | 0 | 3 | 3/2 |

. | . | . | . |

. | . | . | . |

N-4 | 0 | N-4 | (N-4)/2 |

N-3 | 0 | N-3 | (N-3)/2 |

N-2 | 0 | N-2 | (N-2)/2 |

N-1 | 0 | N-1 | (N-1)/2 |

So how many loop iterations are performed by each of the
`locOfLeastGreater()` and `shiftUp()` methods,
in total, including all `N-1` times they are called
by `insertionSort()`? Summing up the numbers in the
last three columns of the table, we find that, in the best case,
zero, in the worst, approximately `N ^{2}/2`,
and, in the average case, approximately

What this tells us is that `insertionSort()` will, like
`selectionSort()`, typically require time proportional
to `N ^{2}` to complete its task, which is to
say that its asymptotic running time is O(N