From Wikipedia, the free encyclopedia  View original article
Shellsort with gaps 23, 10, 4, 1 in action.  
Class  Sorting algorithm 

Data structure  Array 
Worst case performance  O(n^{2}) 
Best case performance  O(n log^{2} n) 
Average case performance  depends on gap sequence 
Worst case space complexity  О(n) total, O(1) auxiliary 
Shellsort with gaps 23, 10, 4, 1 in action.  
Class  Sorting algorithm 

Data structure  Array 
Worst case performance  O(n^{2}) 
Best case performance  O(n log^{2} n) 
Average case performance  depends on gap sequence 
Worst case space complexity  О(n) total, O(1) auxiliary 
Shellsort, also known as Shell sort or Shell's method, is an inplace comparison sort. It can be seen as either a generalization of sorting by exchange (bubble sort) or sorting by insertion (insertion sort).^{[1]} The method starts by sorting pairs of elements far apart from each other, then progressively reducing the gap between elements to be compared. Starting with far apart elements can move some outofplace elements into position faster than a simple nearest neighbor exchange. Donald Shell published the first version of this sort in 1959.^{[2]}^{[3]} The running time of Shellsort is heavily dependent on the gap sequence it uses. For many practical variants, determining their time complexity remains an open problem.
Shellsort is a generalization of insertion sort that allows the exchange of items that are far apart. The idea is to arrange the list of elements so that, starting anywhere, considering every hth element gives a sorted list. Such a list is said to be hsorted. Equivalently, it can be thought of as h interleaved lists, each individually sorted.^{[4]} Beginning with large values of h, this rearrangement allows elements to move long distances in the original list, reducing large amounts of disorder quickly, and leaving less work for smaller hsort steps to do.^{[5]} If the file is then ksorted for some smaller integer k, then the file remains hsorted. Following this idea for a decreasing sequence of h values ending in 1 is guaranteed to leave a sorted list in the end.^{[4]}
An example run of Shellsort with gaps 5, 3 and 1 is shown below.
The first pass, 5sorting, performs insertion sort on separate subarrays (a_{1}, a_{6}, a_{11}), (a_{2}, a_{7}, a_{12}), (a_{3}, a_{8}), (a_{4}, a_{9}), (a_{5}, a_{10}). For instance, it changes the subarray (a_{1}, a_{6}, a_{11}) from (62, 17, 25) to (17, 25, 62). The next pass, 3sorting, performs insertion sort on the subarrays (a_{1}, a_{4}, a_{7}, a_{10}), (a_{2}, a_{5}, a_{8}, a_{11}), (a_{3}, a_{6}, a_{9}, a_{12}). The last pass, 1sorting, is an ordinary insertion sort of the entire array (a_{1},..., a_{12}).
As the example illustrates, the subarrays that Shellsort operates on are initially short; later they are longer but almost ordered. In both cases insertion sort works efficiently.
Shellsort is unstable: it may change the relative order of elements with equal values. It is an adaptive sorting algorithm in that it executes faster when the input is partially sorted.
Using Marcin Ciura's gap sequence, with an inner insertion sort.
# Sort an array a[0...n1]. gaps = [701, 301, 132, 57, 23, 10, 4, 1] # Start with the largest gap and work down to a gap of 1 foreach (gap in gaps) { # Do a gapped insertion sort for this gap size. # The first gap elements a[0..gap1] are already in gapped order # keep adding one more element until the entire array is gap sorted for (i = gap; i < n; i += 1) { # add a[i] to the elements that have been gap sorted # save a[i] in temp and make a hole at position i temp = a[i] # shift earlier gapsorted elements up until the correct location for a[i] is found for (j = i; j >= gap and a[j  gap] > temp; j = gap) { a[j] = a[j  gap] } # put temp (the original a[i]) in its correct location a[j] = temp } }
The question of deciding which gap sequence to use is difficult. Every gap sequence that contains 1 yields a correct sort; however, the properties of thus obtained versions of Shellsort may be very different.
The table below compares most proposed gap sequences published so far. Some of them have decreasing elements that depend on the size of the sorted array (N). Others are increasing infinite sequences, whose elements less than N should be used in reverse order.
General term (k ≥ 1)  Concrete gaps  Worstcase time complexity  Author and year of publication 

[when N=2^{p}]  Shell, 1959^{[2]}  
Frank & Lazarus, 1960^{[6]}  
Hibbard, 1963^{[7]}  
, prefixed with 1  Papernov & Stasevich, 1965^{[8]}  
successive numbers of the form  Pratt, 1971^{[9]}  
, not greater than  Pratt, 1971^{[9]}  
Incerpi & Sedgewick, 1985,^{[10]} Knuth^{[1]}  
, prefixed with 1  Sedgewick, 1986^{[4]}  
Sedgewick, 1986^{[4]}  
?  Gonnet & BaezaYates, 1991^{[11]}  
?  Tokuda, 1992^{[12]}  
unknown  ?  Ciura, 2001^{[13]} 
When the binary representation of N contains many consecutive zeroes, Shellsort using Shell's original gap sequence makes Θ(N^{2}) comparisons in the worst case. For instance, this case occurs for N equal to a power of two when elements greater and smaller than the median occupy odd and even positions respectively, since they are compared only in the last pass.
Although it has higher complexity than the O(NlogN) that is optimal for comparison sorts, Pratt's version lends itself to sorting networks and has the same asymptotic gate complexity as Batcher's bitonic sorter.
Gonnet and BaezaYates observed that Shellsort makes the fewest comparisons on average when the ratios of successive gaps are roughly equal to 2.2.^{[11]} This is why their sequence with ratio 2.2 and Tokuda's sequence with ratio 2.25 prove efficient. However, it is not known why this is so. Sedgewick recommends to use gaps that have low greatest common divisors or are pairwise coprime.^{[14]}
With respect to the average number of comparisons, the best known gap sequences are 1, 4, 10, 23, 57, 132, 301, 701 and similar, with gaps found experimentally. Optimal gaps beyond 701 remain unknown, but good results can be obtained by extending the above sequence according to the recursive formula .
Tokuda's sequence, defined by the simple formula , where , , can be recommended for practical applications.
The following property holds: after h_{2}sorting of any h_{1}sorted array, the array remains h_{1}sorted.^{[15]} Every h_{1}sorted and h_{2}sorted array is also (a_{1}h_{1}+a_{2}h_{2})sorted, for any nonnegative integers a_{1} and a_{2}. The worstcase complexity of Shellsort is therefore connected with the Frobenius problem: for given integers h_{1},..., h_{n} with gcd = 1, the Frobenius number g(h_{1},..., h_{n}) is the greatest integer that cannot be represented as a_{1}h_{1}+ ... +a_{n}h_{n} with nonnegative integer a_{1},..., a_{n}. Using known formulae for Frobenius numbers, we can determine the worstcase complexity of Shellsort for several classes of gap sequences.^{[16]} Proven results are shown in the above table.
With respect to the average number of operations, none of proven results concerns a practical gap sequence. For gaps that are powers of two, Espelid computed this average as .^{[17]} Knuth determined the average complexity of sorting an Nelement array with two gaps (h, 1) to be .^{[1]} It follows that a twopass Shellsort with h = Θ(N^{1/3}) makes on average O(N^{5/3}) comparisons. Yao found the average complexity of a threepass Shellsort.^{[18]} His result was refined by Janson and Knuth:^{[19]} the average number of comparisons made during a Shellsort with three gaps (ch, cg, 1), where h and g are coprime, is in the first pass, in the second pass and in the third pass. ψ(h, g) in the last formula is a complicated function asymptotically equal to . In particular, when h = Θ(N^{7/15}) and g = Θ(h^{1/5}), the average time of sorting is O(N^{23/15}).
Based on experiments, it is conjectured that Shellsort with Hibbard's gap sequence runs in O(N^{5/4}) average time,^{[1]} and that Gonnet and BaezaYates's sequence requires on average 0.41NlnN(ln lnN+1/6) element moves.^{[11]} Approximations of the average number of operations formerly put forward for other sequences fail when sorted arrays contain millions of elements.
The graph below shows the average number of element comparisons in various variants of Shellsort, divided by the theoretical lower bound, i.e. log_{2}N!, where the sequence 1, 4, 10, 23, 57, 132, 301, 701 has been extended according to the formula .
Applying the theory of Kolmogorov complexity, Jiang, Li, and Vitányi proved the following lower bounds for the order of the average number of operations in an mpass Shellsort: Ω(mN^{1+1/m}) when m≤log_{2}N and Ω(mN) when m>log_{2}N.^{[20]} Therefore Shellsort has prospects of running in an average time that asymptotically grows like NlogN only when using gap sequences whose number of gaps grows in proportion to the logarithm of the array size. It is, however, unknown whether Shellsort can reach this asymptotic order of averagecase complexity, which is optimal for comparison sorts.
The worstcase complexity of any version of Shellsort is of higher order: Plaxton, Poonen, and Suel showed that it grows at least as rapidly as Ω(N(logN/log logN)^{2}).^{[21]}
Shellsort is now rarely used in serious applications. It performs more operations and has higher cache miss ratio than quicksort. However, since it can be implemented using little code and does not use the call stack, some implementations of the qsort function in the C standard library targeted at embedded systems use it instead of quicksort. Shellsort is, for example, used in the uClibc library.^{[22]} For similar reasons, an implementation of Shellsort is present in the Linux kernel.^{[23]}
Shellsort can also serve as a subalgorithm of introspective sort, to sort short subarrays and to prevent a pathological slowdown when the recursion depth exceeds a given limit. This principle is employed, for instance, in the bzip2 compressor.^{[24]}
The Wikibook Algorithm implementation has a page on the topic of: Shell sort 
