Bootstrap Tutorial

The tutorial was presented at ICASSP 99.

The Bootstrap: A Powerful Tool for Statistical Signal Processing with Small Sample Sets

Abdelhak M. Zoubir

The bootstrap is a computer-based approach for calculating the performance of a parameter estimator or a detector. It can also be used to choose an estimator, a detector or a model among a number of candidates. The bootstrap is very useful because with only little in the way of assumptions and modeling it yields more accurate results than Gaussian approximation.

Today, more and more complex statistical models are being introduced in engineering applications that often do not provide satisfactory solutions to many practical problems. In place of the formulae and tables of parametric and non-parametric procedures based on complicated mathematics, tools such as asymptotic approximations or Monte Carlo simulations are often invoked. The problem with Monte Carlo simulations is that they are inapplicable when the underlying distribution is unknown. On the other hand, asymptotic approximations are impractical when the analysis is to be performed with a small set of data. The bootstrap which has revolutionized 1990's statistics, is a remedy for the above problems. It is a method for determining the performance of a parameter estimator or detector and other everyday inferential problems. We believe that the bootstrap will prove useful for many signal processing practitioners.

The purpose of the tutorial is (i) to provide a systematic introduction to the theory of the bootstrap, (ii) to provide guidelines for signal processing practitioners so that “misuse” is avoided in situations where theoretical confirmation is unavailable, and (iii) to stimulate the use of the bootstrap and further developments of resampling techniques. An outline of the tutorial follows. We first introduce bootstrap methods. In particular, we discuss the parametric and the non-parametric bootstrap. We then treat dependent and independent data bootstrap methods and show how they can be used for variance and confidence interval estimation, signal detection, and model selection. Finally, we present some real-life applications. Specifically, we discuss the estimation of the noise floor in radar and confidence intervals for flight parameters from an aircraft's acoustic emission. All examples will be interactively presented using a bootstrap Matlab demo.

Tutorial Slides (with corrections)

slides (opens in new tab) (1.2M)

List of Errata (for original slides)

errata.ps.gz (35K) or errata.zip (35K)

Matlab Demo

bootdemo.tar.gz (35K) or bootdemo.zip (53K)

Please send comments to Prof. Abdelhak Zoubir.