Jon A. Wellner

This book provides an account of weak convergence theory, empirical processes, and their application to a wide variety of problems in statistics. The first part of the book presents a thorough treatment of stochastic convergence in its various forms. Part 2 brings together the theory of empirical processes in a form accessible to statisticians and probabilists. In Part 3, the authors cover a range of applications in statistics including rates of convergence of estimators; limit theorems for M- and Z-estimators; the bootstrap; the functional delta-method and semiparametric estimation. Most of the chapters conclude with “problems and complements.” Some of these are exercises to help the reader’s understanding of the material, whereas others are intended to supplement the text. This second edition includes many of the new developments in the field since publication of the first edition in 1996: Glivenko-Cantelli preservation theorems; new bounds on expectations ofsuprema of empirical processes; new bounds on covering numbers for various function classes; generic chaining; definitive versions of concentration bounds; and new applications in statistics including penalized M-estimation, the lasso, classification, and support vector machines. The approximately 200 additional pages also round out classical subjects, including chapters on weak convergence in Skorokhod space, on stable convergence, and on processes based on pseudo-observations.

This book tries to do three things. The first goal is to give an exposition of certain modes of stochastic convergence, in particular convergence in distribution. The classical theory of this subject was developed mostly in the 1950s and is well summarized in Billingsley (1968). During the last 15 years, the need for a more general theory allowing random elements that are not Borel measurable has become well established, particularly in developing the theory of empirical processes. Part 1 of the book, Stochastic Convergence, gives an exposition of such a theory following the ideas of J. Hoffmann-J!1Jrgensen and R. M. Dudley. A second goal is to use the weak convergence theory background devel­ oped in Part 1 to present an account of major components of the modern theory of empirical processes indexed by classes of sets and functions. The weak convergence theory developed in Part 1 is important for this, simply because the empirical processes studied in Part 2, Empirical Processes, arenaturally viewed as taking values in nonseparable Banach spaces, even in the most elementary cases, and are typically not Borel measurable. Much of the theory presented in Part 2 has previously been scattered in the journal literature and has, as a result, been accessible only to a relatively small number of specialists. In view of the importance of this theory for statis­ tics, we hope that the presentation given here will make this theory more accessible to statisticians as well as to probabilists interested in statistical applications.

Originally published in 1986, this valuable reference provides a detailed treatment of limit theorems and inequalities for empirical processes of real-valued random variables. It also includes applications of the theory to censored data, spacings, rank statistics, quantiles, and many functionals of empirical processes, including a treatment of bootstrap methods, and a summary of inequalities that are useful for proving limit theorems. At the end of the Errata section, the authors have supplied references to solutions for 11 of the 19 Open Questions provided in the book's original edition.

This book is about estimation in situations where we believe we have enough knowledge to model some features of the data parametrically, but are unwilling to assume anything for other features. Such models have arisen in a wide variety of contexts in recent years, particularly in economics, epidemiology, and astronomy. The complicated structure of these models typically requires us to consider nonlinear estimation procedures which often can only be implemented algorithmically. The theory of these procedures is necessarily based on asymptotic approximations.