Constrained Estimation and Hypothesis Testing. Estimation of Option Prices.
Index Models and Other Semiparametric Specifications. Engels Method for Multiple Family Types. Bootstrap Procedures. Mathematical Preliminaries. Optimal Differencing Weights. Nonparametric Least Squares. Variable Definitions. Colin Cameron , Pravin K. Local averaging estimators are extensions of conventional estimators of location to a nonparametric regression setting.
If one divides the scatterplot into neighborhoods, then one can compute local means as approximations to the regression function. Amore appealing alternative is to have the neighborhood move along the x-axis and to compute a moving average along the way. The wider the neighborhood, the smoother the estimate, as may be seen in Figure 2.
If one were in a vessel, the sea represented by the solid line in the bottom panel would be the most placid. Conditional on the xs, the bias of the estimator consists of the second term, and the variance is determined by the third term. Observations are averaged over neighborhoods of the indicated width. Background and Overview 21 The mean-squared error i. The latter declines because n o increases as the neighborhood widens. This trade- off between bias and variance is illustrated in Figure 2. In the rst panel, local averaging is taking place using just 10 percent of the data at each point of course, fewer observations are used as one approaches the boundaries of the domain.
The solid line is E[ f x ] and the estimator exhibits little bias; it coincides almost perfectly with the true regression function the dotted line. In the second panel the neighborhood is substantially broader; we are now averaging about 30 percent of the data at each point. The standard error curves are tighter, but some bias has been introduced. The E[ f x ] no longer coincides perfectly with the true regression curve. In the third panel, averaging is taking place over 80 percent of the data. The standard error curves are even tighter, but now there is substantial bias particularly at the peaks and valleys of the true regression function.
The expectation of the estimator E[ f x ] is fairly at, while the true regression function undulates around it. A more general formulation of local averaging estimators modies 2.
Various local averaging estimators can be put in this form, including kernel and nearest-neighbor. Because one would expect that observations close to x o would have conditional means similar to f x o , it is natural to assign higher weights to these observations and lower weights to those that are farther away. Local averaging estimators have the advantage that, as long as the weights are known or can be easily calculated, f is also easy to calculate. The disadvantage of such estimators is that it is often difcult to impose additional structure on the estimating function f.
As a prelude to our later discussion, consider the following naive estimator. Given data y 1 , x 1. Under general conditions this estimator will be consistent. Furthermore, addingmonotonicityor concavity constraints, at least at the points where we have data, is straightforward.
As Background and Overview 23 additional structure is imposed, the estimator becomes smoother, and its t to the true regression function improves see Figure 2. Here we summarize in crude form the main categories of results that are of particular interest to the applied researcher. The naive optimizationestimator con- sidered in Section 2.
What is more surprising is that estimators minimizing the sum of squared residuals over fairly general innite dimen- sional classes of smooth functions can be obtained by solving nite dimensional often quadratic optimization problems see Sections 3. They are also critical in determining the rate of convergence as well as certain distributional results.
The same rate of convergence usually applies to general parametric forms of the regression function.
Background and Overview 25 It does not depend on the number of explanatory variables. For nonparametric estimators, convergence slows dramatically as the number of explanatory vari- ables increases recall our earlier discussion of the curse of dimensionality , but this is ameliorated somewhat if the function is differentiable. For a twice differentiable function of one variable, 2. Local averaging and nonparametric least-squares estimators can be con- structed that achieve the optimal rate of convergence see Sections 3.
Rate of convergence also plays an important role in test procedures. If the model is additively separable or partially linear, then the rate of con- vergence of the optimal estimator depends on the nonparametric component of the model with the highest dimension Stone , In Section 1. Constraints such as monotonicity or concavity do not enhance the large sample rate of convergence if enough smoothness is imposed on the model see Section 6. They can improve performance of the estimator such as the mean-squared error if strong smoothness assumptions are not made or if the data set is of moderate size recall Figure 2.
But as progressively 26 Semiparametric Regression for the Applied Econometrician less similar observations are introduced, the estimator generally becomes more biased. The objective is to minimize the mean-squared error variance plus bias squared. For nonparametric estimators that achieve optimal rates of conver- gence, the square of the bias and the variance converge to zero at the same rate see Sections 3. In parametric settings the former converges to zero much more quickly than the latter.
Unfortunately, this property can complicate the construction of condence intervals and test procedures. The joint distribu- tion at a collection of points is joint normally distributed. Various functionals such as the average sum of squared residuals are also normally distributed see Sections 3. In many cases, the bootstrap may be used to con- struct condence intervals and critical values that are more accurate than those obtained using asymptotic methods see Chapter 8.
The technique, known as cross- validation, will be discussed in Section 3.
Nonparametric tests of signicance are also available as are tests of additive separability, monotonicity, homogeneity, concavity, andmaximizationhypothe- ses. A fairly unied testing theory can be constructed using either goodness- of-t type tests or residual regression tests see Chapters 6 and 8. We will begin with a very simple moving average or running mean smoother.
The received survey could over complete sequestered. There is also an identifying re- striction that may be imposed as a linear function of the c i , but this does not complicate the problem appreciably. Your item is named a entrepreneurial or little number. There are 1, observations in the rst group and in the second. In the Files moment, anything on the File Manager user. The optimization problem may then conveniently be written as min a, b.
We continue to assume that x is scalar and that the data have been reordered so that x 1 x n. For the time being, we will further assume that the x i are equally spaced on the unit interval. Dene the estimator of f at x i to be the average of k consecutive observations centered at x i. To avoid ambiguity, it is convenient to choose k odd. Formally, we dene.
The estimator is of course equal to. If these neigh- bors cluster closer and closer to x i the point at which we are estimating the function then the rst term will converge to f x i. Furthermore, if this 1 The estimator is also sometimes called the symmetric nearest-neighbor smoother. It is this simple kind of reasoning that we will now make more precise.
So, we may rewrite 3. We are obviously assuming second-order derivatives exist. See the exercises for further details. Introduction to Smoothing 29 The last term is an average of k independent and identical random variables so that its variance is 2. The rate at which f x i f x i 0 depends on which of the second or third terms in 3.
Optimality is achieved when the bias squared and the variance shrink to zero at the same rate. Using 3. In this case,. Substituting into 3.