BasisFunctions The Basics of Basis Functions The Characteristics of Spline Functions The Characteristics of Spline Functions II -What is the General Idea -How does this Smoothing Operation Work(cont.) -Do we have any other extensions of these ideas? Goods Growth Weather
 The Characteristics of Spline Functions II Previous - 1 - 2 - 3 - Next Expertise: Intermediate Do we have any other extensions of these ideas? Indeed we do. So far we have assumed that the breakpoints are a strictly increasing sequence. But in fact we can actually place more than one breakpoint at the a specific value t. The argument gets rather complicated, and de Boor (2002) is the standard reference, but what happens is simple: For each additional breakpoint ξ equal to t, the spline function loses one degree of smoothness, and thus has one less smooth derivative. In the extreme, if there are m coincident breakpoints, the resulting spline function can be discontinuous at that point. In this way, we can accommodate even abrupt changes of level in the spline family. The term knot is used routinely in the literature on splines to refer to what we have called a breakpoint here. Strictly speaking, however, the term knot should be reserved for possibly coincident breakpoints, and the term breakpoint then means technically the strictly increasing sequence of t-values at which knots are located. However, there is another whole world beyond even this. Here we have confined our attention to spline functions that are piecewise polynomials. Suppose, however, that we work with some other set of primitive basis functions other than powers of t. Suppose, for the sake of argument, that the functions shall be piecewise Fourier functions instead. Or piecewise sums of exponentials with a sequence of rate constants. The possibilities are unlimited. But enough is enough. Or, if not, see the wonderful book by L. Schumaker(1981). What makes spline functions easy to fit to data? Notice that a spline basis function of order m will be positive over at most m adjacent subintervals. In fact, the spline basis functions next to the boundaries are positive over a decreasing number of subintervals to the point where the extreme basis functions are positive over a only the end intervals. This means that a matrix of values Z of spline basis functions will be band structured in the sense that the nonzero entries in the matrix will be only found in a band running from the top left to the lower right. Now the equation defining the coefficient vector a has the following form: a = (Z'Z + λR)-1Z'y where λ is a scalar smoothing parameter and matrix R is called the roughness penalty matrix. The band-structure is also found in the cross-product matrix Z'Z and in the roughness penalty matrix R. When thousands of points of observation are used, this can result in a huge saving of computer time for algorithms making use of this band structure over algorithms which assume that any entries can be nonzero. Conclusion Now look for the shift-scale-smooth operations in other basis function families, and you will find them. The Fourier series are already infinitely differentiable and therefore no further smoothness is possible, so only shift-scale is involved. Moreover, only a single shift is involved, and that is the shift by πω units to convert sin(ωt) into cos(ωt). The rescaling is the multiplication of the base frequency ω by the successive natural numbers. Polynomials involve neither shifting or scaling, but smoothness is achieved by exactly the same tilting operation that we used for splines. This is not surprising since splines are actually polynomials over sub-intervals. Wavelets are a result of playing especially clever and beautiful games with these three operations. References de Boor, C. (2002) A Practical Guide to Splines. Second Edition. New York: Springer. Schumaker, L. (1981) Spline Functions: Basic Theory. New York: John Wiley and Sons. Previous - 1 - 2 - 3 - Next