KALIF - Kernel Algorithms for Learning in Feature Spaces
by Bernhard Schölkopf
Kernel Algorithms represent a novel means for nonlinear analysis of complex real-world data. By providing a mathematical foundation for previous algorithms such as Neural Nets, they support systematic and principled improvements of learning algorithms for pattern recognition, function estimation, and feature extraction.
Learning algorithms are particularly useful in situations where an empirically observable dependency cannot be modelled explicitly, but ample empirical data are available. Examples thereof include pattern recognition problems ranging from pedestrian or traffic sign detection via internet text categorization to problems of financial time series analysis.
The KALIF project (Kernel Algorithms for Learning in Feature Spaces) develops and applies statistical methods for data analysis in nonlinear feature spaces. One can show that a certain class of kernel functions computes dot products in high-dimensional feature spaces nonlinearly related to input space. This way, a number of learning algorithms that can be cast in terms of dot products can implicitly be carried out in feature spaces - without the need for complex calculations in these high-dimensional spaces.
An example thereof are Support Vector methods for classification (see Figure) and regression. Moreover, we have recently generalized a standard algorithm for high-dimensional data analysis, the algorithm of statistical principal component analysis (PCA), to the nonlinear case. To this end, we carry out standard linear PCA in the feature space, using kernels. The algorithms boils down to a matrix diagonalization. It can, for instance, be used for nonlinear feature extraction and dimensionality reduction.
It is precisely the close encounter of theory and practice that renders this recently emerging research area particularly appealing: using complexity concepts of statistical learning theory and methods of functional analysis, we can now theoretically analyze and further develop a number of successful heuristical methods of data analysis. This has already led to record results on several benchmark problems, such as the digit recognition database of the American National Institute of Standards and Technology (NIST).
Potentially more important, however, is the fact that the increased accuracy of these algorithms helps opening up new application fields, which previously had not been accessible to machine learning such as, for instance, very high-dimensional data sets.
More information at: http://svm.first.gmd.de/kalif.html
Please contact:
Bernhard Schölkopf - GMD
Tel: +49 30 6392 1875
E-mail: bs@first.gmd.de
Klaus-Robert Müller - GMD
Tel:+49 30 6392 1860
E-mail: klaus@first.gmd.de
