Incorporating feature selection into a classification
or regression method often carr
ies a number of advantages. In this
paper we formalize feature selection specifically from a
discriminative perspective of improving classification/regression
accuracy. The feature selection method is developed as an extension to
the recently proposed maximum entropy discrimination (MED)
framework. We describe MED as a flexible (Bayesian)
regularization approach that subsumes, e.g., support vector
classification, regression and exponential family models. For brevity,
we restrict ourselves primarily to feature selection in the context of
linear classification/regression methods and demonstrate that the
proposed approach indeed carries substantial improvements in
practice. Moreover, we discuss and develop various extensions of
feature selection, including the problem of dealing with example
specific but unobserved degrees of freedom -- alignments or
invariants.
Compressed Postscript .
PDF .
Full list of tech reports