Alternative Learning Methods

Many applications involve heterogeneous data, such that labeled training data can be naturally divided into several groups.
Two new methodologies for utilizing such group information include Learning with Structured Data (aka SVM+) and Multi-Task Learning.

Learning with Structured Data (Vapnik, 2006)

• Training Data: originates from several groups

• Goal of learning: to estimate a single predictive model.

Multi-Task Learning

• Training Data: originates from several groups (tasks)

• Goal of learning: to estimate several related predictive models

Universum Learning or Learning through Contradictions (Vapnik, 2006)

• Incorporate a priori information, in the form of Universum data samples, into learning.

Example: Classifying gender of human faces

1. Additional information about labeled data: young, middle-age, old faces
   Learning with Structured Data (training data can be partitioned into groups) or Multi-Task Learning

2. Additional unlabeled data samples: Universum which does not belong to either class
   Learning through contradictions

References

• V. Vapnik, Empirical Inference Science: Afterword of 2008, Springer 2006

• L. Liang and V. Cherkassky, Connection between SVM+ and Multi-Task Learning, Proc. IJCNN 2008.

• V. Cherkassky, F. Cai, and L. Liang, Predictive Learning with Sparse Heterogeneous Data, Proc. IJCNN 2009.

• F. Cai and V. Cherkassky, SVM+ Regression and Multi-Task Learning, Proc. IJCNN 2009.

• J. Weston, R. Collobert, F. Sinz, L. Bottou, and V. Vapnik, Inference with the Universum. Proceedings of the Twenty-third International Conference on Machine Learning (ICML 2006), IMLS/ICML, 2006.

• V. Cherkassky and W. Dai, Empirical Study of the Universum SVM Learning for High-Dimensional Data, Proc. ICANN 2009.

• S. Dhar and V. Cherkassky, Visualization and Interpretation of SVM Classifiers, Data Mining and Konwledge Discovery 2010.