Date of Completion

5-6-2016

Embargo Period

11-2-2016

Keywords

Biconvex optimization, Learning from crowds, Data annotation ambiguity, Multitask learning, Sparse modeling, Feature learning

Major Advisor

Jinbo Bi

Associate Advisor

Alexander Russell

Associate Advisor

Laurent Michel

Associate Advisor

Fei Wang

Field of Study

Computer Science and Engineering

Degree

Doctor of Philosophy

Open Access

Open Access

Abstract

Modern technologies have enabled us to collect large quantities of data. The proliferation of such data has facilitated knowledge discovery using machine learning techniques. However, it has also imposed great challenges for human annotators to label the massive data, which then offers proper supervision to learning algorithms. How we can improve learning accuracy with limited or imprecise supervision has recently become a much focused research subject. In this dissertation, we design effective algorithms based on modern machine learning theories to address the following two problems of: (1) learning from inconsistent labels collected from multiple annotators with varying expertise; and (2) knowledge transfer among related tasks to build more accurate predictive models. The proposed solutions will be evaluated not only on benchmark datasets but also in real-world scenarios from across disciplines.

In the first direction, we develop bi-convex optimization algorithms to address annotation ambiguity from inconsistent labels. We extend the well-known support vector machine (SVM) algorithm and optimize SVM classifiers with respect to a weighted consensus of different labelers' labels. The weights in the consensus are also automatically learned by our learning formulation. A variety of bi-convex programs are derived corresponding to different assumptions on the labeler competencies. Adding another layer of annotation ambiguity is that a labeler's label may be associated with a set of data points rather than each individual one. We then further generalize the formulation to deal with the multiple points' labels. Empirical results on benchmark datasets with synthetic labelers and real-life crowdsourced labels demonstrate the superior performance of our methods over the state of the art.

Along the second direction, we develop new Multitask Learning (MTL) algorithms. MTL improves the generalization of the estimated models for multiple related learning tasks by capturing and exploiting the task relationships. We investigate a general framework of multiplicative multi-task feature learning which decomposes each task's model parameters into a multiplication of two components. One component is used across all tasks and the other is task specific. Several previous methods have been proposed as special cases of our framework. We prove that this framework is mathematically equivalent to the widely used multi-task feature learning methods that are based on a joint regularization of all model parameters, but with a more general form of regularizers. Two new learning formulations are proposed by varying the parameters in the proposed framework.

We further study the method to learn task grouping with multiplicative feature sharing patterns in each group of tasks. We cluster tasks into one cluster if they select the same subset of features. We formulate an optimization problem to jointly optimize the models and grouping structure of the tasks. Empirical studies have revealed the advantages of our formulations by comparing with the state of the art.

Download

Share

COinS