Han Xiao

We consider a novel distributed learning problem: A server receives potentially unlimited data from clients in a sequential manner, but only a small initial fraction of these data are labeled. Because communication bandwidth is expensive, each client is limited to sending the server only a small (high-priority) fraction of the unlabeled data it generates, and the server is limited in the amount of prioritization hints it sends back to the client. The goal is for the server to learn a good model… Mehr anzeigen

We consider a novel distributed learning problem: A server receives potentially unlimited data from clients in a sequential manner, but only a small initial fraction of these data are labeled. Because communication bandwidth is expensive, each client is limited to sending the server only a small (high-priority) fraction of the unlabeled data it generates, and the server is limited in the amount of prioritization hints it sends back to the client. The goal is for the server to learn a good model of all the client data from the labeled and unlabeled data it receives. This setting is frequently encountered in real-world applications and has the characteristics of online, semi-supervised, and active learning. However, previous approaches are not designed for the client-server setting and do not hold the promise of reducing communication costs. We present a novel framework for solving this learning prob- lem in an effective and communication-efficient manner. On the server side, our solution combines two diverse learners working collaboratively, yet in distinct roles, on the partially labeled data stream. A compact, online graph-based semi-supervised learner is used to predict labels for the unlabeled data arriving from the clients. Samples from this model are used as ongoing training for a linear classifier. On the client side, our solution prioritizes data based on an active-learning metric that favors instances that are close to the classifier’s decision hyperplane and yet far from each other. To reduce communication, the server sends the classifier’s weight-vector to the client only periodically. Experimental results on real-world data sets show that this particular combination of techniques outperforms other approaches, and in particular, often outperforms (communication expensive) approaches that send all the data to the server. Weniger anzeigen

Machine learning algorithms are increasingly being applied in security-related tasks such as spam and malware detection, although their security properties against deliberate attacks have not yet been widely understood. Intelligent and adaptive attackers may indeed exploit specific vulnerabilities exposed by machine learning techniques to violate system security. Being robust to adversarial data manipulation is thus an important, additional requirement for machine learning algorithms to… Mehr anzeigen

Machine learning algorithms are increasingly being applied in security-related tasks such as spam and malware detection, although their security properties against deliberate attacks have not yet been widely understood. Intelligent and adaptive attackers may indeed exploit specific vulnerabilities exposed by machine learning techniques to violate system security. Being robust to adversarial data manipulation is thus an important, additional requirement for machine learning algorithms to successfully operate in adversarial settings. In this work, we evaluate the security of Support Vector Machines (SVMs) to well-crafted, adversarial label noise attacks. In particular, we consider an attacker that aims to maximize the SVM’s classification error by flipping a number of labels in the training data. We formalize a corresponding optimal attack strategy, and solve it by means of heuristic approaches to keep the computational complexity tractable. We report an extensive experimental analysis on the effectiveness of the considered attacks against linear and non-linear SVMs, both on synthetic and real-world datasets. We finally argue that our approach can also provide useful insights for developing more secure SVM learning algorithms, and also novel techniques in a number of related research areas, such as semi-supervised and active learning. Weniger anzeigen

Nowadays machine learning is considered as a vital tool for data analysis and automatic decision making in many modern enterprise systems. However, there is an emerging threat that adversaries can mislead the decision of the learning algo- rithm by introducing security faults into the system. Previous security research did not closely examined the vulnerabilities of the learning algorithms to adversarial manipulations. Understanding these threats is the only way to build robust learn- ing… Mehr anzeigen

Nowadays machine learning is considered as a vital tool for data analysis and automatic decision making in many modern enterprise systems. However, there is an emerging threat that adversaries can mislead the decision of the learning algo- rithm by introducing security faults into the system. Previous security research did not closely examined the vulnerabilities of the learning algorithms to adversarial manipulations. Understanding these threats is the only way to build robust learn- ing algorithms for security-sensitive applications. This dissertation is organized in three parts. Each part contributes the new results in adversarial, reliable and scalable machine learning, respectively. Weniger anzeigen

Sequence prediction is a key task in machine learning and data mining. It
involves predicting the next symbol in a sequence given its previous symbols.
Our motivating application is predicting the execution path of a process on an
operating system in real-time. In this case, each symbol in the sequence
represents a system call accompanied with arguments and a return value. We
propose a novel online algorithm for predicting the next system call by
leveraging both… Mehr anzeigen

Sequence prediction is a key task in machine learning and data mining. It
involves predicting the next symbol in a sequence given its previous symbols.
Our motivating application is predicting the execution path of a process on an
operating system in real-time. In this case, each symbol in the sequence
represents a system call accompanied with arguments and a return value. We
propose a novel online algorithm for predicting the next system call by
leveraging both context and side information. The online update of our
algorithm is efficient in terms of time cost and memory consumption.
Experiments on real-world data sets showed that our method outperforms
state-of-the-art online sequence prediction methods in both accuracy and
efficiency, and incorporation of side information does significantly improve
the predictive accuracy. Weniger anzeigen

A significant problem of Gaussian process (GP) is its
unfavorable scaling with a large amount of data. To overcome this issue, we
present a novel GP approximation scheme for online regression. Our model is
based on a combination of multiple GPs with random hyperparameters. The model is
trained by incrementally allocating new examples to a selected subset of GPs.
The selection is carried out efficiently by optimizing a submodular function.
Experiments on real-world data sets… Mehr anzeigen

A significant problem of Gaussian process (GP) is its
unfavorable scaling with a large amount of data. To overcome this issue, we
present a novel GP approximation scheme for online regression. Our model is
based on a combination of multiple GPs with random hyperparameters. The model is
trained by incrementally allocating new examples to a selected subset of GPs.
The selection is carried out efficiently by optimizing a submodular function.
Experiments on real-world data sets showed that our method outperforms existing
online GP regression methods in both accuracy and efficiency. The applicability
of the proposed method is demonstrated by the mouse-trajectory prediction in an
Internet banking scenario. Weniger anzeigen