Efficient protocols for distributed classification and optimization

Authors:
Hal Daumé;Jeff M. Phillips;Avishek Saha;Suresh Venkatasubramanian
Affiliations:
University of Maryland, CP, MD;University of Utah, SLC, UT;University of Utah, SLC, UT;University of Utah, SLC, UT
Venue:
ALT'12 Proceedings of the 23rd international conference on Algorithmic Learning Theory
Year:
2012

Citing 16
Cited 0

Approximations and optimal geometric divide-and-conquer

STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
The discrepancy method: randomness and complexity

The discrepancy method: randomness and complexity
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Fast Algorithms for Approximate Semide.nite Programming using the Multiplicative Weights Update Method

FOCS '05 Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science
Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms

EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10
Data streams: algorithms and applications

Foundations and Trends® in Theoretical Computer Science
Multi-Pass Geometric Algorithms

Discrete & Computational Geometry
Algorithms for distributed functional monitoring

Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms
Rank minimization via online learning

Proceedings of the 25th international conference on Machine learning
Bundle Methods for Regularized Risk Minimization

The Journal of Machine Learning Research
Neural Network Learning: Theoretical Foundations

Neural Network Learning: Theoretical Foundations
Optimal sampling from distributed streams

Proceedings of the twenty-ninth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Distributed training strategies for the structured perceptron

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)
Scaling up Machine Learning: Parallel and Distributed Approaches

Scaling up Machine Learning: Parallel and Distributed Approaches
Optimal distributed online prediction using mini-batches

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

A recent paper [1] proposes a general model for distributed learning that bounds the communication required for learning classifiers with ε error on linearly separable data adversarially distributed across nodes. In this work, we develop key improvements and extensions to this basic model. Our first result is a two-party multiplicative-weight-update based protocol that uses O(d2 log1/ε) words of communication to classify distributed data in arbitrary dimension d, ε-optimally. This extends to classification over k nodes with O(kd2 log1/ε) words of communication. Our proposed protocol is simple to implement and is considerably more efficient than baselines compared, as demonstrated by our empirical results. In addition, we show how to solve fixed-dimensional and high-dimensional linear programming with small communication in a distributed setting where constraints may be distributed across nodes. Our techniques make use of a novel connection from multipass streaming, as well as adapting the multiplicative- weight-update framework more generally to a distributed setting.