Kindle app logo image

Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required.

Read instantly on your browser with Kindle for Web.

Using your mobile phone camera - scan the code below and download the Kindle app.

QR code to download the Kindle App

Follow the author

Something went wrong. Please try your request again later.

Boosting: Foundations and Algorithms (Adaptive Computation and Machine Learning)


Boosting is an approach to machine learning based on the idea of creating a highly accurate predictor by combining many weak and inaccurate "rules of thumb." A remarkably rich theory has evolved around boosting, with connections to a range of topics, including statistics, game theory, convex optimization, and information geometry. Boosting algorithms have also enjoyed practical success in such fields as biology, vision, and speech processing. At various times in its history, boosting has been perceived as mysterious, controversial, even paradoxical.

This book, written by the inventors of the method, brings together, organizes, simplifies, and substantially extends two decades of research on boosting, presenting both theory and applications in a way that is accessible to readers from diverse backgrounds while also providing an authoritative reference for advanced researchers. With its introductory treatment of all material and its inclusion of exercises in every chapter, the book is appropriate for course use as well. The book begins with a general introduction to machine learning algorithms and their analysis; then explores the core theory of boosting, especially its ability to generalize; examines some of the myriad other theoretical viewpoints that help to explain and understand boosting; provides practical extensions of boosting for more complex learning problems; and finally presents a number of advanced theoretical topics. Numerous applications and practical illustrations are offered throughout.


Amazon First Reads | Editors' picks at exclusive prices

Editorial Reviews

About the Author

Robert E. Schapire is Professor of Computer Science at Princeton University. Yoav Freund is Professor of Computer Science at the University of California, San Diego. For their work on boosting, Freund and Schapire received both the Gödel Prize in 2003 and the Kanellakis Theory and Practice Award in 2004.

Product details

  • Publisher ‏ : ‎ Mit Pr (January 1, 2012)
  • Language ‏ : ‎ English
  • Hardcover ‏ : ‎ 526 pages
  • ISBN-10 ‏ : ‎ 0262017180
  • ISBN-13 ‏ : ‎ 978-0262017183
  • Item Weight ‏ : ‎ 2.19 pounds
  • Dimensions ‏ : ‎ 7.25 x 1.25 x 9.25 inches
  • Customer Reviews:

About the author

Follow authors to get new release updates, plus improved recommendations.
Robert E. Schapire
Brief content visible, double tap to read full content.
Full content visible, double tap to read brief content.

Discover more of the author’s books, see similar authors, read author blogs and more

Customer reviews

4.3 out of 5 stars
4.3 out of 5
30 global ratings

Top reviews from the United States

Reviewed in the United States on November 4, 2014
It's a kind of problem I come across way too often: in trying to determine whether a specific condition exists, a number of symptoms might help the diagnosis. But, some of the symptoms can also appear when something else is going on, instead. On the other hand, not all of the symptoms necessarily appear when the condition in fact is active. The question then becomes, given a number of indicators that have some diagnostic value, and given that all of them are inaccurate some of the time, how do I combine the indicators' answers to get the best diagnosis of that condition?

This book proposes "Boosting" as the answer. Start with remarkably few technical requirements on the diagnostic indicators, plus some number of cases in which the condition's presence or absence is already known. Given that, Boosting iteratively determines the weight to assign each indicator. One requirement is that each indicator suggests presence or absence of the condition at least somewhat differently than random guessing - and being wrong most of the time is just as useful as being right more often than not, since the algorithm automatically assigns negative weights to such indicators.

After that, the authors present rigorous development in a number of directions. I emphasize "rigor" - this text offers detailed development, analysis, and formal proof of the algorithm and its properties, far beyond the needs of someone who just wants to implement the technique. Implementable detail is there, but you'll spend a fair bit of time teasing it out of the dense notation used here.

Then, once basics have been established, the discussion branches out. The authors offer game-theoretic analysis of the algorithm, along with comparisons to related optimization techniques. After tightening some of the technical requirements, they apply it to decision trees with binary outcomes, to N-ary classification problems for N>2, and to more complex kinds of tasks, always with the same mathematical rigor.

Frankly, it's a bit much for someone just looking to get the broad picture or someone looking to code something up fast and try it out. But, it's not meant for those readers. To get the most out of it, you should come prepared to take your time, work out the meaning of each equation, follow the development carefully, and maybe even do some of the exercises. If you've already plowed through some "yellow books," you'll know what I mean, even though this doesn't come from the Springer yellow series. For the prepared and patient reader, this has my highest recommendation

-- wiredweird
4 people found this helpful
Report
Reviewed in the United States on July 29, 2016
It's a quite comprehensive book, describing lots of different ways to look at the AdaBoost family of algorithms. For all I can tell, the authors have collected all the state-of-the-art knowledge about boosting at the time the book was written, from the publications developed both by them and by the other people. So if you want to learn pretty much everything about boosting (up to the publication date), this is the book to read.

But be prepared that it's not a quick and casual introduction, it's a collection of the in-depth mathematical papers. It's a book that takes a very long time and much effort to read thoroughly and understand. You can skim over the proofs but it still takes a long time, after all it's pretty much everything known about boosting. I actually highly recommend not spending too much time on the proofs when you read it for the first time. This will give you a good overall picture, and then if you want to go deeper, read the book for the second time, the mathematics will make more sense on the second reading. You can also skip chapters depending on your interests, if you're not out to learn everything about boosting.

Probably the only really annoying thing from the engineering standpoint is that the algorithms in the book are what the mathematicians and the fans of functional programming call "algorithms", not the algorithms in the normal engineering sense. It takes some deciphering to turn them into a straightforward readable and understandable form. Bug again, it's a book about math, not engineering. I've had some of the deciphering I've done posted to a blog but I probably can't post a link to it here.
5 people found this helpful
Report
Reviewed in the United States on December 21, 2012
This is an very interesting book written by famous people in this area.

However, I found it hard to follow if I do not have previous knowledge about machine learning technology.

It is more focus on practical uses and ideas behind scenes, but not a good textbook for teaching courses for collage students.
2 people found this helpful
Report
Reviewed in the United States on August 1, 2013
Love the book. Some of the detailed mathematics and margin theory is outside my expertise, but the first chapter makes it worth all the trouble. I implemented the algorithm on page 5 and it is working fine.

I have had a copy out of the library, and finally ordered my own copy.

Chapter 1 is the best writing I've ever seen as an introduction to a technical book. It's a beautiful work of art.

Excuse me while I go read chapter 2 and on into Margin Theory...
6 people found this helpful
Report
Reviewed in the United States on December 3, 2019
The authors are good at making complex ML theory accessible. But the book has some prerequisites. But it is well written and and a pleasure to read.
2 people found this helpful
Report
Reviewed in the United States on August 20, 2016
I was expecting something more practical. Even the theory doesn't address some of the questions I had.
Reviewed in the United States on March 9, 2015
Aside from being a deep dive in a very useful machine learning technique, this book also provides a great education in the fundamentals of machine learning theory. It is written by the original inventors of Boosting and they do a fine job organizing the book. I am reading it for the second time now.
4 people found this helpful
Report
Reviewed in the United States on March 17, 2015
Clearly wriitten intuition and rigorously proved theorems. Narrowly and brilliantly focused on an important topic.
One person found this helpful
Report

Top reviews from other countries

Mario Marchand
5.0 out of 5 stars Five Stars
Reviewed in Canada on March 2, 2016
Excellent book. Convers all important aspects of Boosting. Very-well written, clear and detailed explenations. A must!
Aldo
5.0 out of 5 stars Buon libro.
Reviewed in Italy on October 28, 2012
Un buon libro per chi vuole capire e approfondire l'argomento del boosting. Tratta l'argomento da varie angolazioni in maniera abbastanza accessibile, anche se alcuni capitoli sono abbastanza impegnativi. Tratta principalmente la parte teorica con qualche accenno anche agli algoritmi che ne derivano. Non penso sia un libro adatto a chi cerca solo un manuale pratico sull'argomento.
One person found this helpful
Report
Tibor Tot
5.0 out of 5 stars Five Stars
Reviewed in Canada on July 18, 2018
Perfect.
Trading Central
2.0 out of 5 stars Long on Proofs, Short on Practical Implementation Algorithms
Reviewed in Canada on September 19, 2013
This is a traditional academic based text that misses the mark on both sides of the equation.

The text is really nothing more than an elementary introduction to this subject area that is better dealt with by other authors in a detailed way that will satisfy the academic communities need for indepth rigor that will satisfy their colleagues peer review standards.

As for the practitioner, there is nothing new and a lack of practical implementation strategies that will assist with the day to day usage for this book. Modern day texts provide the pseudo code based on R or the equivalent to assist the practitioner. This book is useless as a reference title for the practitioner.

Overall. better treatment of the subject area can be found elsewhere and for those thinking the book is a bargain, it will soon end up in the local library used book bin to be used as a coaster or monitor boost for the advanced programmer.
One person found this helpful
Report