Stacked Ensembles in H2O
•
7 likes•2,123 views
Ensemble machine learning methods are often used when the true prediction function is not easily approximated by a single algorithm. Practitioners may prefer ensemble algorithms when model performance is valued above other factors such as model complexity and training time. The Super Learner algorithm, also called "stacking", learns the optimal combination of the base learner fits. The latest version of H2O now contains a "Stacked Ensemble" method, which allows the user to stack H2O models into a Super Learner. The Stacked Ensemble method is the the native H2O version of stacking, previously only available in the h2oEnsemble R package, and now enables stacking from all the H2O APIs: Python, R, Scala, etc. Erin is a Statistician and Machine Learning Scientist at H2O.ai. Before joining H2O, she was the Principal Data Scientist at Wise.io (acquired by GE Digital) and Marvin Mobile Security (acquired by Veracode) and the founder of DataScientific, Inc. Erin received her Ph.D. from University of California, Berkeley. Her research focuses on ensemble machine learning, learning from imbalanced binary-outcome data, influence curve based variance estimation and statistical computing.
Report
Share
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Stacked Ensembles in H2O
Similar to Stacked Ensembles in H2O (20)
More from Sri Ambati
More from Sri Ambati (20)
Recently uploaded
Recently uploaded (20)
Stacked Ensembles in H2O
- 1. Stacked Ensembles in H2O Erin LeDell Ph.D. Machine Learning Scientist February 2017
- 2. Agenda • Who/What is H2O? • Ensemble Learning Overview • Stacking / Super Learner • Why Stacking? • Grid Search & Stacking • Stacking with Third-party Algos • AutoML and Stacking
- 3. H2O.ai H2O.ai, the Company H2O, the Platform • Founded in 2012 • Stanford & Purdue Math & Systems Engineers • Headquarters: Mountain View, California, USA • Open Source Software (Apache 2.0 Licensed) • R, Python, Scala, Java and Web Interfaces • Distributed algorithms that scale to “Big Data”
- 4. Scientific Advisory Council • John A. Overdeck Professor of Mathematics, Stanford University • PhD in Statistics, Stanford University • Co-author, The Elements of Statistical Learning: Prediction, Inference and Data Mining • Co-author with John Chambers, Statistical Models in S • Co-author, Generalized Additive Models Dr. Trevor Hastie • Professor of Statistics and Health Research and Policy, Stanford University • PhD in Statistics, Stanford University • Co-author, The Elements of Statistical Learning: Prediction, Inference and Data Mining • Author, Regression Shrinkage and Selection via the Lasso • Co-author, An Introduction to the Bootstrap Dr. Robert Tibshirani • Professor of Electrical Engineering and Computer Science, Stanford University • PhD in Electrical Engineering and Computer Science, UC Berkeley • Co-author, Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers • Co-author, Linear Matrix Inequalities in System and Control Theory • Co-author, Convex Optimization Dr. Steven Boyd
- 5. H2O Distributed Computing H2O Cluster H2O Frame • Multi-node cluster with shared memory model. • All computations in memory. • Each node sees only some rows of the data. • No limit on cluster size. • Distributed data frames (collection of vectors). • Columns are distributed (across nodes) arrays. • Works just like R’s data.frame or Python Pandas DataFrame
- 7. Ensemble Learning In statistics and machine learning, ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained by any of the constituent algorithms. — Wikipedia
- 8. Common Types of Ensemble Methods • Also reduces variance and increases accuracy • Not robust against outliers or noisy data • Flexible — can be used with any loss function Bagging Boosting Stacking • Reduces variance and increases accuracy • Robust against outliers or noisy data • Often used with Decision Trees (i.e. Random Forest) • Used to ensemble a diverse group of strong learners • Involves training a second-level machine learning algorithm called a “metalearner” to learn the optimal combination of the base learners
- 9. Stacking (aka Super Learner Algorithm) • Start with design matrix, X, and response, y • Specify L base learners (with model params) • Specify a metalearner (just another algorithm) • Perform k-fold CV on each of the L learners “Level-zero” data
- 10. Stacking (aka Super Learner Algorithm) • Collect the predicted values from k-fold CV that was performed on each of the L base learners • Column-bind these prediction vectors together to form a new design matrix, Z • Train the metalearner using Z, y “Level-one” data
- 11. Stacking vs. Parameter Tuning/Search • A common task in machine learning is to perform model selection by specifying a number of models with different parameters. • An example of this is Grid Search or Random Search. • The first phase of the Super Learner algorithm is computationally equivalent to performing model selection via cross-validation. • The latter phase of the Super Learner algorithm (the metalearning step) is just training another single model (no CV). • With Stacking, your computation does not go to waste!
- 13. How to Win Kaggle https://www.kaggle.com/c/GiveMeSomeCredit/leaderboard/private
- 14. How to Win Kaggle https://www.kaggle.com/c/GiveMeSomeCredit/forums/t/1166/congratulations-to-the-winners/7229#post7229
- 15. How to Win Kaggle https://www.kaggle.com/c/GiveMeSomeCredit/forums/t/1166/congratulations-to-the-winners/7230#post7230
- 16. h2oEnsemble R package & Stacked Ensemble in h2o
- 17. Evolution of H2O Ensemble • h2oEnsemble R package in 2015 • Ensemble logic ported to Java in late 2016 • Stacked Ensemble method in h2o in early 2017 • R & Python APIs supported • In progress: Custom metalearners • In progress: MOJO for production use
- 18. Stacking with Random Grids
- 19. H2O Cartesian Grid Search
- 20. H2O Random Grid Search
- 21. Stacking with Random Grids (h2o R)
- 22. Stacking with Random Grids (h2o Python)
- 23. H2O Stacking Resources H2O Stacked Ensembles docs & code demo: http://tinyurl.com/h2o-stacked-ensembles h2oEnsemble R package homepage on Github: http://tinyurl.com/github-h2o-ensemble
- 25. Ensemble H2O with Anything • XGBoost will be available in the next major release of H2O, so you can use it with the Stacked Ensemble method • https://github.com/h2oai/h2o-3/pull/699 A powerful combo: H2O + XGBoost Third party stacking with H2O: • SuperLearner, subsemble, mlr & caret R packages support stacking with H2O for small/medium data
- 26. AutoML
- 27. H2O AutoML Public code coming soon! • AutoML stands for “Automatic Machine Learning” • The idea here is to remove most (or all) of the parameters from the algorithm, as well as automatically generate derived features that will aid in learning. • Single algorithms are tuned automatically using a carefully constructed random grid search. • Optionally, a Stacked Ensemble can be constructed.
- 28. H2O Resources • H2O Online Training: http://learn.h2o.ai • H2O Tutorials: https://github.com/h2oai/h2o-tutorials • H2O Meetup Materials: https://github.com/h2oai/h2o-meetups • H2O Video Presentations: https://www.youtube.com/user/0xdata • H2O Community Events & Meetups: https://h2o.ai/events
- 29. Thank you! @ledell on Github, Twitter erin@h2o.ai http://www.stat.berkeley.edu/~ledell