KGM Mastering Classification and Regression with LLMs: Insights from Kaggle Competitions
•Download as PPTX, PDF•
0 likes•38 views
This document discusses using large language models (LLMs) for text classification tasks. It begins by describing how LLMs are commonly used for text generation and question answering. For classification, models are usually trained supervised on labeled data. The document then explores using LLMs for zero-shot classification without training, and techniques like fine-tuning LLMs on tasks to improve performance. It provides an example of fine-tuning an LLM on a financial sentiment dataset. The document concludes by describing H2O.ai's LLM Studio tool for fine-tuning and a few Kaggle competitions where LLMs achieved success in text classification.
Report
Share
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
More Related Content
Similar to KGM Mastering Classification and Regression with LLMs: Insights from Kaggle Competitions
Similar to KGM Mastering Classification and Regression with LLMs: Insights from Kaggle Competitions (20)
More from Sri Ambati
More from Sri Ambati (20)
Recently uploaded
Recently uploaded (20)
KGM Mastering Classification and Regression with LLMs: Insights from Kaggle Competitions
- 1. v H2O.ai Confidential Senior Principal Data Scientist at H2O.ai ● Vienna / Austria ● Kaggle GM ● All things deep learning ● I love training models ● H2O Hydrogen Torch ● H2O LLM Studio Philipp Singer
- 2. H2O.ai Confidential Mastering Classification and Regression with LLMs: Insights from Kaggle Competitions
- 3. v H2O.ai Confidential The typical LLM use case Text generation Q&A / RAG Labeling Agents
- 4. v H2O.ai Confidential What about classification? Common business use-case Classify text into two or more categories Sentiment classification Document categorization Spam detection Language detection Topic classification
- 5. v H2O.ai Confidential The common way Supervised training Train a model on labeled data, predict on unlabeled data Bag of word approach Vectorize into fixed vocabulary, train gradient boosting models Transformer approach Train transformer models like BERT, Roberta, Deberta
- 6. v H2O.ai Confidential The LLM way Zero-shot classification Ask a LLM model for the prediction without training Zero-shot engineering Can we improve zero-shot prediction quality? Fine-tuning LLMs Fine-tune a LLM model for task-specific classification
- 8. v H2O.ai Confidential Financial sentiment data huggingface.co/datasets/financial_phrasebank ● Data from FiQA and Financial PhraseBank ● Subset with >=75% annotator agreement ● Data statistics ○ 3,453 rows ○ Train: 2,589 rows ○ Val: 864 rows ● Labels ○ ~62% neutral ○ ~26% positive ○ ~12% negative ● Simplest majority baseline ○ 0.622 Accuracy Cramo and Peab have signed exclusive five-year rental agreements in Finland and have extended their existing rental agreements in the Swedish market for another five years. Construction work on the Helsinki Music Centre is to start this autumn, with the total cost of the project estimated at 140 million euros. The company's profit totaled 578,100 in H1 2007, down 30.9% year- on-year.
- 9. v H2O.ai Confidential Let’s ask h2oGPT
- 10. v H2O.ai Confidential Zero-shot streamlining is difficult No training needed No labels needed Easy to get started Prompt engineering tricky Difficult to automate into business processes Runtime expensive Often interpretable results Evaluation still required → labels
- 11. v H2O.ai Confidential Zero-shot logits approach GPT Your task is to analyze the message below and predict whether it has negative, neutral or positive sentiment. Return on investment was 16.6% compared to 15.8% in 2004. The sentiment is
- 12. v H2O.ai Confidential Zero-shot logits approach
- 13. v H2O.ai Confidential Zero-shot logits approach
- 14. v H2O.ai Confidential Classification fine-tuning GPT Negative Neutral Positive Classification Head Finetune
- 15. v H2O.ai Confidential H2O LLM Studio github.com/h2oai/h2o-llmstudio ● Easily and effectively fine-tune LLMs ● CLI & No-Code GUI ● SOTA techniques ○ LoRA ○ Quantization ○ Hyperparameter tuning ○ Experiment tracking and advanced evaluation ○ SFT & RLHF ● Newest Problem Type: Causal Classification ● Fully open-sourced
- 18. H2O.ai Confidential Kaggle success stories
- 19. v H2O.ai Confidential H2O.ai Predict the LLM
- 20. v H2O.ai Confidential CommonLit - Evaluate Student Summaries Competition
- 21. v H2O.ai Confidential LLM Science Exam Competition