Building LLM Solutions using Open Source and Closed Source Solutions in Coherent Manner
•Download as PPTX, PDF•
0 likes•143 views
Sandeep Singh, Head of Applied AI Computer Vision, Beans.ai H2O Open Source GenAI World SF 2023 In the modern era of machine learning, leveraging both open-source and closed-source solutions has become paramount for achieving cutting-edge results. This talk delves into the intricacies of seamlessly integrating open-source Large Language Model (LLM) solutions like Vicuna, Falcon, and Llama with industry giants such as ChatGPT and Google's Palm. As the demand for fine-tuned and specialized datasets grows, it is imperative to understand the synergy between these tools. Attendees will gain insights into best practices for building and enriching datasets tailored for fine-tuning tasks, ensuring that their LLM projects are both robust and efficient. Through real-world examples and hands-on demonstrations, this talk will equip attendees with the knowledge to harness the power of both open and closed-source tools in a coherent and effective manner.
Report
Share
Recommended for you
Recommended for you
Recommended for you
Recommended for you
More Related Content
Similar to Building LLM Solutions using Open Source and Closed Source Solutions in Coherent Manner
Similar to Building LLM Solutions using Open Source and Closed Source Solutions in Coherent Manner (20)
More from Sri Ambati
More from Sri Ambati (20)
Recently uploaded
Recently uploaded (20)
Building LLM Solutions using Open Source and Closed Source Solutions in Coherent Manner
- 1. H2O.ai Confidential SANDEEP SINGH Head of Applied AI, Beans.AI
- 2. H2O.ai Confidential Building LLM Solutions using Open Source and Closed Source Solutions in Coherent Manner
- 3. H2O.ai Confidential Agenda Introduction Ecosystem of Open Source LLM Tools The Significance of blending Open and Closed Source LLMs Open Source LLMs: Features and Benefits Closed Source LLMs: Advantages and Use-Cases Integration Strategies: Methodologies Case Study Highlights and Learning from the Industry Dataset Enrichment. Low to No Code Fine-Tuning Techniques Conclusion
- 4. v H2O.ai Confidential About Me • Head of Applied AI/Computer Vision, Beans AI • Beans AI based out of Palo Alto, CA • We are Location Intelligence Platform. • Hyper-Accurate Maps, Much more accurate than Google, Apple and Bing for apartments. • Computer Vision and Image based Synthesis is inherent part of Innovation at Beans Maps • I deal with Satellite Imagery, Location Data, Convexity Optimization domains at my day to day job. • Holds Masters from Georgia Tech.
- 5. v H2O.ai Confidential Ecosystem of LLMs GPT-4 PaLM SageMaker Neo IBM Watson Salesforce Einstein Wu Dao 2.0 Clarifai Cohere Anthropic Claude MT-NLG LLaMA 2 Falcon- 40B/180B Vicuna 33B MPT-30B GPT-NeoX- 20B CodeGen GPT-J OPT-175B BLOOM Baichuan- 13B Proprietary/Closed Source Open Source
- 6. v H2O.ai Confidential Open Source LLMs: Benefits ● Enhanced data security and privacy: Self-hosted deployment ● Cost savings: No licensing/subscription fees and no API calls expenditure. ● No External Dependency: No reliance on select few vendor avoiding lock-in. ● Code transparency and Constructive Collaboration/Validation: Underlying code and methodologies are vetted for functionality by community. ● Language Model Customization: Domain Adaptation is more manageable with open-source LLMs by Fine- tuning. ● Active Community Support: Often thriving communities , Quicker issue resolution, access to resources and collaborative. ● Fosters innovation: Open-source LLMs encourage innovation by enabling organizations to experiment and build upon existing models. ● Boon for Startups: Leverage models as a foundation for creative and unique applications.
- 7. v H2O.ai Confidential Closed Source LLMs: Advantages ● Support and Reliability: Vendor Support, Professional assistance, Maintenance, Troubleshooting, SLA Requirements ● Customization for Specific Business Needs: Accommodate Unique requirements of a business ● Security and Data Privacy: May offer more robust security features and privacy assurances critical for industries with sensitive data. ● Performance: Regular optimizations and enhancements for better performance for specific tasks or industry ● Integration with Proprietary Systems: Dedicated tooling support to use existing proprietary software stacks within an organization to avoid extensive re-engineering. ● Compliance and Liability: For regulated industries greater assurance of compliance. ● Guardrails Ownership: Responsibility for compliance often falls on the vendor, reducing the legal and financial risks for the user. ● Continuous Development and Updates: Dedicated teams to keep up-to-date cutting edge ● Commercial Viability: Better viability for businesses with limited resource and investments, Enable quicker feature developments.
- 8. v H2O.ai Confidential Best of Both Worlds ● Number of options available. ● Possibility of exploiting more than one solution. ● In-house Performance comparison for “your” task, Not just a benchmark. ● Different baselines for particular domain adaptation. ● Amount of fine tuning needs are not same for similar tasks. ● Possibility of using specific LLM solution for specific task in pipeline. ● Combinations available to hyper ensemble these LLM solutions. ● Ability to pick and choose LLMs without affecting other LLMs in play.
- 9. v H2O.ai Confidential Integration Strategies ● Grunualize the task at hand: Break LLM “initiative” into LLM “tasks” ● Categorize the tasks by Stochasticity Tolerance and Criticality: Different LLM solutions pose varying degree of temperature sensitivity. ● Less tolerant tasks are candidate for Proprietary Off-the-shelf solutions. ● More tolerant tasks are candidate for Open Source with no or less fine tuning needs.
- 10. v H2O.ai Confidential Case Study Highlights ● At Beans.AI, we use combination of approaches like: Few tasks are achieved using Prompt Engineering/RAG based approach. Few tasks are achieved using Limited to moderate Fine Tuning. ● Both Closed source and Open Source LLMs are used. ● Responses from Closed Source LLMs are used by Open Source LLMs and vice versa in pipeline. ● Used for automated support, Insights from dashboard, automated email order etc.
- 11. v H2O.ai Confidential Dataset Enrichment ● No, NOT THAT data enrichment! ● Most of the time: For “your” purpose, you need “your” data. ● “Your” data is limited by: Quantity, Quality and Variety ● LLMs are used to overcome: Quantity: By creating more samples of data Quality: By working with humans in the loop type setup Variety: By revising and rewriting intents in many different possible ways.
- 12. v H2O.ai Confidential Dataset Enrichment(cond.) Example: ● Task: Question Answering Bot for your particular app. Say: Delivery Support App. ● Interaction: Delivery Driver asks a question in the app and expects “how-to” type response. Question: How do I mark an address not deliverable in the app? Candidate Answer: Explains the steps to do the same. ● Current Training Data: Set of Questions and Answers in knowledge article. Enrichment Step: Prompt engineered app to create variations of your domain specific questions as: “Ask the above question in 20 different ways” All these new 20 ways of asking the “same” question, create new training examples for you.
- 13. v H2O.ai Confidential Dataset Enrichment(cond.) All of these questions below ask the EXACT same thing! ● What's the process for labeling an address as undeliverable within the application? ● Can you guide me through the steps to indicate that an address is non-deliverable in the app? ● How can I flag an address as undeliverable when using the app? ● What is the method for setting an address to 'not deliverable' status in the application? ● Is there a way to mark an address as 'cannot be delivered to' in the app interface? ● Could you explain how to designate an address as not deliverable on the app? ● I'm looking to mark an address as non-deliverable in the app; how do I do that? ● How does one go about indicating that an address is not serviceable in the app? ● In the app, what are the steps to mark an address as one that can't be delivered to? ● What’s the procedure to flag an address as 'not deliverable' in the app's system?
- 14. v H2O.ai Confidential Case Optimization ● We at Beans.AI use LLMs o analyze the pipeline to be used on the fly. ● E.g. Super Set of Tasks For the Jobs: Task 1, Task 2, Task 3, Task 4, Task 5, Task 6 Set of Tasks actually need for “job” instance: Task 2, Task 4 and Task 6 Only. ● Proprietary LLMs with higher reasoning and guardrails is used to find the Tasks needed to be run. ● Short-listed Tasks can be run with actual sensitive data uses locally deployed Open Source LLM.
- 15. v H2O.ai Confidential Low to No Code Fine-Tuning Techniques ● H2O LLM Studio: Equivalent to Stable Diffusion’s Automatic1111 or ComfyUI. ● Fine-tune Open Source LLMs without any coding, While can extensible with code. ● GUI specially for LLMs. ● Support for hyperparameters specific to finetune of LLMs. ● Support Low-Rank Adaptation (LoRA) and lower quantization to achieve lean memory footprint. ● Model Performance Tracking in UI. ● Test the fine-tuned model by testing it to get instant feedback. ● Most Important Enabler: Almost touch-less export to Hugging Face Hub.
- 16. v H2O.ai Confidential Low to No Code Fine-Tuning Techniques(cond.) My first fine-tuning using LLM Studio took almost same time as this presentation!