Shivendra Upadhyay’s Post

Why open source LLMs are going to win, in simple terms: Apps built on LLMs require a "good enough" threshold to be passed with regards to the language model. Today, GPT 4 passes this threshold for a lot of use cases. It's the only one, really, so OpenAI is winning. In all likelihood, it's a matter of time before an open source model passes the same threshold. That's when open source will win, just like it usually does with developer-oriented infrastructure. And monetization will once again come from the ecosystem of tools and services around it, creating a spacious market. Far fetched? Some serious companies are already betting on this. Notable mentions are of course Hugging Face and Mistral, but I've seen many pop out recently, and very promising ones. So, anyway, don't bet everything on OpenAI. Don't mindlessly build the core tech of your product on top of their API, hopefully not even assuming some emergent capabilities of their models. Just a piece of advice if you don't want to be left behind, vendor locked to a legacy service in 2-3 years. IMO OpenAI is likely to focus on other things in the future; maybe the GPTs ecosystem. But, at least according to the logic above, their API play will find it hard to last long as a flagship offering.

🚀 Enhancing LLMs with Retrieval-Augmented Generation (RAG) using Qdrant 🚀 Many people often ask me how to retrain LLM models for their specific use cases. Retraining LLM models requires massive computational resources and can be time-consuming. Usually, I ask them, "What is your problem or use case?" Most of the time, they just want to add additional knowledge to the model. However, this additional knowledge is often confidential, and they prefer not to share it with outsiders. My first suggestion is to perform Retrieval-Augmented Generation (RAG), which employs a vector database to add additional knowledge to an existing model. To help with this, I've spent some time creating a simple Proof of Concept (PoC) on RAG with Qdrant. I know some of you might be concerned about sending private knowledge to an external provider when encoding the knowledge into vectors and while querying the knowledge. Therefore, my suggestion is to use alternative open-source models instead of proprietary ones like OpenAI or Google Gemini. For example, BAAI's general embedding for text embedding and C4AI's Command R+ for chat models are great options. However, keep in mind that these still require sufficient hardware for inference. You can check out the repository here: 🔗 https://lnkd.in/gCxuBctg I hope you find it useful! 😊 #AI #MachineLearning #DataScience #NaturalLanguageProcessing #OpenSource #Qdrant #RAG #LLM #KnowledgeAugmentation #DataPrivacy #ProofOfConcept

If I would summarize the speculations about OpenAI’s secret project Q* (possibly the (2) breakthrough in GPT-5 Sam Altman hinted about before his (3) ouster—three things which we don’t know anything about really), many believe that it could be an early sign of their algorithm improving its ability to reason—in other words, using logic to solve new problems. 🧐 🥸 Even though LLMs today appear to be reasoning, they are actually just mimicking plausible sounding language very effectively. Solving a math problem is really just writing the most probable words given that very specific context, hopefully correctly. They can’t really “take a step back”, explore the solution space, plan multiple steps ahead using strategies, and abstractly reason about how to solve a problem using logic. 🤖 These are components believed to be the “breakthrough” of Q*, possibly part of the ongoing GPT-5 development and the safety reason behind Sam Altmans ousting. An early development of an algorithm which can solve problems it haven’t seen—at the moment trivial grade-school maths; but in the exponential progress we’re witnessing, could grade-school maths quickly become PhD maths, and so on? 🤷🏼♂️ At the end, speculators virtually don’t have any information on it, so we just don’t know. As the saying goes about tech industry’s “open secrets”: ship it or zip it. Ps. Talking about shipping, AWS announced their LLM chatbot yesterday evening called ➔ “Q”. It’s for sure the hottest letter in tech ATM. 🤓

𝗠𝗮𝗻𝘆 𝗼𝗳 𝘆𝗼𝘂 𝗵𝗮𝘃𝗲 𝗿𝗲𝗮𝗰𝗵𝗲𝗱 𝗼𝘂𝘁 𝘁𝗼 𝗺𝗲 𝗮𝗯𝗼𝘂𝘁 𝗰𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗮 𝗽𝗼𝘀𝘁 𝗼𝗻 𝗟𝗟𝗠𝗢𝗽𝘀, 𝗮𝗻𝗱 𝗵𝗲𝗿𝗲 𝗶𝘁 𝗶𝘀! LLMOps, short for Large Language Model Operations, is a framework that helps developers create, deploy, and manage large language models (LLMs). The image depicts the eight steps involved in the LLMOps process: 1. 𝗟𝗟𝗠 𝗦𝗲𝗹𝗲𝗰𝘁𝗶𝗼𝗻: This involves choosing the right LLM for the task at hand. The image mentions some examples, such as Azure OpenAI models, Llama, Claude, or any models from Hugging Face. You may also consider fine-tuning a model to improve its performance for your specific use case. 2. 𝗚𝗼𝗹𝗱𝗲𝗻 𝗗𝗮𝘁𝗮𝘀𝗲𝘁: Here, you'll collect and prepare a high-quality dataset to train your LLM. This dataset should be relevant to the task you want the LLM to perform. 3. 𝗣𝗿𝗼𝗺𝗽𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴: This involves creating clear and concise instructions for the LLM, along with any security measures you want to put in place. These prompts will guide the LLM in generating the desired outputs. 4. 𝗗𝗮𝘁𝗮 & 𝗦𝗲𝗿𝘃𝗶𝗰𝗲𝘀: You can enrich your LLM models with domain-specific data or enable in-context learning with use case-specific examples. 5. 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁: This involves developing one or more evaluators that can assess the LLM's response based on real-world data. 6. 𝗠𝗼𝗻𝗶𝘁𝗼𝗿: Here, you'll establish performance metrics to monitor the LLM's flow, including data collection, drift detection, and communication of the model's performance to stakeholders. 7. 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 & 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗶𝗻𝗴: This involves packaging and deploying the LLM flow as a scalable container for making predictions. The image also mentions enabling Blue/Green deployment with traffic routing control for A/B testing of the LLM flow. 8. 𝗢𝗻𝗹𝗶𝗻𝗲 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻: Finally, you'll conduct an online evaluation of the LLM batch to assess its performance, potential risks, etc. This evaluation will be based on the collected LLM data (query and response). By following these steps, you can build, deploy, and manage LLMs effectively. I hope this post clarifies the concept of LLMOps! Feel free to leave a comment if you have any questions.

Do you find it hard to keep up with the latest ML research? Are you overwhelmed with the massive amount of papers about LLMs, vector databases, or RAGs? 🤯 In this post, I will show how to build an AI assistant that mines this large amount of information efficiently. You’ll ask it your questions in natural language and it’ll answer according to relevant papers it finds on Papers with Code ⚙️ On the backend side, this assistant relies on a scalable serverless vector database, an embedding model from VertexAI, and an LLM from OpenAI. 💻 📱 On the front-end side, the assistant will be integrated into an interactive web app built with Streamlit and deployed on Google Cloudrun. Every step of this process will be detailed below with an accompanying source code that you can reuse and adapt👇. #programming #machinelearning #datascience #MLOps #LLMs

Lately, I've come across many GitHub projects lacking well-documented READMEs, which, in my opinion, is far from ideal. As a result of this, I looked for solutions, and found a tool uses Chat GPT api and could be helpful. You can easily create comprehensive and visually appealing READMEs using the tool. Just generate an access token from your OpenAI profile, then run the command in the 'How to use' section of the readme .It's a great tool to enhance the quality of your project documentation, so any developer should use it. The readme-ai repo: https://lnkd.in/dWqzZe_G To create a token, follow these steps: https://lnkd.in/d2sfE8dV

How Can Open Source LLMs catch up to GPT-4V and Google's Gemini? Open-source LLMs are getting really good. However, they are not as powerful as GPT-4 right yet.

Explore how Gloo Gateway can be configured to support LLM APIs, focusing on accessing the OpenAI API. https://lnkd.in/gE89D3Xj

Did you know that when you prompt DALL·E 3, it may not be you who is actually doing the prompting?  It turns out that DALL·E 3 revises the prompts written by the users, in case they are too simple to produce a good result (that’s just me paraphrasing the API’s documentation). This is a case of a relatively novel phenomenon we can call aisplaining: when an artificial system decides that what humans do is not good enough for it. As part of the ridiculous software project, I have decided to properly introduce this phenomenon to the world on the website https://aisplaining.us. There you can read a short essay about this phenomenon, look at the data I collected during my research, and even try an interactive version of aisplaining: https://gpt.aisplaining.us (I am paying myself for the openAI tokens, so I’m crossing fingers this doesn’t go viral).

🚀 Exciting news from OpenAI's dev day! Now you can make the OpenAI API return JSON. Here's how: 1️⃣ Modify your prompt: Specify that the response should be in JSON format and define the structure of the JSON object. 2️⃣ Pass response_format: When calling the API, specify the response_format as a JSON object. 3️⃣ Parse JSON response: Once you receive the response, parse it as JSON to extract the desired information. Check out the example in the playground to see it in action. #OpenAI #API #JSON #ArtificialIntelligence #MachineLearning #DeepLearning #ComputerVision #NLP #DataScience #BigData