Babak Pahlavan’s Post

Upleveling my favorite part of this post: "Leveraging Neuron SDK 2.19: we got early access to the latest Neuron SDK from AWS AI team and it's awesome! Magically, our speed improved and we needed 30% less of our Trainium fleet (speed + throughput improved!). "

🧑🍳 We just dropped Dify v0.6.3 with some cool new integrations. 🧾 Check out the list we put together for you. New Tools 1. JinaReader: Convert any URL to an LLM-friendly input. Experience improved output for your agent and RAG systems at no cost. 2. SearXNG search: A free internet metasearch engine provides extra information for your AI agents. Models 1. NVIDIA CodeGemma-7B, a code LLM based on Gemma, excels in mathematical reasoning and coding capabilities. 2. Two embedding models available on AWS Bedrock offer additional options for building your RAG pipeline： ① amazon.titan-embed-text-v1 ② cohere.embed-english-v3 3. Refactor TONGYI models to add support for function calling and image recognition. Minor enhancements 1. The knowledge base now supports uploading EPUB and XLS files for RAG extractors. 2. New Relyt vector database for the Community Edition. 3. The OpenAI-API-compatible interface integrated model now supports Stream function call, resulting in smoother AI agent output. ❤️ Huge thanks to all our contributors! Check our release log for details: https://lnkd.in/gyM6SzPT

https://lnkd.in/eTncijET anyone can train gpt4 omni competetive models by creating synthetic training data with nemotron-4 340b. the cost of the data generation would be about 42 dollars an hour on google colab. the cost of training the new model would be free on google colab using the t4 compute engine. we can create datasets using nemotron that produce 7b and other flavors of lightweight models from pretrained, opensource weights that compete with gpt4 omni. companies like apple are contracting openai for millions to service models.

We sure live in interesting AI times ! Are we going to witness a renaissance of RNNs and LSTM ? the original author of LSTM just published a paper "xLSTM: Extended Long Short-Term Memory". Let's see how it stacks up against all mighty transformers ! Interestingly universal superiority of transformers as compared to other DL networks architectures has been hyped so much so it almost became common sense. However, in the "An Impartial Take to the CNN vs Transformer Robustness Contest" paper authors show that: 1. "Simplicity bias experiment. Transformers, just like CNNs, also suffer from the so-called simplicity bias". 2. "We show that for out-of-distribution detection task, CNNs and Transformers perform equally well". 3. "In-distribution calibration of the best performing CNN model (in terms of accuracy) is better than the best performing Transformer." 4. "Again, there is no clear winner in detecting misclassified inputs". #AIforall #Aiforgood #AIeverywhere Yury Gorbachev Ekaterina Aidova Aleksa Gordićć Sergey Lyalin Alexander Kozlov Andrei Kochin Roman Kazantsev Evgeniia Nugmanova Anisha Udayakumar Raymond Lo 👓🤖 Paula Ramos Adrian Boguszewski Paul Wealls Ravi Panchumarthy, Ph.D., Junwen Wu Anna Gubenkova Anna Likholat https://lnkd.in/dwJCX3h5 https://lnkd.in/drgZwxBC

🚀 A lightweight update for Dify v0.5.10 adds new models and tools. Here's the list: 🧙🏼♀️ New model providers: 1. NVIDIA: - LLMs: mixtral-8x7b, gemma-7b, llama2-70b, and an additional multimodal model for image recognition named fuyu-8b. - Embedding & Rank model: NV-Embed-QA and nv-rerank-qa-mistral-4b:1, designed to assist in building the RAG pipeline. 2. 01.AI: - Introduces the yi-chat-34b series models. ✨ New models: 1. Anthropic Claude-3-Haiku-20240307 2. AWS Bedrock Claude3 3. Azure OpenAI LLM 0125 version 4. TONGYI text-embedding-v1 and text-embedding-v2 🛠️ New tools: 1. A tool has been added for calculating the weekday of a given date. 2. New tools for open weather search and image generation using the Spark API have been introduced. 😋 A Huge thanks to all our contributors! Check our release log for details: https://lnkd.in/gGMCGPV5.

📢 HuggingFace: Welcome Gemma - Google's new open LLM Summary: Google recently released Gemma, a family of open-source language models including 7B and 2B parameter models. Both sizes have base and instruction-tuned versions and can be run on various consumer hardware. The 7B model performs well, comparable to other top models of similar size. Gemma models are available on the Hugging Face Hub and can be used with Transformers, Jax, or Flax. Google Cloud and Hugging Face have partnered to enable the deployment, training, and tuning of Gemma on Google Cloud platforms. Additionally, Gemma can be deployed on Hugging Face's Inference Endpoints. The Hugging Face ecosystem provides tools like TRL to fine-tune Gemma on consumer GPUs efficiently. These releases and integrations showcase Google's commitment to open-source AI and Hugging Face's comprehensive support for Gemma. Source 🔗: https://lnkd.in/dc7ek5Qv. Curated with ❤️ via https://lnkd.in/dAQMAegz, aggregating 100+ #tech blogs on a single feed! ✨ Subscribe to Techflix Weekly: https://lnkd.in/gDnfiRrb

Phind-70B: Closing Code Quality Gap with Speed and Accuracy Phind has introduced Phind-70B, a model that aims to close the code quality gap with GPT-4 Turbo while running four times faster. It’s designed to provide high-quality answers for technical topics quickly, enhancing the user experience for developers. The model is based on the CodeLlama-70B model and has been fine-tuned on an additional 50 billion tokens. It supports a context window of 32K tokens and scores 82.3% on HumanEval. Phind-70B is also faster than GPT-4 Turbo, running at 80+ tokens per second. The model is available for use without a login, and higher limits are available with a Phind Pro subscription. #Phind #Phind70B #GPT4Turbo #CodeQuality #DeveloperExperience #CodeLlama #TechnicalTopics #AI #MachineLearning #DeveloperTools #PhindPro

Phind's 70B code model scores 83% on HumanEval, matching GPT-4 performance. Crucially it has significantly fewer lazy responses and refusals. The 34B model will be released soon - the larger 70B model will be released in time. #ml #llms #gpt4 #gpt3 #phind https://lnkd.in/dFXwXJBJ

Microsoft is making a significant bet on AI in Windows, emphasizing natural language and semantic computing as the future. During Build 2024, Microsoft announced the Windows Copilot Runtime, which moves AI functionality from Azure to users’ own PCs, leveraging local AI accelerators like neural processing units (NPUs) in desktop. This shift aims to enhance AI experiences for Windows users by bringing powerful AI models closer to the edge.

Are LoRA adapters adept ? Recently at #WWDC2024, Apple announced "Apple Intelligence", a suite of AI tools which seamlessly integrates into everyday workflow (the best kind of AI tools). With Apple's strong focus on privacy, one of the key highlights is their on-device handling of AI models. Particularly (https://lnkd.in/eJv2X5ym), they present how they use LoRA adapters to handle different tasks using a single foundational model.  If you have been in the LLM space, you might also know about NVIDIA's NIM, which provides LoRA adapter swarm (https://lnkd.in/evSZQrSw), which allows you to host a single model with multiple adapters fine-tuned for specific tasks. The benefits are obvious: 1) One model deployment can handle multiple specialized tasks. 2) Less resource requirements for deployment and fine-tuning. 3) Lower model management overhead. Seems like a great idea and something you should leverage, right ? Well, not necessarily... From a technical perspective, (read https://lnkd.in/ehFbsXQm): 1) LoRA in most cases, is just a style transfer, it shifts the token probability distribution. The tasks that we are fine-tuning using LoRA, already need to be in the knowledge base of the foundational model.    2) The fine-tuning does not scale with dataset size.  Adding more data samples to fine-tuning will have diminishing results on model performance. And in the case of out-of-distribution tasks (tasks not seen by base model), we might not even get any improvements. From a use-case perspective: 1) If our use-case has fundamentally different tasks (language translation vs code translation), then it does not make sense to use LoRA adapters. 2) In case we want to have efficient resource utilization and want to provide more functionality, provided we won't have concurrent tasks, this arrangement fits perfectly. Not to say standard fine-tuning is better, far from it. In most cases, full fine-tuning of LLMs actually has more issues. But that is a discussion for another time. For now, let's make lemonade. #artificialintelligence #llm #machinelearning #data