Khalid Salama

Thanks for sharing Baris Yuksel this great site on machine learning trends. https://lnkd.in/d3DTuRBp Next generation AI Systems will need to innovate across different domains at once: Systems - How do we build and run these large systems (cost to train, cost to run) Algorithms - How do we change the architecture such that we can get much better scaling laws.

9

How to optimize Ollama for CPU-only inference? I am wondering if anyone has documentation, blog posts, ideas, ... really anything(!) about optimizing inference for CPU-only. The reason is that most people do not have nice GPU(s) in their notebooks and rely on API providers or remote hardware. Would be great to give those people a good hint on which model they could use locally. We are currently on a good way of doing a massive benchmark about CPU-only inference so i appreciate your input greatly!

"#ApacheIceberg is the most popular #opensource format for storing massive data sets and making them available to a variety of engines.   Now that the biggest #lakehouse vendors have embraced Iceberg, it’s finally possible to write data with one system and read the same data with another.  This gives developers tremendous flexibility to pick the engine of their choice for various workloads on the same data." --Dave Eyler, VP of Product at SingleStore. https://lnkd.in/eixbb3bE #BigData #DataPlatforms

18

https://lnkd.in/dBkJGXwN A really detailed paper comparing latest results in linear mat-mul free progress. Worth reading and if you are a Ph.D. Researcher something to consider as a great investment for your carreer

6

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

From IBM Granite Code Models: A Family of Open Foundation Models for Code Intelligence https://lnkd.in/easN99Pz - Introduces Granite Code models, a series of code foundation models ranging from 3B to 34B parameters, for code intelligence tasks like code generation, fixing bugs, explaining code, etc. - Trained using a two-phase strategy: Phase 1 on 3-4 trillion tokens of 116 programming languages, Phase 2 on 500 billion tokens of code and natural language data. - Two main variants released: Granite Code Base (foundation models) and Granite Code Instruct (finetuned on instructions using curated data). - Evaluated extensively on diverse tasks like code generation, explanation, fixing, editing, translation across major programming languages. - Shows strong performance across tasks and languages, often exceeding recent models like CodeGemma, StarCoder2 despite being smaller. - Larger models like 34B can handle complex tasks like application modernization, while smaller ones suitable for on-device uses. - Advantages: versatile for enterprise software tasks, trained ethically on permissible data, released under Apache 2.0 license. abs; Large Language Models (LLMs) trained on code are revolutionizing the software development process. Increasingly, code LLMs are being integrated into software development environments to improve the productivity of human programmers, and LLM-based agents are beginning to show promise for handling complex tasks autonomously. Realizing the full potential of code LLMs requires a wide range of capabilities, including code generation, fixing bugs, explaining and documenting code, maintaining repositories, and more. In this work, we introduce the Granite series of decoder-only code models for code generative tasks, trained with code written in 116 programming languages. The Granite Code models family consists of models ranging in size from 3 to 34 billion parameters, suitable for applications ranging from complex application modernization tasks to on-device memory-constrained use cases. Evaluation on a comprehensive set of tasks demonstrates that Granite Code models consistently reaches state-of-the-art performance among available open-source code LLMs. The Granite Code model family was optimized for enterprise software development workflows and performs well across a range of coding tasks (e.g. code generation, fixing and explanation), making it a versatile all around code model. We release all our Granite Code models under an Apache 2.0 license for both research and commercial use.

19

🌟 GraphRAG, a new tool for complex data discovery, is now available on GitHub for free! 🚀 from MicroSoft GraphRAG surpasses traditional RAG chunking by offering hierarchical summaries that precisely outline the semantic structure of data. It stands out by generating comprehensive and diverse answers, addressing global questions effectively while lowering token costs. The community summaries it produces enhance data discovery, making it an efficient tool for complex queries. Check out more details about this innovative tool here: 🔗 https://lnkd.in/gTB-y86a #DataDiscovery #GraphRAG #GitHub #DataAnalysis #Innovation #TechNews

14

Microsoft is training its own large language model, internally labaled MAI-1, with Mustafa Suleyman leading the effort. The model is around 500B parameters and could compete directly with LLMs from Google, OpenAI, etc. This marks the first time Microsoft has developed an in-house AI model of this magnitude since investing over $10 billion in OpenAI for the rights to reuse the startup's AI models. OpenAI's GPT-4 powers not only ChatGPT but also Microsoft Copilot. The development of MAI-1 is being led by Mustafa Suleyman, the former Google AI leader who recently served as CEO of the AI startup Inflection before Microsoft acquired the majority of the startup's staff and intellectual property for $650 million in March. Although MAI-1 may build on techniques brought over by former Inflection staff, it is reportedly an entirely new large language model (LLM) With approximately 500 billion parameters, MAI-1 will be significantly larger than Microsoft's previous open source models (such as Phi-3).

48

3 Comments

These findings motivate our design of a novel framework to investigate hidden representations in LLMs with LLMs, which we call Patchscopes. The key idea behind this framework is to use LLMs to provide natural language explanations of their own internal hidden representations. Patchscopes unifies and extends a broad range of existing interpretability techniques, and it enables answering questions that were difficult or impossible before. For example, it offers insights into how an LLM's hidden representations capture nuances of meaning in the model's input, making it easier to fix certain types of reasoning errors. While we initially focus the application of Patchscopes to the natural language domain and the autoregressive Transformer model family, its potential applications are broader. For example, we are excited about its applications to detection and correction of model hallucinations, the exploration of multimodal (image and text) representations, and the investigation of how models build their predictions in more complex scenarios. Google DeepMind

8

🚀 Exciting News in AI! 🌐 Snowflake AI Research has recently unveiled Arctic, an innovative open-source large language model (LLM) designed for enterprise applications. This breakthrough model boasts a staggering 480 billion parameters and utilizes a Dense-MoE Hybrid transformer architecture to efficiently manage tasks like SQL generation, coding, and instruction following. What sets Arctic apart is its cost-effectiveness and accessibility. Developed to operate under lower computing budgets, it allows businesses to adopt cutting-edge AI without the hefty price tag typically associated. The model is hosted on platforms such as Hugging Face, AWS, and NVIDIA AI Catalog, making it easily accessible and ready for enterprise integration. Snowflake Arctic is not only a testament to technological advancement but also a beacon for open-source collaboration. It is licensed under Apache 2.0, promoting transparency and community engagement. This strategic move is poised to spur further innovation and extend the model's capabilities through community-driven development. As businesses continue to seek competitive advantages, Arctic offers a scalable and adaptable solution that caters to diverse enterprise needs. Its release marks a significant leap forward in making advanced AI more feasible and impactful for businesses of all sizes. Stay tuned as we continue to follow the developments of this exciting project! #AI #MachineLearning #OpenSource #SnowflakeArctic #TechnologyTrends #BusinessInnovation https://lnkd.in/dZ3NkJgP

16

Interesting post by João Salgado about partitioning vs. sharding in #Postgres which lists trade-offs for each. It's great looking at the list of cons for partitioning and seeing that #TimescaleDB actually helps with most of them - Complexity ⬇️ because we transparently manage partition creation and lifecycle for you as data is ingested - Optimization Challenges ⬇️ because we have planner hooks that help with handling data exclusion (you still do need to pay attention to your partitioning key though). - Storage Overhead ⬇️ (although Postgres doesn't really have much overhead for partitions) because you can just choose to compress your data (we often see a 95x reduction in size). Timescale 🚀

18

1 Comment

Closing thoughts on PyData Eindhoven, data scientists have a very good understanding regarding good uses cases for Generative AI, and, as well, use cases that don't really fit Generative AI and are better served by other approaches. For those use cases, the discussion is about monitoring, quality control and management of AI applications. My talk was the first talk in the back-up list, however, none of the titular roster speakers needed to excuse him/her self, so it would be next year. For those wondering what my talk was to be about, check the following repo :) https://lnkd.in/ehwC7h4H

12

AI researchers write about "GRAG: Graph Retrieval-Augmented Generation" as a way to improve results from large language models beyond what "Retrieval-Augmented Generation (RAG)" already enables https://lnkd.in/eVt2DJDV. #airesearch #graphtechnology #retrievalaugmentedgeneration #generativeai #artificialintelligence

2

**Introducing GraphRAG by Microsoft Research!** GraphRAG, now available on GitHub, is a tool for complex data discovery. It leverages a large language model to create rich knowledge graphs from text documents, enabling precise question-answering over private datasets. This approach outperforms naive retrieval-augmented generation (RAG) by providing structured information retrieval and comprehensive response generation. Ideal for answering global questions, GraphRAG supports informed decision-making with higher comprehensiveness and diversity. 🔗 [Read more](https://lnkd.in/eEcukH_A)

56

Meta to release the Llama 3.1 series of models on Databricks – available to customers today. These models expand context length to 128K, add multilingual capabilities, and include Llama 3.1 405B — the largest and highest-quality openly available foundation model.    With this release, customers can serve and fine-tune the Llama 3.1 models, connect them to our RAG and agentic systems, easily generate synthetic data to better customize their models, and leverage them for scalable evaluation.    Databricks has integrated Llama 3.1 throughout our Data Intelligence Platform.

31

In the last two years, we have seen the emergence of Large Language Models(LLMs), specifically in enterprises, there is a trend of either building or leveraging small open source language models to handle targeted use-cases with necessary optimization both in terms of system resource usage & accuracy. We are working on some exciting stuff here & our technical disclosure on ‘multi-stage fine-tuning process for optimizing small LLMs in RAG applications’ just got published! Happy reading!!! #SLMs #GenAI https://lnkd.in/gBvHZvHt  

18

6 Comments