Dr. Aditya Raj’s Post

Cornell University: "Scalable MatMul-free Language Modeling" by Rui-Jie Zhu and team. They found a way to eliminate matrix multiplication in large language models while keeping performance high. Their method reduces memory use significantly and even outperforms traditional models at large scales. They also created a GPU-efficient version that cuts memory use by over 10x. This work is pushing LLMs closer to brain-like efficiency. #AI #MachineLearning #LanguageModeling #Efficiency #Innovation #CornellUniversity https://lnkd.in/gJcTHWQy

Interesting paper on the relationship between Generative AI and compression. https://lnkd.in/eKpCzhqy

More Tech for Good Research - Nature PrePrint Chemistry specific Agents fuelling domain specific capability whilst leveraging the power of LLMs. Large language models can be queried to perform chain-of-thought reasoning on text descriptions of data or computational tools, which can enable flexible and au… #TechforGood - reasons to be cheerful. #AI Source: Nature

This Paper Introduces AQLM: A Machine Learning Algorithm that Helps in the Extreme Compression of Large Language Models via Additive Quantization Quick read: https://lnkd.in/g4V7jJtx Paper: https://lnkd.in/grd_7K_f Github: https://lnkd.in/g4-ezNs3 #artificialintelligence #machinelearning #ai #largelanguagemodels

SCALABLE MATMUL-FREE LANGUAGE MODELING Failure to pay attention to this paper could mean missing out on a breakthrough that could shape the future of AI. This paper, which addresses the replacement of matrix multiplication (MatMul) operations in large-scale language models (LLMs) with matrix addition (MatAdd) operations, should not go unnoticed, due to the enormous potential impact it can have on the development of AI, both at the theoretical level (R&D&I) and at the level of product development. This approach has the potential to outperform current methods by a significant margin, paving the way for a new generation of AI models. Just a data point: With this new method, efficient GPU implementation would reduce memory usage by up to 61% and accelerate LLM training by 25.6%.

I am excited to share my latest article on the Transformer architecture for Large Language Models (LLMs)! If you are curious about the technology behind advanced language models, this article provides a comprehensive overview of the architecture and its applications. #ArtificialIntelligence #MachineLearning #NaturalLanguageProcessing #Transformers #DeepLearning #AI #DataScience #LLMs #TechInnovation #AIResearch

Just finished the course “Introduction to Prompt Engineering for Generative AI” by Ronnie Sheer! Great course to learn how and why things work when you use GPTs and large language models. Check it out: https://lnkd.in/gmTkHbBu #generativeai #naturallanguageprocessing.

I just published "Part 8 — Mathematical Explanation of Why It’s Hard for LLMs to Memorize" Contrary to popular belief, LLMs (or transformers in general) cannot be tuned for memorization, a task that appears straightforward but is mathematically complex. In this detailed exploration I dive into the mathematical principles underlying these models, highlighting why exact memorization is a formidable challenge with LLMs. You know what is easy though? Hallucination. No you cannot have it both ways where you accuse LLMs for memorization and hallucination. This is not mathematically possible (no they are not orthogonal) #AI #artficialintelligence #machinelearning #LLM #GPT https://lnkd.in/gTEMY8Er

Is anyone else frustrated by the lack of transparency in AI research? This amazing OLMo paper just made my day! It opens up a whole new world of open-source language models, complete with data, training code, eval code, adaptaion code, weights, and even wandb logs! Researchers can finally dig deeper, understand biases, and steer AI development in the right direction. Who's in for a paradigm shift? #OpenLM #LanguageModels #AIResearch https://lnkd.in/dMCCYsjb

I used to refer to these as "not-so-large-language-models"....Small Language Models (SLMs) is much better! "Small Language Models, which are compact generative AI models, are distinguished by their small neural network size, number of parameters, and volume of training data. SLMs require less memory and processing power than Large Language Models, which makes them perfect for on-premises and on-device deployments."