Joaquin Quiñonero Candela

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

Small models becoming smarter! Phi-3 reinforces the importance of data quality.

24

I’m thrilled to announce the launch of my new Substack, Silicon Sand Stack, with my very first article: "Today's Large Language Models Will Never Be Responsible, Safe, or Green: The Need for Investment in Revolutionary New AI Paradigms" In this debut piece, I dig into the critical flaws of current large language models (LLMs) and why they fall short of being responsible, safe, or environmentally friendly. I examine the inherent issues like hallucinations, lack of real-world grounding, unreliable reasoning, and potential biases that plague these models. 🔍 Key insights include: - The technical and ethical limitations of today's LLMs. - The necessity for radically new AI paradigms. - The role of venture capital in fostering innovative AI approaches that prioritize safety, responsibility, and sustainability. Join us as we embark on a journey to rethink AI and advocate for groundbreaking advancements that align with human values and societal needs. Read the full article and subscribe here: https://lnkd.in/eDcTniTg #ArtificialIntelligence #AI #GenerativeAI #TechInnovation #ResponsibleAI #GreenAI #FutureOfAI #SiliconSandStudio

46

1 Comment

If you're interested in understanding the inner layers of LLM models, the #Anthropic blog is amazing. They tore down the Claude 3.0 Sonnet LLM models apart and peeked into its model view and perspectives. It's a long read paper, but an amazing read! By successfully extracting millions of features from the middle layer of their Claude 3.0 Sonnet model, they have uncovered a conceptual map of its internal representations, revealing how it encodes diverse concepts like cities, scientific fields, and even abstract notions supporting the security, various bias and power-seeking behavior etc... Good Read for a long weekend. https://lnkd.in/gNm8qA3W

7

2 Comments

Come see this talk, which I view as the culmination of several years of discussions and co-teaching with George Fairbanks. It's a bit of an overview, a bit of an explainer, a bit of a how-to, but mostly a sneaky way to make some important vocabulary from design become load-bearing in how you think about software testing.

86

1 Comment

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

Excited to be open-Sourcing long context (128K) Llama-3 70B - Llama-3-Giraffe-70B and Giraffe-Instruct From Abacus.AI The biggest issue with the Llama-3 family is that they are fairly small context models. So, even though they are as performant as GPT-4 on critical benchmarks, they can only be used in the real world with a bigger context. We just open-sourced two 128K long-context fine-tunes for Llama-3-70B, which effectively puts it on par with GPT-4 regarding context length support and real-world usage. At Abacus AI, we already use Llama-3 on a few production workloads as a great base model for our customer fine-tunes. Links to both models - https://lnkd.in/gPX_XpD6 https://lnkd.in/gaeEq94t

3

Lin Guan and Yifan Zhou have led a very interesting study investigating the effectiveness of Vision Language Models in critiquing robot behaviors in terms of the desirability of their "style." If you are wondering how our work on LLM's inability to verify correctness squares with this study on using them as style critics, see my earlier post https://lnkd.in/gBYAr5qq ! This is the reason why being a style critic is one of the roles LLMs can play in LLM-Modulo framework: https://lnkd.in/gyAyKx4E Check out the details of the work @ https://lnkd.in/gfVZ4EYW ( Lin Guan also has a nice twitter thread on the work here: https://lnkd.in/gnieU-qM )

88

2 Comments

AI at Meta has released the most powerful open-source model yet as of today: 𝑳𝑳𝑨𝑴𝑨3 - 8B and 70B models at 8k context length! They highlight improvements in each of the following aspects as the key differentiator: a) Model Architecture b) Pretraining data c) scaling up pre-training d) Instruction fine-tuning 💥 𝐏𝐫𝐞𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 & 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 𝐮𝐩: ✈️ Vocab of 128k tokens & GQA across both 8B and 70B, 8k native context length ✈️ Pretrained over 15T tokens, 4 times more code than LLAMA2 ✈️ Multi-lingual : over 5% of pretraining data consists of high-quality non English data that covers over 30 languages: don't expect same performance as English ✈️ LLAMA2 generates training data (like in Self-Instruct) to generate text classifiers that filter data powering LLAMA3 ✈️ Both 8B and 70B improve even after the model is trained on 2 orders of magnitude more data i.e log-linear improvement after 15T tokens training ✈️ Training runs on 2 custom-built 24k GPU clusters with an overall training time efficiency of 95% ✈️ 𝘖𝘷𝘦𝘳𝘢𝘭𝘭, 3 𝘵𝘪𝘮𝘦𝘴 𝘮𝘰𝘳𝘦 𝘦𝘧𝘧𝘪𝘤𝘪𝘦𝘯𝘵 𝘵𝘩𝘢𝘯 𝘓𝘓𝘈𝘔𝘈2 💥 𝑰𝒏𝒔𝒕𝒓𝒖𝒄𝒕𝒊𝒐𝒏 𝑭𝒊𝒏𝒆𝒕𝒖𝒏𝒊𝒏𝒈: ✈️ Combination of SFT + PPO + DPO + Rejection Sampling ✈️ 𝑸𝒖𝒂𝒍𝒊𝒕𝒚 𝒐𝒇 𝒑𝒓𝒐𝒎𝒑𝒕𝒔 𝒖𝒔𝒆𝒅 𝒊𝒏 𝑺𝑭𝑻, 𝒑𝒓𝒆𝒇𝒆𝒓𝒆𝒏𝒄𝒆 𝒓𝒂𝒏𝒌𝒊𝒏𝒈𝒔 𝒖𝒔𝒆𝒅 𝒊𝒏 𝑫𝑷𝑶, 𝑷𝑷𝑶 𝒉𝒂𝒔 𝒂𝒏 𝒐𝒖𝒕𝒔𝒊𝒛𝒆𝒅 𝒊𝒎𝒑𝒂𝒄𝒕 𝒐𝒏 𝒎𝒐𝒅𝒆𝒍 𝒑𝒆𝒓𝒇𝒐𝒓𝒎𝒂𝒏𝒄𝒆 ✈️ Careful data curation and multiple rounds of QA on annotations from human annotators ✈️Training on preference ranking improved the model performance greatly on coding and reasoning tasks 💥 𝑳𝑳𝑨𝑴𝑨3 𝑮𝒂𝒖𝒓𝒅𝒓𝒂𝒊𝒍𝒔: ✈️ Updated and new safety tools : LLAMA Gaurd2 & cybersecEval 2 ✈️ CodeShield - inference time gaurdrail for filtering insecure code ✈️ Instruction Fine-tuned model redteamed for safety : By generating adverserial prompts that try to elicit problematic responses. ✈️ LLAMAGaurd - foundational for prompt and response safety, can be easily finetuned to create a new taxonomy 💥 𝑰𝒏𝒇𝒆𝒓𝒆𝒏𝒄𝒆 & 𝑭𝒖𝒕𝒖𝒓𝒆 ✈️ Despite LLAMA3 having 1B more params than LLAMA2 7B, the improved tokenizer efficiency and GQA contribute to maintaining the inference efficiency on par with Llama 2 7B. ✈️ 400B+ parameters, Multilingual, Multimodal, and longer context window LLAMA3 would be available on AWS soon. Try it out! #generativeAI #llm #llama3 #aws #bedrock #sota

33

1 Comment

Hey AV, Self-driving, and ML Infra (:wink:) communities! I'm proud to announce a new Nuro technical publication called "FTL Model Compiler Framework". The previous time we shared our efforts on scaling the ML Training (link: https://lnkd.in/gXBszEME ). This time, let's talk about the inference. Can we integrate multiple ML sub-compilers to provide multi-GPU serving, execution priority control, and even custom GPU kernel injection? The answer is "Yes" with our Faster Than Light (FTL) framework - https://lnkd.in/g_vryPRn ! Thanks to the authors - Ali Boubezari, Muyang Yu, Nick Korovaiko, and Hongze Z.!

61

🚨 Excited about AI's potential but worried about its safety? Perhaps you're struggling to convince decision-makers at your institution of its benefits? In the wonderfully wise words of Douglas Adams: "Don't Panic!" Instead, Join us on Tuesday, April 30th at 4 PM BST / 11 AM EST for a discussion on making AI safe for Higher Education. ⭐️ Sign up for free today (using the link below this post) and secure your spot. I'm excited to connect with you and address these important issues together! #AI #HigherEducation #Webinar #AISafety #Gecko

16

1 Comment

Interesting article covered in the New York Times that describes experiments by Anthropic to understand what's really going on inside of neural network-based Large Language Models (LLMs). Researchers identified clusters of millions of neurons related to "concepts." Although neural networks are "inspired by the human brain" and are not an actual brain, the ability to excise these clusters to remove a behavior or to stimulate/focus on them to emphasize a concept does sound a lot like brain surgery. The ability to identify such clusters does kind of sound like Functional MRI. It will be interesting to see whether research into a computer model that generates language teaches us something about our own biology. Because of course we are totally not just computer code running on someone's computer. https://lnkd.in/eEM_fEzX

8

2 Comments

Anthropic is trying to figure out what they actually built. They have made a major breakthrough (or is it???) in AI interpretability with their latest paper "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet". There are a bunch of “first times” mentioned in the paper, like for the first time they have extracted millions of understandable features from the middle layers of a state-of-the-art production language model. The researchers found a wide range of interpretable features corresponding to concepts like famous people, cities, scientific fields, code syntax, and more abstract ideas like security vulnerabilities and gender bias. Remarkably, they were able to manipulate the model's behavior by amplifying or suppressing specific features, like inducing it to self-identify as the Golden Gate Bridge. While there is still much more work to be done, this research lays critical foundations for making AI systems safer and more understandable. Anyhoo I know what I'll be diving into this weekend 😛 #ai #ml #aimusings

22

Sebastian Raschka's blog provides invaluable insights and comprehensive information on #LLMs and #LLMOps, making it a valuable resource for anyone seeking knowledge and understanding in this field. #NLP #LLMS #LLMOps

9

#RAG DOES NOT REDUCE HALLUCINATIONS IN #LLMS: Think I've called this out a few times, but the validation is nice - https://lnkd.in/eAi6zxW2. The article makes two well established points: 1) "Hallucination is a problem of reasoning and not relevance! Any amount of relevant text fed through RAG to a machine will retain the original perplexity and entropy in the system to hallucinate independent of the text"; and 2) "The initial challenge arises from the retrieval phase’s reliance on semantic similarity, which is quantified by inner product spaces or cosine similarities. This metric, while capturing semantic closeness, does not differentiate between factually accurate and inaccurate information." This should be clear to everyone, but the industry still shills flimsy RAG scaffolding anyway (shovels to gold miners).

104

34 Comments

𝕎𝕙𝕪 #𝔸𝕀 𝕗𝕠𝕝𝕜𝕤 𝕟𝕖𝕖𝕕 𝕒 𝕓𝕣𝕠𝕒𝕕 𝕓𝕒𝕤𝕖𝕕 𝕀𝕟𝕥𝕣𝕠 𝕥𝕠 #𝔸𝕀 👉 As I go around giving talks/tutorials on the planning and reasoning abilities of LLMs, I am constantly surprised at the rather narrow ML-centric background of grad students/young researchers have about #AI. This seems to be especially the case with those who think LLMs are already doing planning and reasoning etc. Most of them don't seem to know much about the many topics that are taught in a broad-based Intro to #AI course--such as combinatorial search, logic, CSP, difference between inductive vs. deductive reasoning (aka learning vs. inference), soundness vs. completeness of inference/reasoning etc. I can understand why a strong background in ML and DL is sine qua non these days in using/applying the current #AI technology. That doesn't however mean that other things, that are typically not covered in ML courses, but are covered in Intro #AI courses, are expendable. If you don't know those concepts, you are more likely than not to re-invent crooked wheels (see this for examples of how people get tripped up: https://lnkd.in/gUPPb7s4) All this is particularly relevant for those busy building empirical scaffolds over LLMs (the "LLMs are Zero-shot <XXX>" variety). Most often, these young researchers are coming from NLP. At one point, NLP used to be NLU and students had quite a firm grasp of logic (e..g Montague Semantics!). But over the years, NLU became NLP which in turn has become Applied Machine Learning, and students don't quite have the background in logic/reasoning etc. Now that LLMs have basically "solved" the "processing" tasks--such as information extraction, format conversion etc., NLP folks are trying to turn to reasoning tasks--but often lack the necessary background. (See this unsolicited advice to NLP students: https://lnkd.in/gKTdsH2P) Background in the standard Intro AI topics like search/CSP/logic are useful even if you don't plan on directly using those techniques (e.g. because you want differentiable everything to make use of your SGD hammer). Like MDPs, they provide a normative basis for many deeper reasoning tasks AI systems would have to carry out when they broaden their scope beyond statistical learning. Without that background, you will likely try to pigeon hole everything into "in/out of distribution" framework, when what you need to think of is "in/out of deductive/knowledge closure; see https://lnkd.in/gTWVibdt ) One of the other things that you get exposed to in the standard Intro #AI is computational complexity of the various reasoning tasks. People who jumped in directly via applied ML might understand a bit of sample complexity (maybe?), but are not that attuned to reasoning complexity. (Contd. in the comment below)

374

46 Comments

This new little product demo is awesome! Our new Rev AI Transcript Assistant will save you TONS of time. Easily pull out nuggets of info from your transcripts, making valuable content more accessible than ever. https://lnkd.in/gzzFpcDa #transcription #asr #accessibility #transcript #summarization #ai #assistant

31

2 Comments

I remember learning category theory in undergrad and wondering: when would this ever be useful? Turns out it's very useful for program synthesis and software modeling -- thanks Kristopher for the great talk!

9