Aaron Roth

Stole a few points from Prof Christopher Manning, who talked about LLMs and language modeling in general in the latest TWIML AI Podcast: 📌 Humans acquire language skills in a way very different from LLMs. We need millions of words, compared to billions or even trillions of tokens for LLMs. LLM researchers may want to investigate and learn from how humans acquire language skills. 📌 LLMs cannot reason. However, there are other deep learning models, such as AlphaGo, that can. LLM researchers may want to look into how to integrate that type of reasoning/searching/planning capability into LLMs. 📌 LLMs' world models should enable search and discovery. Although Prof. Manning didn’t call this out explicitly, my understanding is more similar to a knowledge graph type of structure. 📌 Next-gen LLM idea: a soft form of locality and hierarchy. Transformers attend every token to every other token, which is very inefficient, while human language can be modeled by n-grams most of the time. #artificialintelligence #machinelearning #deeplearning https://lnkd.in/euzwMQ6p

84

18 Comments

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

CRITICAL Reading prompt template: Foundational work on critical reading/thinking for verifying reasoning quality of LLMs using the Socratic method, paring with inductive and deductive reasoning. https://lnkd.in/ghRekxe5

13

1 Comment

Separating Fact from Logic: Test of Time ToT Benchmark Isolates Reasoning Skills in LLMs for Improved Temporal Understanding Quick read: https://lnkd.in/gbc5CqN8 Paper: https://lnkd.in/giu72CwW HF Page: https://lnkd.in/g7H6dgY9

17

If you didn't get a chance to attend the NSF CISE Career Workshop the slides from all of ours talks are posted on the materials page: https://lnkd.in/g3dRcxEx And Josiah made a great Overleaf template: https://lnkd.in/gNdUG2kY Some key takeaways from our CAREER Awardee Talks: 1. The first page should tell your whole story - this is the first impression for the reviewer. You don't want them to have to look any further into the proposal to be sold on your idea. 2. We all had different degrees of preliminary work. Unfortunately, I don't think there is a hard metric for this - and I think it further emphasizes that the CAREER is more about your vision than anything else. 3. Have a baby? Sleep deprivation may lead to transformative research ideas - caveat: this is based on a small sample size (3 out of 4) so maybe not a strong recommendation d=

42

If you are interested in Large Language Models and fine-tuning space, check out our paper where we study different fine-tuning scenarios and the interaction of various fine-tuning optimizations in such scenarios. In collaboration with Arjun Singh, Anup A. Shirgaonkar, Pavan Manoj and Vijay Aski.

27

Following Jee's meesage, please let me share a new related work jointly done with Tomi Kinnunen, Kong Aik Lee, Paul-Gauthier Noé, and Junichi Yamagishi. 📜 Title Revisiting and Improving Scoring Fusion for Spoofing-aware Speaker Verification Using Compositional Data Analysis (arxiv link https://lnkd.in/g568pKXB) ⚒ Use it now !? The proposed fusion method can be used in the ongoing ASVspoof 5 challenge, Track 2 (https://www.asvspoof.org/). 🔗 link to the paper: https://lnkd.in/g568pKXB 🔗 a Jupyter notebook: https://lnkd.in/gW8bPdfJ 🔗 code of SASV system: https://lnkd.in/gPyh6NPh There will be a package on ASVspoof5 github site as well https://lnkd.in/gU6XUhhG (currently in private) 🔖 Background:  The paper done with Jee showed how ASV systems trained on bona fide data (human speech data) could defend themselves from spoofing attacks to some extent. I understand this potential is consistent with the expectation of many researchers, but currently, it may still be helpful to have spoofing-robust ASV systems, either built upon the combination of a spoofing countermeasure (CM) and an ASV system or trained using the bona fide and spoofing data in an end-to-end manner. 🆕 Key question: how should we fuse CM and ASV scores? Our new work revisits this question and produces findings from ** a decision theory point of view **. We describe a non-linear fusion of ASV and CM log-likelihood-ratios-like quantities, which minimizes the conditional risk in decision making for the task. 📚 The new paper also links to many pieces of work by others, from which I learned a lot. I will write more in the threads below. My understanding on related topics is still shallow. Looking forward to everyones' comments!

31

8 Comments

It's always funny to see when a top-tier research group like the folks at Stanford under Christopher Manning do something, and then quickly (and quietly, but not so quietly that I didn't notice) back-peddle. For those who don't know, PyReft is a library for *extremely efficiently* fine-tuning language models. You might not realize it yet, but techniques like it are going to blow regular PeFT techniques like LoRa out of the water hard very soon. The authors, in order to prove how good their technique is, have done many experiments. One of these experiments was to jailbreak the model and make it "unaligned". I think that these kind of experiments are fine/good, and certainly in line with the kind being done by the lesswrong community (who are AI doomers) with orthogonalization - but it's interesting to see that someone else decided that this kind of script was risky. For anyone who cares, you can access the scripts easily via this link (click "view file" on the "deleted" files), and see how researchers are tricking these models to become unhinged (or german speaking, or instruction tuned, or funny, etc) with surgical modifications to only ~0.0001% of the parameters. https://lnkd.in/gGtQTFNU

153

11 Comments

Excited to share the topics of our two just accepted IEEE/RSJ IROS 2024 papers, as both represent key directions for our efforts on resilient autonomy. 1. Marvin Chayton Harms, Mihir Kulkarni, Nikhil Vijay Khedekar, Martin Jacquet, Kostas Alexis, "Neural Control Barrier Functions for Safe Navigation", 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024), Abu Dhabi, UAE Video: https://lnkd.in/dcYbnzne >> In this work we introduce the concept of Neural CBFs for safe autonomous navigation exploiting exteroceptive data and without the need for a map or consistent odometry estimation. 2. Mohit Singh, Kostas Alexis, "Online Refractive Camera Model Calibration in Visual Inertial Odometry", 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024), Abu Dhabi, UAE Video: https://lnkd.in/d_iyQdgp >> In this work we present online visual-inertial odometry for underwater systems that explicitly accounts for refractive effects and co-estimates the refractive index of a medium in real-time thus eliminating the need for medium-specific camera calibration. #robotics #autonomy #ntnu

242

1 Comment

Andrej Karpathy introduced the concept of LLMs not as chatbots, but the kernel process of a new Operating System, where it orchestrates: - input & output across modalities (text, audio, vision) - code interpreter, ability to write & run programs - browser / internet access - embeddings database for files and internal memory storage & retrieval I created LLM-OS which local implementation of the LLM-OS idea where multiple AI agents orchestrate and delegates tasks amongst each other in order to come up with a more robust and factually correct response. Check it out here: https://lnkd.in/gdDmrJu6

19

PARIKSHA update! TLDR; We did 90k human evaluations (thanks to Karya!) across 10 Indian languages and 30 models (probably the largest multilingual human evaluation of LLMs so far?). GPT-4o consistently performs best and Llama-3 70B is close behind. We also performed LLM-based evaluations and found that agreement with human evaluation is higher in some cases, particularly pairwise evaluation in some languages, while agreement on prompts containing cultural nuances is lower (so don't use LLM evaluators for this yet!). Abstract: Evaluation of multilingual Large Language Models (LLMs) is challenging due to a variety of factors -- the lack of benchmarks with sufficient linguistic diversity, contamination of popular benchmarks into LLM pre-training data and the lack of local, cultural nuances in translated benchmarks. In this work, we study human and LLM-based evaluation in a multilingual, multi-cultural setting. We evaluate 30 models across 10 Indic languages by conducting 90K human evaluations and 30K LLM-based evaluations and find that models such as GPT-4o and Llama-3 70B consistently perform best for most Indic languages. We build leaderboards for two evaluation settings - pairwise comparison and direct assessment and analyse the agreement between humans and LLMs. We find that humans and LLMs agree fairly well in the pairwise setting but the agreement drops for direct assessment evaluation especially for languages such as Bengali and Odia. We also check for various biases in human and LLM-based evaluation and find evidence of self-bias in the GPT-based evaluator. Our work presents a significant step towards scaling up multilingual evaluation of LLMs. Preprint: https://lnkd.in/g_m5beGJ We will continue to add more prompts, models and languages in future rounds of Pariksha. Work done with the fantastic team at Microsoft Research India and Karya - Ishaan Watts, Varun Gumma, Aditya Yadavalli, Vivek Seshadri, Swami Manohar. #multilingual #evaluation #genai #indic

349

8 Comments

"A Robot Walks into a Bar" - I heard this title so much I made it into a socio-technical systems and human-computer interaction study at Google DeepMind, with my fantastic co-authors Juliette Love, Kory W. Mathewson and Shakir Mohamed. Fresh out of press on arXiv and just presented at ACM FAccT. "A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians" We interviewed 20 comedians who use AI in their artistic process, as part of 3-hour workshops on "AI x Comedy", conducted at the Edinburgh Festival Fringe in August 2023 (and online): * Comedy writing session with large language models (LLMs) * a human-computer interaction questionnaire to assess the Creativity Support Index of AI as a writing tool * and a focus group interrogating the comedians' processes of using AI * as well as their ethical concerns about bias and cultural appropriation, censorship and cultural erasure, and copyright. Participants noted that existing moderation strategies used in safety filtering and instruction-tuned LLMs reinforced hegemonic viewpoints, by erasing minority groups and their perspectives - a form of censorship. At the same time, most participants felt the LLMs did not succeed as a creativity support tool, by producing bland and biased comedy tropes, akin to "cruise ship comedy material from the 1950s, but a bit less racist". Our work extends scholarship about the subtle difference between, one the one hand, harmful speech, and on the other hand, "offensive" language as a practice of resistance, satire and "punching up". We also interrogate the global value alignment behind such language models, and discuss the importance of community-based value alignment and data ownership to build AI tools that better suit artists' needs. https://lnkd.in/ejg8e_Xf

183

41 Comments

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

I'm happy to share our new paper, "Probabilistic Byzantine Fault Tolerance," which was just published on ACM PODC'24: https://lnkd.in/dTpQwDeJ The paper introduces a new way to scale BFT/blockchain consensus without giving up optimal latency (3 steps) and resilience (3t+1) by embracing probabilistic guarantees. It describes a new protocol called ProBFT, which can be seen as a version of PBFT employing probabilistic quorum systems. Our key result is that ProBFT has a message complexity of O(n*sqrt(n)) instead of O(n^2), as in the original PBFT. We believe the same fundamental techniques employed in ProBFT can be used to scale other decentralized protocols. (Image source: Morrison & Troug, Animal Man n. 8, DC Comics, 1989.)

104

4 Comments

If you’re attending EUROSYS this year, check out Zhongshu Gu’s presentation of our paper DeTA: Minimizing Data Leaks in Federated Learning via Decentralized and Trustworthy Aggregation (https://lnkd.in/eSeduQBH) in Session C (https://lnkd.in/eAhgurSY) It is not always feasible or desirable to centralize data in a single data center to train models, which necessitates the use of decentralized model training techniques. In this context, our work is a nifty combination of randomized model partitioning, parameter shuffling and the use of confidential computing that enables private federated learning at a much lower overhead compared to homomorphic encryption and secure multi party computation protocols. Our technique exploits the fact that parameter order in a model is seldom necessary for aggregation but is always needed for reverse engineering attacks. Cheng Pau-chen Kevin Eykholt Zhongshu Gu Ray Valdez Hani Jamjoom Ashish Verma

21

Joint work with Aby Philip, Soorya Rethinasamy, and Vincent Russo now published in Quantum - the open journal for quantum science: https://lnkd.in/giwDBREV Title: Schrödinger as a Quantum Programmer: Estimating Entanglement via Steering Popular Summary: Entanglement is a unique feature of quantum mechanics, initially brought to light by Einstein, Podolsky, and Rosen. Determining whether a general quantum state is entangled or not, known as the separability problem, is of fundamental interest and relevant to various fields of physics: condensed matter, quantum gravity, quantum optics, and quantum information science. Motivated by the rapid development of quantum computers, we provide a variational quantum steering algorithm (VQSA) to address the separability problem. The quantum steering effect, discovered by Schrödinger, is central to the design of our algorithm, which is implementable on quantum computers that are available today. We provide benchmarks and simulations of our algorithm and establish a meaningful connection between steering, entanglement, quantum algorithms, and quantum computational complexity theory.

78

4 Comments

Thrilled to announce that our new open-source tool Patchsorter has officially been published in Nature Digital Medicine If you're on the hook for labeling large quantities of objects, give it a try- it won't disappoint! : ) Abstract The discovery of patterns associated with diagnosis, prognosis, and therapy response in digital pathology images often requires intractable labeling of large quantities of histological objects. Here we release an open-source labeling tool, PatchSorter, which integrates deep learning with an intuitive web interface. Using >100,000 objects, we demonstrate a >7x improvement in labels per second over unaided labeling, with minimal impact on labeling accuracy, thus enabling high-throughput labeling of large datasets. https://lnkd.in/eVxSvHHV https://lnkd.in/eqK7X4b5

135

6 Comments

Researchers from UC Berkeley and Anyscale have introduced RouteLLM, an open-source framework for cost-effective LLM routing. RouteLLM balances price and performance by intelligently directing queries to appropriate models based on their capabilities. Key features: • Uses preference data to train routers • Achieves 95% of GPT-4's performance while reducing calls by 74% • Offers up to 48% cost reduction compared to random routing • Outperforms some commercial routing systems by 40% in cost-effectiveness • Generalizes well to different model pairs without retraining RouteLLM demonstrates significant potential for optimizing LLM deployment in real-world applications, balancing quality and cost. #AI #MachineLearning #RouteLLM​​​​​​​​​​​​​​​​ https://lnkd.in/g5UCT7Kt

111

6 Comments