📣 It's been an incredible 4 months since starting at Snorkel AI. Surfacing from getting up to speed / firehose of AI infrastructure, I'm in absolute AWE of the practical foresight the team consistently demonstrates 😍 Alexander Ratner recent post is another example. If you are interested in learning how enterprises will drive long term value from AI, here's a succinct summary for how we make foundational LLMs work in accordance with enterprise policies, regulatory conditions and brand ... absolutely nails 🔥 '...what matters most now, as the algorithms and infrastructure for doing fine-tuning has already standardized and commoditized? The LABELED DATA!" #genai #LLMs #enterpriseAI
Some thoughts on Alignment: - It's just fine tuning (basically)! - It's all about the labeled data - Some recent work Snorkel AI on what we call "enterprise alignment" + a mini rant on what we need to focus on to actually make AI work :) There have been a plethora of terms to describe some form of tuning an LLM with respect to a labeled dataset: - "Fine tuning": Tuning on labeled training data - "Instruction tuning": Tuning on prompt, response pairs. - "Alignment": Tuning on some form of preference data (e.g. LLM A vs. B, yes/no, written feedback, etc.) Conceptually though: it's just tuning LLMs for a labeled dataset either way! For the above reasons- Snorkel AI we just prefer to call the above all "fine-tuning" for simplicity; and, believe that this (+ prompting) will all eventually converge and become lower-level details of LLM adaptation. What will matter then? And what matters most now, as the algorithms and infrastructure for doing fine-tuning has already standardized and commoditized? The labeled data! This is evinced by BigTech and LLM providers spending $ Bs on labeled data for fine-tuning and alignment, and organizations like OAI beginning to research more scalable approaches for future superalignment objectives, like weak-to-strong generalization (which we started working on 8+ years ago, https://lnkd.in/gnBqS422). The problem for enterprises: the use cases, objectives, policies, regulations, tone of voice, etc. that they need their AI models to align with are *not* the same as the generic ones these LLM providers are tuning/aligning to! We call the challenge of aligning to enterprises' specific objectives/policies enterprise alignment. It's difficult especially because private data & internal subject matter experts are usually required to do the data labeling (i.e. can't just be outsourced, even if there were $ Bs to spend!) Snorkel AI we've show that our programmatic approach to data labeling & development can solve this challenge, leading to custom alignment in a few hours of development- for ex., aligning a finance chatbot to be compliant with financial advisory policies (+20.7 pts. above baseline) in several hours. Check out our work here: https://lnkd.in/gW5hbqJF As for the "mini rant" - mostly bait to read a long post on a niche technical topic :). However, many of us in AI have a strong view that: - (A) Superhuman AGIs are not right around the corner. - (B) Enterprises are struggling to deploy AI to impactful production settings- and the party's over for all of us (and all the positive impact that we *know* AI can have) if we don't deliver value ASAP. - (C) Safe systems at scale are usually built through years/decades of incremental and collective engineering (e.g. think air travel), vs. secretive leaps forward. That is: let's all learn to walk safely first- then we'll get to flying saucer seatbelts!
Data Scientist | Freelance Writer for Data, AI, B2B & SaaS | Content in v7, Encord, y42, Wisecube | Blogs | Whitepapers | Developer Advocate | Technical Writer | Content Marketer 💪
1moAbsolutely right Cate Lochead Labeled Data is the answer!