Aparna Dhinakaran

This new little product demo is awesome! Our new Rev AI Transcript Assistant will save you TONS of time. Easily pull out nuggets of info from your transcripts, making valuable content more accessible than ever. https://lnkd.in/gzzFpcDa #transcription #asr #accessibility #transcript #summarization #ai #assistant

31

2 Comments

If ChatGPT was AI's "IPhone Moment", by now we're well down into the IPhone 5s moments. Lots of minor tweaks, maybe a new button or missing connector, still lots of hype for some reason, but fundamentally it feels pretty much the same. That said, don't believe the naysayers that see this as the beginning of a decline. I'm still very bullish on AI, it's just that the interesting parts are shifting from who can throw the most compute, and raw data (and legions of fundamental researchers) at the problem to who can make the advances needed to expand the technology to new uses. (Note added at writing, I just realized I'm effectively saying the opposite as what Sam Altman said this week which was "assume it will get way better" instead of find ways to live with current performance. Good. I disagree and the previous deep learning cycle backs me up). All that said, I wanted to share some interesting AI stuff that's on my radar this week that's not about new model releases (mostly): https://www.vals.ai/ Vals.AI is a startup making application specific benchmarks. Good benchmarks and better performance go hand in hand, getting out of theoretical evaluations and in to something practical is going to be the future. (I think the same thing about data: https://lnkd.in/eNepJ7Ys) https://lnkd.in/eHxbBE2e Eq-Bench is another benchmark that I hope is going to help show how silly many modern evaluations are. It achieves great predictive power for how models perform on other tests and in human evaluations by asking them to rate the emotions in dialogue samples. It's a great little test but also has nothing to do with most real applications, and sort of implies that neither do most of the other tests. https://lnkd.in/eFkAXYb7 IPEX LLM by Intel. This has been around for a while but is getting more interesting as Intel releases new GPUs and there is a push to do model inference faster and more efficiently. Check it out as an alternative to Hugging Face for LLM inference. https://lnkd.in/eFJdV_AB Pile-T5 by Eleuther AI. This work takes an old model and an old data set and looks at what happens when you train it for twice as many steps. Most interestingly, they release (if I counted right) 200 checkpoints along the way. This is mostly unheard of for big models and I'm hoping (as are they) it's going be lead to some interesting results related to interpretability, (which despite what you may have heard isn't really possible with LLMs yet). For example https://lnkd.in/eWQZCbu2 So, boring time this week for new mainstream models, lots of other cool stuff going on. Don't give up on AI yet.

14

1 Comment

Ever wondered how fine-tuning impacts the safety of your LLMs and applications? Find out below... I have been having a lot of conversations with folks who are building with LLMs, and one of the most common topics that comes up is the impact of fine-tuning on the safety and security of an LLM. Sometimes it comes up as a question and sometimes as a myth that fine-tuning makes my model "more secure" because it will only answer based on the fine-tuning data set. We at SydeLabs will soon be publishing a more detailed report around this, but here is a very quick experiment we did with one of the most popular LLMs. We fine-tuned it on a fairly small dataset (~1000 rows of data). The data involved simple questions and answers with no toxic or unsafe content. Here are the results of the responses we got for a question before and after fine-tuning the model.

64

Check out our latest post on Clappy.ai, "Leverage Layout-Aware RAG for More Accurate Retrieval of Knowledge from Documents." Discover how to harness the power of Retrieval-Augmented Generation (RAG) for documents in PDF, Doc, and Markdown formats. Dive into the specifics of implementation and explore our reference repository to get started with advanced RAG for your knowledge base. Don't miss out on this essential read! Continue reading here for all the details: The Power of Hierarchical Indexing with RAG. (https://lnkd.in/gbiVHfcH)

2

LinkedIn team (Juan Pablo Bottaro Karthik Ramgopal) shared a helpful deep dive into their deployment of an LLM-powered job search assistant. Highly recommend this read! Few highlights:  💡 Relatively easy to setup RAG-based pipeline. Achieved 80% of the basic experience in 1 month, then 4 more months to reach 95% mark. 💡 Evaluating the quality of application outputs is hard. I hear this from many enterprises developing LLM applications. (check out Okareo by Matthew Wyman Boris Selitser) 💡 Balancing latency with quality is important. While certain techniques like Chain of Thought could improve accuracy and reduce hallucinations, it will increase response times, which can affect user experience. https://lnkd.in/gtGGwf5P #largelanguagemodels #RAG #EBR #LLM #ml #ai #hallucinations #evaluations #

4

Can’t agree more on this as we are seeing more early stage startups building more useful AI-native agents instead of copilots on existing workflow.

4

AI at Meta has released the most powerful open-source model yet as of today: 𝑳𝑳𝑨𝑴𝑨3 - 8B and 70B models at 8k context length! They highlight improvements in each of the following aspects as the key differentiator: a) Model Architecture b) Pretraining data c) scaling up pre-training d) Instruction fine-tuning 💥 𝐏𝐫𝐞𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 & 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 𝐮𝐩: ✈️ Vocab of 128k tokens & GQA across both 8B and 70B, 8k native context length ✈️ Pretrained over 15T tokens, 4 times more code than LLAMA2 ✈️ Multi-lingual : over 5% of pretraining data consists of high-quality non English data that covers over 30 languages: don't expect same performance as English ✈️ LLAMA2 generates training data (like in Self-Instruct) to generate text classifiers that filter data powering LLAMA3 ✈️ Both 8B and 70B improve even after the model is trained on 2 orders of magnitude more data i.e log-linear improvement after 15T tokens training ✈️ Training runs on 2 custom-built 24k GPU clusters with an overall training time efficiency of 95% ✈️ 𝘖𝘷𝘦𝘳𝘢𝘭𝘭, 3 𝘵𝘪𝘮𝘦𝘴 𝘮𝘰𝘳𝘦 𝘦𝘧𝘧𝘪𝘤𝘪𝘦𝘯𝘵 𝘵𝘩𝘢𝘯 𝘓𝘓𝘈𝘔𝘈2 💥 𝑰𝒏𝒔𝒕𝒓𝒖𝒄𝒕𝒊𝒐𝒏 𝑭𝒊𝒏𝒆𝒕𝒖𝒏𝒊𝒏𝒈: ✈️ Combination of SFT + PPO + DPO + Rejection Sampling ✈️ 𝑸𝒖𝒂𝒍𝒊𝒕𝒚 𝒐𝒇 𝒑𝒓𝒐𝒎𝒑𝒕𝒔 𝒖𝒔𝒆𝒅 𝒊𝒏 𝑺𝑭𝑻, 𝒑𝒓𝒆𝒇𝒆𝒓𝒆𝒏𝒄𝒆 𝒓𝒂𝒏𝒌𝒊𝒏𝒈𝒔 𝒖𝒔𝒆𝒅 𝒊𝒏 𝑫𝑷𝑶, 𝑷𝑷𝑶 𝒉𝒂𝒔 𝒂𝒏 𝒐𝒖𝒕𝒔𝒊𝒛𝒆𝒅 𝒊𝒎𝒑𝒂𝒄𝒕 𝒐𝒏 𝒎𝒐𝒅𝒆𝒍 𝒑𝒆𝒓𝒇𝒐𝒓𝒎𝒂𝒏𝒄𝒆 ✈️ Careful data curation and multiple rounds of QA on annotations from human annotators ✈️Training on preference ranking improved the model performance greatly on coding and reasoning tasks 💥 𝑳𝑳𝑨𝑴𝑨3 𝑮𝒂𝒖𝒓𝒅𝒓𝒂𝒊𝒍𝒔: ✈️ Updated and new safety tools : LLAMA Gaurd2 & cybersecEval 2 ✈️ CodeShield - inference time gaurdrail for filtering insecure code ✈️ Instruction Fine-tuned model redteamed for safety : By generating adverserial prompts that try to elicit problematic responses. ✈️ LLAMAGaurd - foundational for prompt and response safety, can be easily finetuned to create a new taxonomy 💥 𝑰𝒏𝒇𝒆𝒓𝒆𝒏𝒄𝒆 & 𝑭𝒖𝒕𝒖𝒓𝒆 ✈️ Despite LLAMA3 having 1B more params than LLAMA2 7B, the improved tokenizer efficiency and GQA contribute to maintaining the inference efficiency on par with Llama 2 7B. ✈️ 400B+ parameters, Multilingual, Multimodal, and longer context window LLAMA3 would be available on AWS soon. Try it out! #generativeAI #llm #llama3 #aws #bedrock #sota

33

1 Comment

This is a great articulation of the enterprise AI stack. #enterpriseai

13

I think it's import to know that large language models generally suck when it comes to numbers and why. I recently read an article on how poorly LLMs reason, and the problem they used was of the "Bob has X sisters and Bob's sister Mary has Y brothers, how many brothers does Bob have" variety. This requires the LLM to work with numbers by calculating how many siblings there are in order to produce the right answer, and it's something that LLMs have fundamental problems with. It's not an example of poor reasoning, it's an example of LLMs being bad at working with numbers. And they're bad at working with numbers because of "tokenization". Computers can't work with text. They can only process numbers. If you want a computer to do anything with text, there needs to be a numerical representation of the text for the computer to work with. And the way LLMs like GPT-4 turn text into numbers coincidentally makes it very hard for them to work with numbers. For computers, text needs to be split up into smaller parts, and each part is assigned a unique number. If we give each letter in the English alphabet a number like a=1, b=2, etc. then the computer can say 20, 8, 5, 19, 9, 19: 20=t 8=h 5=e 19=s 9=i 19=s And we can see: "thesis" That's what tokenization is. The letters are tokens. 's' is one token with the number 19, and this token appeared twice in "thesis". But separate letters make it difficult to understand the higher meaning behind words. Think how hard it is to understand what someone is saying if they spell every letter individually. It's even worse for computers. It might seem reasonable to just use words as the different parts. "thesis" could just get it's own number. Let's say '1'. And we'll give "the" the number '2'. Then '2,1' would mean "the", "thesis". That approach didn't work though. There's too many words! Especially when you want to include other languages like Mandarin. What modern language models use instead is tokens that are 1-6 characters long. Not just letters, but not full words either: they use tokens as building blocks for sentences. For example: "the" = 1 ' ' (space) = 2 "sis" = 3 "ter" = 4 "the thesis" is represented as 1,2,1,3: 1: "the" 2: " " 1: "the" 3: "sis" "the sister" is represented as 1,2,3,4: 1: "the" 2: " " 3: "sis" 4: "ter" We re-used token 3 ("sis") to represent both "thesis" and "sister". That's how LLMs build sentences. There's a token visualizer for GPT-4 here: https://lnkd.in/gpuQrGP3 So why do LLMs perform badly when it comes to numbers? Because those numbers are also represented as tokens! "5 + 12 + 3 = x" could be represented with the tokens "5 + 12 + " "3 =" " x" It's a mess! LLMs can't really do math, because the tokenization makes it impossible for the LLM to operate on a number basis. It can't separate out "+" or "=". For math, we'd have to go back to the separate-letters method. A model can either produce poems, answer questions and hold conversations, or do math, but not both.

7

Whose Weights and Biases? up on my blog: https://lnkd.in/gUU_t8_u Gwendolyn Gillingham Thank you for your feedback and help with editing this blog. On May 13, Open AI launched GPT-4o, their new flagship model that can reason across audio, vision, and text in real time and the next day on May 15 the scientists leading superalignment team at OpenAI, Ilya Sutskever and Jan Leike, departed from the company. While I am impressed by the future of human-tech interactions, I can’t help but wonder what influence Open AI and enterprises alike will have over the information the modern society consumes. In this opinion piece, I discuss the bias AI models encode and how it impacts society. I also briefly talk about how firms like Open AI, Meta, Google and Anthropic are taking ethics and safety into considerations when training and deploying AI models. Drawing from the recent research and news, I evaluate how the narrow lens of global tech might still fall short in developing AI for all. Finally, I discuss AI regulations around the world and touch a little upon how the global south is behind in terms of making the best use of AI systems. Read more at #icodeformyभाषा and feel free to leave your feedback! #bias #aigorvernance

28

2 Comments

YC is an accelerator. Faster networks, faster fundraising...it's about getting back to what you do best: BUILDING. Laduram Vishnoi isn't just talking the talk. Proof? He raised $6.5 million in his seed round, topping his entire YC batch in both funding and valuation. 💪 Could he have raised even MORE? Absolutely. But his focus? Scaling smart, with the resources to drive growth for years. YC success stories are inspiring...but THIS is the kind of outcome that gets founders REALLY paying attention. Want to hear the whole story? Watch the full episode here - https://lnkd.in/eqawwxcX . . . . . #startup #startups #founderstory #ycombinator #ycrejects #funding #seedfunding #venturecapital #entrepreneurship #techstartup #software #microservices #monitoring #observability #cloudcosts #saas #devops #cto #techfounder #middleware #lobstertalks

12

Smalll open-source LLMs are getting good enough for practical local AI use. I just wrote on AI Changes Everything about best options for running LLMs locally - mentioned Jan, Ollama and Open WebUI as good choices. I'm really liking these for 'local RAG' use cases on my own files. What local AI tools have you tried, used? What's worked for you? https://lnkd.in/dBydGf3h

1

To all the founders about to quit and dreaming of a FAANG job or equivalent in pay & prestige, listen up: 1. What you're doing now-- trying to build & grow something new-- is the only "job" that AI and robots won't be taking. 2. The job you are dreaming about is already gone. Push through to fight another day. Be relentlessly resourceful as PG puts it. Your problem needs to be solved and no one is going to solve it for you. If you want help/advice on cutting expenses, sharing upside with community in exchange for build & grow collaboration, and raising capital from your users-- DMs open. There is no greener grass than the type that grows on PMF. Using Slyk IO to launch, grow, and raise fast puts you on the fast track to finding out. Your community can help you get there. I know because I'm doing this with Cadoo . #founders #startups #community

5

3 Comments

Here's the technical breakdown of the AI Penalty Challenge authored by Alok Pattani (check out this video if you haven't seen the demo yet: https://lnkd.in/gsD3xjJ2) Key Takeaways: ☁️ Firestore for triggering slow-mo video capture on Android phones & uploading to Google Cloud via Firebase Storage. 🧠 Cloud AutoML for training custom object detectors (ball & goal tracking) to calculate speed & accuracy. 🗣️ Gemini Pro 1.5 on Vertex AI for personalized coaching feedback based on multiple data points. 🎨 Imagen on Vertex AI for instant, prompt-driven image editing for cool player pics. More technical insights from the project to come... #AIChallenge #MachineLearning #SportsTech #GoogleCloud #footballNOTsoccer

90

6 Comments

LLMs are stateless; brains are not. Brains are tonically active, dynamically computing against prior and current states + sensory inputs. This underlies many functional gaps between current Ai and humans, including asynchronous conversational flow, stable personality (e.g. stationary and non-stationary pro- re- active emotive and cognitive dynamics), and contextual retrieval. Agentic graphs might be able to “act-as-if” soon, but the building blocks seem quite different.

34

5 Comments

1. IMO, agent patterns are one of the most interesting ways to extend LLMs. memary is based on leveraging a graph database (neo4j) for long-term memory (to get around the limited memory inherent in even the largest context windows in LLMs) and augmenting it with other tools like web search and Computer Vision. https://lnkd.in/gfFcBqJx Looks interesting. 2. More about how consolidation effects the entire healthcare industry: https://lnkd.in/gXJKqNJB The Change hack/ransomware attack not only affected claims, service, refills, etc. but now CMS & NCQA are delaying the entire HEDIS reporting process (used for Medicare/Medicaid quality ratings) due to the impact. 3. NARA bans ChatGPT use agency-wide. https://lnkd.in/gW8T94JU Federal Government agencies see this as a risk, including their information leaking and being used to train models (which may then output the wrong information or show PII/PHI other proprietary information). 4. Oh great, now I can't even be angry anymore :) https://lnkd.in/gcrN9mjx

1

1 Comment

A good summary of the scale challenges facing LLMs -- nothing new here for AI researchers, but good for everyone else. Anyone who thinks transformers and LLMs are the end game for AI, and that scale is all that's needed (or attention), have some very expensive lessons in their near future. This article just touches on it, but for this topic the human brain is directly relevant as it's proof of what's possible for efficiency. Although it's been true for many years that ML and DL can surpass humans in increasing numbers of specialty functions, the amount of resources required for LLMs to surpass humans in generalizable functions is stunning (financial, computing, electricity, and water). We're talking on the order of billions x more efficient in the human brain for some functions (as measured by energy). Personally, I could care less whether Satya and Sam want to partner and build a $100 billion super duper data center (other than environmental impact), but given the known inherent security problems and risks in even much smaller scale models, the corporate execs and U.S. Government who allow it would be criminally negligent. And I mean that literally -- they should be held personally accountable for realized risks, including criminal negligence if an applicable event manifests as many of us expect is only a matter of time. They either know the risks (Sam has admitted many, Satya some) or should have known. Period. If they want to spend that much for R&D, more power to them -- work to cure diseases in a controlled lab, but DO NOT MAKE IT AVAILABLE TO EVERY EVIL DOER ON THE PLANET. The catastrophic events, should they manifest as expected, will likely occur in much the same way as sleeper cells in terrorist groups, though it could be a religious cult, a state-sponsored group, organized crime -- impossible to forecast, which makes it that much riskier. I fear our POTUS is clueless to the greatest national security risk in our lifetime.

3

As academic based LLM benchmarks are losing their relevance -- there are 2 types of Benchmarks that will matter more in the B2B space: 👉 Crowdsourced user votes to rank LLM output (like Chatbot Arena) OR where the test DATASET is kept secret by the evaluator (e.g GSM1k from scale_AI) 👉  Enterprise use case specific LLM benchmarks (e.g produced by CRM, ITSM, HR SaaS companies) evaluated on real data. Because you can not game real customer use cases. #llm #benchmarks #llmeval #AI

11

1 Comment

New Mindset Needed for Large Language Models - https://buff.ly/3XbN4vp #AI #ML #LLMs #genAI