Sravan Bodapati

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

Ever wonder why some companies can ship software at lightning speeds while others seem stuck in the mud? 🚀⏳ . . . . . . . . . Daksh Gupta of Greptile collected real-world insights from engineers at many of the world's largest tech companies, unpacking the intricacies that either propel or impede the speed of shipping code. The big reveal? Older and larger companies tend to move more slowly because the costs of missteps become greater. In some cases, those costs are fairly non-negotiable (e.g. ensuring software for customers in production remains secure) while in other cases, the costs are more optional and cultural (e.g. wanting a lot of accurate estimates, forcing a manager's vs. maker's schedule, etc). #SoftwareEngineering #Tech #DevelopmentVelocity #Innovation #Productivity

1

The latest State of #AI report by #StanfordHAI showcases AI's remarkable progress. AI has surpassed human performance on some benchmarks like image classification and language understanding, but still lags behind on complex cognitive skills. Interestingly, new benchmarks like MMMU for general reasoning are emerging, with most recent tests showing close-to-vertical lines! However, it's essential to understand the limitations and potential pitfalls of such benchmarks. They lack broader context and are narrowly focused on evaluating a single capability. Rigorous and multi-faceted evaluations are essential for enterprise use cases, including meaningful and comprehensive benchmarks tied to real-world applications using business domain-specific datasets. https://lnkd.in/eUuK65u3

26

1 Comment

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

14

3 Comments

Have you wondered about the striking synergies between Large Language Models (LLMs) and Automated Machine Learning(Auto ML) Models? A few years ago, AutoML models made a lot of buzz in the AI domain. Even bigger buzz has been made by LLMs in recent times. There are striking synergies between LLMs and AutoML models. The performance of both LLMs and AutoML models improves with the quantity and quality of data. Both aim to automate aspects of machine learning. LLMs automate text-related tasks, while AutoML models automate the entire machine-learning pipeline. While LLMs are suitable for applications requiring deep language understanding and generation, AutoML models are ideal for users who need efficient and scalable ML solutions across different domains. As the use of LLM applications continues to grow, the next decade is poised to witness a significant shift towards the automation and productization of these models rather than their stand-alone deployment. This evolution in technology is a fascinating prospect, and if you share this interest, I invite you to stay tuned and follow along for more. #artificialintelligence #generativeai #LLMs #AutoML #leadership #futurisits #reviewwithsrujana

92

4 Comments

Amazon-to-Google-to-Amazon #a2g2a thought of the day: The Intensity, oh, the Intensity!!! When I went from Amazon to Google, it felt like I had just gotten off the freeway and was driving on a side street. At the time, I couldn’t quite disentangle whether the slowdown was because this was June of 2020 and covid had turned the entire world upside down or because it was a deeper part of the DNA of the company. Now that I’m back at Amazon, I feel like I was indeed driving on a side street for a while and now I’m accelerating onto the freeway onramp once again. There’s most definitely a difference in intensity between the cultures. Intensity by itself is neither good nor bad, but it depends on the context. As covid shut down the world, I appreciated Google’s emphasis on their employees well-being. In 2020, googlers received a generous 14 weeks of additional time off, fully paid, to be used any way we needed given the extraordinary world circumstances. I had two boys that we were now suddenly homeschooling, so I very much appreciated the benefit. The other side of that is that at times I wanted to step on the gas and it was quite hard. You can’t single-handedly do that, you need people around you to be as excited as you are and to follow you on the journey. Working backwards from firm dates is a bit of a foreign concept to Google and it’s table stakes at Amazon. "Intensity" is NOT necessarily the same thing as "working long hours" (although it is often confused). I have a family and my time with them is very important to me. It's about what you do with the hours that you do spend at work. The last few weeks, the intensity has been healthy. I’ve felt more alive. It’s energizing. I see the freeway up ahead and the adrenaline is beginning to kick in again!

306

28 Comments

Collaborative AI is gaining momentum at both MNC and Startups, they are investing into this EvolutionNext Robotics AI Transformative Experiential Learning Journey. I am extremely glad to put Singapore on the World map, by anchoring Robotics AI companies here, creating Jobs of the FutureNow as Roboticist. Do support my Leadership Initiative for Singapore as National Interest; help me enable our next-generation Singaporeans to take on Science, Technology, Engineering and Mathematics STEM Education and longer-term Career Practice into Industry 5.0 as Machine Intelligence of Human-Robot Collaborative, Human-centric Autonomy. * My STEM OneCareer Pinnacle and Finale as a 55-years old, always just another very ordinary Singapore-born Singaporean Scientist Engineer in Singapore, essentially a Nobody here 🌸

13

2 Comments

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

16

Understanding relationship between brands is helpful for many search applications (e.g., helping shopper discover brands similar to a brand they had in their search query). But it is incredibly hard to find labeled data capturing similarities between brands. In 2022, we had an interesting idea -- a graph neural network (GNN) can process how shoppers are interacting with multiple brands during the same search session, and learn the relationship graph connecting different brands. Happy to share the idea has been accepted as a patent now! Amazon Science

18

2 Comments

Quentin H. and Yanwei (Wayne) Zhang on the Faire Data team recently did some great work fine-tuning Llama3 model to enhance search relevance. Here are some key insights from the work: • While prompt engineering is valuable, LLM fine-tuning is essential for a domain specific task like understanding semantic search relevance. • Fine-tuned open-source LLMs like Llama3 can deliver impressive performance, scalability, and cost-efficiency compared to using proprietary models. • Effective application of a fine-tuned LLM demands a well-defined problem, robust, high-quality labeled data, and ongoing model refinement. Ground-truth labeled data is crucial for maximizing model performance. • Fine-tuned Llama3-8b achieves performance on par with the larger Llama2–13b model. Read the blog post to dive deeper into these findings! https://lnkd.in/ecP_3E7S We are also hiring (https://lnkd.in/dHfRZcPi) #LLM #AI #Finetuning #LLAMA3 #SearchRelevance #Hiring

88

3 Comments

Evaluating Large Language Model Applications As LLMs revolutionize applications like chatbots and content generation, ensuring their quality, reliability, and effectiveness is crucial yet complex. In this comprehensive guide on Medium, I explore: ✅ Automated evaluation techniques (rule-based and model-graded) ✅ Key evaluation criteria (context adherence, relevance, correctness, bias) ✅ Human evaluation factors ✅ Tailoring strategies for continuous improvement Whether you're building LLM apps, researching LLMs, or making decisions in this space, this article aims to provide valuable insights into implementing robust evaluation processes. Read the full guide on Medium: https://lnkd.in/g83kzxnY Chip Huyen CircleCI Rob Zuber Andrew Ng #llm #evaluation #ML

27

one of the challenges with RAG is evaluating it. check out this code from Amazon Science if you are building a RAG based system Romina Sharifpour, Ph.D. Baichuan Sun

23

5 Comments

In the last two years, we have seen the emergence of Large Language Models(LLMs), specifically in enterprises, there is a trend of either building or leveraging small open source language models to handle targeted use-cases with necessary optimization both in terms of system resource usage & accuracy. We are working on some exciting stuff here & our technical disclosure on ‘multi-stage fine-tuning process for optimizing small LLMs in RAG applications’ just got published! Happy reading!!! #SLMs #GenAI https://lnkd.in/gBvHZvHt  

18

6 Comments

Fantastic news today on the general availability of Amazon Q. Amazon Q is the most capable generative AI-powered assistant for accelerating software development and leveraging companies’ internal data. It eliminates tedious work for developers and employees across organizations. #AmazonQ #GenerativeAI

15

"Demystifying Real-Time Analytics, Search and Hybrid Search" I just released a legendary episode with the legendary Dhruba Borthakur, CTO at Rockset. Before Rockset he used to work at Meta, created #RocksDB and did major contributions to the #hadoop ecosystem before that. Dhruba explained so many #DistributedSystems concepts and more importantly how they are implemented in a real world system #RockSet. We went deep into Rockset's internals. Consistent Hashing, compute-compute separation, compute-storage separation, Data partitioning, Cloud native elasticity, Converged index, Avoiding write amplification are just a few examples. The episode is packed with a lot more. So give it a watch. https://lnkd.in/eGYmtEe8 I also want to make y'all aware about a conference called Index 2024, which is going to be packed with amazing tech talks. So do check it out and register for free. Conference link: https://lnkd.in/eNUadqft

42

Last November the news was on fire, with speculation that an OpenAI model called Q* was getting close to AGI. Today Adam Selipsky announced their new product Amazon Q, coincidence? I think not. This used to be a product called CodeWhisperer. But it looks like it's merged into a product suite under the name Q. The website for CodeWhisperer had an announcement listed: "Amazon CodeWhisperer is now Amazon Q Developer, which is generally available. All of the functionality that CodeWhisperer provides is part of Amazon Q Developer – in-line code suggestions, security scanning, open-source license tracking, and more. This website will remain live until mid May. For the latest information and to learn more about Amazon Q Developer, visit the website." Any how, it looks like the play is to connect to internal data sources and employees to make generative AI apps based by their own apps. They note in their FAQs "When you sign up for Amazon Q Developer Pro or Amazon Q Business plans, we don't use content to improve underlying models for others. Your data is your differentiator." But it doesn't look like there are separate instances of the model but rather a super-model (yeah I made that pun). So if you trust that your data won't get integrated than you should be fine. Just like the trainwreck you couldn't look away from on the news. 🙄 P.S. 👍 If you found these insights useful, please like this post to support more content like this! 🔁 Feel free to share with your network. P.P.S. Subscribe to my newsletter, The Artificially Intelligent Enterprise, for weekly insights into how to succeed with AI in Business - https://lnkd.in/eqJNArYx

211

17 Comments

Evaluating Large Language Models (LLMs) effectively remains a significant challenge in NLP. 🧠 A recent study sheds light on the common pitfalls and best practices in LLM evaluation. Here are the key takeaways: 1️⃣ Challenges in Evaluation: Evaluating LLMs involves dealing with many semantically equivalent but syntactically different ways of expressing the same idea. This complexity, termed the "Key Problem," makes it difficult to assess the correctness of natural language responses accurately. 2️⃣ Best Practices: Share Prompts and Code: For reproducible evaluation runs, sharing the exact prompts and code used is essential. Avoid Copying Results: Results should not be copied from other papers unless the exact same code has been used to run the experiments. Provide Model Outputs: Sharing model outputs alongside evaluation code allows for recalculating scores and performing statistical significance testing. 3️⃣ Automated Metrics: While human evaluations are costly and time-consuming, automated metrics like BLEU and ROUGE can offer reproducible and scalable solutions. However, these metrics also come with their own set of challenges. 4️⃣ Benchmark Design: Designing effective benchmarks requires considering the latest advances in LLM usage, such as in-context learning and chat interactions, to ensure validity and relevance. 5️⃣ Reproducibility and Transparency: The Language Model Evaluation Harness (lm-eval) aims to address these issues by providing a flexible evaluation library that enables researchers to run any benchmark on any model easily, promoting a more robust evaluation ecosystem. This study highlights the importance of rigorous evaluation practices to ensure fair comparisons and reliable progress tracking in the field of NLP. By addressing these challenges, we can move towards more accurate and meaningful evaluations of LLMs. Paper: https://lnkd.in/eCY4f-Ks #AI #NLP #LLMs #Evaluation #Reproducibility #Benchmarking

1

Had a great time at the Uber Gen AI Meetup in Bangalore! As I'm starting my Gen AI learning journey, it was amazing to see how experts are using AI to solve real problems. Following are some of the highlights from the event: - Jayaram Valliyur set the stage with their welcome and keynote on Uber's advancements in GenAI - Vivasvat Keswani introduced Uber's Michelangelo: Platform to build, deploy, and operate machine learning/ GenAI solutions at Uber’s scale - Sameera Poduri showed us how Uber is using Generative AI to make customer service better. - Varsha Mahadevan from Coinbase discussed the importance of keeping AI safe and secure in applications, focusing on prompt injection and implementing guardrails to prevent misuse. - Lokesh Agarwal from Kognitos discussed how AI could change our future - Pratyush Kumar from Sarvam AI emphasized the importance of using Indian languages in AI, making it more inclusive and also discussed building a full stack GenAI platform - Rush Tehrani from Uber focused on scaling ML infrastructure, deployment and serving of LLMs at Scale - Manyata Goyal and Geeti Oberoi introduced us to Uber’s tool, uLabel, which is Uber's labeling framework / data annotation solution that helps with organising data. - Ayush Garg, Chanakya Malireddy, Radhesh Sarma, and Georgy Mathew brought to light various advanced techniques like RAG-based chatbots, dynamic few-shot prompting, auto evaluation framework, showcasing Uber's leadership in applying AI to solve complex problems. - Smit Shah spoke on extracting actionable insights from LLMs, another testament to the practical uses of AI in business operations. - Ajeet Ganga explained how they're leveraging Gen AI to help manage Uber's workforce better - Panel discussion on "Unlocking the Value of Gen AI" moderated by Sameera Poduri featured Rohit Chatter from Walmart, @Vivasvat Keswani from Uber and Saurabh Dash from Cohere, along with Pratyush Kumar from Sarvam AI, offering diverse perspectives on the integration and impact of AI technologies across various sectors. Grateful to have been part of an event that showcased groundbreaking discussions on Gen AI. Big Shoutout to all the hosts at Uber for organising such a well-coordinated event. Looking forward to more such events! #UberGenAIMeetup #AI #Learning

68

2 Comments