Siddharth T.’s Post

#Update 4: I am almost ready to give #Kimbho full insitu compiler and interpreter. Today, I have tested JSX, Python and R, adding few more languages and smoothening out few quirks. Once done, this will be extremely powerful for quick ideation, preview and then fat prototyping. In an ideal scenario, imagine you are a huge dev shop, and you give all your developers access to a tool like this, their ability to ideate, get to a prototype, pivot a design just goes 1000 fold up. From months and weeks you get to hours. Add collaboration tools to this, which is extremely easy and you entire pod can take components, build prototype, branch out harden and commit, all within minutes and hours, if not days. Productivity like never before ! Are you investing in these innovations ? let me know in comments below ! #GenerativeUI #AI #LLMs

#Kimbho Update 3: Today I gave Kimbho full "copilot" capabilities ( I call it “Ko!Pro”). Now it works with the user to understand the real intent behind an ask and what actions are supposed to be taken post answering. Observations: 99% accurate search and thesis from publoc web 94% accurate Retrieval Ko!Pro is result of fine-tuned GPT2 and T5 working together supported by GPT-4o for final intent generation only.

I was building my database of companies from EDGAR archives and found this Gem from #JeffBezos. Its from his 2018 annual shareholder’s letter. I didnt read anything like this in a long long time, spare a few minutes and do read it. I am so inspired but this, that I have decided to build an extractive abstractive summaries of shareholders letters. Thank you for this amazing writeup Jeff ! https://lnkd.in/gBmzQrD8

Today I gave #Kimbho, ability to generate #InfoCards for quick information capture with image reference (Redux and NextUI component). Building a product is addictive, once you get going, you just get going. Kimbho uses a generatorLLM and a validator tool to fetch, validate, ground and then generate labeled infocards dynamically. #Kimbho can currently semantically expand the query to make it specific. Next I am going to give it '#copilot' capabilities to better understand user intent to search, act and provide actionable insights. Lets use LLMs to accelerate action and decision making. PS: Card and Chat message is orchestrated by Claude. I find Claude models so good at following instructions. Great Job hashtag #Anthropic

Its very important to create right system architecture and give right tools to your LLMs. Once built the right way, with right foundational architecture, they are extremely powerful, but if not, they are just the next token predictors, full of hallucination and crippled by the tokenizer and alignment. Left side is Haiku with right tools, fucntions and checks vs right side is Opus using its own KB and attention architecture. Every answer on left are accurate as the Model has access to a scientific calculator, and Wolfram for formulae, whereas on the right Opus doesn't. Its also important considering the cost aspect of these models, you may use an expensive model for your usecase, but still continue to get wrong hallucinatory answers. Give your LLM “All it needs”, so that it can do “All what you need”, effectively. so, how are you architecting your complex real world applications ? How are you bringing the business need closer to the tech ? Leave a comment. PS: See how Opus fails to accurately calculate 3^73 and tries to burry the answer in bunch of unnecessary explanation.

Three most important contributions to tech industry by Meta are: 1. React 2. Flux ( leading to Redux) 3. PyTorch If you know basics of Javascript, you can just stitch magic !

Over Last 3 months, my personal self built Assistant #Kimbho has become so much better and productive. I started building it as I wanted to keep my information and searches personal and honestly I wanted to make sure I used an automated approach to have LLMs do more effective and accurate work for me. Today every month I spend ~$85 in expenses, and it makes me 10X of myself. Currently it has following ability: 1. 5 Domain fine-tuned self hosted models which do most of the work : $3 a month in electricity bill ( 3 llama3:8b, 1 llama3:70b and 1 Mixtral8x22b, finetuning cost was $540) 2. GPT-4o for Query understanding and classification: $2 a month in API cost 3. GPT-3.5 for function calline especially stocks, weather, Itinerary builder and Email summarization: ~$3 a month in cost 4. Claude-haiku for report and thesis generation: $4 on an average 5. Bing Websearch API: $15 a month 6. Polygon.io for Detailed realtime stock: $244 a year 7. Wolfram Alpha for financial modeling and complex Maths: $25 a month 8. Edgar curated dataset of 22000 companies ( self curated): $0 9. Full access to Arxiv for academic research and querying: (included in Bing Search) 10. Full access to semantically composed knowledge base I learned and created in last 12 13 years including 9 books, several research papers, rough notes etc ( close to 7 TB of data) 11. Has an evolving access to Octave for scientific computation, OpenCV/yolo for image segmentation My Assistant has following capabilities: - Search Web for real time data - Create extremely nice JS and NextUI based presentations for me to deliver presentations, build my thesis, etc instantly from multiple sources at once - Plan complete trip for me - Summarize my personal emails locally, create a summary of todos in airtable and push notifications about important things directly to my phone - Perform Video summarization or Video to dashboard from YT transcripts grabbing relevant important timelines - Perform proper company research and analysis - Perform complex modeling using Wolfram Alpha for maths and complex queries - Look at complete 10Ks and provide valuable thesis - Convert Earning transcripts into interactive dashboards for quick understanding - Leverage my own models for valuations, Market statistics etc - Has a completely composable archietcture to add infinite functions and tools. - All in all, there is no limit to what I can add and currently what I wan to ask. - Narrates stories to my Kids too 😊 This has made me extremely productive, smarter about things and efficient at completing things faster. I am continuously adding more features to it. Zero agents, no need for all the langchain, llamindex, langgraph etc, etc. Over 6800 lines of code, MaterialUI, NextUI, React, Redux for fast rendering, PGSQL for data and history (locally hosted and federated), Material UI, Cookie and JWT, NginNX, redis etc. There is nothing better than going hands on !

🚨 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐀𝐈 𝐌𝐢𝐬𝐮𝐬𝐞: 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬 𝐚𝐧𝐝 𝐓𝐚𝐱𝐨𝐧𝐨𝐦𝐲 🚨 A recent paper by Google DeepMind, Jigsaw, and Google.org delves into the misuse of Generative AI (GenAI). Here’s what you need to know: 🔍 𝐊𝐞𝐲 𝐅𝐢𝐧𝐝𝐢𝐧𝐠𝐬: 𝐌𝐚𝐧𝐢𝐩𝐮𝐥𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐇𝐮𝐦𝐚𝐧 𝐋𝐢𝐤𝐞𝐧𝐞𝐬𝐬: Most prevalent misuse tactic. Examples include AI-generated audio clips impersonating President Biden to suppress votes and fake social media accounts defending UAE’s presidency of a climate summit. 𝐄𝐱𝐩𝐥𝐨𝐢𝐭𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐄𝐚𝐬𝐢𝐥𝐲 𝐀𝐜𝐜𝐞𝐬𝐬𝐢𝐛𝐥𝐞 𝐂𝐚𝐩𝐚𝐛𝐢𝐥𝐢𝐭𝐢𝐞𝐬: Many incidents involve simple tactics requiring minimal technical expertise. 𝐄𝐦𝐞𝐫𝐠𝐢𝐧𝐠 𝐄𝐭𝐡𝐢𝐜𝐚𝐥 𝐂𝐨𝐧𝐜𝐞𝐫𝐧𝐬: New forms of misuse blur the lines between authenticity and deception, such as AI-generated intimate imagery without consent. 💡 𝐓𝐚𝐱𝐨𝐧𝐨𝐦𝐲 𝐨𝐟 𝐌𝐢𝐬𝐮𝐬𝐞 𝐓𝐚𝐜𝐭𝐢𝐜𝐬: 𝟏. 𝐈𝐦𝐩𝐞𝐫𝐬𝐨𝐧𝐚𝐭𝐢𝐨𝐧: AI robocalls impersonate President Biden to influence public opinion. 𝟐. 𝐀𝐩𝐩𝐫𝐨𝐩𝐫𝐢𝐚𝐭𝐞𝐝 𝐋𝐢𝐤𝐞𝐧𝐞𝐬𝐬: Photos of detained protesters altered to show them smiling. 𝟑. 𝐒𝐨𝐜𝐤𝐩𝐮𝐩𝐩𝐞𝐭𝐢𝐧𝐠: Fake social media accounts defending political stances. 𝟒. 𝐍𝐨𝐧-𝐜𝐨𝐧𝐬𝐞𝐧𝐬𝐮𝐚𝐥 𝐈𝐧𝐭𝐢𝐦𝐚𝐭𝐞 𝐈𝐦𝐚𝐠𝐞𝐫𝐲 (𝐍𝐂𝐈𝐈): Explicit images generated using a person’s likeness. 𝟓. 𝐂𝐡𝐢𝐥𝐝 𝐒𝐞𝐱𝐮𝐚𝐥 𝐀𝐛𝐮𝐬𝐞 𝐌𝐚𝐭𝐞𝐫𝐢𝐚𝐥 (𝐂𝐒𝐀𝐌): Deepfake CSAI found for sale. 𝟔. 𝐅𝐚𝐥𝐬𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧: AI-generated images falsely depicting events, like explosions at the Pentagon. 𝟕. 𝐈𝐧𝐭𝐞𝐥𝐥𝐞𝐜𝐭𝐮𝐚𝐥 𝐏𝐫𝐨𝐩𝐞𝐫𝐭𝐲 𝐈𝐧𝐟𝐫𝐢𝐧𝐠𝐞𝐦𝐞𝐧𝐭: Unauthorized AI-generated books appearing on Amazon. 𝟖. 𝐂𝐨𝐮𝐧𝐭𝐞𝐫𝐟𝐞𝐢𝐭: Fraudulent replicas of AI models like ChatGPT appearing online. 𝟗. 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 & 𝐀𝐦𝐩𝐥𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧: GPT-3 used to mass email state legislators. 𝟏𝟎. 𝐓𝐚𝐫𝐠𝐞𝐭𝐢𝐧𝐠 & 𝐏𝐞𝐫𝐬𝐨𝐧𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧: WormGPT crafting effective phishing emails. 📊 𝐅𝐢𝐧𝐝𝐢𝐧𝐠𝐬: 𝐏𝐫𝐞𝐯𝐚𝐥𝐞𝐧𝐜𝐞 𝐨𝐟 𝐌𝐢𝐬𝐮𝐬𝐞 𝐓𝐚𝐜𝐭𝐢𝐜𝐬: 𝐈𝐦𝐩𝐞𝐫𝐬𝐨𝐧𝐚𝐭𝐢𝐨𝐧: 25% of cases, using text-to-speech and video generation tools. 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 & 𝐀𝐦𝐩𝐥𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧: 13% of cases, creating large volumes of content. 𝐅𝐚𝐥𝐬𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧: 12% of cases, fabricating evidence or documents. 𝐆𝐨𝐚𝐥𝐬 𝐚𝐧𝐝 𝐒𝐭𝐫𝐚𝐭𝐞𝐠𝐢𝐞𝐬: 𝐎𝐩𝐢𝐧𝐢𝐨𝐧 𝐌𝐚𝐧𝐢𝐩𝐮𝐥𝐚𝐭𝐢𝐨𝐧: 27% of cases, including generating fake political endorsements. 𝐌𝐨𝐧𝐞𝐭𝐢𝐳𝐚𝐭𝐢𝐨𝐧 & 𝐏𝐫𝐨𝐟𝐢𝐭: 21% of cases, such as generating low-quality content for ad revenue. 𝐒𝐜𝐚𝐦 & 𝐅𝐫𝐚𝐮𝐝: 18% of cases, including impersonating trusted individuals to extort money. 🔗 Read the full paper for detailed insights and recommendations on addressing these challenges in AI governance and safety evaluations: https://lnkd.in/gG8ukG9q 𝐂𝐫𝐞𝐚𝐭𝐞𝐝 𝐛𝐲: 𝐊𝐢𝐦𝐛𝐡𝐨𝐀𝐈 𝐰𝐢𝐭𝐡 𝐡𝐮𝐦𝐚𝐧 𝐬𝐮𝐩𝐞𝐫𝐯𝐢𝐬𝐢𝐨𝐧 #Trust #aisafety #GenerativeAI

Happy #independenceDay everyone ! We have all come along a long way since the #July4th back in 1996, when Sabeer Bhatia and Jack smith launched #Hotmail, which changed the way we communicated. 18 months later #Microsoft acquired it for $400 million worth of stocks. #FunFact: Launch on Independence day was not the only amazing thing, they initially stylized it as “HoTMaiL”, to signify the might of “HTML” it used to make it 100% web based. Happy 4th of July🇺🇸 #july4th

𝗗𝗲𝘀𝗶𝗴𝗻 𝗦𝗲𝘃𝗲𝗻 𝗕𝗶𝗹𝗹𝗶𝗼𝗻 𝗣𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿 𝗠𝘂𝗹𝘁𝗶𝗺𝗼𝗱𝗮𝗹 𝗠𝗼𝗱𝗲𝗹: Handles both speech input and output. 🎙️   𝗕𝗶𝗱𝗶𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝗮𝗹 𝗜/𝗢: Provides continuous streaming of text tokens and audio codecs. 🔄   𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲: Achieves 160ms latency with a Real-Time Factor of 2. ⚡   𝗖𝗼𝗿𝗲 𝗠𝗼𝗱𝗲𝗹: The foundational text language model, Helium 7B, developed from scratch. 💡   𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗲𝗱 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴: Trained with both text and audio codecs simultaneously. 🧠   𝗔𝘂𝗱𝗶𝗼 𝗖𝗼𝗺𝗽𝗿𝗲𝘀𝘀𝗶𝗼𝗻: Uses Mimi, the proprietary audio compression model. 🎧   𝗖𝗼𝗺𝗽𝗿𝗲𝘀𝘀𝗶𝗼𝗻 𝗖𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆: Mimi’s VQ-VAE enables a 300x compression rate, incorporating both semantic and acoustic data. 📈   𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗧𝗧𝗦 𝗘𝗻𝗴𝗶𝗻𝗲: Capable of expressing 70 different emotions and styles, including whispering and accents. 🎭   𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 𝗜𝗻𝗶𝘁𝗶𝗮𝗹 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴: Utilizes 100K transcripts created by Helium, annotated with emotional and stylistic details. 📜   𝗘𝗻𝗵𝗮𝗻𝗰𝗲𝗱 𝗧𝗧𝗦 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴: Further refined with 20 hours of licensed audio from Alice. 🎙️   𝗤𝘂𝗶𝗰𝗸 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴: Requires less than 30 minutes of additional audio for customization. ⏱️   𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗠𝗲𝗮𝘀𝘂𝗿𝗲𝘀: Generated audio includes watermarks and is cataloged in a database. 🔒   𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗦𝗲𝘁𝘂𝗽: Conducted on a Scaleway cluster with 1000 H100 GPUs. 🖥️   𝗠𝗼𝗱𝗲𝗹 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 𝗗𝗲𝗺𝗼 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁: Capable of batch size 2 at 24GB VRAM, hosted on Scaleway and Hugging Face. 🚀   𝗤𝘂𝗮𝗻𝘁𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗢𝗽𝘁𝗶𝗼𝗻𝘀: Supports both 4-bit and 8-bit quantization. 🔢   𝗖𝗿𝗼𝘀𝘀-𝗣𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗖𝗼𝗺𝗽𝗮𝘁𝗶𝗯𝗶𝗹𝗶𝘁𝘆: Functions across CUDA, Metal, and CPU environments. 🛠️   𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Inference code optimized with Rust for efficiency. 🏎️   𝗙𝘂𝘁𝘂𝗿𝗲 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻𝘀: Potential improvements with better KV caching and prompt caching. 🔄   𝗙𝘂𝘁𝘂𝗿𝗲 𝗗𝗶𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀 𝗨𝗽𝗰𝗼𝗺𝗶𝗻𝗴 𝗥𝗲𝗽𝗼𝗿𝘁𝘀 𝗮𝗻𝗱 𝗥𝗲𝗹𝗲𝗮𝘀𝗲𝘀: Plans to release a technical report and open models. 📄   𝗢𝗽𝗲𝗻 𝗔𝗰𝗰𝗲𝘀𝘀: Will include the inference codebase, 7B model, audio codec, and optimized stack. 🌐   𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗜𝗺𝗽𝗿𝗼𝘃𝗲𝗺𝗲𝗻𝘁: Ongoing scaling and refinement based on feedback, targeting versions Moshi 1.1, 1.2, and 2.0. 📈   𝗙𝗹𝗲𝘅𝗶𝗯𝗹𝗲 𝗟𝗶𝗰𝗲𝗻𝘀𝗶𝗻𝗴: Aiming for the most permissive licensing possible. 📝   𝗧𝗲𝗮𝗺 A dedicated team of eight has brought this project to life. Experiencing the system’s rapid response feels almost magical, opening new possibilities in research, brainstorming, language learning, and more, all while being fully on-device. 🌟   Kudos to Kyutai and the team for delivering a seamless and publicly available version! 🎉   𝙽𝚘𝚝𝚎: 𝙲𝚛𝚎𝚊𝚝𝚎𝚍 𝚋𝚢 𝙺𝚒𝚖𝚋𝚑𝚘��𝙸 𝚠𝚒𝚝𝚑 𝚑𝚞𝚖𝚊𝚗 𝚜𝚞𝚙𝚎𝚛𝚟𝚒𝚜𝚒𝚘𝚗 Try: https://lnkd.in/gtkFYHBW