G. Nicolas Guénon

What TikTok Teaches Us on Algorithm Transparency 📱🔍 There’s a reasonably obvious joke in data circles that ‘whatever developers don’t want to explain is called The Algorithm’. 🤔 And to a certain extent that’s correct - and entirely unacceptable. I’ve heard the name of The Algorithm taken in vain by lots of people - whether they’re talking about ‘not wanting to raise their kids on The Algorithm’, or ‘we must make content which does well in The Algorithm’. Hell, there’s entire industries out there predicated on having a better guess as to what happens inside the recommendation engines of big tech firms. 🏢 But what are we actually talking about? Can you imagine if you went into a business meeting having made a key decision, and the only thing you had to say as to your reasoning was ‘The Algorithm said so’. Obviously that’s madness and AI cult behaviour of the highest order - you have to understand the mechanisms behind your decision making process. 🧠 So what can we learn from TikTok? On the face of it, not a lot. However, I’ve been regularly posting on the platform for over a year now - and have some guesses as to what’s going on under the hood. (Also, my handle is @MLGonzo1). 📱 Firstly, one of the main predictors of success within the platform is categorization of your videos. Whenever you interact with a video, TikTok will try to recommend more videos for you to watch, and one of the main drivers of this is through a pre-filled search bar to discover other relevant content. High performing videos have two things in common: they fit into one of the search categories which TikTok has, and they are in a high performing category (I believe ‘cats of TikTok’ is one of the highest performing, obviously). If you fit into one of the categories that TikTok’s model has determined to exist, then that algorithm ‘knows’ (quotes intentional) what to do with your video. Conversely, if you’re like me and make obscure videos about obsolete audio formats - then the algorithm has no clue what you’re doing and so your videos don’t gain the same traction. 🐱 Because of this insight, we effectively have a feedback loop whereby users can create the sorts of videos that keep people on the platform. This transparency is actually A Good Thing. Imagine the converse situation where videos were promoted entirely at random. Creators would have absolutely no idea what did and didn’t work - and would either be driven mad or give up. 🎥 In your business, you don’t have all the proprietary problems which TikTok has regarding secrets and model theft (or at least internally I hope you don’t!), and so the more information your users have about the levers to pull to get either better insights or predictions from your AI the better. More education is always good. 📚 ❗️ Want more insights like this? Check out our free, Ultimate Guide to AI. Hit the link in the comments! 🔔 Want more of this content in your feed? Don’t forget to hit the bell on my profile.

18

3 Kommentare

I'm always surprised by how much CCNet was ahead of its time in terms of high-quality data extraction from webdata (especially CommonCrawl) to train language models. Even very recent open pipelines are basically minor modifications of it. I wrote an article about it and other pipelines here: https://lnkd.in/eisPXtAN.

17

You can ask the data questions AI: Hi there! I can answer any questions related to HSIC - Earnings call Q1 2024.Me: Is the bad news behind themAI:Searching... Is the bad news behind HSIC - Earnings call Q1 2024Based on the information from the Q1 2024 earnings call, it appears that HSIC is optimistic about moving past its recent challenges and is seeing positive developments:Execution of Strategic Plan: HSIC is making progress on executing its BOLD+1 strategic plan. The company has made acquisitions that contributed to profitability in the first quarter, indicating a strategic move towards growth and recovery.Financial Performance and Guidance: The company has affirmed its expectations for 2024 non-GAAP diluted EPS and adjusted EBITDA growth while tightening expectations for 2024 total sales growth. This suggests confidence in their financial outlook and operational stability.Recovery from Cyber Incident: HSIC's projected sales growth reflects continued recovery from last year's cyber incident. The management mentioned a strong pipeline of new specialty products and software innovation contributing to higher sales growth in the second half of the year, indicating that the impact of the cyber incident is being mitigated.Product and Service Growth: There is an expectation of modest overall equipment sales growth for the remainder of the year, with traditional equipment sales in line with last year and strong growth in Technical Services and CAD/CAM equipment. This diversification in product and service offerings could be a positive sign for future revenue streams.Impact of Acquisitions: Sales guidance includes sales from acquisitions completed to date, which are contributing to the company's profitability. This indicates that HSIC's strategy to grow through acquisitions is starting to bear fruit.In summary, while HSIC faced challenges such as a cyber incident affecting its operations, the management's commentary during the Q1 2024 earnings call suggests optimism about overcoming these hurdles. The company's strategic plan execution, financial performance guidance, recovery efforts from past incidents, product innovation, and successful integration of acquisitions indicate that HSIC believes the worst may be behind them. However, as with any investment decision, it's important to consider broader market conditions and perform due diligence beyond company-provided updates.

2

🚀 New Article Alert! 🚀 I'm thrilled to share my latest article, "Open File Formats: Parquet"! 📄✨ In this article, I dive deep into the world of Parquet, an open-source, columnar storage format designed for efficient data processing. Whether you're a data engineer, analyst, or simply interested in data storage solutions, this piece will provide valuable insights into: 🔍 The advantages of using Parquet 📊 How it optimizes data storage and retrieval ⚙️ Practical applications and use cases with Python and Apache Arrow 📈 Performance benefits for large-scale data analytics Check it out to understand how Parquet can enhance your data workflows and drive better performance in your projects. https://lnkd.in/d6TYNaAa #DataScience #DataEngineering #BigData #Parquet #OpenSource #DataStorage #TechInsights #DataAnalytics #ApacheArrow #Data #Technology

6

According to Anthropic's announcement, Claude 3.5 surpassed all the competitors' top-tier models. But does it actually perform better in an actual use case in our hands? Here is my quick experiment using LangChain's Agent framework. #llm #claude #gpt #genAI #langchain

13

1 Kommentar

🎓 💙 What do iris flowers, handwritten digits, boston housing prices and the titanic have in common? Only data scieentists know... 🤔 😉 For everyone who does not remember or know instantly: I'm talking about basically THE toy datasets that most of us have started their data science journey with. And I like that quite a lot. For example when I do interviews, I can use one of these and chances are high that my applicant has at least heard of them before. So today I'm celebrating these problems that have helped data scientists get into the field, try out new algorithms and methods, experiment with feature engineering techniques or use them as examples in talks, papers, etc.! What other datasets are your go-to examples? There for sure are more of them...alone with taking iconic kaggle datasets into account :D

14

The 3-Step Guide to Making Your Company AI-Forward 🚀: 1. Provide Access to ChatGPT Plus / Claude Pro($20/user/month) 2. Empower People to Experiment 💡 3. Encourage Sharing 📢 Sure, it sounds simple—maybe too simple—but many executives are overthinking it, bogged down by "CEO guides." Every day, people are discovering nuanced AI workflows that collectively save weeks of time across the company. This isn't a far-off dream; it’s happening now. Why wait until 2040? The AI tools you need are available today. And guess what? Early adopters are eager to experiment. The excitement around AI tools is still very much alive. Let your team experiment. Give them a platform to share their insights. It's that simple! Transform your business with AI today! 🚀 #ArtificialIntelligence #Startups #Technology #AIFuture #Innovation

5

1 Kommentar

Five months ago, I created the Reinforcement Learning Monitoring Feedback (RLMF) system for Aurélia AI, the AI that writes our prospecting emails. The idea is straightforward: in sales, you want to send the best possible email to your target audience. However, it’s challenging to know in advance what the best email is. For example, even if you spend hours searching the internet to craft the perfect personalized icebreaker for one specific person, the likelihood that your email ends up in the trash is still considerable (ofc, Aurélia does the job in your place, that's the point). This is where A/B testing comes into play. By testing a combination of hundreds of email templates, each personalized for the target, you can determine the most effective one. In data science, the system I created is Reinforcement Learning Monitoring Feedback, which uses ponderation to determine the importance of the template email for each persona. We select the best email for a particular persona and continuously improve the model. Over time, the AI will be able to send the most successful emails per persona. Today, I discovered a paper called "ORPO" that describes a similar approach. The ORPO algorithm emphasizes the critical role of Supervised Fine-Tuning (SFT) in preference alignment, highlighting that a minor penalty for disfavored generation styles is sufficient for achieving preference-aligned SFT (1): ORPO introduces a novel, reference model-free approach called odds ratio preference optimization (equivalent to our ponderation), which efficiently aligns models without the need for an additional preference alignment phase. This method penalizes undesired generation styles and provides strong adaptation signals for preferred responses. ORPO's empirical and theoretical results demonstrate its effectiveness in surpassing state-of-the-art models in various evaluations, proving its potential in combining Supervised Fine-Tuning with Reinforcement Learning techniques for optimal performance. Here the paper: https://lnkd.in/eaRhNSxN Here Aurélia AI: https://lnkd.in/exh9Y6S7 (1) More about SFT: https://lnkd.in/emF9JmHJ

2

Data Scientists, This is BIG! Snowflake's Arctic LLM is Here – and Open-Source! 🤯 Get your hands on it now: Interactive Demo: Explore Arctic's capabilities with this Streamlit app → https://lnkd.in/gJ7y38Ng Replicate: Ready to experiment? Find it on Replicate → https://lnkd.in/gyDXiMju Why This Matters: Arctic lets you quickly build demos of your models. This is a game-changer for showcasing your work and making your models more accessible! Huge thanks to Snowflake for making this happen! Are you excited to try Arctic? Let me know in the comments! 👇 #LLM #OpenSource #DataScience #Streamlit #Snowflake

8

From D Hwang [Google LLC] (2024) FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information https://lnkd.in/ecCv_Ccs - Adam optimizer can be considered as an approximation of natural gradient descent through the use of diagonal empirical Fisher information matrix (FIM). - The diagonal FIM results in loss of covariance information but retains variance. This approximation seems to work well in practice. - The second momentum term in Adam is actually the diagonal empirical FIM. However, Adam uses the square root of FIM instead of the direct inverse. This is to get a more stable estimate. - The FIM relies on empirical approximations using training data rather than model distribution. This works better for discrete distributions (like text) than continuous (like images). - Some flaws in original Adam algorithm are uncovered based on the analysis. Proposed fixes are: use invariant natural gradient for momentum, adjust bias corrections, clip gradients. - Weight decay should be applied as a natural gradient too based on FIM. This explains the success of AdamW which decouples weight decay. - Overall modified algorithm is named Fisher Adam (FAdam) which shows strong performance on text, speech and image tasks. abs: This paper establishes a mathematical foundation for the Adam optimizer, elucidating its connection to natural gradient descent through Riemannian and information geometry. We rigorously analyze the diagonal empirical Fisher information matrix (FIM) in Adam, clarifying all detailed approximations and advocating for the use of log probability functions as loss, which should be based on discrete distributions, due to the limitations of empirical FIM. Our analysis uncovers flaws in the original Adam algorithm, leading to proposed corrections such as enhanced momentum calculations, adjusted bias corrections, and gradient clipping. We refine the weight decay term based on our theoretical framework. Our modified algorithm, Fisher Adam (FAdam), demonstrates superior performance across diverse domains including LLM, ASR, and VQ-VAE, achieving state-of-the-art results in ASR. #AI #LLM

24

Oana O. pointed out in her short note that we already have too many GenAI tools, but most companies rely on AWS, consultants and ninja engineers. The main problem is the same as with ML, which leads to the fractioning and (mostly) failure of MLOps: Most problems worth solving are custom business-related problems that are hard to solve with one-size-fits-all tools. So tools are either too low-level for non-engineer DSes or too high-level for having a good TAM. The same will happen with GenAI. Solution: Either educate your DSes to be better engineers or figure out some unicorn alignment by mixing SWEs and DSes in the same team (practically unheard of, no idea why).

24

19 Kommentare

You don't need statistical theory to design A/B tests.

5

Hi there, Network! 👩‍💼 I am really enjoying LangChain at the moment, so today I bring you my latest article on LangChain v0.2 {Agents} and {Tools}! I want to help you build powerful assistants that use ChatGPT. Imagine having a model that can understand user inputs and call other functions, including your own models! Check it out here: https://lnkd.in/dvFgB9kv Let's make our models more accessible and useful! 🔗 #LangChain #ChatGPT #MachineLearning #AI #LLM #DataScience #RealEstate

49

7 Kommentare

AI: Hi there! I can answer any questions related to PLTR - Earnings call Q1 2024.Me: What are the future risksAI:Searching... What are the future risks for Palantir Technologies?The future risks for Palantir Technologies, as discussed in their Q1 2024 earnings call, can be categorized into several key areas:Societal and Cultural Risks: Alexander Karp, CEO of Palantir, highlighted the risk posed by a regressive way of thinking that he describes as corrupting and corroding institutions. This mindset, which he refers to as "woke" but critiques as a form of thin Pagan religion, is seen as a danger to society and by extension to Palantir. The concern is that if such ideologies are not openly discussed and addressed, they could negatively impact Palantir's operations and its alignment with customers.Competition and Market Positioning: While Karp expressed confidence in Palantir's unique product offerings and market position, stating that there is no real competition due to the specialized nature of their software infrastructure, this also implies an inherent risk. The company's success relies heavily on maintaining its technological edge and the unique value proposition of its products like Apollo foundry and ontology. Any failure to innovate or adapt could potentially erode its competitive advantage.Scaling Challenges: There are indications of challenges related to scaling up operations. The company acknowledges difficulties in expanding its workforce at a rate that matches the demand for its services. This limitation could hinder growth prospects if not effectively managed.Geopolitical Sensitivities: Palantir's outspoken stance on geopolitics and its defense of what it sees as Western values could pose risks in terms of public perception and relationships with international clients or governments with differing viewpoints. Such positions might lead to controversies or backlash affecting the company's global operations.Economic Headwinds in Europe: The company has identified economic stagnation in Europe, where it conducts 16% of its business, as a potential problem. With Europe gliding towards 0% GDP growth over the next couple of years, Palantir faces challenges in these markets that may affect its revenue streams from this region.In summary, while Palantir Technologies exhibits strong confidence in its product offerings and market position, it acknowledges several risks ranging from societal ideologies affecting corporate culture, challenges related to scaling operations efficiently, geopolitical sensitivities due to outspoken stances on certain issues, to economic headwinds in significant markets like Europe. Addressing these risks effectively will be crucial for sustaining growth and maintaining competitive advantage.

2

Hello there! Here is a little story about narrow-angle optics for today. We've been busy making our regular queries run fast on #ORC tables in #Vertica. And we achieved some good stuff (you can read it in the posts below). But the strike came out of the real-life queries that had something distinct from the regular ones. When the real user digs in to discover the data in your table, he starts with something like `select * from your_table where random_field = random_value`. In the case of a rare random value from an unsorted column, we end up reading all the data we have. What a bummer. It can be hundred billion rows to return like 10 records. As stupid as it sounds. A single query can significantly increase pressure on our S3 in one shot. So, a little laziness in column reading is what we do now. We just read the predicate columns first. As a result, we reduced the spikes by half by avoiding unnecessary reads. Here is a day before and a day after in the life of our object storage. Use metrics to keep your perspective wide. 

13

1 Kommentar