Micha Hershman’s Post

#MachineLearning directly in the database! With #ClickHouse you can train a ML model (stochasticLinearRegression or stochasticLogisticRegression) and then use it to perform predictions! The flow below also uses the s3 function to read the data from a bucket Check it out with Aiven for ClickHouse

Shout out for the tool of the month: ---- (Note: My tool of the month posts are not sponsored in any way. These are just recommendations from a happy user) Statsig is an experimentation and analytics platform that comes feature packed with all the tools required for A/B & multi-variant testing, feature flagging, dynamic configs and much more. what I liked about it was the ease of integration, clear documentation and the generous free-tier that let's you try the platform to make an informed decision before committing to it. https://www.statsig.com/

Clean data = Better outcomes. Versium sheets cleans your data directly in Google Sheets.

The final installment of the three-part series: How can StarTree's indexing help your day-to-day business? Part three focuses on understanding the impact of real-world customer use cases. Please take time to read through Sandeep Dabade's post as he wraps up his three-part series. #knowyourdata #apachepinot #realtimeanytics https://lnkd.in/ewpegWBW

Introducing veryLLM, an open-source toolkit, empowering developers and data scientists to build a universally needed transparency layer into #LLMs. Try it out and get the code here: https://prn.to/3ZnSimz

Giving kudos to your own workforce in a simple .txt file may seem futile. But it’s the small things that can really make a difference. Discover why you should create your own humans.txt in our newest blog. https://lnkd.in/efhBu8BA #HumansTxt #EmployeeRecognition #SmallThingsMatter #WebsiteCredits #CompanyCulture #StructuredData #WebData #dataprovider #DataproviderInsights #InternetBasics

🏗️ 𝗗𝗿𝘂𝗶𝗱 𝗗𝗲𝗽𝗿𝗲𝗰𝗮𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗖𝗹𝗶𝗰𝗸𝗛𝗼𝘂𝘀𝗲 𝗔𝗱𝗼𝗽𝘁𝗶𝗼𝗻 𝗮𝘁 𝗟𝘆𝗳𝘁 🏗️ At Lyft, we have used systems like ClickHouse and Apache Druid for near real-time and sub-second analytics. Sub-second query systems allow for near real-time data explorations and low latency, high throughput queries, which are particularly well-suited for handling time-series data. For our customers, this means faster analytics on near real-time data and decision making. This is crucial for use cases like market signaling and forecasting which benefit from, and depend upon, the most up-to-date information. Overall, these systems allow us to make timely decisions for the business with accurate predictions. In this particular blog post, we explain how Druid has been used at Lyft and what led us to adopt ClickHouse for our sub-second analytic system. https://lnkd.in/gCc7TYrA #DataEngineering #DataArchitecture #RealTime

Like many of you, we've faced the challenge of storing large text and querying it efficiently. After trying various solutions, we ended up using our current ClickHouse implementation. We’ve written a blog post detailing how we did it, focusing on: Optimizing Query Performance 🚀 Benchmarking Solutions 🔍 If you've been wrestling with massive texts, check the article we've written with Nathan Ostgard. https://lnkd.in/dYRvJe5C

Easily enrich leads and enhance lists with data from popular providers like @Apolloio for context and email verification from @hunter-io + more! Trust us, you're going to love enriching from any #data provider, at any scale! https://buff.ly/3K6Dxy5 #growthhacks #dataenrichment #salesops #spreadsheet

🚀 I'm a fan of RAG, but there is potentially untapped alpha in fine-tuning, especially in Life Sciences where proprietary data and nuanced terminology are common, the effectiveness of general LLMs can be limited. With Snowflake Cortex-finetuning, it took me less than 5 minutes to fine-tune the LLaMA-8B, creating a new model, Device_LLaMA_8B, to accurately answer questions based on proprietary sample data in a table. 🔍 The challenge? Analyze raw customer complaint texts from the devices sold by MediSnow (a fictitious company) and determine which Business Unit (BU) manufactured each device. 📉 LLaMA-8B by itself: Small, cheaper to run, but not very powerful. 💪 LLaMA-70B: More powerful, but lacks context. For instance, it has no clue which Business Unit manufactures the QuantumNova X5, a device MediSnow produces and sells. 💡 The solution: Fine-tune the smaller LLaMA-8B, making it both affordable and knowledgeable about MediSnow-specific data. 💬 Again, RAG is a good approach, but remember, you pay for tokens IN and OUT. So, in some cases, front-loading that cost by fine-tuning could be a better approach as opposed to building increasingly complex prompt pipelines with RAG. 🔧 Follow this quick tutorial step-by-step (can be done in under 15 minutes): https://lnkd.in/gcyuXJqQ