Hans Beck

Inexpensive token generation and agentic workflows introduce fresh prospects for training Large Language Models (LLMs) with synthetic data, potentially elevating output quality. Direct pretraining of LLMs on their generated responses proves ineffective, but employing agentic workflows could yield superior results, akin to humans learning from their own thinking processes. LLM training typically comprises pretraining on vast amounts of unlabeled text data, followed by fine-tuning and alignment to human values. However, sourcing adequate training data remains a hurdle, prompting exploration of innovative methods like training smaller models on the output of larger ones. Despite the substantial cost of token generation, agentic workflows present promising avenues for creating high-quality synthetic data, ultimately enhancing LLM performance.

11

AnchorGT: A Novel Attention Architecture for Graph Transformers as a Flexible Building Block to Improve the Scalability of a Wide Range of Graph Transformer Models Transformers have taken the machine learning world by storm with their powerful self-attention mechanism, achieving state-of-the-art results in areas like natural language processing and computer vision. However, when it came to graph data, which is... https://lnkd.in/eedeAnTa #AI #ML #Automation

2

Use Case: Design of predictive soft sensors Read all about how we optimised operations and quality control effectively, without interrupting the normal plant exploitation. https://lnkd.in/eDjrhGjn #DataStories #Datascience #usecase

7

A new paper with a LoRA-like method for efficient LLM finetuning is out. In "MoRA: High-Rank Updating for Parameter-Efficient Finetuning (https://lnkd.in/guhdrKYd)," the authors take a related yet opposite approach to low-rank adaptation. Why another LoRA alternative, now with "high" ranks? 

1

AEs (Variational Autoencoders) and AEs (Autoencoders) are neural network architectures used for dimensionality reduction and data compression. They share a similar structure with an encoder and a decoder, but they differ in how they handle the latent space, which is a compressed representation of the input data. Here is a good article explaining the difference between AEs and VAEs. #AE #VAE #neuralnetworks #deeplearning #computervision I like to share my Data Science journey with my LinkedIn friends… I believe learning everyday is part of life. Follow me if you like my posts. keep reading. keep learning.

9

💎 𝐊𝐀𝐍: 𝐊𝐨𝐥𝐦𝐨𝐠𝐨𝐫𝐨𝐯-𝐀𝐫𝐧𝐨𝐥𝐝 𝐍𝐞𝐭𝐰𝐨𝐫𝐤𝐬 💡 This work proposes Kolmogorov-Arnold Networks (#KANs) as promising alternatives to Multi-Layer Perceptrons (#MLPs). ✅ While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). ✅ KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. 🚀 It is showed that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. 🚀 KANs are promising alternatives for MLPs, opening opportunities for further improving today's deep learning models which rely heavily on MLPs. 👉 Paper https://lnkd.in/d9Sq7ErF ♻️𝒌𝒊𝒏𝒅𝒍𝒚 𝒓𝒆𝒑𝒐𝒔𝒕♻️ #ai #deeplearning #neuralnetworks

23

Using Normalization Layers to Improve Deep Learning Models https://zurl.co/LW2h

50

Here's a complete end-to-end demo of what Dr. James McCaffrey of Microsoft Research says is arguably the simplest possible classification technique. https://lnkd.in/esNJtpab #DataScience #CSharp #MachineLearning #Centroid #Classification #Developers #DevTips #TechTips

1

Demystifying Explainable AI with TCAV: A Deep Dive into Concept Activation Vectors #ml #ai

1

New repository for applying Constrained Deep Learning! Check out this blog post to learn how to design constrained models, and how constrained deep learning makes neural networks easier to verify 👉 https://spr.ly/6045gmKQv

207

1 Kommentar

Eryk Lewinson brings us 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁 𝗧𝗿𝗮𝗰𝗸𝗶𝗻𝗴 & 𝗛𝘆𝗽𝗲𝗿𝗽𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿 𝗧𝘂𝗻𝗶𝗻𝗴: 𝗞𝗲𝗲𝗽𝗶𝗻𝗴 𝗬𝗼𝘂𝗿 𝗧𝗿𝗶𝗮𝗹𝘀 𝗢𝗿𝗴𝗮𝗻𝗶𝘇𝗲𝗱 𝘄𝗶𝘁𝗵 𝗗𝗩𝗖 ️ As data scientists or machine learning engineers, keeping track of experiments, especially during 𝗛𝘆𝗽𝗲𝗿𝗽𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿 𝗧𝘂𝗻𝗶𝗻𝗴 (𝗛𝗣𝗧), can be challenging. In this 𝗮𝗿𝘁𝗶𝗰𝗹𝗲, Eryk explores how DVC can be leveraged to ensure reproducibility and organization throughout the entire ML experiment lifecycle, including HPT on Towards Data Science. He explored: 🔹𝗠𝗮𝗻𝘂𝗮𝗹 𝗛𝗣𝗧: Versioning 𝑝𝘢𝑟𝘢𝑚𝘴.𝘺𝑎𝘮𝑙 for control and tracking. 🔹𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗲𝗱 𝗔𝗽𝗽𝗿𝗼𝗮𝗰𝗵𝗲𝘀: Exhaustive/randomized grid search with  𝑑𝘷𝑐 𝑟𝘶𝑛 𝑒𝘹𝑝. 🔹𝗕𝗮𝘆𝗲𝘀𝗶𝗮𝗻 𝗚𝗿𝗶𝗱 𝗦𝗲𝗮𝗿𝗰𝗵: Utilizing Optuna with 𝐷𝘝𝐶𝘓𝑖𝘷𝑒𝘊𝑎𝘭𝑙𝘣𝑎𝘤𝑘 for seamless tracking. By combining these techniques with DVC's core functionalities, we can ensure all experiments, including HPT trials, are fully reproducible. This article is a valuable resource for anyone looking to streamline their HPT workflows and maintain control over their ML experiments. Follow DVC.ai

7

1 Kommentar

Besides supporting your coding as an assistant, function calling using LLMs is an incredibly powerful and fun tool to use in your software projects. My first mini-project to try this out was to create a 'textual calculator'. All it did was convert textual equations into a JSON format that contains the detected operands and operators. This allowed my software to call a math function based on the JSON data. This was run locally with a last generation GPU on a relatively tiny model. Although LLMs can do easy math without function calling, this is not advisable for anything reliable. Can't wait to take the course below!

3

0017: Quantum Optimization - Solving Max-Cut Problems with QAOA Quantum Approximate Optimization Algorithm (QAOA) is a quantum algorithm used to find approximate solutions to combinatorial optimization problems. One of its applications is solving the Max-Cut problem, which is a classic problem in graph theory. QAOA is a promising algorithm for tackling combinatorial optimization problems, leveraging the power of quantum computing to find approximate solutions efficiently. MaxCut refers to a classic problem where the goal is to divide a graph's vertices into two groups such that the number of edges between these groups (cut edges) is maximized. This problem is known to be NP-hard but is fundamental in various optimization contexts, including quantum computing. In quantum optimization using tools like Qiskit, MaxCut is often used as a benchmark problem to demonstrate the capabilities of quantum algorithms, such as the Quantum Approximate Optimization Algorithm (QAOA). QAOA aims to find approximate solutions to combinatorial optimization problems like MaxCut by leveraging quantum circuits and classical optimization techniques. The idea is to encode the MaxCut problem into a Hamiltonian whose ground state corresponds to the optimal division of the graph's vertices. Quantum algorithms then attempt to find this ground state by evolving a quantum state using quantum gates and measuring to optimize the cut. Below is an implementation in IBM's Qiskit. Check comment section for link to repository for more insights and code. #QuantumComputing #Qiskit #QAOA #Optimization #GraphTheory #TechInnovation

6

1 Kommentar

🤝 Optimizing vector retrieval with advanced graph-based metadata techniques using LangChain and Neo4j 🤝 Text embeddings and vector similarity search help us find documents by understanding their meanings and how similar they are to each other. However, text embeddings aren’t as effective when sorting information based on specific criteria like dates or categories -- for example, if you need to find all documents created in a particular year or documents tagged under a specific category like “science fiction.” This is where metadata filtering or filtered vector search comes into play, as it can effectively handle those structured filters, allowing users to narrow their search results according to specific attributes. 💡 Metadata filtering followed by vector similarity search can increase the accuracy and relevance of the search results. Graph databases like Neo4j can store highly complex and connected structured data alongside unstructured data. Together, you can greatly narrow down the relevant document subset using a structured graph-based metadata filter. Read more in this blog post to learn how to implement graph-based metadata filtering using LangChain and Neo4j ➡ https://lnkd.in/gYUj4WEF

675

17 Kommentare

Today's machine learning models are so complex that traditional mapping techniques like SHAP and LIME are no longer practical without unlimited budgets and time. Despite claims that SHAP is sufficient for large language models (LLMs), it's challenging to explain results to non-experts and too time-consuming. Read: don't bother, you'll burn hours, and end up with everyone confused and having more questions than answers. Recently, we discovered Google's Model Explorer (https://lnkd.in/gJ_G8ia3) , which significantly simplifies the visualization of large-scale models. It supports hierarchical views, multi-framework compatibility, and advanced debugging with minimal setup, making it an invaluable tool for researchers and engineers. Model Explorer addresses these challenges with hierarchical views and GPU-accelerated rendering. Key Features - Hierarchical Visualization: Organizes model structures into intuitive layers, simplifying the understanding of complex architectures. - Multi-Framework Support: Compatible with popular ML frameworks like TensorFlow, PyTorch, and JAX. - Advanced Debugging: Facilitates side-by-side graph comparisons to debug conversion and performance issues. Model Explorer leverages techniques from 3D graphics to maintain smooth interaction, even with extensive models. Something to try over the weekend for those so inclined.

21

2 Kommentare

QUANT PRACAR by Paanduv Applications - Democratizing Quantum AI Education. Register for software access (Free): https://lnkd.in/gYf-_JqC (Note: join our webinar on Quantum AI: https://bit.ly/3NvJczp) Quant Pracar's mission is to make quantum computing accessible to everyone. This innovative software offers students and professionals a unique opportunity to: -Engage with cutting-edge quantum machine learning algorithms: Dive deep into the world of quantum AI and explore its potential to revolutionize various fields. -Learn by doing: Upload your own datasets and train them using quantum machine learning techniques, gaining practical experience with this revolutionary technology. -Embrace open-source and cross-platform support: Quant Pracar's commitment to open-source development ensures everyone has access to this powerful tool. Additionally, its cross-platform compatibility makes it usable on a variety of devices. Are you ready to embark on your quantum AI journey? Stay tuned for more information on the official release of Quant Pracar by Paanduv Applications! #quantum #AI #machinelearning #innovation #quantumcomputing

7