Slava Kisilevich

Explainability or Mechanistic Interpretability in LLMs: In the last couple of months covers have been lifted off this topic by Anthropic and OpenAI. While the two have used different approaches to identify which features best explain the results (Dictionary Learning and k-Sparse Autoencoders respectively), it is commendable to see the efforts in this much needed area. OpenAI's approach relies on concepts that have been around but the thinking around choosing top k Vs L0, reducing dead latents and using N2G (Neuron to Graph) is simply brilliant! A dense paper but worth the time. https://lnkd.in/e_dYadpA

43

Happy to share that our next AI seminar at Technology Innovation Institute will be given by Prof. Lenka Zdeborova from EPFL. Prof. Zdeborova will walk us through phase transitions between positional and semantic learning in the attention mechanism using a mathematically tractable dot-product attention model. 📆 Date & time: 25th June, 2024, 3:00pm (GST) 👉🏻 Register at https://lnkd.in/dwJVZibg 👩‍🏫 Lenka Zdeborová is a Professor of Physics and Computer Science at École Polytechnique Fédérale de Lausanne, where she leads the Statistical Physics of Computation Laboratory. She received a PhD in physics from the University of Paris-Sud and Charles University in Prague in 2008. She spent two years in the Los Alamos National Laboratory as the Director's Postdoctoral Fellow. Between 2010 and 2020, she was a researcher at CNRS, working in the Institute of Theoretical Physics in CEA Saclay, France. In 2014, she was awarded the CNRS bronze medal, in 2016 Philippe Meyer prize in theoretical physics and an ERC Starting Grant, in 2018 the Irène Joliot-Curie prize, in 2021 the Gibbs lectureship of AMS and the Neuron Fund award. She was an editorial board member for the Journal of Physics A, Physical Review E, Physical Review X, SIMODS, Machine Learning: Science and Technology, and Information and Inference. Lenka's expertise is in the application of concepts from statistical physics, such as advanced mean field methods, the replica method, and related message-passing algorithms, to problems in machine learning, signal processing, inference, and optimization. She enjoys erasing the boundaries between theoretical physics, mathematics and computer science. #attention #transformers #LLMs #AI #seminar #theory

69

Call for Papers: First International Conference on Machine Learning and Knowledge Discovery (MLKD 2024) October 23-24, 2024 Amirkabir University of Technology Tehran Iran https://mlkd.aut.ac.ir   We invite you to submit your original and unpublished research to the First International Conference on Machine Learning and Knowledge Discovery (MLKD 2024)! MLKD 2024 is a premier forum for researchers and practitioners to present and discuss the latest advances in machine learning, knowledge discovery, and their diverse applications across various domains. The conference aims to bring together a vibrant community of researchers, developers, and industry leaders to share knowledge, foster collaboration, and explore the future directions of these rapidly evolving fields. The topics of the conference include, but are not limited to the following:  - Supervised Learning - Unsupervised Learning - Reinforcement Learning - Graph Neural Networks - Deep Learning - Natural Language Processing - Computer Vision - Speech Recognition - Robotics - Big Data Analytics and Algorithms - Knowledge Representation - Data Mining - Information Retrieval - Social Computing - Explainable AI (XAI) - Ethical Considerations in Machine Learning - Applications in Healthcare, Finance, Retail, Manufacturing, Media & Entertainment, Transportation, Neuroscience, Bioinformatics, Geoinformatics, Agriculture, Environment Important Dates: Opening submissions: August 1, 2024 Submission deadline: August 23, 2024 Acceptance notification: September 23, 2024 Conference: October 23-24, 2024 Submission Guidelines: - The submission must be prepared with LaTeX, using our MLKD 2024 LaTeX template; here is the link of our template: https://lnkd.in/dwSaU_fP  - The submission must be submitted through Microsoft’s Conference Management Toolkit via this link: https://lnkd.in/dxz_STsH Publication Opportunities: Authors of selected, accepted papers will be invited to submit extended article versions for publication in JMM, AJMC, CMDE, and MIR. Join us at MLKD 2024 and be part of shaping the future of Machine Learning and Knowledge Discovery! Mohammad Ali Abam MOSTAFA ABBASZADEH Mohammad Akbari Jamal Amani Rad Hadi Amiri Bagher BabaAli Hamid Beigy Ali Darvishi Boloorani Ardeshir Dolati Hadi Farahani Mehdi Ghatee Philippe Giabbanelli Dr. Ali Al-Hemoud Nazanin H. Majid Jaberi Kazem Jahanbakhsh, Ph.D. Hossein Jowhari Amirreza Khodadadian Ali Masoudi-Nejad, Professor Babak Nasersharif Taher Rahgooy Mostafa Rezazad Mansoor Rezghi Bahram Sadeghi Bigham Hamed Shahbazi Mostafa Shamsi Mehrnoush Shamsfard Yasser Shekofteh Dunhui Xiao Mohsen Yousefnezhad Hamid Zarrabi-Zadeh

16

1 Kommentar

[New on our blog] How to Automate ML Experiment Management With CI/CD Authors: Kilian Kluge and Dhruvil Karani Reading time: 8 min — (link to the full article in the comments) #ml #machinelearning #mlops

2

1 Kommentar

The tendency of large language models to “hallucinate” continues to trouble #CIOs eyeing production use-cases – even as efforts around fine-tuning and RAG-based optimisations continue. Among those concerned about this “inconsistency” is the US Department of Defense. The Stack took a look at a new programme as well as some fresh research on #LLM validation. 👉 https://lnkd.in/eEuntxqB #artificialintelligence #nationalsecurity #llms #hallucination

2

#EarlyCareer #talk at #IJCAI2024 by🗣️Julia Neidhart, Technische Universität Wien: Transforming Recommender Systems: Balancing Personalization, Fairness, and Human Values ➡️https://lnkd.in/dK3Nieyr Abstract: Recent advancements in recommender systems highlight metrics beyond accuracy, including diversity, serendipity and fairness. This paper examines the integration of Large Language Models into recommender systems, focusing on challenges such as interpretability, bias, and user autonomy. LLMs offer enhanced personalization capabilities but also introduce concerns regarding transparency and ethical implications. This work discusses ongoing research efforts aiming to develop transparent, fair, and contextually aware systems. Our approach seeks to prioritize user wellbeing and inclusivity, contributing to a more equitable and functional digital environment through advanced technologies and interdisciplinary insights. #XAI #LLMs #InclusiveTech #NextGenAI #AI TU Wien Informatics

3

Our small research team (main researcher Zimin Chen and our PO Michal Kovacik) is working on a research project at Deutsche Telekom: the Advanced Coding Assistant. https://lnkd.in/eaupSM_M This second generation Coding Assistant based on extensive research utilizes knowledge graphs and abstract syntax trees (ASTs) to enhance coding support. Here’s what our Advanced Coding Assistant aims to bring to developers: - Enhanced Contextual Understanding: By combining syntactic and semantic information, the chatbot gains a comprehensive understanding of the codebase, offering more accurate suggestions and identifying potential issues that might be missed with a purely syntactic or semantic analysis (https://lnkd.in/ebPWWmsZ). - Improved Code Quality: With a deep understanding of code structures and relationships, the chatbot suggests best practices, design patterns, and optimizations, leading to cleaner, more maintainable, and efficient code (https://lnkd.in/eYxfpSFG). - Bug Detection and Prevention: The chatbot’s high-level reasoning about code enables it to catch subtle bugs and logical errors, reducing the risk of costly mistakes and improving overall software reliability (https://lnkd.in/eGn7xhdj). - Personalized Assistance: Leveraging LLMs and knowledge graphs, the chatbot provides personalized coding support tailored to individual developers' or teams' specific needs and preferences (https://lnkd.in/e_hdEuvC). Key Technologies behind our Advanced Coding Chatbot: LangChain4J: Enables exploration of different LLM integrations. Neo4J: A state-of-the-art graph database. TreeSitter: Builds concrete syntax trees for source files. Jim Webber Thank you for your encouragement! (https://neo4j.com/) Tomaz Bratanic Thank you for your awesome engineering articles! (https://lnkd.in/eYHnvrA6) Lize Raes and Dmytro Liubarskyi thank you for the great LangChain4J framework, that enables us to even realize Advanced Coding Assistant vision (https://lnkd.in/eridNyxN) I would like to thank our leadership at #AI4Coding for continual support: Juraj Girman, Michael Goeb, Arne Clauß and Dr. Simon Stolz Peter Leukert #AI #Coding #TechInnovation #DeutscheTelekom #KnowledgeGraphs #ASTs #LLM #AIinCoding #SoftwareDevelopment #AI4Coding #LangChain4J #Neo4J

86

1 Kommentar

From The Max Planck Institute for the Science of Light Scimuse Generation and human-expert evaluation of interesting research ideas using knowledge graphs and large language models X Gu, M Krenn [Max Planck Institute for the Science of Light] (2024) https://lnkd.in/e6PmE4XM - The paper introduces SciMuse, a system to generate personalized research ideas using a knowledge graph built from 58 million papers and GPT-4. - Over 100 research group leaders evaluated 4000+ personalized suggestions on interest level from 1-5. - No significant difference in interest between projects generated with random concepts, high-impact concepts, and no concepts. - Negative correlation between concept connectivity/citations and interest level. Positive correlation between concept similarity and interest. - ML model predicts research interest from graph features with high precision, over 50% for top 15 suggestions. - Work demonstrates potential of AI to suggest highly interesting interdisciplinary research ideas and collaborations. abs: Advanced artificial intelligence (AI) systems with access to millions of research papers could inspire new research ideas that may not be conceived by humans alone. However, how interesting are these AI-generated ideas, and how can we improve their quality? Here, we introduce SciMuse, a system that uses an evolving knowledge graph built from more than 58 million scientific papers to generate personalized research ideas via an interface to GPT-4. We conducted a large-scale human evaluation with over 100 research group leaders from the Max Planck Society, who ranked more than 4,000 personalized research ideas based on their level of interest. This evaluation allows us to understand the relationships between scientific interest and the core properties of the knowledge graph. We find that data-efficient machine learning can predict research interest with high precision, allowing us to optimize the interest-level of generated research ideas. This work represents a step towards an artificial scientific muse that could catalyze unforeseen collaborations and suggest interesting avenues for scientists. #AI #computation #language #digitallibraries #machinelearning #knowledgegraph #LLM

13

#FineTuning is perhaps one of the most discussed technical aspects when it comes to Large Language Models. Most people understand that training these models is expensive and requires significant capital investment, so it is exciting to see that you can create a model that is somewhat unique by taking a pre-existing model and fine-tuning it with your own data. There are multiple methods to fine-tune a model, but one of the most consistently popular currently is the LoRA method (short for Low Rank Adaptation) discussed in the “LoRA: Low-Rank Adaptation of Large Language Models” paper. Before we dive into the mechanics behind LoRA, we need to understand some Matrix background and some of the basics of fine-tuning a machine learning model. https://lnkd.in/gnkWQYby

5

1 Kommentar

🔔 NEW RESEARCH BLOG: Since we are at ICLR this week, we're publishing a series of research blogs focused on our accepted papers to give you a deeper look at this exciting research. First up, "ConR: Contrastive Regularizer for Deep Imbalanced Regression" by M. Keramati, L. Meng, R. David Evans - A contrastive regularizer models global and local label similarities in feature space and prevents the features of minority samples from being collapsed into their majority neighbours. You can read the blog here ➡️ https://bit.ly/4dsTKeb. #ICLR #ImbalancedRegression #ICLR2024 #ConR #ICLRpapers #AIresearch #ML #ICLRpapers #ICLRinVienna #ContrastiveRegularizer

13

Separating Fact from Logic: Test of Time ToT Benchmark Isolates Reasoning Skills in LLMs for Improved Temporal Understanding Quick read: https://lnkd.in/gbc5CqN8 Paper: https://lnkd.in/giu72CwW HF Page: https://lnkd.in/g7H6dgY9

17

Our next video is out! This is the first of a 2-part video in which Prof. Semih Salihoğlu explains about Resource Description Framework (RDF), a popular data model in graph DBMSs. If you're curious about graph modeling and are intimidated by the terminology used (especially when it comes to RDF), this video has you covered! ✅ Fundamentals of RDF: What are resources, URIs, literals, and (subject, predicate, object) triples, and how do these form a graph? ✅ Advantages of using RDF as a data model for complex and irregular domains and its flexibility in handling irregularities and connections between objects ✅ How RDF allows you to query both data and schema information in a uniform way. In the 2nd video (coming soon!), we will also go over RDF's powerful knowledge representation and reasoning capabilities, and their potential in the age of AI. Stay tuned! https://lnkd.in/gZM79U44

30

From Harvard University E(n) Equivariant Topological Neural Networks C Battiloro, E Karaismailoğlu, M Tec, G Dasoulas… [Harvard University] (2024) https://lnkd.in/eRf9QZEK - Introduces E(n) Equivariant Topological Neural Networks (ETNNs), message passing networks on combinatorial complexes that incorporate geometric node features while respecting rotation and translation equivariance. - Combinatorial complexes flexibly represent arbitrary hierarchical higher-order interactions, allowing ETNNs to handle settings with heterogeneous interactions. - Provides theoretical analysis showing improved expressiveness of ETNNs over E(n) equivariant graph methods. - Derives several E(n) equivariant variants of existing topological deep learning models from the ETNN framework. - Demonstrates broad applicability through molecular property prediction on QM9 benchmark and hyperlocal estimation of air pollution with multi-resolution irregular geospatial data. - Experiment results validate ETNNs and the benefits of principled geometric inductive bias for learning from richly structured data. abs: Graph neural networks excel at modeling pairwise interactions, but they cannot flexibly accommodate higher-order interactions and features. Topological deep learning (TDL) has emerged recently as a promising tool for addressing this issue. TDL enables the principled modeling of arbitrary multi-way, hierarchical higher-order interactions by operating on combinatorial topological spaces, such as simplicial or cell complexes, instead of graphs. However, little is known about how to leverage geometric features such as positions and velocities for TDL. This paper introduces E(n)-Equivariant Topological Neural Networks (ETNNs), which are E(n)-equivariant message-passing networks operating on combinatorial complexes, formal objects unifying graphs, hypergraphs, simplicial, path, and cell complexes. ETNNs incorporate geometric node features while respecting rotation and translation equivariance. Moreover, ETNNs are natively ready for settings with heterogeneous interactions. We provide a theoretical analysis to show the improved expressiveness of ETNNs over architectures for geometric graphs. We also show how several E(n) equivariant variants of TDL models can be directly derived from our framework. The broad applicability of ETNNs is demonstrated through two tasks of vastly different nature: i) molecular property prediction on the QM9 benchmark and ii) land-use regression for hyper-local estimation of air pollution with multi-resolution irregular geospatial data. The experiment results indicate that ETNNs are an effective tool for learning from diverse types of richly structured data, highlighting the benefits of principled geometric inductive bias. #AI #machinelearning #EvolutionaryComputing #NeuralComputing #topology #neuralnetworks

16

Want to excel as a machine learning engineer? In his recent video Boris Meinardus a student researcher at TU Darmstadt, summarizes the seven most important skill sets based on insights from ML engineers at top companies like Google, Meta, NVIDIA, and OpenAI. Key Skills: 1. Programming Skill 2. Mathematics 3. Soft Skills 4. Familiarity with Core Libraries 5. Machine Learning Knowledge 6. Model Evaluation 7. Software Engineering Skills

1

In this Technical Deep Dive, we share our approaches in using Large Language Models (#LLMs) to make numerical-type data(sets) like DHS, JMP, HDX more accessible with the example of WASH data. In this 15 min. video Olivier Mills and team cover: 1️⃣ Basic scripting to extract core data from datasets from API endpoints 2️⃣ How to use LLMs to generate LLM-friendly schema definitions 3️⃣ How to structure a AI-friendly database store that can scale to thousands of datasets and millions of data indicators 4️⃣ How to use vector querying to retrieve relevant indicators from the store based off of a natural language (english) query 5️⃣ How to build a prompt that generates valid SQL query using LLMs with the above retrieved data schema. The various components form a single AI system combining data ingestion and data exploration using a software layer, a data layer and multiple LLM layers. ⏭️ Next steps will be to expand the datasets, automate other data source ingestion and further test the system with more complex plain english queries. We plan to incorporate this as a new feature for WASH AI after testing. Feel free to reach out if you would like a copy of the Notebooks, sample prompts and data schema definitions. We are #opensource The initial work was done in collaboration with Data for Good YYC (see https://lnkd.in/g3AijS93)

3

1 Kommentar

We just published another video by Prof. Semih Salihoğlu - this one addresses the burning question of "When do you need a graph database?", and how GDBMSs are different from RDBMSs. If you're coming from the world of relational databases, this is a very pertinent question to ask, and we think you'll find answers :). ✅ Recursive querying and the notion of paths ✅ Flexible join sequences ✅ Querying patterns in data as graph patterns ✅ Answering entity-oriented questions ✅ Modeling complex & irregular domains (where the data is too complex and irregular to structure as tables) pip install kuzu to get started And please subscribe to our YouTube channel and stay tuned for more in this playlist! https://lnkd.in/empa6ysG #graph #database #rdbms

9

🎙️ How do the statistics and ML communities differ? What does current research in trend filtering and conformal prediction look like? Daniel Bashir speaks with Professor Ryan Tibshirani.

6

1 Kommentar

🔍 Don't Miss Out on This Insightful Session! 🔍 Join us at Data Week Leipzig for a dynamic presentation on ""Industrial Knowledge Graphs in action"" by Nikolay Krustev from eccenca's partner Ontotext 🕒 When: April 18th, between 13:00 - 14:30 📍 Where: Sitzungssaal 🎙️ Language: English Nikolay Krustev, leading Ontotext's Industrial Vertical, brings over a decade of experience in driving strategic business development across Automotive, Manufacturing, AECO, and Energy sectors. In this session, Nikolay will unveil real-world applications and use cases, demonstrating the power of knowledge graphs in transforming industries. From enhancing data product catalogs to architecting modern data stacks for commercial analytics, Nikolay's insights promise to inspire and enlighten. Don't miss this opportunity to delve into the world of industrial knowledge graphs and gain valuable insights from Nikolay and other industry experts. See you there! #KnowledgeGraphs #DataWeekLeipzig #IndustrialInsights #eccenca #Ontotext

23