Showing 1–13 of 13 results for author: Georgiev, D

Hyperspectral unmixing for Raman spectroscopy via physics-constrained autoencoders

Authors: Dimitar Georgiev, Álvaro Fernández-Galiana, Simon Vilms Pedersen, Georgios Papadopoulos, Ruoxiao Xie, Molly M. Stevens, Mauricio Barahona

Abstract: Raman spectroscopy is widely used across scientific domains to characterize the chemical composition of samples in a non-destructive, label-free manner. Many applications entail the unmixing of signals from mixtures of molecular species to identify the individual components present and their proportions, yet conventional methods for chemometrics often struggle with complex mixture scenarios encoun… ▽ More Raman spectroscopy is widely used across scientific domains to characterize the chemical composition of samples in a non-destructive, label-free manner. Many applications entail the unmixing of signals from mixtures of molecular species to identify the individual components present and their proportions, yet conventional methods for chemometrics often struggle with complex mixture scenarios encountered in practice. Here, we develop hyperspectral unmixing algorithms based on autoencoder neural networks, and we systematically validate them using both synthetic and experimental benchmark datasets created in-house. Our results demonstrate that unmixing autoencoders provide improved accuracy, robustness and efficiency compared to standard unmixing methods. We also showcase the applicability of autoencoders to complex biological settings by showing improved biochemical characterization of volumetric Raman imaging data from a monocytic cell. △ Less

Submitted 7 March, 2024; originally announced March 2024.

The Deep Equilibrium Algorithmic Reasoner

Authors: Dobrik Georgiev, Pietro Liò, Davide Buffelli

Abstract: Recent work on neural algorithmic reasoning has demonstrated that graph neural networks (GNNs) could learn to execute classical algorithms. Doing so, however, has always used a recurrent architecture, where each iteration of the GNN aligns with an algorithm's iteration. Since an algorithm's solution is often an equilibrium, we conjecture and empirically validate that one can train a network to sol… ▽ More Recent work on neural algorithmic reasoning has demonstrated that graph neural networks (GNNs) could learn to execute classical algorithms. Doing so, however, has always used a recurrent architecture, where each iteration of the GNN aligns with an algorithm's iteration. Since an algorithm's solution is often an equilibrium, we conjecture and empirically validate that one can train a network to solve algorithmic problems by directly finding the equilibrium. Note that this does not require matching each GNN iteration with a step of the algorithm. △ Less

Submitted 9 April, 2024; v1 submitted 9 February, 2024; originally announced February 2024.

RamanSPy: An open-source Python package for integrative Raman spectroscopy data analysis

Authors: Dimitar Georgiev, Simon Vilms Pedersen, Ruoxiao Xie, Álvaro Fernández-Galiana, Molly M. Stevens, Mauricio Barahona

Abstract: Raman spectroscopy is a non-destructive and label-free chemical analysis technique, which plays a key role in the analysis and discovery cycle of various branches of science. Nonetheless, progress in Raman spectroscopic analysis is still impeded by the lack of software, methodological and data standardisation, and the ensuing fragmentation and lack of reproducibility of analysis workflows thereof.… ▽ More Raman spectroscopy is a non-destructive and label-free chemical analysis technique, which plays a key role in the analysis and discovery cycle of various branches of science. Nonetheless, progress in Raman spectroscopic analysis is still impeded by the lack of software, methodological and data standardisation, and the ensuing fragmentation and lack of reproducibility of analysis workflows thereof. To address these issues, we introduce RamanSPy, an open-source Python package for Raman spectroscopic research and analysis. RamanSPy provides a comprehensive library of ready-to-use tools for spectroscopic analysis, which streamlines day-to-day tasks, integrative analyses, as well as novel research and algorithmic development. RamanSPy is modular and open source, not tied to a particular technology or data format, and can be readily interfaced with the burgeoning ecosystem for data science, statistical analysis and machine learning in Python. △ Less

Submitted 5 July, 2023; originally announced July 2023.

Parallel Algorithms Align with Neural Execution

Authors: Valerie Engelmayer, Dobrik Georgiev, Petar Veličković

Abstract: Neural algorithmic reasoners are parallel processors. Teaching them sequential algorithms contradicts this nature, rendering a significant share of their computations redundant. Parallel algorithms however may exploit their full computational power, therefore requiring fewer layers to be executed. This drastically reduces training times, as we observe when comparing parallel implementations of sea… ▽ More Neural algorithmic reasoners are parallel processors. Teaching them sequential algorithms contradicts this nature, rendering a significant share of their computations redundant. Parallel algorithms however may exploit their full computational power, therefore requiring fewer layers to be executed. This drastically reduces training times, as we observe when comparing parallel implementations of searching, sorting and finding strongly connected components to their sequential counterparts on the CLRS framework. Additionally, parallel versions achieve (often strongly) superior predictive performance. △ Less

Submitted 3 January, 2024; v1 submitted 8 July, 2023; originally announced July 2023.

Comments: 13 pages, 7 figures, Proceedings of the Second Learning on Graphs Conference (LoG 2023), PMLR 231

Accelerating Molecular Graph Neural Networks via Knowledge Distillation

Authors: Filip Ekström Kelvinius, Dimitar Georgiev, Artur Petrov Toshev, Johannes Gasteiger

Abstract: Recent advances in graph neural networks (GNNs) have enabled more comprehensive modeling of molecules and molecular systems, thereby enhancing the precision of molecular property prediction and molecular simulations. Nonetheless, as the field has been progressing to bigger and more complex architectures, state-of-the-art GNNs have become largely prohibitive for many large-scale applications. In th… ▽ More Recent advances in graph neural networks (GNNs) have enabled more comprehensive modeling of molecules and molecular systems, thereby enhancing the precision of molecular property prediction and molecular simulations. Nonetheless, as the field has been progressing to bigger and more complex architectures, state-of-the-art GNNs have become largely prohibitive for many large-scale applications. In this paper, we explore the utility of knowledge distillation (KD) for accelerating molecular GNNs. To this end, we devise KD strategies that facilitate the distillation of hidden representations in directional and equivariant GNNs, and evaluate their performance on the regression task of energy and force prediction. We validate our protocols across different teacher-student configurations and datasets, and demonstrate that they can consistently boost the predictive accuracy of student models without any modifications to their architecture. Moreover, we conduct comprehensive optimization of various components of our framework, and investigate the potential of data augmentation to further enhance performance. All in all, we manage to close the gap in predictive accuracy between teacher and student models by as much as 96.7% and 62.5% for energy and force prediction respectively, while fully preserving the inference throughput of the more lightweight models. △ Less

Submitted 28 October, 2023; v1 submitted 26 June, 2023; originally announced June 2023.

Comments: Accepted as a conference paper at NeurIPS 2023

Quantitative dynamics of design thinking and creativity perspectives in company context

Authors: Georgi V. Georgiev, Danko D. Georgiev

Abstract: This study is intended to provide in-depth insights into how design thinking and creativity issues are understood and possibly evolve in the course of design discussions in a company context. For that purpose, we use the seminar transcripts of the Design Thinking Research Symposium 12 (DTRS12) dataset "Tech-centred Design Thinking: Perspectives from a Rising Asia," which are primarily concerned wi… ▽ More This study is intended to provide in-depth insights into how design thinking and creativity issues are understood and possibly evolve in the course of design discussions in a company context. For that purpose, we use the seminar transcripts of the Design Thinking Research Symposium 12 (DTRS12) dataset "Tech-centred Design Thinking: Perspectives from a Rising Asia," which are primarily concerned with how Korean companies implement design thinking and what role designers currently play. We employed a novel method of information processing based on constructed dynamic semantic networks to investigate the seminar discussions according to company representatives and company size. We compared the quantitative dynamics in two seminars: the first involved managerial representatives of four companies, and the second involved specialized designers and management of a design center of single company. On the basis of dynamic semantic networks, we quantified the changes in four semantic measures -- abstraction, polysemy, information content, and pairwise word similarity -- in chronologically reconstructed individual design-thinking processes. Statistical analyses show that design thinking in the seminar with four companies, exhibits significant differences in the dynamics of abstraction, polysemy, and information content, compared to the seminar with the design center of single company. Both the decrease in polysemy and abstraction and the increase in information content in the individual design-thinking processes in the seminar with four companies indicate that design managers are focused on more concrete design issues, with more information and less ambiguous content to the final design product. By contrast, specialized designers manifest more abstract thinking and appear to exhibit a slightly higher level of divergence in their design processes. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Comments: 24 pages, 7 figures, 1 table

Journal ref: Technology in Society 2023; 74: 102292

Neural Algorithmic Reasoning for Combinatorial Optimisation

Authors: Dobrik Georgiev, Danilo Numeroso, Davide Bacciu, Pietro Liò

Abstract: Solving NP-hard/complete combinatorial problems with neural networks is a challenging research area that aims to surpass classical approximate algorithms. The long-term objective is to outperform hand-designed heuristics for NP-hard/complete problems by learning to generate superior solutions solely from training data. Current neural-based methods for solving CO problems often overlook the inheren… ▽ More Solving NP-hard/complete combinatorial problems with neural networks is a challenging research area that aims to surpass classical approximate algorithms. The long-term objective is to outperform hand-designed heuristics for NP-hard/complete problems by learning to generate superior solutions solely from training data. Current neural-based methods for solving CO problems often overlook the inherent "algorithmic" nature of the problems. In contrast, heuristics designed for CO problems, e.g. TSP, frequently leverage well-established algorithms, such as those for finding the minimum spanning tree. In this paper, we propose leveraging recent advancements in neural algorithmic reasoning to improve the learning of CO problems. Specifically, we suggest pre-training our neural model on relevant algorithms before training it on CO instances. Our results demonstrate that by using this learning setup, we achieve superior performance compared to non-algorithmically informed deep learning models. △ Less

Submitted 13 February, 2024; v1 submitted 18 May, 2023; originally announced June 2023.

Global Concept-Based Interpretability for Graph Neural Networks via Neuron Analysis

Authors: Han Xuanyuan, Pietro Barbiero, Dobrik Georgiev, Lucie Charlotte Magister, Pietro Lió

Abstract: Graph neural networks (GNNs) are highly effective on a variety of graph-related tasks; however, they lack interpretability and transparency. Current explainability approaches are typically local and treat GNNs as black-boxes. They do not look inside the model, inhibiting human trust in the model and explanations. Motivated by the ability of neurons to detect high-level semantic concepts in vision… ▽ More Graph neural networks (GNNs) are highly effective on a variety of graph-related tasks; however, they lack interpretability and transparency. Current explainability approaches are typically local and treat GNNs as black-boxes. They do not look inside the model, inhibiting human trust in the model and explanations. Motivated by the ability of neurons to detect high-level semantic concepts in vision models, we perform a novel analysis on the behaviour of individual GNN neurons to answer questions about GNN interpretability, and propose new metrics for evaluating the interpretability of GNN neurons. We propose a novel approach for producing global explanations for GNNs using neuron-level concepts to enable practitioners to have a high-level view of the model. Specifically, (i) to the best of our knowledge, this is the first work which shows that GNN neurons act as concept detectors and have strong alignment with concepts formulated as logical compositions of node degree and neighbourhood properties; (ii) we quantitatively assess the importance of detected concepts, and identify a trade-off between training duration and neuron-level interpretability; (iii) we demonstrate that our global explainability approach has advantages over the current state-of-the-art -- we can disentangle the explanation into individual interpretable concepts backed by logical descriptions, which reduces potential for bias and improves user-friendliness. △ Less

Submitted 8 March, 2023; v1 submitted 22 August, 2022; originally announced August 2022.

Comments: 9 pages, 5 figures

HEAT: Hyperedge Attention Networks

Authors: Dobrik Georgiev, Marc Brockschmidt, Miltiadis Allamanis

Abstract: Learning from structured data is a core machine learning task. Commonly, such data is represented as graphs, which normally only consider (typed) binary relationships between pairs of nodes. This is a substantial limitation for many domains with highly-structured data. One important such domain is source code, where hypergraph-based representations can better capture the semantically rich and stru… ▽ More Learning from structured data is a core machine learning task. Commonly, such data is represented as graphs, which normally only consider (typed) binary relationships between pairs of nodes. This is a substantial limitation for many domains with highly-structured data. One important such domain is source code, where hypergraph-based representations can better capture the semantically rich and structured nature of code. In this work, we present HEAT, a neural model capable of representing typed and qualified hypergraphs, where each hyperedge explicitly qualifies how participating nodes contribute. It can be viewed as a generalization of both message passing neural networks and Transformers. We evaluate HEAT on knowledge base completion and on bug detection and repair using a novel hypergraph representation of programs. In both settings, it outperforms strong baselines, indicating its power and generality. △ Less

Submitted 5 September, 2022; v1 submitted 28 January, 2022; originally announced January 2022.

Comments: Published in TMLR

Algorithmic Concept-based Explainable Reasoning

Authors: Dobrik Georgiev, Pietro Barbiero, Dmitry Kazhdan, Petar Veličković, Pietro Liò

Abstract: Recent research on graph neural network (GNN) models successfully applied GNNs to classical graph algorithms and combinatorial optimisation problems. This has numerous benefits, such as allowing applications of algorithms when preconditions are not satisfied, or reusing learned models when sufficient training data is not available or can't be generated. Unfortunately, a key hindrance of these appr… ▽ More Recent research on graph neural network (GNN) models successfully applied GNNs to classical graph algorithms and combinatorial optimisation problems. This has numerous benefits, such as allowing applications of algorithms when preconditions are not satisfied, or reusing learned models when sufficient training data is not available or can't be generated. Unfortunately, a key hindrance of these approaches is their lack of explainability, since GNNs are black-box models that cannot be interpreted directly. In this work, we address this limitation by applying existing work on concept-based explanations to GNN models. We introduce concept-bottleneck GNNs, which rely on a modification to the GNN readout mechanism. Using three case studies we demonstrate that: (i) our proposed model is capable of accurately learning concepts and extracting propositional formulas based on the learned concepts for each target class; (ii) our concept-based GNN models achieve comparative performance with state-of-the-art models; (iii) we can derive global graph concepts, without explicitly providing any supervision on graph-level concepts. △ Less

Submitted 15 July, 2021; originally announced July 2021.

Comments: preprint

Enhancing user creativity: Semantic measures for idea generation

Authors: Georgi V. Georgiev, Danko D. Georgiev

Abstract: Human creativity generates novel ideas to solve real-world problems. This thereby grants us the power to transform the surrounding world and extend our human attributes beyond what is currently possible. Creative ideas are not just new and unexpected, but are also successful in providing solutions that are useful, efficient and valuable. Thus, creativity optimizes the use of available resources an… ▽ More Human creativity generates novel ideas to solve real-world problems. This thereby grants us the power to transform the surrounding world and extend our human attributes beyond what is currently possible. Creative ideas are not just new and unexpected, but are also successful in providing solutions that are useful, efficient and valuable. Thus, creativity optimizes the use of available resources and increases wealth. The origin of human creativity, however, is poorly understood, and semantic measures that could predict the success of generated ideas are currently unknown. Here, we analyze a dataset of design problem-solving conversations in real-world settings by using 49 semantic measures based on WordNet 3.1 and demonstrate that a divergence of semantic similarity, an increased information content, and a decreased polysemy predict the success of generated ideas. The first feedback from clients also enhances information content and leads to a divergence of successful ideas in creative problem solving. These results advance cognitive science by identifying real-world processes in human problem solving that are relevant to the success of produced solutions and provide tools for real-time monitoring of problem solving, student training and skill acquisition. A selected subset of information content (IC Sánchez-Batet) and semantic similarity (Lin/Sánchez-Batet) measures, which are both statistically powerful and computationally fast, could support the development of technologies for computer-assisted enhancements of human creativity or for the implementation of creativity in machines endowed with general artificial intelligence. △ Less

Submitted 18 June, 2021; originally announced June 2021.

Comments: 42 pages, 10 figures, 2 tables

Journal ref: Knowledge-Based Systems 2018; 151: 1-15

PyTorch, Explain! A Python library for Logic Explained Networks

Authors: Pietro Barbiero, Gabriele Ciravegna, Dobrik Georgiev, Franscesco Giannini

Abstract: "PyTorch, Explain!" is a Python module integrating a variety of state-of-the-art approaches to provide logic explanations from neural networks. This package focuses on bringing these methods to non-specialists. It has minimal dependencies and it is distributed under the Apache 2.0 licence allowing both academic and commercial use. Source code and documentation can be downloaded from the github rep… ▽ More "PyTorch, Explain!" is a Python module integrating a variety of state-of-the-art approaches to provide logic explanations from neural networks. This package focuses on bringing these methods to non-specialists. It has minimal dependencies and it is distributed under the Apache 2.0 licence allowing both academic and commercial use. Source code and documentation can be downloaded from the github repository: https://github.com/pietrobarbiero/pytorch_explain. △ Less

Submitted 23 July, 2021; v1 submitted 25 May, 2021; originally announced May 2021.

Neural Bipartite Matching

Authors: Dobrik Georgiev, Pietro Liò

Abstract: Graph neural networks (GNNs) have found application for learning in the space of algorithms. However, the algorithms chosen by existing research (sorting, Breadth-First search, shortest path finding, etc.) usually align perfectly with a standard GNN architecture. This report describes how neural execution is applied to a complex algorithm, such as finding maximum bipartite matching by reducing it… ▽ More Graph neural networks (GNNs) have found application for learning in the space of algorithms. However, the algorithms chosen by existing research (sorting, Breadth-First search, shortest path finding, etc.) usually align perfectly with a standard GNN architecture. This report describes how neural execution is applied to a complex algorithm, such as finding maximum bipartite matching by reducing it to a flow problem and using Ford-Fulkerson to find the maximum flow. This is achieved via neural execution based only on features generated from a single GNN. The evaluation shows strongly generalising results with the network achieving optimal matching almost 100% of the time. △ Less

Submitted 11 July, 2024; v1 submitted 22 May, 2020; originally announced May 2020.