-
Strong Copyright Protection for Language Models via Adaptive Model Fusion
Authors:
Javier Abad,
Konstantin Donhauser,
Francesco Pinto,
Fanny Yang
Abstract:
The risk of language models unintentionally reproducing copyrighted material from their training data has led to the development of various protective measures. In this paper, we propose model fusion as an effective solution to safeguard against copyright infringement. In particular, we introduce Copyright-Protecting Fusion (CP-Fuse), an algorithm that adaptively combines language models to minimi…
▽ More
The risk of language models unintentionally reproducing copyrighted material from their training data has led to the development of various protective measures. In this paper, we propose model fusion as an effective solution to safeguard against copyright infringement. In particular, we introduce Copyright-Protecting Fusion (CP-Fuse), an algorithm that adaptively combines language models to minimize the reproduction of protected materials. CP-Fuse is inspired by the recently proposed Near-Access Free (NAF) framework and additionally incorporates a desirable balancing property that we demonstrate prevents the reproduction of memorized training data. Our results show that CP-Fuse significantly reduces the memorization of copyrighted content while maintaining high-quality text and code generation. Furthermore, we demonstrate how CP-Fuse can be integrated with other techniques for enhanced protection.
△ Less
Submitted 29 July, 2024;
originally announced July 2024.
-
Extracting Training Data from Document-Based VQA Models
Authors:
Francesco Pinto,
Nathalie Rauschmayr,
Florian Tramèr,
Philip Torr,
Federico Tombari
Abstract:
Vision-Language Models (VLMs) have made remarkable progress in document-based Visual Question Answering (i.e., responding to queries about the contents of an input document provided as an image). In this work, we show these models can memorize responses for training samples and regurgitate them even when the relevant visual information has been removed. This includes Personal Identifiable Informat…
▽ More
Vision-Language Models (VLMs) have made remarkable progress in document-based Visual Question Answering (i.e., responding to queries about the contents of an input document provided as an image). In this work, we show these models can memorize responses for training samples and regurgitate them even when the relevant visual information has been removed. This includes Personal Identifiable Information (PII) repeated once in the training set, indicating these models could divulge memorised sensitive information and therefore pose a privacy risk. We quantitatively measure the extractability of information in controlled experiments and differentiate between cases where it arises from generalization capabilities or from memorization. We further investigate the factors that influence memorization across multiple state-of-the-art models and propose an effective heuristic countermeasure that empirically prevents the extractability of PII.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
Towards Certification of Uncertainty Calibration under Adversarial Attacks
Authors:
Cornelius Emde,
Francesco Pinto,
Thomas Lukasiewicz,
Philip H. S. Torr,
Adel Bibi
Abstract:
Since neural classifiers are known to be sensitive to adversarial perturbations that alter their accuracy, \textit{certification methods} have been developed to provide provable guarantees on the insensitivity of their predictions to such perturbations. Furthermore, in safety-critical applications, the frequentist interpretation of the confidence of a classifier (also known as model calibration) c…
▽ More
Since neural classifiers are known to be sensitive to adversarial perturbations that alter their accuracy, \textit{certification methods} have been developed to provide provable guarantees on the insensitivity of their predictions to such perturbations. Furthermore, in safety-critical applications, the frequentist interpretation of the confidence of a classifier (also known as model calibration) can be of utmost importance. This property can be measured via the Brier score or the expected calibration error. We show that attacks can significantly harm calibration, and thus propose certified calibration as worst-case bounds on calibration under adversarial perturbations. Specifically, we produce analytic bounds for the Brier score and approximate bounds via the solution of a mixed-integer program on the expected calibration error. Finally, we propose novel calibration attacks and demonstrate how they can improve model calibration through \textit{adversarial calibration training}.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Directed Isoperimetry and Monotonicity Testing: A Dynamical Approach
Authors:
Renato Ferreira Pinto Jr
Abstract:
This paper explores the connection between classical isoperimetric inequalities, their directed analogues, and monotonicity testing. We study the setting of real-valued functions $f : [0,1]^d \to \mathbb{R}$ on the solid unit cube, where the goal is to test with respect to the $L^p$ distance. Our goals are twofold: to further understand the relationship between classical and directed isoperimetry,…
▽ More
This paper explores the connection between classical isoperimetric inequalities, their directed analogues, and monotonicity testing. We study the setting of real-valued functions $f : [0,1]^d \to \mathbb{R}$ on the solid unit cube, where the goal is to test with respect to the $L^p$ distance. Our goals are twofold: to further understand the relationship between classical and directed isoperimetry, and to give a monotonicity tester with sublinear query complexity in this setting.
Our main results are 1) an $L^2$ monotonicity tester for $M$-Lipschitz functions with query complexity $\widetilde O(\sqrt{d} M^2 / ε^2)$ and, behind this result, 2) the directed Poincaré inequality $\mathsf{dist}^{\mathsf{mono}}_2(f)^2 \le C \mathbb{E}[|\nabla^- f|^2]$, where the "directed gradient" operator $\nabla^-$ measures the local violations of monotonicity of $f$.
To prove the second result, we introduce a partial differential equation (PDE), the directed heat equation, which takes a one-dimensional function $f$ into a monotone function $f^*$ over time and enjoys many desirable analytic properties. We obtain the directed Poincaré inequality by combining convergence aspects of this PDE with the theory of optimal transport. Crucially for our conceptual motivation, this proof is in complete analogy with the mathematical physics perspective on the classical Poincaré inequality, namely as characterizing the convergence of the standard heat equation toward equilibrium.
△ Less
Submitted 27 April, 2024;
originally announced April 2024.
-
Sensory Glove-Based Surgical Robot User Interface
Authors:
Leonardo Borgioli,
Ki-Hwan Oh,
Alberto Mangano,
Alvaro Ducas,
Luciano Ambrosini,
Federico Pinto,
Paula A Lopez,
Jessica Cassiani,
Milos Zefran,
Liaohai Chen,
Pier Cristoforo Giulianotti
Abstract:
Robotic surgery has reached a high level of maturity and has become an integral part of standard surgical care. However, existing surgeon consoles are bulky and take up valuable space in the operating room, present challenges for surgical team coordination, and their proprietary nature makes it difficult to take advantage of recent technological advances, especially in virtual and augmented realit…
▽ More
Robotic surgery has reached a high level of maturity and has become an integral part of standard surgical care. However, existing surgeon consoles are bulky and take up valuable space in the operating room, present challenges for surgical team coordination, and their proprietary nature makes it difficult to take advantage of recent technological advances, especially in virtual and augmented reality. One potential area for further improvement is the integration of modern sensory gloves into robotic platforms, allowing surgeons to control robotic arms directly with their hand movements intuitively. We propose one such system that combines an HTC Vive tracker, a Manus Meta Prime 3 XR sensory glove, and God Vision wireless smart glasses. The system controls one arm of a da Vinci surgical robot. In addition to moving the arm, the surgeon can use fingers to control the end-effector of the surgical instrument. Hand gestures are used to implement clutching and similar functions. In particular, we introduce clutching of the instrument orientation, a functionality not available in the da Vinci system. The vibrotactile elements of the glove are used to provide feedback to the user when gesture commands are invoked. A preliminary evaluation of the system shows that it has excellent tracking accuracy and allows surgeons to efficiently perform common surgical training tasks with minimal practice with the new interface; this suggests that the interface is highly intuitive. The proposed system is inexpensive, allows rapid prototyping, and opens opportunities for further innovations in the design of surgical robot interfaces.
△ Less
Submitted 20 March, 2024;
originally announced March 2024.
-
As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?
Authors:
Anjun Hu,
Jindong Gu,
Francesco Pinto,
Konstantinos Kamnitsas,
Philip Torr
Abstract:
Foundation models pre-trained on web-scale vision-language data, such as CLIP, are widely used as cornerstones of powerful machine learning systems. While pre-training offers clear advantages for downstream learning, it also endows downstream models with shared adversarial vulnerabilities that can be easily identified through the open-sourced foundation model. In this work, we expose such vulnerab…
▽ More
Foundation models pre-trained on web-scale vision-language data, such as CLIP, are widely used as cornerstones of powerful machine learning systems. While pre-training offers clear advantages for downstream learning, it also endows downstream models with shared adversarial vulnerabilities that can be easily identified through the open-sourced foundation model. In this work, we expose such vulnerabilities in CLIP's downstream models and show that foundation models can serve as a basis for attacking their downstream systems. In particular, we propose a simple yet effective adversarial attack strategy termed Patch Representation Misalignment (PRM). Solely based on open-sourced CLIP vision encoders, this method produces adversaries that simultaneously fool more than 20 downstream models spanning 4 common vision-language tasks (semantic segmentation, object detection, image captioning and visual question-answering). Our findings highlight the concerning safety risks introduced by the extensive usage of public foundational models in the development of downstream systems, calling for extra caution in these scenarios.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
Distribution Testing with a Confused Collector
Authors:
Renato Ferreira Pinto Jr.,
Nathaniel Harms
Abstract:
We are interested in testing properties of distributions with systematically mislabeled samples. Our goal is to make decisions about unknown probability distributions, using a sample that has been collected by a confused collector, such as a machine-learning classifier that has not learned to distinguish all elements of the domain. The confused collector holds an unknown clustering of the domain a…
▽ More
We are interested in testing properties of distributions with systematically mislabeled samples. Our goal is to make decisions about unknown probability distributions, using a sample that has been collected by a confused collector, such as a machine-learning classifier that has not learned to distinguish all elements of the domain. The confused collector holds an unknown clustering of the domain and an input distribution $μ$, and provides two oracles: a sample oracle which produces a sample from $μ$ that has been labeled according to the clustering; and a label-query oracle which returns the label of a query point $x$ according to the clustering.
Our first set of results shows that identity, uniformity, and equivalence of distributions can be tested efficiently, under the earth-mover distance, with remarkably weak conditions on the confused collector, even when the unknown clustering is adversarial. This requires defining a variant of the distribution testing task (inspired by the recent testable learning framework of Rubinfeld & Vasilyan), where the algorithm should test a joint property of the distribution and its clustering. As an example, we get efficient testers when the distribution tester is allowed to reject if it detects that the confused collector clustering is "far" from being a decision tree.
The second set of results shows that we can sometimes do significantly better when the clustering is random instead of adversarial. For certain one-dimensional random clusterings, we show that uniformity can be tested under the TV distance using $\widetilde O\left(\frac{\sqrt n}{ρ^{3/2} ε^2}\right)$ samples and zero queries, where $ρ\in (0,1]$ controls the "resolution" of the clustering. We improve this to $O\left(\frac{\sqrt n}{ρε^2}\right)$ when queries are allowed.
△ Less
Submitted 23 November, 2023;
originally announced November 2023.
-
Directed Poincaré Inequalities and $L^1$ Monotonicity Testing of Lipschitz Functions
Authors:
Renato Ferreira Pinto Jr
Abstract:
We study the connection between directed isoperimetric inequalities and monotonicity testing. In recent years, this connection has unlocked breakthroughs for testing monotonicity of functions defined on discrete domains. Inspired the rich history of isoperimetric inequalities in continuous settings, we propose that studying the relationship between directed isoperimetry and monotonicity in such se…
▽ More
We study the connection between directed isoperimetric inequalities and monotonicity testing. In recent years, this connection has unlocked breakthroughs for testing monotonicity of functions defined on discrete domains. Inspired the rich history of isoperimetric inequalities in continuous settings, we propose that studying the relationship between directed isoperimetry and monotonicity in such settings is essential for understanding the full scope of this connection.
Hence, we ask whether directed isoperimetric inequalities hold for functions $f : [0,1]^n \to \mathbb{R}$, and whether this question has implications for monotonicity testing. We answer both questions affirmatively. For Lipschitz functions $f : [0,1]^n \to \mathbb{R}$, we show the inequality $d^{\mathsf{mono}}_1(f) \lesssim \mathbb{E}\left[\|\nabla^- f\|_1\right]$, which upper bounds the $L^1$ distance to monotonicity of $f$ by a measure of its "directed gradient". A key ingredient in our proof is the monotone rearrangement of $f$, which generalizes the classical "sorting operator" to continuous settings. We use this inequality to give an $L^1$ monotonicity tester for Lipschitz functions $f : [0,1]^n \to \mathbb{R}$, and this framework also implies similar results for testing real-valued functions on the hypergrid.
△ Less
Submitted 5 July, 2023;
originally announced July 2023.
-
PILLAR: How to make semi-private learning more effective
Authors:
Francesco Pinto,
Yaxi Hu,
Fanny Yang,
Amartya Sanyal
Abstract:
In Semi-Supervised Semi-Private (SP) learning, the learner has access to both public unlabelled and private labelled data. We propose a computationally efficient algorithm that, under mild assumptions on the data, provably achieves significantly lower private labelled sample complexity and can be efficiently run on real-world datasets. For this purpose, we leverage the features extracted by networ…
▽ More
In Semi-Supervised Semi-Private (SP) learning, the learner has access to both public unlabelled and private labelled data. We propose a computationally efficient algorithm that, under mild assumptions on the data, provably achieves significantly lower private labelled sample complexity and can be efficiently run on real-world datasets. For this purpose, we leverage the features extracted by networks pre-trained on public (labelled or unlabelled) data, whose distribution can significantly differ from the one on which SP learning is performed. To validate its empirical effectiveness, we propose a wide variety of experiments under tight privacy constraints ($ε= 0.1$) and with a focus on low-data regimes. In all of these settings, our algorithm exhibits significantly improved performance over available baselines that use similar amounts of public data.
△ Less
Submitted 6 June, 2023;
originally announced June 2023.
-
Distribution Testing Under the Parity Trace
Authors:
Renato Ferreira Pinto Jr.,
Nathaniel Harms
Abstract:
Distribution testing is a fundamental statistical task with many applications, but we are interested in a variety of problems where systematic mislabelings of the sample prevent us from applying the existing theory. To apply distribution testing to these problems, we introduce distribution testing under the parity trace, where the algorithm receives an ordered sample $S$ that reveals only the leas…
▽ More
Distribution testing is a fundamental statistical task with many applications, but we are interested in a variety of problems where systematic mislabelings of the sample prevent us from applying the existing theory. To apply distribution testing to these problems, we introduce distribution testing under the parity trace, where the algorithm receives an ordered sample $S$ that reveals only the least significant bit of each element. This abstraction reveals connections between the following three problems of interest, allowing new upper and lower bounds:
1. In distribution testing with a confused collector, the collector of the sample may be incapable of distinguishing between nearby elements of a domain (e.g. a machine learning classifier). We prove bounds for distribution testing with a confused collector on domains structured as a cycle or a path.
2. Recent work on the fundamental testing vs. learning question established tight lower bounds on distribution-free sample-based property testing by reduction from distribution testing, but the tightness is limited to symmetric properties. The parity trace allows a broader family of equivalences to non-symmetric properties, while recovering and strengthening many of the previous results with a different technique.
3. We give the first results for property testing in the well-studied trace reconstruction model, where the goal is to test whether an unknown string $x$ satisfies some property or is far from satisfying that property, given only independent random traces of $x$.
Our main technical result is a tight bound of $\widetilde Θ\left((n/ε)^{4/5} + \sqrt n/ε^2\right)$ for testing uniformity of distributions over $[n]$ under the parity trace, leading also to results for the problems above.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators
Authors:
Jianhao Yuan,
Francesco Pinto,
Adam Davies,
Philip Torr
Abstract:
Neural image classifiers are known to undergo severe performance degradation when exposed to inputs that are sampled from environmental conditions that differ from their training data. Given the recent progress in Text-to-Image (T2I) generation, a natural question is how modern T2I generators can be used to simulate arbitrary interventions over such environmental factors in order to augment traini…
▽ More
Neural image classifiers are known to undergo severe performance degradation when exposed to inputs that are sampled from environmental conditions that differ from their training data. Given the recent progress in Text-to-Image (T2I) generation, a natural question is how modern T2I generators can be used to simulate arbitrary interventions over such environmental factors in order to augment training data and improve the robustness of downstream classifiers. We experiment across a diverse collection of benchmarks in single domain generalization (SDG) and reducing reliance on spurious features (RRSF), ablating across key dimensions of T2I generation, including interventional prompting strategies, conditioning mechanisms, and post-hoc filtering. Our extensive empirical findings demonstrate that modern T2I generators like Stable Diffusion can indeed be used as a powerful interventional data augmentation mechanism, outperforming previously state-of-the-art data augmentation techniques regardless of how each dimension is configured.
△ Less
Submitted 3 June, 2024; v1 submitted 21 December, 2022;
originally announced December 2022.
-
An Impartial Take to the CNN vs Transformer Robustness Contest
Authors:
Francesco Pinto,
Philip H. S. Torr,
Puneet K. Dokania
Abstract:
Following the surge of popularity of Transformers in Computer Vision, several studies have attempted to determine whether they could be more robust to distribution shifts and provide better uncertainty estimates than Convolutional Neural Networks (CNNs). The almost unanimous conclusion is that they are, and it is often conjectured more or less explicitly that the reason of this supposed superiorit…
▽ More
Following the surge of popularity of Transformers in Computer Vision, several studies have attempted to determine whether they could be more robust to distribution shifts and provide better uncertainty estimates than Convolutional Neural Networks (CNNs). The almost unanimous conclusion is that they are, and it is often conjectured more or less explicitly that the reason of this supposed superiority is to be attributed to the self-attention mechanism. In this paper we perform extensive empirical analyses showing that recent state-of-the-art CNNs (particularly, ConvNeXt) can be as robust and reliable or even sometimes more than the current state-of-the-art Transformers. However, there is no clear winner. Therefore, although it is tempting to state the definitive superiority of one family of architectures over another, they seem to enjoy similar extraordinary performances on a variety of tasks while also suffering from similar vulnerabilities such as texture, background, and simplicity biases.
△ Less
Submitted 22 July, 2022;
originally announced July 2022.
-
Sample-dependent Adaptive Temperature Scaling for Improved Calibration
Authors:
Tom Joy,
Francesco Pinto,
Ser-Nam Lim,
Philip H. S. Torr,
Puneet K. Dokania
Abstract:
It is now well known that neural networks can be wrong with high confidence in their predictions, leading to poor calibration. The most common post-hoc approach to compensate for this is to perform temperature scaling, which adjusts the confidences of the predictions on any input by scaling the logits by a fixed value. Whilst this approach typically improves the average calibration across the whol…
▽ More
It is now well known that neural networks can be wrong with high confidence in their predictions, leading to poor calibration. The most common post-hoc approach to compensate for this is to perform temperature scaling, which adjusts the confidences of the predictions on any input by scaling the logits by a fixed value. Whilst this approach typically improves the average calibration across the whole test dataset, this improvement typically reduces the individual confidences of the predictions irrespective of whether the classification of a given input is correct or incorrect. With this insight, we base our method on the observation that different samples contribute to the calibration error by varying amounts, with some needing to increase their confidence and others needing to decrease it. Therefore, for each input, we propose to predict a different temperature value, allowing us to adjust the mismatch between confidence and accuracy at a finer granularity. Furthermore, we observe improved results on OOD detection and can also extract a notion of hardness for the data-points. Our method is applied post-hoc, consequently using very little computation time and with a negligible memory footprint and is applied to off-the-shelf pre-trained classifiers. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets, showing that producing per-data-point temperatures is beneficial also for the expected calibration error across the whole test set. Code is available at: https://github.com/thwjoy/adats.
△ Less
Submitted 22 July, 2022; v1 submitted 13 July, 2022;
originally announced July 2022.
-
RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustness
Authors:
Francesco Pinto,
Harry Yang,
Ser-Nam Lim,
Philip H. S. Torr,
Puneet K. Dokania
Abstract:
We show that the effectiveness of the well celebrated Mixup [Zhang et al., 2018] can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss. This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup in mos…
▽ More
We show that the effectiveness of the well celebrated Mixup [Zhang et al., 2018] can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss. This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup in most cases under various forms of covariate shifts and out-of-distribution detection experiments. In fact, we observe that Mixup yields much degraded performance on detecting out-of-distribution samples possibly, as we show empirically, because of its tendency to learn models that exhibit high-entropy throughout; making it difficult to differentiate in-distribution samples from out-distribution ones. To show the efficacy of our approach (RegMixup), we provide thorough analyses and experiments on vision datasets (ImageNet & CIFAR-10/100) and compare it with a suite of recent approaches for reliable uncertainty estimation.
△ Less
Submitted 6 February, 2023; v1 submitted 29 June, 2022;
originally announced June 2022.
-
Enhanced Behavioral Cloning with Environmental Losses for Self-Driving Vehicles
Authors:
Nelson Fernandez Pinto,
Thomas Gilles
Abstract:
Learned path planners have attracted research interest due to their ability to model human driving behavior and rapid inference. Recent works on behavioral cloning show that simple imitation of expert observations is not sufficient to handle complex driving scenarios. Besides, predictions that land outside drivable areas can lead to potentially dangerous situations. This paper proposes a set of lo…
▽ More
Learned path planners have attracted research interest due to their ability to model human driving behavior and rapid inference. Recent works on behavioral cloning show that simple imitation of expert observations is not sufficient to handle complex driving scenarios. Besides, predictions that land outside drivable areas can lead to potentially dangerous situations. This paper proposes a set of loss functions, namely Social loss and Road loss, which account for modelling risky social interactions in path planning. These losses act as a repulsive scalar field that surrounds non-drivable areas. Predictions that land near these regions incur in a higher training cost, which is minimized using backpropagation. This methodology provides additional environment feedback to the traditional supervised learning set up. We validated this approach on a large-scale urban driving dataset. The results show the agent learns to imitate human driving while exhibiting better safety metrics. Furthermore, the proposed methodology has positive effects on inference without the need to artificially generate unsafe driving examples. The explanability study suggests that the benefits obtained are associated with a higher relevance of non-drivable areas in the agent's decisions compared to classical behavioral cloning.
△ Less
Submitted 4 February, 2022;
originally announced February 2022.
-
Study of Linear Precoding and Power Allocation for Large Multiple-Antenna Systems with Coarsely Quantized Signals
Authors:
S. F. Pinto,
R. C. de Lamare
Abstract:
This work studies coarse quantization-aware BD (${\scriptstyle\mathrm{CQA-BD}}$) and coarse quantization-aware RBD (${\scriptstyle\mathrm{CQA-RBD}}$) precoding algorithms for large-scale MU-MIMO systems with coarsely quantized signals and proposes the coarse-quantization most advantageous allocation strategy (${\scriptstyle\mathrm{CQA-MAAS}}$) power allocation algorithm for linearly-precoded MU-MI…
▽ More
This work studies coarse quantization-aware BD (${\scriptstyle\mathrm{CQA-BD}}$) and coarse quantization-aware RBD (${\scriptstyle\mathrm{CQA-RBD}}$) precoding algorithms for large-scale MU-MIMO systems with coarsely quantized signals and proposes the coarse-quantization most advantageous allocation strategy (${\scriptstyle\mathrm{CQA-MAAS}}$) power allocation algorithm for linearly-precoded MU-MIMO systems. An analysis of the sum-rate along with studies of computational complexity is also carried out. Finally, comparisons between existing precoding and its power allocated version are followed by conclusions.
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
Encoder-Decoder Architectures for Clinically Relevant Coronary Artery Segmentation
Authors:
João Lourenço Silva,
Miguel Nobre Menezes,
Tiago Rodrigues,
Beatriz Silva,
Fausto J. Pinto,
Arlindo L. Oliveira
Abstract:
Coronary X-ray angiography is a crucial clinical procedure for the diagnosis and treatment of coronary artery disease, which accounts for roughly 16% of global deaths every year. However, the images acquired in these procedures have low resolution and poor contrast, making lesion detection and assessment challenging. Accurate coronary artery segmentation not only helps mitigate these problems, but…
▽ More
Coronary X-ray angiography is a crucial clinical procedure for the diagnosis and treatment of coronary artery disease, which accounts for roughly 16% of global deaths every year. However, the images acquired in these procedures have low resolution and poor contrast, making lesion detection and assessment challenging. Accurate coronary artery segmentation not only helps mitigate these problems, but also allows the extraction of relevant anatomical features for further analysis by quantitative methods. Although automated segmentation of coronary arteries has been proposed before, previous approaches have used non-optimal segmentation criteria, leading to less useful results. Most methods either segment only the major vessel, discarding important information from the remaining ones, or segment the whole coronary tree based mostly on contrast information, producing a noisy output that includes vessels that are not relevant for diagnosis. We adopt a better-suited clinical criterion and segment vessels according to their clinical relevance. Additionally, we simultaneously perform catheter segmentation, which may be useful for diagnosis due to the scale factor provided by the catheter's known diameter, and is a task that has not yet been performed with good results. To derive the optimal approach, we conducted an extensive comparative study of encoder-decoder architectures trained on a combination of focal loss and a variant of generalized dice loss. Based on the EfficientNet and the UNet++ architectures, we propose a line of efficient and high-performance segmentation models using a new decoder architecture, the EfficientUNet++, whose best-performing version achieved average dice scores of 0.8904 and 0.7526 for the artery and catheter classes, respectively, and an average generalized dice score of 0.9234.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
-
Automated Detection of Coronary Artery Stenosis in X-ray Angiography using Deep Neural Networks
Authors:
Dinis L. Rodrigues,
Miguel Nobre Menezes,
Fausto J. Pinto,
Arlindo L. Oliveira
Abstract:
Coronary artery disease leading up to stenosis, the partial or total blocking of coronary arteries, is a severe condition that affects millions of patients each year. Automated identification and classification of stenosis severity from minimally invasive procedures would be of great clinical value, but existing methods do not match the accuracy of experienced cardiologists, due to the complexity…
▽ More
Coronary artery disease leading up to stenosis, the partial or total blocking of coronary arteries, is a severe condition that affects millions of patients each year. Automated identification and classification of stenosis severity from minimally invasive procedures would be of great clinical value, but existing methods do not match the accuracy of experienced cardiologists, due to the complexity of the task. Although a number of computational approaches for quantitative assessment of stenosis have been proposed to date, the performance of these methods is still far from the required levels for clinical applications. In this paper, we propose a two-step deep-learning framework to partially automate the detection of stenosis from X-ray coronary angiography images. In the two steps, we used two distinct convolutional neural network architectures, one to automatically identify and classify the angle of view, and another to determine the bounding boxes of the regions of interest in frames where stenosis is visible. Transfer learning and data augmentation techniques were used to boost the performance of the system in both tasks. We achieved a 0.97 accuracy on the task of classifying the Left/Right Coronary Artery (LCA/RCA) angle view and 0.68/0.73 recall on the determination of the regions of interest, for LCA and RCA, respectively. These results compare favorably with previous results obtained using related approaches, and open the way to a fully automated method for the identification of stenosis severity from X-ray angiographies.
△ Less
Submitted 4 March, 2021;
originally announced March 2021.
-
Towards Automated Satellite Conjunction Management with Bayesian Deep Learning
Authors:
Francesco Pinto,
Giacomo Acciarini,
Sascha Metz,
Sarah Boufelja,
Sylvester Kaczmarek,
Klaus Merz,
José A. Martinez-Heras,
Francesca Letizia,
Christopher Bridges,
Atılım Güneş Baydin
Abstract:
After decades of space travel, low Earth orbit is a junkyard of discarded rocket bodies, dead satellites, and millions of pieces of debris from collisions and explosions. Objects in high enough altitudes do not re-enter and burn up in the atmosphere, but stay in orbit around Earth for a long time. With a speed of 28,000 km/h, collisions in these orbits can generate fragments and potentially trigge…
▽ More
After decades of space travel, low Earth orbit is a junkyard of discarded rocket bodies, dead satellites, and millions of pieces of debris from collisions and explosions. Objects in high enough altitudes do not re-enter and burn up in the atmosphere, but stay in orbit around Earth for a long time. With a speed of 28,000 km/h, collisions in these orbits can generate fragments and potentially trigger a cascade of more collisions known as the Kessler syndrome. This could pose a planetary challenge, because the phenomenon could escalate to the point of hindering future space operations and damaging satellite infrastructure critical for space and Earth science applications. As commercial entities place mega-constellations of satellites in orbit, the burden on operators conducting collision avoidance manoeuvres will increase. For this reason, development of automated tools that predict potential collision events (conjunctions) is critical. We introduce a Bayesian deep learning approach to this problem, and develop recurrent neural network architectures (LSTMs) that work with time series of conjunction data messages (CDMs), a standard data format used by the space community. We show that our method can be used to model all CDM features simultaneously, including the time of arrival of future CDMs, providing predictions of conjunction event evolution with associated uncertainties.
△ Less
Submitted 22 December, 2020;
originally announced December 2020.
-
Spacecraft Collision Risk Assessment with Probabilistic Programming
Authors:
Giacomo Acciarini,
Francesco Pinto,
Sascha Metz,
Sarah Boufelja,
Sylvester Kaczmarek,
Klaus Merz,
José A. Martinez-Heras,
Francesca Letizia,
Christopher Bridges,
Atılım Güneş Baydin
Abstract:
Over 34,000 objects bigger than 10 cm in length are known to orbit Earth. Among them, only a small percentage are active satellites, while the rest of the population is made of dead satellites, rocket bodies, and debris that pose a collision threat to operational spacecraft. Furthermore, the predicted growth of the space sector and the planned launch of megaconstellations will add even more comple…
▽ More
Over 34,000 objects bigger than 10 cm in length are known to orbit Earth. Among them, only a small percentage are active satellites, while the rest of the population is made of dead satellites, rocket bodies, and debris that pose a collision threat to operational spacecraft. Furthermore, the predicted growth of the space sector and the planned launch of megaconstellations will add even more complexity, therefore causing the collision risk and the burden on space operators to increase. Managing this complex framework with internationally agreed methods is pivotal and urgent. In this context, we build a novel physics-based probabilistic generative model for synthetically generating conjunction data messages, calibrated using real data. By conditioning on observations, we use the model to obtain posterior distributions via Bayesian inference. We show that the probabilistic programming approach to conjunction assessment can help in making predictions and in finding the parameters that explain the observed data in conjunction data messages, thus shedding more light on key variables and orbital characteristics that more likely lead to conjunction events. Moreover, our technique enables the generation of physically accurate synthetic datasets of collisions, answering a fundamental need of the space and machine learning communities working in this area.
△ Less
Submitted 18 December, 2020;
originally announced December 2020.
-
VC Dimension and Distribution-Free Sample-Based Testing
Authors:
Eric Blais,
Renato Ferreira Pinto Jr.,
Nathaniel Harms
Abstract:
We consider the problem of determining which classes of functions can be tested more efficiently than they can be learned, in the distribution-free sample-based model that corresponds to the standard PAC learning setting. Our main result shows that while VC dimension by itself does not always provide tight bounds on the number of samples required to test a class of functions in this model, it can…
▽ More
We consider the problem of determining which classes of functions can be tested more efficiently than they can be learned, in the distribution-free sample-based model that corresponds to the standard PAC learning setting. Our main result shows that while VC dimension by itself does not always provide tight bounds on the number of samples required to test a class of functions in this model, it can be combined with a closely-related variant that we call "lower VC" (or LVC) dimension to obtain strong lower bounds on this sample complexity.
We use this result to obtain strong and in many cases nearly optimal lower bounds on the sample complexity for testing unions of intervals, halfspaces, intersections of halfspaces, polynomial threshold functions, and decision trees. Conversely, we show that two natural classes of functions, juntas and monotone functions, can be tested with a number of samples that is polynomially smaller than the number of samples required for PAC learning.
Finally, we also use the connection between VC dimension and property testing to establish new lower bounds for testing radius clusterability and testing feasibility of linear constraint systems.
△ Less
Submitted 7 December, 2020;
originally announced December 2020.
-
Individual Factors that Influence Effort and Contributions on Wikipedia
Authors:
Luiz F. Pinto,
Carlos Denner dos Santos,
Silvia Onoyama
Abstract:
In this work, we aim to analyze how attitude, self-efficacy, and altruism influence effort and active contributions on Wikipedia. We propose a new conceptual model based on the theory of planned behavior and findings from the literature on online communities. This model differs from other models that have been previously proposed by considering altruism in its various facets (identification, recip…
▽ More
In this work, we aim to analyze how attitude, self-efficacy, and altruism influence effort and active contributions on Wikipedia. We propose a new conceptual model based on the theory of planned behavior and findings from the literature on online communities. This model differs from other models that have been previously proposed by considering altruism in its various facets (identification, reciprocity, and reputation), and by treating effort as a factor prior to performance results, which is measured in terms of active contributions, according to the organizational literature. To fulfill the study specific objectives, Wikipedia surveyed community members and collected secondary data. After excluding outliers, we obtained a final sample with 212 participants. We applied exploratory factor analysis and structural equation modeling, which resulted in a model with satisfactory fit indices. The results indicate that effort influences active contributions, and attitude, altruism by reputation, and altruism by identification influence effort. None of the proposed factors are directly related to active contributions. Experience directly influences self-efficacy while it positively moderates the relation between effort and active contributions. Finally, we present the conclusions via several implications for the literature as well as suggestions for future research.
△ Less
Submitted 14 July, 2020;
originally announced July 2020.
-
Text-based inference of moral sentiment change
Authors:
Jing Yi Xie,
Renato Ferreira Pinto Jr.,
Graeme Hirst,
Yang Xu
Abstract:
We present a text-based framework for investigating moral sentiment change of the public via longitudinal corpora. Our framework is based on the premise that language use can inform people's moral perception toward right or wrong, and we build our methodology by exploring moral biases learned from diachronic word embeddings. We demonstrate how a parameter-free model supports inference of historica…
▽ More
We present a text-based framework for investigating moral sentiment change of the public via longitudinal corpora. Our framework is based on the premise that language use can inform people's moral perception toward right or wrong, and we build our methodology by exploring moral biases learned from diachronic word embeddings. We demonstrate how a parameter-free model supports inference of historical shifts in moral sentiment toward concepts such as slavery and democracy over centuries at three incremental levels: moral relevance, moral polarity, and fine-grained moral dimensions. We apply this methodology to visualizing moral time courses of individual concepts and analyzing the relations between psycholinguistic variables and rates of moral sentiment change at scale. Our work offers opportunities for applying natural language processing toward characterizing moral sentiment change in society.
△ Less
Submitted 20 January, 2020;
originally announced January 2020.
-
Motivic clustering schemes for directed graphs
Authors:
Facundo Mémoli,
Guilherme Vituri F. Pinto
Abstract:
Motivated by the concept of network motifs we construct certain clustering methods (functors) which are parametrized by a given collection of motifs (or representers).
Motivated by the concept of network motifs we construct certain clustering methods (functors) which are parametrized by a given collection of motifs (or representers).
△ Less
Submitted 6 January, 2020; v1 submitted 1 January, 2020;
originally announced January 2020.
-
Automatic Model Monitoring for Data Streams
Authors:
Fábio Pinto,
Marco O. P. Sampaio,
Pedro Bizarro
Abstract:
Detecting concept drift is a well known problem that affects production systems. However, two important issues that are frequently not addressed in the literature are 1) the detection of drift when the labels are not immediately available; and 2) the automatic generation of explanations to identify possible causes for the drift. For example, a fraud detection model in online payments could show a…
▽ More
Detecting concept drift is a well known problem that affects production systems. However, two important issues that are frequently not addressed in the literature are 1) the detection of drift when the labels are not immediately available; and 2) the automatic generation of explanations to identify possible causes for the drift. For example, a fraud detection model in online payments could show a drift due to a hot sale item (with an increase in false positives) or due to a true fraud attack (with an increase in false negatives) before labels are available. In this paper we propose SAMM, an automatic model monitoring system for data streams. SAMM detects concept drift using a time and space efficient unsupervised streaming algorithm and it generates alarm reports with a summary of the events and features that are important to explain it. SAMM was evaluated in five real world fraud detection datasets, each spanning periods up to eight months and totaling more than 22 million online transactions. We evaluated SAMM using human feedback from domain experts, by sending them 100 reports generated by the system. Our results show that SAMM is able to detect anomalous events in a model life cycle that are considered useful by the domain experts. Given these results, SAMM will be rolled out in a next version of Feedzai's Fraud Detection solution.
△ Less
Submitted 12 August, 2019;
originally announced August 2019.
-
A Framework for Analyzing Fog-Cloud Computing Cooperation Applied to Information Processing of UAVs
Authors:
Milena F. Pinto,
André L. M. Marcato,
Aurélio G. Melo,
Leonardo M. Honório,
Cristina Urdiales
Abstract:
Unmanned aerial vehicles (UAVs) are a relatively new technology. Their application can often involve complex and unseen problems. For instance, they can work in a cooperative-based environment under the supervision of a ground station to speed up critical decision-making processes. However, the amount of information exchanged among the aircraft and ground station is limited by high distances, low…
▽ More
Unmanned aerial vehicles (UAVs) are a relatively new technology. Their application can often involve complex and unseen problems. For instance, they can work in a cooperative-based environment under the supervision of a ground station to speed up critical decision-making processes. However, the amount of information exchanged among the aircraft and ground station is limited by high distances, low bandwidth size, restricted processing capability, and energy constraints. These drawbacks restrain large-scale operations such as large area inspections. New distributed state-of-the-art processing architectures, such as fog computing, can improve latency, scalability, and efficiency to meet time constraints via data acquisition, processing, and storage at different levels. Under these amendments, this research work proposes a mathematical model to analyze distribution-based UAVs topologies and a fog-cloud computing framework for large-scale mission and search operations. The tests have successfully predicted latency and other operational constraints, allowing the analysis of fog-computing advantages over traditional cloud-computing architectures.
△ Less
Submitted 10 January, 2019;
originally announced January 2019.
-
Multi-Step Knowledge-Aided Iterative ESPRIT for Direction Finding
Authors:
S. F. B. Pinto,
R. C. de Lamare
Abstract:
In this work, we propose a subspace-based algorithm for DOA estimation which iteratively reduces the disturbance factors of the estimated data covariance matrix and incorporates prior knowledge which is gradually obtained on line. An analysis of the MSE of the reshaped data covariance matrix is carried out along with comparisons between computational complexities of the proposed and existing algor…
▽ More
In this work, we propose a subspace-based algorithm for DOA estimation which iteratively reduces the disturbance factors of the estimated data covariance matrix and incorporates prior knowledge which is gradually obtained on line. An analysis of the MSE of the reshaped data covariance matrix is carried out along with comparisons between computational complexities of the proposed and existing algorithms. Simulations focusing on closely-spaced sources, where they are uncorrelated and correlated, illustrate the improvements achieved.
△ Less
Submitted 30 April, 2018;
originally announced May 2018.
-
autoBagging: Learning to Rank Bagging Workflows with Metalearning
Authors:
Fábio Pinto,
Vítor Cerqueira,
Carlos Soares,
João Mendes-Moreira
Abstract:
Machine Learning (ML) has been successfully applied to a wide range of domains and applications. One of the techniques behind most of these successful applications is Ensemble Learning (EL), the field of ML that gave birth to methods such as Random Forests or Boosting. The complexity of applying these techniques together with the market scarcity on ML experts, has created the need for systems that…
▽ More
Machine Learning (ML) has been successfully applied to a wide range of domains and applications. One of the techniques behind most of these successful applications is Ensemble Learning (EL), the field of ML that gave birth to methods such as Random Forests or Boosting. The complexity of applying these techniques together with the market scarcity on ML experts, has created the need for systems that enable a fast and easy drop-in replacement for ML libraries. Automated machine learning (autoML) is the field of ML that attempts to answers these needs. Typically, these systems rely on optimization techniques such as bayesian optimization to lead the search for the best model. Our approach differs from these systems by making use of the most recent advances on metalearning and a learning to rank approach to learn from metadata. We propose autoBagging, an autoML system that automatically ranks 63 bagging workflows by exploiting past performance and dataset characterization. Results on 140 classification datasets from the OpenML platform show that autoBagging can yield better performance than the Average Rank method and achieve results that are not statistically different from an ideal model that systematically selects the best workflow for each dataset. For the purpose of reproducibility and generalizability, autoBagging is publicly available as an R package on CRAN.
△ Less
Submitted 28 June, 2017;
originally announced June 2017.
-
Survey of Inter-satellite Communication for Small Satellite Systems: Physical Layer to Network Layer View
Authors:
Radhika Radhakrishnan,
William Edmonson,
Fatemeh Afghah,
R. Rodriguez-Osorio,
Frank Pinto,
Scott Burleigh
Abstract:
Small satellite systems enable whole new class of missions for navigation, communications, remote sensing and scientific research for both civilian and military purposes. As individual spacecraft are limited by the size, mass and power constraints, mass-produced small satellites in large constellations or clusters could be useful in many science missions such as gravity mapping, tracking of forest…
▽ More
Small satellite systems enable whole new class of missions for navigation, communications, remote sensing and scientific research for both civilian and military purposes. As individual spacecraft are limited by the size, mass and power constraints, mass-produced small satellites in large constellations or clusters could be useful in many science missions such as gravity mapping, tracking of forest fires, finding water resources, etc. Constellation of satellites provide improved spatial and temporal resolution of the target. Small satellite constellations contribute innovative applications by replacing a single asset with several very capable spacecraft which opens the door to new applications. With increasing levels of autonomy, there will be a need for remote communication networks to enable communication between spacecraft. These space based networks will need to configure and maintain dynamic routes, manage intermediate nodes, and reconfigure themselves to achieve mission objectives. Hence, inter-satellite communication is a key aspect when satellites fly in formation. In this paper, we present the various researches being conducted in the small satellite community for implementing inter-satellite communications based on the Open System Interconnection (OSI) model. This paper also reviews the various design parameters applicable to the first three layers of the OSI model, i.e., physical, data link and network layer. Based on the survey, we also present a comprehensive list of design parameters useful for achieving inter-satellite communications for multiple small satellite missions. Specific topics include proposed solutions for some of the challenges faced by small satellite systems, enabling operations using a network of small satellites, and some examples of small satellite missions involving formation flying aspects.
△ Less
Submitted 27 September, 2016; v1 submitted 27 September, 2016;
originally announced September 2016.
-
Latent Dirichlet Allocation Uncovers Spectral Characteristics of Drought Stressed Plants
Authors:
Mirwaes Wahabzada,
Kristian Kersting,
Christian Bauckhage,
Christoph Roemer,
Agim Ballvora,
Francisco Pinto,
Uwe Rascher,
Jens Leon,
Lutz Ploemer
Abstract:
Understanding the adaptation process of plants to drought stress is essential in improving management practices, breeding strategies as well as engineering viable crops for a sustainable agriculture in the coming decades. Hyper-spectral imaging provides a particularly promising approach to gain such understanding since it allows to discover non-destructively spectral characteristics of plants gove…
▽ More
Understanding the adaptation process of plants to drought stress is essential in improving management practices, breeding strategies as well as engineering viable crops for a sustainable agriculture in the coming decades. Hyper-spectral imaging provides a particularly promising approach to gain such understanding since it allows to discover non-destructively spectral characteristics of plants governed primarily by scattering and absorption characteristics of the leaf internal structure and biochemical constituents. Several drought stress indices have been derived using hyper-spectral imaging. However, they are typically based on few hyper-spectral images only, rely on interpretations of experts, and consider few wavelengths only. In this study, we present the first data-driven approach to discovering spectral drought stress indices, treating it as an unsupervised labeling problem at massive scale. To make use of short range dependencies of spectral wavelengths, we develop an online variational Bayes algorithm for latent Dirichlet allocation with convolved Dirichlet regularizer. This approach scales to massive datasets and, hence, provides a more objective complement to plant physiological practices. The spectral topics found conform to plant physiological knowledge and can be computed in a fraction of the time compared to existing LDA approaches.
△ Less
Submitted 16 October, 2012;
originally announced October 2012.
