Showing 1–50 of 72 results for author: Shen, L

Segmenting Medical Images with Limited Data

Authors: Zhaoshan Liua, Qiujie Lv, Chau Hung Lee, Lei Shen

Abstract: While computer vision has proven valuable for medical image segmentation, its application faces challenges such as limited dataset sizes and the complexity of effectively leveraging unlabeled images. To address these challenges, we present a novel semi-supervised, consistency-based approach termed the data-efficient medical segmenter (DEMS). The DEMS features an encoder-decoder architecture and in… ▽ More While computer vision has proven valuable for medical image segmentation, its application faces challenges such as limited dataset sizes and the complexity of effectively leveraging unlabeled images. To address these challenges, we present a novel semi-supervised, consistency-based approach termed the data-efficient medical segmenter (DEMS). The DEMS features an encoder-decoder architecture and incorporates the developed online automatic augmenter (OAA) and residual robustness enhancement (RRE) blocks. The OAA augments input data with various image transformations, thereby diversifying the dataset to improve the generalization ability. The RRE enriches feature diversity and introduces perturbations to create varied inputs for different decoders, thereby providing enhanced variability. Moreover, we introduce a sensitive loss to further enhance consistency across different decoders and stabilize the training process. Extensive experimental results on both our own and three public datasets affirm the effectiveness of DEMS. Under extreme data shortage scenarios, our DEMS achieves 16.85\% and 10.37\% improvement in dice score compared with the U-Net and top-performed state-of-the-art method, respectively. Given its superior data efficiency, DEMS could present significant advancements in medical segmentation under small data regimes. The project homepage can be accessed at https://github.com/NUS-Tim/DEMS. △ Less

Submitted 12 July, 2024; originally announced July 2024.

Comments: Neural Networks Accepted

Global Spatial-Temporal Information-based Residual ConvLSTM for Video Space-Time Super-Resolution

Authors: Congrui Fu, Hui Yuan, Shiqi Jiang, Guanghui Zhang, Liquan Shen, Raouf Hamzaoui

Abstract: By converting low-frame-rate, low-resolution videos into high-frame-rate, high-resolution ones, space-time video super-resolution techniques can enhance visual experiences and facilitate more efficient information dissemination. We propose a convolutional neural network (CNN) for space-time video super-resolution, namely GIRNet. To generate highly accurate features and thus improve performance, th… ▽ More By converting low-frame-rate, low-resolution videos into high-frame-rate, high-resolution ones, space-time video super-resolution techniques can enhance visual experiences and facilitate more efficient information dissemination. We propose a convolutional neural network (CNN) for space-time video super-resolution, namely GIRNet. To generate highly accurate features and thus improve performance, the proposed network integrates a feature-level temporal interpolation module with deformable convolutions and a global spatial-temporal information-based residual convolutional long short-term memory (convLSTM) module. In the feature-level temporal interpolation module, we leverage deformable convolution, which adapts to deformations and scale variations of objects across different scene locations. This presents a more efficient solution than conventional convolution for extracting features from moving objects. Our network effectively uses forward and backward feature information to determine inter-frame offsets, leading to the direct generation of interpolated frame features. In the global spatial-temporal information-based residual convLSTM module, the first convLSTM is used to derive global spatial-temporal information from the input features, and the second convLSTM uses the previously computed global spatial-temporal information feature as its initial cell state. This second convLSTM adopts residual connections to preserve spatial information, thereby enhancing the output features. Experiments on the Vimeo90K dataset show that the proposed method outperforms state-of-the-art techniques in peak signal-to-noise-ratio (by 1.45 dB, 1.14 dB, and 0.02 dB over STARnet, TMNet, and 3DAttGAN, respectively), structural similarity index(by 0.027, 0.023, and 0.006 over STARnet, TMNet, and 3DAttGAN, respectively), and visually. △ Less

Submitted 11 July, 2024; originally announced July 2024.

ImageFlowNet: Forecasting Multiscale Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images

Authors: Chen Liu, Ke Xu, Liangbo L. Shen, Guillaume Huguet, Zilong Wang, Alexander Tong, Danilo Bzdok, Jay Stewart, Jay C. Wang, Lucian V. Del Priore, Smita Krishnaswamy

Abstract: The forecasting of disease progression from images is a holy grail for clinical decision making. However, this task is complicated by the inherent high dimensionality, temporal sparsity and sampling irregularity in longitudinal image acquisitions. Existing methods often rely on extracting hand-crafted features and performing time-series analysis in this vector space, leading to a loss of rich spat… ▽ More The forecasting of disease progression from images is a holy grail for clinical decision making. However, this task is complicated by the inherent high dimensionality, temporal sparsity and sampling irregularity in longitudinal image acquisitions. Existing methods often rely on extracting hand-crafted features and performing time-series analysis in this vector space, leading to a loss of rich spatial information within the images. To overcome these challenges, we introduce ImageFlowNet, a novel framework that learns latent-space flow fields that evolve multiscale representations in joint embedding spaces using neural ODEs and SDEs to model disease progression in the image domain. Notably, ImageFlowNet learns multiscale joint representation spaces by combining cohorts of patients together so that information can be transferred between the patient samples. The dynamics then provide plausible trajectories of progression, with the SDE providing alternative trajectories from the same starting point. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features, latent space organization, and trajectory smoothness. We then demonstrate ImageFlowNet's effectiveness through empirical evaluations on three longitudinal medical image datasets depicting progression in retinal geographic atrophy, multiple sclerosis, and glioblastoma. △ Less

Submitted 12 July, 2024; v1 submitted 20 June, 2024; originally announced June 2024.

Comments: Fixed some typos. Merged multibib

Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions

Authors: Anfeng Xu, Kevin Huang, Tiantian Feng, Lue Shen, Helen Tager-Flusberg, Shrikanth Narayanan

Abstract: Speech foundation models, trained on vast datasets, have opened unique opportunities in addressing challenging low-resource speech understanding, such as child speech. In this work, we explore the capabilities of speech foundation models on child-adult speaker diarization. We show that exemplary foundation models can achieve 39.5% and 62.3% relative reductions in Diarization Error Rate and Speaker… ▽ More Speech foundation models, trained on vast datasets, have opened unique opportunities in addressing challenging low-resource speech understanding, such as child speech. In this work, we explore the capabilities of speech foundation models on child-adult speaker diarization. We show that exemplary foundation models can achieve 39.5% and 62.3% relative reductions in Diarization Error Rate and Speaker Confusion Rate, respectively, compared to previous speaker diarization methods. In addition, we benchmark and evaluate the speaker diarization results of the speech foundation models with varying the input audio window size, speaker demographics, and training data ratio. Our results highlight promising pathways for understanding and adopting speech foundation models to facilitate child speech understanding. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: Interspeech 2024

Subject-Adaptive Transfer Learning Using Resting State EEG Signals for Cross-Subject EEG Motor Imagery Classification

Authors: Sion An, Myeongkyun Kang, Soopil Kim, Philip Chikontwe, Li Shen, Sang Hyun Park

Abstract: Electroencephalography (EEG) motor imagery (MI) classification is a fundamental, yet challenging task due to the variation of signals between individuals i.e., inter-subject variability. Previous approaches try to mitigate this using task-specific (TS) EEG signals from the target subject in training. However, recording TS EEG signals requires time and limits its applicability in various fields. In… ▽ More Electroencephalography (EEG) motor imagery (MI) classification is a fundamental, yet challenging task due to the variation of signals between individuals i.e., inter-subject variability. Previous approaches try to mitigate this using task-specific (TS) EEG signals from the target subject in training. However, recording TS EEG signals requires time and limits its applicability in various fields. In contrast, resting state (RS) EEG signals are a viable alternative due to ease of acquisition with rich subject information. In this paper, we propose a novel subject-adaptive transfer learning strategy that utilizes RS EEG signals to adapt models on unseen subject data. Specifically, we disentangle extracted features into task- and subject-dependent features and use them to calibrate RS EEG signals for obtaining task information while preserving subject characteristics. The calibrated signals are then used to adapt the model to the target subject, enabling the model to simulate processing TS EEG signals of the target subject. The proposed method achieves state-of-the-art accuracy on three public benchmarks, demonstrating the effectiveness of our method in cross-subject EEG MI classification. Our findings highlight the potential of leveraging RS EEG signals to advance practical brain-computer interface systems. The code is available at https://github.com/SionAn/MICCAI2024-ResTL. △ Less

Submitted 9 July, 2024; v1 submitted 17 May, 2024; originally announced May 2024.

Comments: Early Accepted at MICCAI 2024

QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge

Authors: Hongwei Bran Li, Fernando Navarro, Ivan Ezhov, Amirhossein Bayat, Dhritiman Das, Florian Kofler, Suprosanna Shit, Diana Waldmannstetter, Johannes C. Paetzold, Xiaobin Hu, Benedikt Wiestler, Lucas Zimmer, Tamaz Amiranashvili, Chinmay Prabhakar, Christoph Berger, Jonas Weidner, Michelle Alonso-Basant, Arif Rashid, Ujjwal Baid, Wesam Adel, Deniz Ali, Bhakti Baheti, Yingbin Bai, Ishaan Bhatt, Sabri Can Cetindag , et al. (55 additional authors not shown)

Abstract: Uncertainty in medical image segmentation tasks, especially inter-rater variability, arising from differences in interpretations and annotations by various experts, presents a significant challenge in achieving consistent and reliable image segmentation. This variability not only reflects the inherent complexity and subjective nature of medical image interpretation but also directly impacts the de… ▽ More Uncertainty in medical image segmentation tasks, especially inter-rater variability, arising from differences in interpretations and annotations by various experts, presents a significant challenge in achieving consistent and reliable image segmentation. This variability not only reflects the inherent complexity and subjective nature of medical image interpretation but also directly impacts the development and evaluation of automated segmentation algorithms. Accurately modeling and quantifying this variability is essential for enhancing the robustness and clinical applicability of these algorithms. We report the set-up and summarize the benchmark results of the Quantification of Uncertainties in Biomedical Image Quantification Challenge (QUBIQ), which was organized in conjunction with International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2020 and 2021. The challenge focuses on the uncertainty quantification of medical image segmentation which considers the omnipresence of inter-rater variability in imaging datasets. The large collection of images with multi-rater annotations features various modalities such as MRI and CT; various organs such as the brain, prostate, kidney, and pancreas; and different image dimensions 2D-vs-3D. A total of 24 teams submitted different solutions to the problem, combining various baseline models, Bayesian neural networks, and ensemble model techniques. The obtained results indicate the importance of the ensemble models, as well as the need for further research to develop efficient 3D methods for uncertainty quantification methods in 3D segmentation tasks. △ Less

Submitted 24 June, 2024; v1 submitted 19 March, 2024; originally announced May 2024.

Comments: initial technical report

Multi-Dataset Multi-Task Learning for COVID-19 Prognosis

Authors: Filippo Ruffini, Lorenzo Tronchin, Zhuoru Wu, Wenting Chen, Paolo Soda, Linlin Shen, Valerio Guarrasi

Abstract: In the fight against the COVID-19 pandemic, leveraging artificial intelligence to predict disease outcomes from chest radiographic images represents a significant scientific aim. The challenge, however, lies in the scarcity of large, labeled datasets with compatible tasks for training deep learning models without leading to overfitting. Addressing this issue, we introduce a novel multi-dataset mul… ▽ More In the fight against the COVID-19 pandemic, leveraging artificial intelligence to predict disease outcomes from chest radiographic images represents a significant scientific aim. The challenge, however, lies in the scarcity of large, labeled datasets with compatible tasks for training deep learning models without leading to overfitting. Addressing this issue, we introduce a novel multi-dataset multi-task training framework that predicts COVID-19 prognostic outcomes from chest X-rays (CXR) by integrating correlated datasets from disparate sources, distant from conventional multi-task learning approaches, which rely on datasets with multiple and correlated labeling schemes. Our framework hypothesizes that assessing severity scores enhances the model's ability to classify prognostic severity groups, thereby improving its robustness and predictive power. The proposed architecture comprises a deep convolutional network that receives inputs from two publicly available CXR datasets, AIforCOVID for severity prognostic prediction and BRIXIA for severity score assessment, and branches into task-specific fully connected output networks. Moreover, we propose a multi-task loss function, incorporating an indicator function, to exploit multi-dataset integration. The effectiveness and robustness of the proposed approach are demonstrated through significant performance improvements in prognosis classification tasks across 18 different convolutional neural network backbones in different evaluation strategies. This improvement is evident over single-task baselines and standard transfer learning strategies, supported by extensive statistical analysis, showing great application potential. △ Less

Submitted 22 May, 2024; originally announced May 2024.

Interference Cancellation for UWA Random Access Data Packet Transmission

Authors: Yuriy Zakharov, Lu Shen, Benjamin Henson, Nils Morozs, Paul D. Mitchell

Abstract: In underwater acoustic (UWA) random access communication networks with multiple users and data packet transmissions, the packet collisions are the main cause of the network performance degradation. The aim of this paper is to investigate interference cancellation (IC) techniques capable of resolving such collisions in a low-complexity modem with single-carrier modulation and single transducer. Mor… ▽ More In underwater acoustic (UWA) random access communication networks with multiple users and data packet transmissions, the packet collisions are the main cause of the network performance degradation. The aim of this paper is to investigate interference cancellation (IC) techniques capable of resolving such collisions in a low-complexity modem with single-carrier modulation and single transducer. More specifically, in this modem, the IC is used at multiple stages of the receiver. Firstly, the IC is performed for cancelling the multipath interference to improve the equalization performance in comparison with the linear equalization and Rake combining. Secondly, the IC removes the interference from collided data packets within extracted signal segments after identifying the collisions. Finally, the IC is applied to the received baseband signal to improve the data packet detection. The modem performance is investigated in a lake experiment with intensive multipath channels. The experimental results demonstrate high detection performance of the proposed modem design and show that the proposed IC techniques can significantly improve the throughput of random access UWA networks. △ Less

Submitted 18 January, 2024; originally announced January 2024.

Comments: 13 pages, 13 figures

Performance Evaluation of a Full-Duplex UWA System in Lake Experiments

Authors: Lu Shen, Benjamin Henson, Long Shi, Yuriy Zakharov

Abstract: In this work we present a full-duplex (FD) underwater acoustic (UWA) communication system simultaneously transmitting and receiving acoustic signals in the same frequency bandwidth. To simplify the FD hardware, the system exploits a recently designed transducer capable of simultaneously transmitting and receiving signals. The key challenge of implementing an FD system is to cancel at the near-end… ▽ More In this work we present a full-duplex (FD) underwater acoustic (UWA) communication system simultaneously transmitting and receiving acoustic signals in the same frequency bandwidth. To simplify the FD hardware, the system exploits a recently designed transducer capable of simultaneously transmitting and receiving signals. The key challenge of implementing an FD system is to cancel at the near-end receiver the strong self-interference (SI) from the near-end transmitter. By using advanced adaptive filtering algorithms providing high accuracy channel estimates, a high level of SI cancellation can be achieved when the far-end signal is absent. However, the SI channel estimation performance is limited in FD scenarios since the far-end signal acts as an interference when estimating the near-end SI channel. In this paper, we propose an FD UWA communication system which alternates between the SI cancellation and far-end data demodulation. An adaptive Rake combiner with multipath interference cancellation is implemented to improve the demodulation performance in time-varying multipath channels. The performance of the FD UWA system is evaluated in lake experiments. It is shown that the proposed adaptive Rake combiner with multipath interference cancellation significantly outperforms the conventional Rake combiner in all the experiments. The experimental results demonstrate that, with the new Rake combiner, the detection performance of the proposed FD UWA system is comparable with that of the half-duplex system. △ Less

Submitted 18 January, 2024; originally announced January 2024.

Comments: 9 pages, 15 figures

Target detection using underwater acoustic communication links

Authors: Lu Shen, Yuriy Zakharov, Benjamin Henson, Nils Morozs, Benoît Parrein, Paul D. Mitchell

Abstract: Underwater monitoring and surveillance systems are essential for underwater target detection, localization and classification. The aim of this work is to investigate the possibility of target detection by using data transmission between communication nodes in an underwater acoustic (UWA) network, i.e, re-using acoustic communication signals for target detection. A new target detection method based… ▽ More Underwater monitoring and surveillance systems are essential for underwater target detection, localization and classification. The aim of this work is to investigate the possibility of target detection by using data transmission between communication nodes in an underwater acoustic (UWA) network, i.e, re-using acoustic communication signals for target detection. A new target detection method based on estimation of the time-varying channel impulse response between the communication transmitter(s) and receiver(s) is proposed and investigated. This is based on a lake experiment and numerical experiments using a simulator developed for modeling the time-varying UWA channel in the presence of a moving target. The proposed detection method provides a clear indication of a target crossing the communication link. A good similarity between results obtained in the numerical and lake experiments is observed. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Comments: 12 pages, 14 figures

AI-Enabled Unmanned Vehicle-Assisted Reconfigurable Intelligent Surfaces: Deployment, Prototyping, Experiments, and Opportunities

Authors: Li-Hsiang Shen, Kai-Ten Feng, Ta-Sung Lee, Yuan-Chun Lin, Shih-Cheng Lin, Chia-Chan Chang, Sheng-Fuh Chang

Abstract: The requirement of wireless data demands is increasingly high as the sixth-generation (6G) technology evolves. Reconfigurable intelligent surface (RIS) is promisingly deemed to be one of 6G techniques for extending service coverage, reducing power consumption, and enhancing spectral efficiency. In this article, we have provided some fundamentals of RIS deployment in theory and hardware perspective… ▽ More The requirement of wireless data demands is increasingly high as the sixth-generation (6G) technology evolves. Reconfigurable intelligent surface (RIS) is promisingly deemed to be one of 6G techniques for extending service coverage, reducing power consumption, and enhancing spectral efficiency. In this article, we have provided some fundamentals of RIS deployment in theory and hardware perspectives as well as utilization of artificial intelligence (AI) and machine learning. We conducted an intelligent deployment of RIS (i-Dris) prototype, including dual-band auto-guided vehicle (AGV) assisted RISs associated with an mmWave base station (BS) and a receiver. The RISs are deployed on the AGV with configured incident/reflection angles. While, both the mmWave BS and receiver are associated with an edge server monitoring downlink packets for obtaining system throughput. We have designed a federated multi-agent reinforcement learning scheme associated with several AGV-RIS agents and sub-agents per AGV-RIS consisting of the deployment of position, height, orientation and elevation angles. The experimental results presented the stationary measurement in different aspects and scenarios. The i-Dris can reach up to 980 Mbps transmission throughput under a bandwidth of 100 MHz with comparably low complexity as well as rapid deployment, which outperforms the other existing works. At last, we highlight some opportunities and future issues in leveraging RIS-empowered wireless communication networks. △ Less

Submitted 6 November, 2023; originally announced November 2023.

Downlink Rate Maximization with Reconfigurable Intelligent Surface Assisted Full-Duplex Transmissions

Authors: Li-Hsiang Shen, Chia-Jou Ku, Kai-Ten Feng

Abstract: Reconfigurable intelligent surfaces (RIS) as an effective technique for intelligently manipulating channel paths through reflection to serve desired users. Full-duplex (FD) systems, enabling simultaneous transmission and reception from a base station (BS), offer the theoretical advantage of doubled spectrum efficiency. However, the presence of strong self-interference (SI) in FD systems significan… ▽ More Reconfigurable intelligent surfaces (RIS) as an effective technique for intelligently manipulating channel paths through reflection to serve desired users. Full-duplex (FD) systems, enabling simultaneous transmission and reception from a base station (BS), offer the theoretical advantage of doubled spectrum efficiency. However, the presence of strong self-interference (SI) in FD systems significantly degrades performance, which can be mitigated by leveraging the capabilities of RIS. In this work, we consider joint BS and RIS beamforming for maximizing the downlink (DL) transmission rate while guaranteeing uplink (UL) rate requirement. We propose an FD-RIS beamforming (FRIS) scheme by adopting penalty convex-concave programming. Simulation results demonstrate the UL/DL rate improvements achieved by considering various levels of imperfect CSI. The proposed FRIS scheme validates their effectiveness across different RIS deployments and RIS/BS configurations. FRIS has achieved the highest rate compared to the other approximation method, conventional beamforming techniques, HD systems, and deployment without RIS. △ Less

Submitted 6 November, 2023; originally announced November 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2306.05693

Random Sampling of Bandlimited Graph Signals from Local Measurements

Authors: Lili Shen, Jun Xian, Cheng Cheng

Abstract: The random sampling on graph signals is one of the fundamental topics in graph signal processing. In this letter, we consider the random sampling of k-bandlimited signals from the local measurements and show that no more than O(klogk) measurements with replacement are sufficient for the accurate and stable recovery of any k-bandlimited graph signals. We propose two random sampling strategies based… ▽ More The random sampling on graph signals is one of the fundamental topics in graph signal processing. In this letter, we consider the random sampling of k-bandlimited signals from the local measurements and show that no more than O(klogk) measurements with replacement are sufficient for the accurate and stable recovery of any k-bandlimited graph signals. We propose two random sampling strategies based on the minimum measurements, i.e., the optimal sampling and the estimated sampling. The geodesic distance between vertices is introduced to design the sampling probability distribution. Numerical experiments are included to show the effectiveness of the proposed methods. △ Less

Submitted 17 October, 2023; originally announced October 2023.

D-STAR: Dual Simultaneously Transmitting and Reflecting Reconfigurable Intelligent Surfaces for Joint Uplink/Downlink Transmission

Authors: Li-Hsiang Shen, Po-Chen Wu, Chia-Jou Ku, Yu-Ting Li, Kai-Ten Feng, Yuanwei Liu, Lajos Hanzo

Abstract: The joint uplink/downlink (JUD) design of simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RIS) is conceived in support of both uplink (UL) and downlink (DL) users. Furthermore, the dual STAR-RISs (D-STAR) concept is conceived as a promising architecture for 360-degree full-plane service coverage, including UL/DL users located between the base station (BS) and t… ▽ More The joint uplink/downlink (JUD) design of simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RIS) is conceived in support of both uplink (UL) and downlink (DL) users. Furthermore, the dual STAR-RISs (D-STAR) concept is conceived as a promising architecture for 360-degree full-plane service coverage, including UL/DL users located between the base station (BS) and the D-STAR as well as beyond. The corresponding regions are termed as primary (P) and secondary (S) regions. Both BS/users exist in the P-region, but only users are located in the S-region. The primary STAR-RIS (STAR-P) plays an important role in terms of tackling the P-region inter-user interference, the self-interference (SI) from the BS and from the reflective as well as refractive UL users imposed on the DL receiver. By contrast, the secondary STAR-RIS (STAR-S) aims for mitigating the S-region interferences. The non-linear and non-convex rate-maximization problem formulated is solved by alternating optimization amongst the decomposed convex sub-problems of the BS beamformer, and the D-STAR amplitude as well as phase shift configurations. We also propose a D-STAR based active beamforming and passive STAR-RIS amplitude/phase (DBAP) optimization scheme to solve the respective sub-problems by Lagrange dual with Dinkelbach's transformation, alternating direction method of multipliers (ADMM) with successive convex approximation (SCA), and penalty convex-concave procedure (PCCP). Our simulation results reveal that the proposed D-STAR architecture outperforms the conventional single RIS, single STAR-RIS, and half-duplex networks. The proposed DBAP of D-STAR outperforms the state-of-the-art solutions found in the open literature for different numbers of quantization levels, geographic deployment, transmit power and for diverse numbers of transmit antennas, patch partitions as well as D-STAR elements. △ Less

Submitted 8 February, 2024; v1 submitted 29 July, 2023; originally announced July 2023.

Comments: Accepted by IEEE TCOM

SALC: Skeleton-Assisted Learning-Based Clustering for Time-Varying Indoor Localization

Authors: An-Hung Hsiao, Li-Hsiang Shen, Chen-Yi Chang, Chun-Jie Chiu, Kai-Ten Feng

Abstract: Wireless indoor localization has attracted significant amount of attention in recent years. Using received signal strength (RSS) obtained from WiFi access points (APs) for establishing fingerprinting database is a widely utilized method in indoor localization. However, the time-variant problem for indoor positioning systems is not well-investigated in existing literature. Compared to conventional… ▽ More Wireless indoor localization has attracted significant amount of attention in recent years. Using received signal strength (RSS) obtained from WiFi access points (APs) for establishing fingerprinting database is a widely utilized method in indoor localization. However, the time-variant problem for indoor positioning systems is not well-investigated in existing literature. Compared to conventional static fingerprinting, the dynamicallyreconstructed database can adapt to a highly-changing environment, which achieves sustainability of localization accuracy. To deal with the time-varying issue, we propose a skeleton-assisted learning-based clustering localization (SALC) system, including RSS-oriented map-assisted clustering (ROMAC), cluster-based online database establishment (CODE), and cluster-scaled location estimation (CsLE). The SALC scheme jointly considers similarities from the skeleton-based shortest path (SSP) and the time-varying RSS measurements across the reference points (RPs). ROMAC clusters RPs into different feature sets and therefore selects suitable monitor points (MPs) for enhancing location estimation. Moreover, the CODE algorithm aims for establishing adaptive fingerprint database to alleviate the timevarying problem. Finally, CsLE is adopted to acquire the target position by leveraging the benefits of clustering information and estimated signal variations in order to rescale the weights fromweighted k-nearest neighbors (WkNN) method. Both simulation and experimental results demonstrate that the proposed SALC system can effectively reconstruct the fingerprint database with an enhanced location estimation accuracy, which outperforms the other existing schemes in the open literature. △ Less

Submitted 14 July, 2023; originally announced July 2023.

MedAugment: Universal Automatic Data Augmentation Plug-in for Medical Image Analysis

Authors: Zhaoshan Liu, Qiujie Lv, Yifan Li, Ziduo Yang, Lei Shen

Abstract: Data augmentation (DA) has been widely leveraged in the realm of computer vision to alleviate the data shortage, whereas the DA in medical image analysis (MIA) faces multiple challenges. The prevalent DA approaches in MIA encompass conventional DA, synthetic DA, and automatic DA. However, the utilization of these approaches poses various challenges such as experience-driven design and intensive co… ▽ More Data augmentation (DA) has been widely leveraged in the realm of computer vision to alleviate the data shortage, whereas the DA in medical image analysis (MIA) faces multiple challenges. The prevalent DA approaches in MIA encompass conventional DA, synthetic DA, and automatic DA. However, the utilization of these approaches poses various challenges such as experience-driven design and intensive computation cost. Here, we propose an efficient and effective automatic DA method termed MedAugment. We propose the pixel augmentation space and spatial augmentation space and exclude the operations that can break the details and features within medical images. Besides, we propose a novel sampling strategy by sampling a limited number of operations from the two spaces. Moreover, we present a hyperparameter mapping relationship to produce a rational augmentation level and make the MedAugment fully controllable using a single hyperparameter. These revisions address the differences between natural and medical images. Extensive experimental results on four classification and three segmentation datasets demonstrate the superiority of MedAugment. We posit that the plug-and-use and training-free MedAugment holds the potential to make a valuable contribution to the medical field, particularly benefiting medical experts lacking foundational expertise in deep learning. Code is available at https://github.com/NUS-Tim/MedAugment. △ Less

Submitted 7 November, 2023; v1 submitted 30 June, 2023; originally announced June 2023.

Comments: 26 pages, 10 figures

Distributed Localization and Tracking Control for Nonholonomic Agents with Time-varying Bearing Formation

Authors: Huiming Li, Hao Chen, Xiangke Wang, Mengge Zhang, Lincheng Shen

Abstract: This paper studies the bearing-based time-varying formation control problem for unicycle-type agents without bearing rigidity conditions. In the considered problem, only a small set of agents, named as anchors, can obtain their global positions, and the other agents only have access to the bearing information relative to their neighbors. To address the problem, we propose a novel scheme integratin… ▽ More This paper studies the bearing-based time-varying formation control problem for unicycle-type agents without bearing rigidity conditions. In the considered problem, only a small set of agents, named as anchors, can obtain their global positions, and the other agents only have access to the bearing information relative to their neighbors. To address the problem, we propose a novel scheme integrating the distributed localization algorithm and the observer-based formation tracking controller. The designed localization algorithm estimates the global position by using inter-agent bearing measurements, and the observer-based controller tracks the desired formation with the estimated positions. A key distinction of our approach is extending the localization-and-tracking control scheme to the bearing-based coordination of nonholonomic systems, where the desired inter-agent bearings can be time-varying, instead of the constant ones assumed in most of the existing researches. The asymptotic stability of the coupled localization-and-tracking control system is proved, and simulations are carried out to validate the theoretical analysis. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Comments: Submitted to the 62nd IEEE Conference on Decision and Control

Robust Active and Passive Beamforming for RIS-Assisted Full-Duplex Systems under Imperfect CSI

Authors: Li-Hsiang Shen, Chia-Jou Ku, Kai-Ten Feng

Abstract: The sixth-generation (6G) wireless technology recognizes the potential of reconfigurable intelligent surfaces (RIS) as an effective technique for intelligently manipulating channel paths through reflection to serve desired users. Full-duplex (FD) systems, enabling simultaneous transmission and reception from a base station (BS), offer the theoretical advantage of doubled spectrum efficiency. Howev… ▽ More The sixth-generation (6G) wireless technology recognizes the potential of reconfigurable intelligent surfaces (RIS) as an effective technique for intelligently manipulating channel paths through reflection to serve desired users. Full-duplex (FD) systems, enabling simultaneous transmission and reception from a base station (BS), offer the theoretical advantage of doubled spectrum efficiency. However, the presence of strong self-interference (SI) in FD systems significantly degrades performance, which can be mitigated by leveraging the capabilities of RIS. Moreover, accurately obtaining channel state information (CSI) from RIS poses a critical challenge. Our objective is to maximize downlink (DL) user data rates while ensuring quality-of-service (QoS) for uplink (UL) users under imperfect CSI from reflected channels. To address this, we propose a robust active BS and passive RIS beamforming (RAPB) scheme for RIS-FD, accounting for both SI and imperfect CSI. RAPB incorporates distributionally robust design, conditional value-at-risk (CVaR), and penalty convex-concave programming (PCCP) techniques. Simulation results demonstrate the UL/DL rate improvement are achieved by considering different levels of imperfect CSI. The proposed RAPB schemes validate their effectiveness across different RIS deployments and RIS/BS configurations. Benefited from robust beamforming, RAPB outperforms the existing methods in terms of non-robustness, deployment without RIS, conventional approximation, and half-duplex systems. △ Less

Submitted 19 November, 2023; v1 submitted 9 June, 2023; originally announced June 2023.

Comments: some errors found

Incomplete Multimodal Learning for Complex Brain Disorders Prediction

Authors: Reza Shirkavand, Liang Zhan, Heng Huang, Li Shen, Paul M. Thompson

Abstract: Recent advancements in the acquisition of various brain data sources have created new opportunities for integrating multimodal brain data to assist in early detection of complex brain disorders. However, current data integration approaches typically need a complete set of biomedical data modalities, which may not always be feasible, as some modalities are only available in large-scale research coh… ▽ More Recent advancements in the acquisition of various brain data sources have created new opportunities for integrating multimodal brain data to assist in early detection of complex brain disorders. However, current data integration approaches typically need a complete set of biomedical data modalities, which may not always be feasible, as some modalities are only available in large-scale research cohorts and are prohibitive to collect in routine clinical practice. Especially in studies of brain diseases, research cohorts may include both neuroimaging data and genetic data, but for practical clinical diagnosis, we often need to make disease predictions only based on neuroimages. As a result, it is desired to design machine learning models which can use all available data (different data could provide complementary information) during training but conduct inference using only the most common data modality. We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks to effectively exploit auxiliary modalities available during training in order to improve the performance of a unimodal model at inference. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. Experimental results demonstrate that our approach outperforms the related machine learning and deep learning methods by a significant margin. △ Less

Submitted 25 May, 2023; originally announced May 2023.

Understanding Spoken Language Development of Children with ASD Using Pre-trained Speech Embeddings

Authors: Anfeng Xu, Rajat Hebbar, Rimita Lahiri, Tiantian Feng, Lindsay Butler, Lue Shen, Helen Tager-Flusberg, Shrikanth Narayanan

Abstract: Speech processing techniques are useful for analyzing speech and language development in children with Autism Spectrum Disorder (ASD), who are often varied and delayed in acquiring these skills. Early identification and intervention are crucial, but traditional assessment methodologies such as caregiver reports are not adequate for the requisite behavioral phenotyping. Natural Language Sample (NLS… ▽ More Speech processing techniques are useful for analyzing speech and language development in children with Autism Spectrum Disorder (ASD), who are often varied and delayed in acquiring these skills. Early identification and intervention are crucial, but traditional assessment methodologies such as caregiver reports are not adequate for the requisite behavioral phenotyping. Natural Language Sample (NLS) analysis has gained attention as a promising complement. Researchers have developed benchmarks for spoken language capabilities in children with ASD, obtainable through the analysis of NLS. This paper proposes applications of speech processing technologies in support of automated assessment of children's spoken language development by classification between child and adult speech and between speech and nonverbal vocalization in NLS, with respective F1 macro scores of 82.6% and 67.8%, underscoring the potential for accurate and scalable tools for ASD research and clinical use. △ Less

Submitted 31 May, 2023; v1 submitted 23 May, 2023; originally announced May 2023.

Comments: Accepted to Interspeech 2023, 5 pages

Poisson-Gaussian Holographic Phase Retrieval with Score-based Image Prior

Authors: Zongyu Li, Jason Hu, Xiaojian Xu, Liyue Shen, Jeffrey A. Fessler

Abstract: Phase retrieval (PR) is a crucial problem in many imaging applications. This study focuses on resolving the holographic phase retrieval problem in situations where the measurements are affected by a combination of Poisson and Gaussian noise, which commonly occurs in optical imaging systems. To address this problem, we propose a new algorithm called "AWFS" that uses the accelerated Wirtinger flow (… ▽ More Phase retrieval (PR) is a crucial problem in many imaging applications. This study focuses on resolving the holographic phase retrieval problem in situations where the measurements are affected by a combination of Poisson and Gaussian noise, which commonly occurs in optical imaging systems. To address this problem, we propose a new algorithm called "AWFS" that uses the accelerated Wirtinger flow (AWF) with a score function as generative prior. Specifically, we formulate the PR problem as an optimization problem that incorporates both data fidelity and regularization terms. We calculate the gradient of the log-likelihood function for PR and determine its corresponding Lipschitz constant. Additionally, we introduce a generative prior in our regularization framework by using score matching to capture information about the gradient of image prior distributions. We provide theoretical analysis that establishes a critical-point convergence guarantee for the proposed algorithm. The results of our simulation experiments on three different datasets show the following: 1) By using the PG likelihood model, the proposed algorithm improves reconstruction compared to algorithms based solely on Gaussian or Poisson likelihood. 2) The proposed score-based image prior method, performs better than the method based on denoising diffusion probabilistic model (DDPM), as well as plug-and-play alternating direction method of multipliers (PnP-ADMM) and regularization by denoising (RED). △ Less

Submitted 20 September, 2023; v1 submitted 12 May, 2023; originally announced May 2023.

Multi-Modality Deep Network for Extreme Learned Image Compression

Authors: Xuhao Jiang, Weimin Tan, Tian Tan, Bo Yan, Liquan Shen

Abstract: Image-based single-modality compression learning approaches have demonstrated exceptionally powerful encoding and decoding capabilities in the past few years , but suffer from blur and severe semantics loss at extremely low bitrates. To address this issue, we propose a multimodal machine learning method for text-guided image compression, in which the semantic information of text is used as prior i… ▽ More Image-based single-modality compression learning approaches have demonstrated exceptionally powerful encoding and decoding capabilities in the past few years , but suffer from blur and severe semantics loss at extremely low bitrates. To address this issue, we propose a multimodal machine learning method for text-guided image compression, in which the semantic information of text is used as prior information to guide image compression for better compression performance. We fully study the role of text description in different components of the codec, and demonstrate its effectiveness. In addition, we adopt the image-text attention module and image-request complement module to better fuse image and text features, and propose an improved multimodal semantic-consistent loss to produce semantically complete reconstructions. Extensive experiments, including a user study, prove that our method can obtain visually pleasing results at extremely low bitrates, and achieves a comparable or even better performance than state-of-the-art methods, even though these methods are at 2x to 4x bitrates of ours. △ Less

Submitted 26 April, 2023; originally announced April 2023.

Comments: 13 pages, 14 figures, accepted by AAAI 2023

A New Paradigm for Device-free Indoor Localization: Deep Learning with Error Vector Spectrum in Wi-Fi Systems

Authors: Wen Liu, An-Hung Hsiao, Li-Hsiang Shen, Kai-Ten Feng

Abstract: The demand for device-free indoor localization using commercial Wi-Fi devices has rapidly increased in various fields due to its convenience and versatile applications. However, random frequency offset (RFO) in wireless channels poses challenges to the accuracy of indoor localization when using fluctuating channel state information (CSI). To mitigate the RFO problem, an error vector spectrum (EVS)… ▽ More The demand for device-free indoor localization using commercial Wi-Fi devices has rapidly increased in various fields due to its convenience and versatile applications. However, random frequency offset (RFO) in wireless channels poses challenges to the accuracy of indoor localization when using fluctuating channel state information (CSI). To mitigate the RFO problem, an error vector spectrum (EVS) is conceived thanks to its higher resolution of signal and robustness to RFO. To address these challenges, this paper proposed a novel error vector assisted learning (EVAL) for device-free indoor localization. The proposed EVAL scheme employs deep neural networks to classify the location of a person in the indoor environment by extracting ample channel features from the physical layer signals. We conducted realistic experiments based on OpenWiFi project to extract both EVS and CSI to examine the performance of different device-free localization techniques. Experimental results show that our proposed EVAL scheme outperforms conventional machine learning methods and benchmarks utilizing either CSI amplitude or phase information. Compared to most existing CSI-based localization schemes, a new paradigm with higher positioning accuracy by adopting EVS is revealed by our proposed EVAL system. △ Less

Submitted 25 March, 2023; originally announced April 2023.

WiRiS: Transformer for RIS-Assisted Device-Free Sensing for Joint People Counting and Localization using Wi-Fi CSI

Authors: Wei-Yu Chung, Li-Hsiang Shen, Kai-Ten Feng, Yuan-Chun Lin, Shih-Cheng Lin, Sheng-Fuh Chang

Abstract: Channel State Information (CSI) is widely adopted as a feature for indoor localization. Taking advantage of the abundant information from the CSI, people can be accurately sensed even without equipped devices. However, the positioning error increases severely in non-line-of-sight (NLoS) regions. Reconfigurable intelligent surface (RIS) has been introduced to improve signal coverage in NLoS areas,… ▽ More Channel State Information (CSI) is widely adopted as a feature for indoor localization. Taking advantage of the abundant information from the CSI, people can be accurately sensed even without equipped devices. However, the positioning error increases severely in non-line-of-sight (NLoS) regions. Reconfigurable intelligent surface (RIS) has been introduced to improve signal coverage in NLoS areas, which can re-direct and enhance reflective signals with massive meta-material elements. In this paper, we have proposed a Transformer-based RIS-assisted device-free sensing for joint people counting and localization (WiRiS) system to precisely predict the number of people and their corresponding locations through configuring RIS. A series of predefined RIS beams is employed to create inputs of fingerprinting CSI features as sequence-to-sequence learning database for Transformer. We have evaluated the performance of proposed WiRiS system in both ray-tracing simulators and experiments. Both simulation and real-world experiments demonstrate that people counting accuracy exceeds 90\%, and the localization error can achieve the centimeter-level, which outperforms the existing benchmarks without employment of RIS. △ Less

Submitted 9 November, 2023; v1 submitted 25 March, 2023; originally announced April 2023.

Attention-based Learning for Sleep Apnea and Limb Movement Detection using Wi-Fi CSI Signals

Authors: Chi-Che Chang, An-Hung Hsiao, Li-Hsiang Shen, Kai-Ten Feng, Chia-Yu Chen

Abstract: Wi-Fi channel state information (CSI) has become a promising solution for non-invasive breathing and body motion monitoring during sleep. Sleep disorders of apnea and periodic limb movement disorder (PLMD) are often unconscious and fatal. The existing researches detect abnormal sleep disorders in impractically controlled environments. Moreover, it leads to compelling challenges to classify complex… ▽ More Wi-Fi channel state information (CSI) has become a promising solution for non-invasive breathing and body motion monitoring during sleep. Sleep disorders of apnea and periodic limb movement disorder (PLMD) are often unconscious and fatal. The existing researches detect abnormal sleep disorders in impractically controlled environments. Moreover, it leads to compelling challenges to classify complex macro- and micro-scales of sleep movements as well as entangled similar waveforms of cases of apnea and PLMD. In this paper, we propose the attention-based learning for sleep apnea and limb movement detection (ALESAL) system that can jointly detect sleep apnea and PLMD under different sleep postures across a variety of patients. ALESAL contains antenna-pair and time attention mechanisms for mitigating the impact of modest antenna pairs and emphasizing the duration of interest, respectively. Performance results show that our proposed ALESAL system can achieve a weighted F1-score of 84.33, outperforming the other existing non-attention based methods of support vector machine and deep multilayer perceptron. △ Less

Submitted 26 March, 2023; originally announced April 2023.

Edge Selection and Clustering for Federated Learning in Optical Inter-LEO Satellite Constellation

Authors: Chih-Yu Chen, Li-Hsiang Shen, Kai-Ten Feng, Lie-Liang Yang, Jen-Ming Wu

Abstract: Low-Earth orbit (LEO) satellites have been prosperously deployed for various Earth observation missions due to its capability of collecting a large amount of image or sensor data. However, traditionally, the data training process is performed in the terrestrial cloud server, which leads to a high transmission overhead. With the recent development of LEO, it is more imperative to provide ultra-dens… ▽ More Low-Earth orbit (LEO) satellites have been prosperously deployed for various Earth observation missions due to its capability of collecting a large amount of image or sensor data. However, traditionally, the data training process is performed in the terrestrial cloud server, which leads to a high transmission overhead. With the recent development of LEO, it is more imperative to provide ultra-dense LEO constellation with enhanced on-board computation capability. Benefited from it, we have proposed a collaborative federated learning for low Earth orbit (FELLO). We allocate the entire process on LEOs with low payload inter-satellite transmissions, whilst the low-delay terrestrial gateway server (GS) only takes care for initial signal controlling. The GS initially selects an LEO server, whereas its LEO clients are all determined by clustering mechanism and communication capability through the optical inter-satellite links (ISLs). The re-clustering of changing LEO server will be executed once with low communication quality of FELLO. In the simulations, we have numerically analyzed the proposed FELLO under practical Walker-based LEO constellation configurations along with MNIST training dataset for classification mission. The proposed FELLO outperforms the conventional centralized and distributed architectures with higher classification accuracy as well as comparably lower latency of joint communication and computing. △ Less

Submitted 10 April, 2023; v1 submitted 25 March, 2023; originally announced March 2023.

Hierarchical Multi-Agent Multi-Armed Bandit for Resource Allocation in Multi-LEO Satellite Constellation Networks

Authors: Li-Hsiang Shen, Yun Ho, Kai-Ten Feng, Lie-Liang Yang, Sau-Hsuan Wu, Jen-Ming Wu

Abstract: Low Earth orbit (LEO) satellite constellation is capable of providing global coverage area with high-rate services in the next sixth-generation (6G) non-terrestrial network (NTN). Due to limited onboard resources of operating power, beams, and channels, resilient and efficient resource management has become compellingly imperative under complex interference cases. However, different from conventio… ▽ More Low Earth orbit (LEO) satellite constellation is capable of providing global coverage area with high-rate services in the next sixth-generation (6G) non-terrestrial network (NTN). Due to limited onboard resources of operating power, beams, and channels, resilient and efficient resource management has become compellingly imperative under complex interference cases. However, different from conventional terrestrial base stations, LEO is deployed at considerable height and under high mobility, inducing substantially long delay and interference during transmission. As a result, acquiring the accurate channel state information between LEOs and ground users is challenging. Therefore, we construct a framework with a two-way transmission under unknown channel information and no data collected at long-delay ground gateway. In this paper, we propose hierarchical multi-agent multi-armed bandit resource allocation for LEO constellation (mmRAL) by appropriately assigning available radio resources. LEOs are considered as collaborative multiple macro-agents attempting unknown trials of various actions of micro-agents of respective resources, asymptotically achieving suitable allocation with only throughput information. In simulations, we evaluate mmRAL in various cases of LEO deployment, serving numbers of users and LEOs, hardware cost and outage probability. Benefited by efficient and resilient allocation, the proposed mmRAL system is capable of operating in homogeneous or heterogeneous orbital planes or constellations, achieving the highest throughput performance compared to the existing benchmarks in open literature. △ Less

Submitted 25 March, 2023; originally announced March 2023.

SGDA: Towards 3D Universal Pulmonary Nodule Detection via Slice Grouped Domain Attention

Authors: Rui Xu, Zhi Liu, Yong Luo, Han Hu, Li Shen, Bo Du, Kaiming Kuang, Jiancheng Yang

Abstract: Lung cancer is the leading cause of cancer death worldwide. The best solution for lung cancer is to diagnose the pulmonary nodules in the early stage, which is usually accomplished with the aid of thoracic computed tomography (CT). As deep learning thrives, convolutional neural networks (CNNs) have been introduced into pulmonary nodule detection to help doctors in this labor-intensive task and dem… ▽ More Lung cancer is the leading cause of cancer death worldwide. The best solution for lung cancer is to diagnose the pulmonary nodules in the early stage, which is usually accomplished with the aid of thoracic computed tomography (CT). As deep learning thrives, convolutional neural networks (CNNs) have been introduced into pulmonary nodule detection to help doctors in this labor-intensive task and demonstrated to be very effective. However, the current pulmonary nodule detection methods are usually domain-specific, and cannot satisfy the requirement of working in diverse real-world scenarios. To address this issue, we propose a slice grouped domain attention (SGDA) module to enhance the generalization capability of the pulmonary nodule detection networks. This attention module works in the axial, coronal, and sagittal directions. In each direction, we divide the input feature into groups, and for each group, we utilize a universal adapter bank to capture the feature subspaces of the domains spanned by all pulmonary nodule datasets. Then the bank outputs are combined from the perspective of domain to modulate the input group. Extensive experiments demonstrate that SGDA enables substantially better multi-domain pulmonary nodule detection performance compared with the state-of-the-art multi-domain learning methods. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Comments: Accepted by IEEE/ACM Transactions on Computational Biology and Bioinformatics

Enhance Local Consistency in Federated Learning: A Multi-Step Inertial Momentum Approach

Authors: Yixing Liu, Yan Sun, Zhengtao Ding, Li Shen, Bo Liu, Dacheng Tao

Abstract: Federated learning (FL), as a collaborative distributed training paradigm with several edge computing devices under the coordination of a centralized server, is plagued by inconsistent local stationary points due to the heterogeneity of the local partial participation clients, which precipitates the local client-drifts problems and sparks off the unstable and slow convergence, especially on the ag… ▽ More Federated learning (FL), as a collaborative distributed training paradigm with several edge computing devices under the coordination of a centralized server, is plagued by inconsistent local stationary points due to the heterogeneity of the local partial participation clients, which precipitates the local client-drifts problems and sparks off the unstable and slow convergence, especially on the aggravated heterogeneous dataset. To address these issues, we propose a novel federated learning algorithm, named FedMIM, which adopts the multi-step inertial momentum on the edge devices and enhances the local consistency for free during the training to improve the robustness of the heterogeneity. Specifically, we incorporate the weighted global gradient estimations as the inertial correction terms to guide both the local iterates and stochastic gradient estimation, which can reckon the global objective optimization on the edges' heterogeneous dataset naturally and maintain the demanding consistent iteration locally. Theoretically, we show that FedMIM achieves the $\mathcal{O}(\frac{1}{\sqrt{SKT}})$ convergence rate with a linear speedup property with respect to the number of selected clients $S$ and proper local interval $K$ in communication round $T$ without convex assumption. Empirically, we conduct comprehensive experiments on various real-world datasets and demonstrate the efficacy of the proposed FedMIM against several state-of-the-art baselines. △ Less

Submitted 11 February, 2023; originally announced February 2023.

CECT: Controllable Ensemble CNN and Transformer for COVID-19 Image Classification

Authors: Zhaoshan Liu, Lei Shen

Abstract: The COVID-19 pandemic has resulted in hundreds of million cases and numerous deaths worldwide. Here, we develop a novel classification network CECT by controllable ensemble convolutional neural network and transformer to provide a timely and accurate COVID-19 diagnosis. The CECT is composed of a parallel convolutional encoder block, an aggregate transposed-convolutional decoder block, and a window… ▽ More The COVID-19 pandemic has resulted in hundreds of million cases and numerous deaths worldwide. Here, we develop a novel classification network CECT by controllable ensemble convolutional neural network and transformer to provide a timely and accurate COVID-19 diagnosis. The CECT is composed of a parallel convolutional encoder block, an aggregate transposed-convolutional decoder block, and a windowed attention classification block. Each block captures features at different scales from 28 $\times$ 28 to 224 $\times$ 224 from the input, composing enriched and comprehensive information. Different from existing methods, our CECT can capture features at both multi-local and global scales without any sophisticated module design. Moreover, the contribution of local features at different scales can be controlled with the proposed ensemble coefficients. We evaluate CECT on two public COVID-19 datasets and it reaches the highest accuracy of 98.1% in the intra-dataset evaluation, outperforming existing state-of-the-art methods. Moreover, the developed CECT achieves an accuracy of 90.9% on the unseen dataset in the inter-dataset evaluation, showing extraordinary generalization ability. With remarkable feature capture ability and generalization ability, we believe CECT can be extended to other medical scenarios as a powerful diagnosis tool. Code is available at https://github.com/NUS-Tim/CECT. △ Less

Submitted 31 March, 2024; v1 submitted 5 February, 2023; originally announced February 2023.

Comments: Computers in Biology and Medicine Accepted

Gene-SGAN: a method for discovering disease subtypes with imaging and genetic signatures via multi-view weakly-supervised deep clustering

Authors: Zhijian Yang, Junhao Wen, Ahmed Abdulkadir, Yuhan Cui, Guray Erus, Elizabeth Mamourian, Randa Melhem, Dhivya Srinivasan, Sindhuja T. Govindarajan, Jiong Chen, Mohamad Habes, Colin L. Masters, Paul Maruff, Jurgen Fripp, Luigi Ferrucci, Marilyn S. Albert, Sterling C. Johnson, John C. Morris, Pamela LaMontagne, Daniel S. Marcus, Tammie L. S. Benzinger, David A. Wolk, Li Shen, Jingxuan Bao, Susan M. Resnick , et al. (3 additional authors not shown)

Abstract: Disease heterogeneity has been a critical challenge for precision diagnosis and treatment, especially in neurologic and neuropsychiatric diseases. Many diseases can display multiple distinct brain phenotypes across individuals, potentially reflecting disease subtypes that can be captured using MRI and machine learning methods. However, biological interpretability and treatment relevance are limite… ▽ More Disease heterogeneity has been a critical challenge for precision diagnosis and treatment, especially in neurologic and neuropsychiatric diseases. Many diseases can display multiple distinct brain phenotypes across individuals, potentially reflecting disease subtypes that can be captured using MRI and machine learning methods. However, biological interpretability and treatment relevance are limited if the derived subtypes are not associated with genetic drivers or susceptibility factors. Herein, we describe Gene-SGAN - a multi-view, weakly-supervised deep clustering method - which dissects disease heterogeneity by jointly considering phenotypic and genetic data, thereby conferring genetic correlations to the disease subtypes and associated endophenotypic signatures. We first validate the generalizability, interpretability, and robustness of Gene-SGAN in semi-synthetic experiments. We then demonstrate its application to real multi-site datasets from 28,858 individuals, deriving subtypes of Alzheimer's disease and brain endophenotypes associated with hypertension, from MRI and SNP data. Derived brain phenotypes displayed significant differences in neuroanatomical patterns, genetic determinants, biological and clinical biomarkers, indicating potentially distinct underlying neuropathologic processes, genetic drivers, and susceptibility factors. Overall, Gene-SGAN is broadly applicable to disease subtyping and endophenotype discovery, and is herein tested on disease-related, genetically-driven neuroimaging phenotypes. △ Less

Submitted 25 January, 2023; originally announced January 2023.

BTS: Bifold Teacher-Student in Semi-Supervised Learning for Indoor Two-Room Presence Detection Under Time-Varying CSI

Authors: Li-Hsiang Shen, Kai-Jui Chen, An-Hung Hsiao, Kai-Ten Feng

Abstract: In recent years, indoor human presence detection based on supervised learning (SL) and channel state information (CSI) has attracted much attention. However, existing studies that rely on spatial information of CSI are susceptible to environmental changes which degrade prediction accuracy. Moreover, SL-based methods require time-consuming data labeling for retraining models. Therefore, it is imper… ▽ More In recent years, indoor human presence detection based on supervised learning (SL) and channel state information (CSI) has attracted much attention. However, existing studies that rely on spatial information of CSI are susceptible to environmental changes which degrade prediction accuracy. Moreover, SL-based methods require time-consuming data labeling for retraining models. Therefore, it is imperative to design a continuously monitored model using a semi-supervised learning (SSL) based scheme. In this paper, we conceive a bifold teacher-student (BTS) learning approach for indoor human presence detection in an adjoining two-room scenario. The proposed SSL-based primal-dual teacher-student network intelligently learns spatial and temporal features from labeled and unlabeled CSI datasets. Additionally, the enhanced penalized loss function leverages entropy and distance measures to distinguish drifted data, i.e., features of new datasets affected by time-varying effects and altered from the original distribution. Experimental results demonstrate that the proposed BTS system sustains asymptotic accuracy after retraining the model with unlabeled data. Furthermore, BTS outperforms existing SSL-based models in terms of the highest detection accuracy while achieving the asymptotic performance of SL-based methods. △ Less

Submitted 6 June, 2023; v1 submitted 21 December, 2022; originally announced December 2022.

Five Facets of 6G: Research Challenges and Opportunities

Authors: Li-Hsiang Shen, Kai-Ten Feng, Lajos Hanzo

Abstract: Whilst the fifth-generation (5G) systems are being rolled out across the globe, researchers have turned their attention to the exploration of radical next-generation solutions. At this early evolutionary stage we survey five main research facets of this field, namely {\em Facet~1: next-generation architectures, spectrum and services, Facet~2: next-generation networking, Facet~3: Internet of Things… ▽ More Whilst the fifth-generation (5G) systems are being rolled out across the globe, researchers have turned their attention to the exploration of radical next-generation solutions. At this early evolutionary stage we survey five main research facets of this field, namely {\em Facet~1: next-generation architectures, spectrum and services, Facet~2: next-generation networking, Facet~3: Internet of Things (IoT), Facet~4: wireless positioning and sensing, as well as Facet~5: applications of deep learning in 6G networks.} In this paper, we have provided a critical appraisal of the literature of promising techniques ranging from the associated architectures, networking, applications as well as designs. We have portrayed a plethora of heterogeneous architectures relying on cooperative hybrid networks supported by diverse access and transmission mechanisms. The vulnerabilities of these techniques are also addressed and carefully considered for highlighting the most of promising future research directions. Additionally, we have listed a rich suite of learning-driven optimization techniques. We conclude by observing the evolutionary paradigm-shift that has taken place from pure single-component bandwidth-efficiency, power-efficiency or delay-optimization towards multi-component designs, as exemplified by the twin-component ultra-reliable low-latency mode of the 5G system. We advocate a further evolutionary step towards multi-component Pareto optimization, which requires the exploration of the entire Pareto front of all optiomal solutions, where none of the components of the objective function may be improved without degrading at least one of the other components. △ Less

Submitted 7 November, 2022; originally announced December 2022.

Journal ref: ACM Computing Surveys, 2023

CRONOS: Colorization and Contrastive Learning for Device-Free NLoS Human Presence Detection using Wi-Fi CSI

Authors: Li-Hsiang Shen, Chia-Che Hsieh, An-Hung Hsiao, Kai-Ten Feng

Abstract: In recent years, the demand for pervasive smart services and applications has increased rapidly. Device-free human detection through sensors or cameras has been widely adopted, but it comes with privacy issues as well as misdetection for motionless people. To address these drawbacks, channel state information (CSI) captured from commercialized Wi-Fi devices provides rich signal features for accura… ▽ More In recent years, the demand for pervasive smart services and applications has increased rapidly. Device-free human detection through sensors or cameras has been widely adopted, but it comes with privacy issues as well as misdetection for motionless people. To address these drawbacks, channel state information (CSI) captured from commercialized Wi-Fi devices provides rich signal features for accurate detection. However, existing systems suffer from inaccurate classification under a non-line-of-sight (NLoS) and stationary scenario, such as when a person is standing still in a room corner. In this work, we propose a system called CRONOS (Colorization and Contrastive Learning Enhanced NLoS Human Presence Detection), which generates dynamic recurrence plots (RPs) and color-coded CSI ratios to distinguish mobile and stationary people from vacancy in a room, respectively. We also incorporate supervised contrastive learning to retrieve substantial representations, where consultation loss is formulated to differentiate the representative distances between dynamic and stationary cases. Furthermore, we propose a self-switched static feature enhanced classifier (S3FEC) to determine the utilization of either RPs or color-coded CSI ratios. Our comprehensive experimental results show that CRONOS outperforms existing systems that either apply machine learning or non-learning based methods, as well as non-CSI based features in open literature. CRONOS achieves the highest human presence detection accuracy in vacancy, mobility, line-of-sight (LoS), and NLoS scenarios. △ Less

Submitted 16 August, 2023; v1 submitted 7 November, 2022; originally announced November 2022.

Comments: Accepted by IEEE IoT-J

MARS: Message Passing for Antenna and RF Chain Selection for Hybrid Beamforming in MIMO Communication Systems

Authors: Li-Hsiang Shen, Yen-Chun Lo, Kai-Ten Feng, Sau-Hsuan Wu, Lie-Liang Yang

Abstract: In this paper, we consider a prospective receiving hybrid beamforming structure consisting of several radio frequency (RF) chains and abundant antenna elements in multi-input multi-output (MIMO) systems. Due to conventional costly full connections, we design an enhanced partially connected beamformer employing a low-density parity-check (LDPC)-based structure. As a benefit of the LDPC-based struct… ▽ More In this paper, we consider a prospective receiving hybrid beamforming structure consisting of several radio frequency (RF) chains and abundant antenna elements in multi-input multi-output (MIMO) systems. Due to conventional costly full connections, we design an enhanced partially connected beamformer employing a low-density parity-check (LDPC)-based structure. As a benefit of the LDPC-based structure, information can be exchanged among clustered RF/antenna groups, which results in a low computational complexity order. Advanced message passing (MP) capable of inferring and transferring information among different paths is designed to support the LDPC-based hybrid beamformer. We propose a message-passing enhanced antenna and RF chain selection (MARS) scheme for minimizing the operational power of antennas and RF chains of the receiver as well as hybrid beamforming. Furthermore, sequential and parallel MP schemes for MARS are designed, namely, MARS-S and MARS-P, respectively, to address the convergence speed issue. A heuristic genetic algorithm is designed for receiving hybrid beamforming, comprising gene generation initialization, elite selection, crossover, and mutation. Simulations validate the convergence of both the MARS-P and the MARS-S algorithms. Due to the asynchronous information transfer of MARS-P, it requires higher power than MARS-S, which strikes a compelling balance among power consumption, convergence, and computational complexity. It is also demonstrated that the proposed MARS scheme outperforms the existing benchmarks using the heuristic method of fully/partially connected architectures in the open literature by requiring the lowest power and realizing the highest energy efficiency. △ Less

Submitted 20 May, 2024; v1 submitted 7 November, 2022; originally announced November 2022.

Comments: Accepted by IEEE TCOM

Recent Progress in Transformer-based Medical Image Analysis

Authors: Zhaoshan Liu, Qiujie Lv, Ziduo Yang, Yifan Li, Chau Hung Lee, Lei Shen

Abstract: The transformer is primarily used in the field of natural language processing. Recently, it has been adopted and shows promise in the computer vision (CV) field. Medical image analysis (MIA), as a critical branch of CV, also greatly benefits from this state-of-the-art technique. In this review, we first recap the core component of the transformer, the attention mechanism, and the detailed structur… ▽ More The transformer is primarily used in the field of natural language processing. Recently, it has been adopted and shows promise in the computer vision (CV) field. Medical image analysis (MIA), as a critical branch of CV, also greatly benefits from this state-of-the-art technique. In this review, we first recap the core component of the transformer, the attention mechanism, and the detailed structures of the transformer. After that, we depict the recent progress of the transformer in the field of MIA. We organize the applications in a sequence of different tasks, including classification, segmentation, captioning, registration, detection, enhancement, localization, and synthesis. The mainstream classification and segmentation tasks are further divided into eleven medical image modalities. A large number of experiments studied in this review illustrate that the transformer-based method outperforms existing methods through comparisons with multiple evaluation metrics. Finally, we discuss the open challenges and future opportunities in this field. This task-modality review with the latest contents, detailed information, and comprehensive comparison may greatly benefit the broad MIA community. △ Less

Submitted 25 July, 2023; v1 submitted 13 August, 2022; originally announced August 2022.

Comments: Computers in Biology and Medicine Accepted

MSC Class: I.2.m; I.4.9; I.5.4; J.0

Scale-free and Task-agnostic Attack: Generating Photo-realistic Adversarial Patterns with Patch Quilting Generator

Authors: Xiangbo Gao, Cheng Luo, Qinliang Lin, Weicheng Xie, Minmin Liu, Linlin Shen, Keerthy Kusumam, Siyang Song

Abstract: \noindent Traditional L_p norm-restricted image attack algorithms suffer from poor transferability to black box scenarios and poor robustness to defense algorithms. Recent CNN generator-based attack approaches can synthesize unrestricted and semantically meaningful entities to the image, which is shown to be transferable and robust. However, such methods attack images by either synthesizing local… ▽ More \noindent Traditional L_p norm-restricted image attack algorithms suffer from poor transferability to black box scenarios and poor robustness to defense algorithms. Recent CNN generator-based attack approaches can synthesize unrestricted and semantically meaningful entities to the image, which is shown to be transferable and robust. However, such methods attack images by either synthesizing local adversarial entities, which are only suitable for attacking specific contents or performing global attacks, which are only applicable to a specific image scale. In this paper, we propose a novel Patch Quilting Generative Adversarial Networks (PQ-GAN) to learn the first scale-free CNN generator that can be applied to attack images with arbitrary scales for various computer vision tasks. The principal investigation on transferability of the generated adversarial examples, robustness to defense frameworks, and visual quality assessment show that the proposed PQG-based attack framework outperforms the other nine state-of-the-art adversarial attack approaches when attacking the neural networks trained on two standard evaluation datasets (i.e., ImageNet and CityScapes). △ Less

Submitted 19 November, 2022; v1 submitted 12 August, 2022; originally announced August 2022.

Interpretable Graph Convolutional Network of Multi-Modality Brain Imaging for Alzheimer's Disease Diagnosis

Authors: Houliang Zhou, Lifang He, Yu Zhang, Li Shen, Brian Chen

Abstract: Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies. In this paper, we propose an interpretable Graph Convolutional Network (GCN) framework for the identification and classification of Alzheimer's disease (AD) using multi-modality brain imaging data. Specifically, we extended the Gradient Class Activation Mappi… ▽ More Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies. In this paper, we propose an interpretable Graph Convolutional Network (GCN) framework for the identification and classification of Alzheimer's disease (AD) using multi-modality brain imaging data. Specifically, we extended the Gradient Class Activation Mapping (Grad-CAM) technique to quantify the most discriminative features identified by GCN from brain connectivity patterns. We then utilized them to find signature regions of interest (ROIs) by detecting the difference of features between regions in healthy control (HC), mild cognitive impairment (MCI), and AD groups. We conducted the experiments on the ADNI database with imaging data from three modalities, including VBM-MRI, FDG-PET, and AV45-PET, and showed that the ROI features learned by our method were effective for enhancing the performances of both clinical score prediction and disease status identification. It also successfully identified biomarkers associated with AD and MCI. △ Less

Submitted 27 April, 2022; originally announced April 2022.

Comments: This paper has been published to IEEE 19th International Symposium on Biomedical Imaging (ISBI), 2022

MSC Class: 68T07; 68T20; 68T45 ACM Class: I.2.6; I.2.10; J.3

Self-Supervised Deep Learning to Enhance Breast Cancer Detection on Screening Mammography

Authors: John D. Miller, Vignesh A. Arasu, Albert X. Pu, Laurie R. Margolies, Weiva Sieh, Li Shen

Abstract: A major limitation in applying deep learning to artificial intelligence (AI) systems is the scarcity of high-quality curated datasets. We investigate strong augmentation based self-supervised learning (SSL) techniques to address this problem. Using breast cancer detection as an example, we first identify a mammogram-specific transformation paradigm and then systematically compare four recent SSL m… ▽ More A major limitation in applying deep learning to artificial intelligence (AI) systems is the scarcity of high-quality curated datasets. We investigate strong augmentation based self-supervised learning (SSL) techniques to address this problem. Using breast cancer detection as an example, we first identify a mammogram-specific transformation paradigm and then systematically compare four recent SSL methods representing a diversity of approaches. We develop a method to convert a pretrained model from making predictions on uniformly tiled patches to whole images, and an attention-based pooling method that improves the classification performance. We found that the best SSL model substantially outperformed the baseline supervised model. The best SSL model also improved the data efficiency of sample labeling by nearly 4-fold and was highly transferrable from one dataset to another. SSL represents a major breakthrough in computer vision and may help the AI for medical imaging field to shift away from supervised learning and dependency on scarce labels. △ Less

Submitted 15 March, 2022; originally announced March 2022.

GSDA: Generative Adversarial Network-based Semi-Supervised Data Augmentation for Ultrasound Image Classification

Authors: Zhaoshan Liu, Qiujie Lv, Chau Hung Lee, Lei Shen

Abstract: Medical Ultrasound (US) is one of the most widely used imaging modalities in clinical practice, but its usage presents unique challenges such as variable imaging quality. Deep Learning (DL) models can serve as advanced medical US image analysis tools, but their performance is greatly limited by the scarcity of large datasets. To solve the common data shortage, we develop GSDA, a Generative Adversa… ▽ More Medical Ultrasound (US) is one of the most widely used imaging modalities in clinical practice, but its usage presents unique challenges such as variable imaging quality. Deep Learning (DL) models can serve as advanced medical US image analysis tools, but their performance is greatly limited by the scarcity of large datasets. To solve the common data shortage, we develop GSDA, a Generative Adversarial Network (GAN)-based semi-supervised data augmentation method. GSDA consists of the GAN and Convolutional Neural Network (CNN). The GAN synthesizes and pseudo-labels high-resolution, high-quality US images, and both real and synthesized images are then leveraged to train the CNN. To address the training challenges of both GAN and CNN with limited data, we employ transfer learning techniques during their training. We also introduce a novel evaluation standard that balances classification accuracy with computational time. We evaluate our method on the BUSI dataset and GSDA outperforms existing state-of-the-art methods. With the high-resolution and high-quality images synthesized, GSDA achieves a 97.9% accuracy using merely 780 images. Given these promising results, we believe that GSDA holds potential as an auxiliary tool for medical US analysis. △ Less

Submitted 5 October, 2023; v1 submitted 11 March, 2022; originally announced March 2022.

Comments: Heliyon Accepted

ACM Class: I.2.1; I.2.10; I.4.9; I.5.4; J.0

Deep Co-supervision and Attention Fusion Strategy for Automatic COVID-19 Lung Infection Segmentation on CT Images

Authors: Haigen Hu, Leizhao Shen, Qiu Guan, Xiaoxin Li, Qianwei Zhou, Su Ruan

Abstract: Due to the irregular shapes,various sizes and indistinguishable boundaries between the normal and infected tissues, it is still a challenging task to accurately segment the infected lesions of COVID-19 on CT images. In this paper, a novel segmentation scheme is proposed for the infections of COVID-19 by enhancing supervised information and fusing multi-scale feature maps of different levels based… ▽ More Due to the irregular shapes,various sizes and indistinguishable boundaries between the normal and infected tissues, it is still a challenging task to accurately segment the infected lesions of COVID-19 on CT images. In this paper, a novel segmentation scheme is proposed for the infections of COVID-19 by enhancing supervised information and fusing multi-scale feature maps of different levels based on the encoder-decoder architecture. To this end, a deep collaborative supervision (Co-supervision) scheme is proposed to guide the network learning the features of edges and semantics. More specifically, an Edge Supervised Module (ESM) is firstly designed to highlight low-level boundary features by incorporating the edge supervised information into the initial stage of down-sampling. Meanwhile, an Auxiliary Semantic Supervised Module (ASSM) is proposed to strengthen high-level semantic information by integrating mask supervised information into the later stage. Then an Attention Fusion Module (AFM) is developed to fuse multiple scale feature maps of different levels by using an attention mechanism to reduce the semantic gaps between high-level and low-level feature maps. Finally, the effectiveness of the proposed scheme is demonstrated on four various COVID-19 CT datasets. The results show that the proposed three modules are all promising. Based on the baseline (ResUnet), using ESM, ASSM, or AFM alone can respectively increase Dice metric by 1.12\%, 1.95\%,1.63\% in our dataset, while the integration by incorporating three models together can rise 3.97\%. Compared with the existing approaches in various datasets, the proposed method can obtain better segmentation performance in some main metrics, and can achieve the best generalization and comprehensive performance. △ Less

Submitted 20 December, 2021; originally announced December 2021.

Journal ref: Pattern Recognition,2022,124:108452

Hybrid Controlled User Association and Resource Management for Energy-Efficient Green RANs with Limited Fronthaul

Authors: Li-Hsiang Shen, Chia-Lin Tsai, Chia-Yu Wang, Kai-Ten Feng

Abstract: To alleviate green house effect, high network energy efficiency (EE) has increasingly become an important research target in wireless green communications. Therefore, the investigation for resource management to mitigate the co-tier interference in the small cell network (SCN) is provided. Moreover, with the merits of cloud radio access network (C-RAN), small cell base stations (SBSs) can be decom… ▽ More To alleviate green house effect, high network energy efficiency (EE) has increasingly become an important research target in wireless green communications. Therefore, the investigation for resource management to mitigate the co-tier interference in the small cell network (SCN) is provided. Moreover, with the merits of cloud radio access network (C-RAN), small cell base stations (SBSs) can be decomposed of a central small cell (CSC) and remote small cells (RSCs). To achieve the coordination, the split medium access control (MAC) based functional splitting is adopted with scheduler deployed at CSCs and retransmission functions left at RSCs. However, limited fronthaul has a compelling impact at RSCs due to requirements of user quality-of-service (QoS). Accordingly, a traffic control-based user association and resource allocation (TURA) scheme is proposed for a centralized resource management. To deal with the infeasibility to control all RSCs by CSC, we propose a hybrid controlled user and resource management (HARM) scheme. A CSC performs TURA for RSCs to mitigate intra-group interference within localized C-RANs, whereas the CSCs among separate C-RANs conduct cooperative resource competition (CRC) game for alleviating inter-group interference. Based on regret-based learning algorithm, the proposed schemes are analytically proved to reach the correlated equilibrium (CE). Simulation results have validated the effect of traffic control in TURA scheme and the convergence of CRC. Moreover, the comparison of the proposed TURA, HARM, and CRC schemes with the benchmark is revealed. It is observed that the TURA scheme outperforms the other schemes under ideal fronthaul control, whilst the proposed HARM scheme can sustain EE performance considering feasible implementation. △ Less

Submitted 7 December, 2021; originally announced December 2021.

Journal ref: IEEE Access, 2022

Solving Inverse Problems in Medical Imaging with Score-Based Generative Models

Authors: Yang Song, Liyue Shen, Lei Xing, Stefano Ermon

Abstract: Reconstructing medical images from partial measurements is an important inverse problem in Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Existing solutions based on machine learning typically train a model to directly map measurements to medical images, leveraging a training dataset of paired images and measurements. These measurements are typically synthesized from images using a… ▽ More Reconstructing medical images from partial measurements is an important inverse problem in Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Existing solutions based on machine learning typically train a model to directly map measurements to medical images, leveraging a training dataset of paired images and measurements. These measurements are typically synthesized from images using a fixed physical model of the measurement process, which hinders the generalization capability of models to unknown measurement processes. To address this issue, we propose a fully unsupervised technique for inverse problem solving, leveraging the recently introduced score-based generative models. Specifically, we first train a score-based generative model on medical images to capture their prior distribution. Given measurements and a physical model of the measurement process at test time, we introduce a sampling method to reconstruct an image consistent with both the prior and the observed measurements. Our method does not assume a fixed measurement process during training, and can thus be flexibly adapted to different measurement processes at test time. Empirically, we observe comparable or better performance to supervised learning techniques in several medical imaging tasks in CT and MRI, while demonstrating significantly better generalization to unknown measurement processes. △ Less

Submitted 15 June, 2022; v1 submitted 15 November, 2021; originally announced November 2021.

Comments: Published at ICLR 2022

Uncertainty-Guided Lung Nodule Segmentation with Feature-Aware Attention

Authors: Han Yang, Lu Shen, Mengke Zhang, Qiuli Wang

Abstract: Since radiologists have different training and clinical experiences, they may provide various segmentation annotations for a lung nodule. Conventional studies choose a single annotation as the learning target by default, but they waste valuable information of consensus or disagreements ingrained in the multiple annotations. This paper proposes an Uncertainty-Guided Segmentation Network (UGS-Net),… ▽ More Since radiologists have different training and clinical experiences, they may provide various segmentation annotations for a lung nodule. Conventional studies choose a single annotation as the learning target by default, but they waste valuable information of consensus or disagreements ingrained in the multiple annotations. This paper proposes an Uncertainty-Guided Segmentation Network (UGS-Net), which learns the rich visual features from the regions that may cause segmentation uncertainty and contributes to a better segmentation result. With an Uncertainty-Aware Module, this network can provide a Multi-Confidence Mask (MCM), pointing out regions with different segmentation uncertainty levels. Moreover, this paper introduces a Feature-Aware Attention Module to enhance the learning of the nodule boundary and density differences. Experimental results show that our method can predict the nodule regions with different uncertainty levels and achieve superior performance in LIDC-IDRI dataset. △ Less

Submitted 7 June, 2022; v1 submitted 24 October, 2021; originally announced October 2021.

Comments: 10 pages, 4 figures, 30 references

Enabling variable high spatial resolution retrieval from a long pulse BOTDA sensor

Authors: Zhao Ge, Li Shen, Can Zhao, Hao Wu, Zhiyong Zhao, Ming Tang

Abstract: In the field of Internet of Things, there is an urgent need for sensors with large-scale sensing capability for scenarios such as intelligent monitoring of production lines and urban infrastructure. Brillouin optical time domain analysis (BOTDA) sensors, which can monitor thousands of continuous points simultaneously, show great advantages in these applications. We propose a convolutional neural n… ▽ More In the field of Internet of Things, there is an urgent need for sensors with large-scale sensing capability for scenarios such as intelligent monitoring of production lines and urban infrastructure. Brillouin optical time domain analysis (BOTDA) sensors, which can monitor thousands of continuous points simultaneously, show great advantages in these applications. We propose a convolutional neural network (CNN) to process the data of conventional Brillouin optical time domain analysis (BOTDA) sensors, which achieves unprecedented performance improvement that allows to directly retrieve higher spatial resolution (SR) from the sensing system that use long pump pulses. By using the simulated Brillouin gain spectrums (BGSs) as the CNN input and the corresponding high SR BFS as the output target, the trained CNN is able to obtain a SR higher than the theoretical value determined by the pump pulse width. In the experiment, the CNN accurately retrieves 0.5-m hotspots from the measured BGS with pump pulses from 20 to 50 ns, and the acquired BFS is in great agreement with 45/40 ns differential pulse-width pair (DPP) measurement results. Compared with the DPP technique, the proposed CNN demonstrates a 2-fold improvement in BFS uncertainty with only half the measurement time. In addition, by changing the training datasets, the proposed CNN can obtain tunable high SR retrieval based on conventional BOTDA sensors that use long pulses without any requirement of hardware modifications. The proposed data post-processing approach paves the way to enable novel high spatial resolution BOTDA sensors, which brings substantial improvement over the state-of-the-art techniques in terms of system complexity, measurement time and reliability, etc. △ Less

Submitted 8 September, 2021; originally announced September 2021.

Comments: 7 pages, 6 figures

MSC Class: 78A15 ACM Class: I.2.1

NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction

Authors: Liyue Shen, John Pauly, Lei Xing

Abstract: Image reconstruction is an inverse problem that solves for a computational image based on sampled sensor measurement. Sparsely sampled image reconstruction poses addition challenges due to limited measurements. In this work, we propose an implicit Neural Representation learning methodology with Prior embedding (NeRP) to reconstruct a computational image from sparsely sampled measurements. The meth… ▽ More Image reconstruction is an inverse problem that solves for a computational image based on sampled sensor measurement. Sparsely sampled image reconstruction poses addition challenges due to limited measurements. In this work, we propose an implicit Neural Representation learning methodology with Prior embedding (NeRP) to reconstruct a computational image from sparsely sampled measurements. The method differs fundamentally from previous deep learning-based image reconstruction approaches in that NeRP exploits the internal information in an image prior, and the physics of the sparsely sampled measurements to produce a representation of the unknown subject. No large-scale data is required to train the NeRP except for a prior image and sparsely sampled measurements. In addition, we demonstrate that NeRP is a general methodology that generalizes to different imaging modalities such as CT and MRI. We also show that NeRP can robustly capture the subtle yet significant image changes required for assessing tumor progression. △ Less

Submitted 13 January, 2023; v1 submitted 24 August, 2021; originally announced August 2021.

Comments: Code available at https://github.com/liyues/NeRP

Journal ref: IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022

S2Looking: A Satellite Side-Looking Dataset for Building Change Detection

Authors: Li Shen, Yao Lu, Hao Chen, Hao Wei, Donghai Xie, Jiabao Yue, Rui Chen, Shouye Lv, Bitao Jiang

Abstract: Building-change detection underpins many important applications, especially in the military and crisis-management domains. Recent methods used for change detection have shifted towards deep learning, which depends on the quality of its training data. The assembly of large-scale annotated satellite imagery datasets is therefore essential for global building-change surveillance. Existing datasets al… ▽ More Building-change detection underpins many important applications, especially in the military and crisis-management domains. Recent methods used for change detection have shifted towards deep learning, which depends on the quality of its training data. The assembly of large-scale annotated satellite imagery datasets is therefore essential for global building-change surveillance. Existing datasets almost exclusively offer near-nadir viewing angles. This limits the range of changes that can be detected. By offering larger observation ranges, the scroll imaging mode of optical satellites presents an opportunity to overcome this restriction. This paper therefore introduces S2Looking, a building-change-detection dataset that contains large-scale side-looking satellite images captured at various off-nadir angles. The dataset consists of 5000 bitemporal image pairs of rural areas and more than 65,920 annotated instances of changes throughout the world. The dataset can be used to train deep-learning-based change-detection algorithms. It expands upon existing datasets by providing (1) larger viewing angles; (2) large illumination variances; and (3) the added complexity of rural images. To facilitate {the} use of the dataset, a benchmark task has been established, and preliminary tests suggest that deep-learning algorithms find the dataset significantly more challenging than the closest-competing near-nadir dataset, LEVIR-CD+. S2Looking may therefore promote important advances in existing building-change-detection algorithms. The dataset is available at https://github.com/S2Looking/. △ Less

Submitted 11 January, 2022; v1 submitted 19 July, 2021; originally announced July 2021.

Journal ref: Remote Sens. 2021, 13, 5094

Learning from Pseudo Lesion: A Self-supervised Framework for COVID-19 Diagnosis

Authors: Zhongliang Li, Zhihao Jin, Xuechen Li, Linlin Shen

Abstract: The Coronavirus disease 2019 (COVID-19) has rapidly spread all over the world since its first report in December 2019 and thoracic computed tomography (CT) has become one of the main tools for its diagnosis. In recent years, deep learning-based approaches have shown impressive performance in myriad image recognition tasks. However, they usually require a large number of annotated data for training… ▽ More The Coronavirus disease 2019 (COVID-19) has rapidly spread all over the world since its first report in December 2019 and thoracic computed tomography (CT) has become one of the main tools for its diagnosis. In recent years, deep learning-based approaches have shown impressive performance in myriad image recognition tasks. However, they usually require a large number of annotated data for training. Inspired by Ground Glass Opacity (GGO), a common finding in COIVD-19 patient's CT scans, we proposed in this paper a novel self-supervised pretraining method based on pseudo lesions generation and restoration for COVID-19 diagnosis. We used Perlin noise, a gradient noise based mathematical model, to generate lesion-like patterns, which were then randomly pasted to the lung regions of normal CT images to generate pseudo COVID-19 images. The pairs of normal and pseudo COVID-19 images were then used to train an encoder-decoder architecture based U-Net for image restoration, which does not require any labelled data. The pretrained encoder was then fine-tuned using labelled data for COVID-19 diagnosis task. Two public COVID-19 diagnosis datasets made up of CT images were employed for evaluation. Comprehensive experimental results demonstrated that the proposed self-supervised learning approach could extract better feature representation for COVID-19 diagnosis and the accuracy of the proposed method outperformed the supervised model pretrained on large scale images by 6.57% and 3.03% on SARS-CoV-2 dataset and Jinan COVID-19 dataset, respectively. △ Less

Submitted 23 June, 2021; originally announced June 2021.

Neuron segmentation using 3D wavelet integrated encoder-decoder network

Authors: Qiufu Li, Linlin Shen

Abstract: Motivation: 3D neuron segmentation is a key step for the neuron digital reconstruction, which is essential for exploring brain circuits and understanding brain functions. However, the fine line-shaped nerve fibers of neuron could spread in a large region, which brings great computational cost to the neuron segmentation. Meanwhile, the strong noises and disconnected nerve fibers bring great challen… ▽ More Motivation: 3D neuron segmentation is a key step for the neuron digital reconstruction, which is essential for exploring brain circuits and understanding brain functions. However, the fine line-shaped nerve fibers of neuron could spread in a large region, which brings great computational cost to the neuron segmentation. Meanwhile, the strong noises and disconnected nerve fibers bring great challenges to the task. Results: In this paper, we propose a 3D wavelet and deep learning based 3D neuron segmentation method. The neuronal image is first partitioned into neuronal cubes to simplify the segmentation task. Then, we design 3D WaveUNet, the first 3D wavelet integrated encoder-decoder network, to segment the nerve fibers in the cubes; the wavelets could assist the deep networks in suppressing data noises and connecting the broken fibers. We also produce a Neuronal Cube Dataset (NeuCuDa) using the biggest available annotated neuronal image dataset, BigNeuron, to train 3D WaveUNet. Finally, the nerve fibers segmented in cubes are assembled to generate the complete neuron, which is digitally reconstructed using an available automatic tracing algorithm. The experimental results show that our neuron segmentation method could completely extract the target neuron in noisy neuronal images. The integrated 3D wavelets can efficiently improve the performance of 3D neuron segmentation and reconstruction. Availability: The data and codes for this work are available at https://github.com/LiQiufu/3D-WaveUNet. △ Less

Submitted 8 November, 2021; v1 submitted 1 June, 2021; originally announced June 2021.

Comments: Bioinformatics accepted paper

Learning a Model-Driven Variational Network for Deformable Image Registration

Authors: Xi Jia, Alexander Thorley, Wei Chen, Huaqi Qiu, Linlin Shen, Iain B Styles, Hyung Jin Chang, Ales Leonardis, Antonio de Marvao, Declan P. O'Regan, Daniel Rueckert, Jinming Duan

Abstract: Data-driven deep learning approaches to image registration can be less accurate than conventional iterative approaches, especially when training data is limited. To address this whilst retaining the fast inference speed of deep learning, we propose VR-Net, a novel cascaded variational network for unsupervised deformable image registration. Using the variable splitting optimization scheme, we first… ▽ More Data-driven deep learning approaches to image registration can be less accurate than conventional iterative approaches, especially when training data is limited. To address this whilst retaining the fast inference speed of deep learning, we propose VR-Net, a novel cascaded variational network for unsupervised deformable image registration. Using the variable splitting optimization scheme, we first convert the image registration problem, established in a generic variational framework, into two sub-problems, one with a point-wise, closed-form solution while the other one is a denoising problem. We then propose two neural layers (i.e. warping layer and intensity consistency layer) to model the analytical solution and a residual U-Net to formulate the denoising problem (i.e. generalized denoising layer). Finally, we cascade the warping layer, intensity consistency layer, and generalized denoising layer to form the VR-Net. Extensive experiments on three (two 2D and one 3D) cardiac magnetic resonance imaging datasets show that VR-Net outperforms state-of-the-art deep learning methods on registration accuracy, while maintains the fast inference speed of deep learning and the data-efficiency of variational model. △ Less

Submitted 25 May, 2021; originally announced May 2021.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible