Selfredepth

Electrocardiogram (ECG) is acquired during Magnetic Resonance Imaging (MRI) to monitor patients and synchronize image acquisition with the heart motion. ECG signals are highly distorted during MRI due to the complex electromagnetic environment. Automated ECG analysis is therefore complicated in this context and there is no reference technique in MRI to classify pathological heartbeats. Imaging arrhythmic patients is hence difficult in MRI. Deep Learning based heartbeat classifier have been suggested but require large databases whereas existing annotated sets of ECG in MRI are very small. We proposed a Siamese network to leverage a large database of unannotated ECGs outside MRI. This was used to develop an efficient representation of ECG signals, further used to develop a heartbeat classifier. We extensively tested several data augmentations and self-supervised learning (SSL) techniques and assessed the generalization of the obtained classifier to ECG signals acquired in MRI. These augmentations included random noises and a model simulating MRI specific artefacts. SSL pretraining improved the generalizability of heartbeat classifiers in MRI (F1=0.75) compared to Deep Learning not relying on SSL (F1=0.46) and another classical machine learning approach (F1=0.40). These promising results seem to indicate that the use of SSL techniques can learn efficient ECG signal representation, and are useful for the development of Deep Learning models even when only scarce annotated medical data are available.

Medical ultrasound imaging is a key diagnostic tool across various fields, with computer-aided diagnosis systems benefiting from advances in deep learning. However, its lower resolution and artifacts pose challenges, particularly for non-specialists. The simultaneous acquisition of degraded and high-quality images is infeasible, limiting supervised learning approaches. Additionally, self-supervised and zero-shot methods require extensive processing time, conflicting with the real-time demands of ultrasound imaging. Therefore, to address the aforementioned issues, we propose real-time ultrasound image enhancement via a self-supervised learning technique and a test-time adaptation for sophisticated rotational cuff tear diagnosis. The proposed approach learns from other domain image datasets and performs self-supervised learning on an ultrasound image during inference for enhancement. Our approach not only demonstrated superior ultrasound image enhancement performance compared to other state-of-the-art methods but also achieved an 18% improvement in the RCT segmentation performance.

Due to the limited resolution of depth maps captured by RGB-D sensors, depth map Super Resolution (SR) techniques have received a lot of attention. Intensity guided depth map SR methods based on bilateral filter or guided image filter are commonly used for depth upsampling. Although promising edge-preserving results have been reported in these methods, texture-copying artifacts caused by structure discrepancy between depth map and associated intensity image cannot be addressed, easily. In this paper we aim to balance the trade-off between preserving structure and suppressing texture defects. Based on this, a structure-preserving guided filter is presented that not only keeps the advantages of aforementioned methods, but also overcomes texture-copying artifacts. Unlike conventional guided filtering-based methods which rely on only one guidance, we emphasize on the use of both intensity and depth information as guidance to alleviate the deficiencies of the existing works. We replace the mean filtering scheme in guided filters with a weighted average strategy, where the weights are described by the local depth kernel depended on the input depth map. This enables our method to considerably reduce texture-copying artifacts while preserving 3D structural details. Visual evaluation of results shows that the algorithm can also avoid halo artifacts near the edges whereas traditional guided filters suffer from it. Quantitative results of comprehensive experiments demonstrate the effectiveness of our approach over prior depth map SR works.

Measuring near-field wave elevation with monocular vision using self-supervised learning techniques

The underwater domain presents unique challenges and opportunities for scientific exploration, resource extraction, and environmental monitoring. Autonomous underwater vehicles (AUVs) rely on simultaneous localization and mapping (SLAM) for real-time navigation and mapping in these complex environments. However, traditional SLAM techniques face significant obstacles, including poor visibility, dynamic lighting conditions, sensor noise, and water-induced distortions, all of which degrade the accuracy and robustness of underwater navigation systems. Recent advances in deep learning (DL) have introduced powerful solutions to overcome these challenges. DL techniques enhance underwater SLAM by improving feature extraction, image denoising, distortion correction, and sensor fusion. This survey provides a comprehensive analysis of the latest developments in DL-enhanced SLAM for underwater applications, categorizing approaches based on their methodologies, sensor dependencies, and integration with deep learning models. We critically evaluate the benefits and limitations of existing techniques, highlighting key innovations and unresolved challenges. In addition, we introduce a novel classification framework for underwater SLAM based on its integration with underwater wireless sensor networks (UWSNs). UWSNs offer a collaborative framework that enhances localization, mapping, and real-time data sharing among AUVs by leveraging acoustic communication and distributed sensing. Our proposed taxonomy provides new insights into how communication-aware SLAM methodologies can improve navigation accuracy and operational efficiency in underwater environments. Furthermore, we discuss emerging research trends, including the use of transformer-based architectures, multi-modal sensor fusion, lightweight neural networks for real-time deployment, and self-supervised learning techniques. By identifying gaps in current research and outlining potential directions for future work, this survey serves as a valuable reference for researchers and engineers striving to develop robust and adaptive underwater SLAM solutions. Our findings aim to inspire further advancements in autonomous underwater exploration, supporting critical applications in marine science, deep-sea resource management, and environmental conservation.

Characterizing network flows is essential for security operators to enhance their awareness about cyber-threats targeting their networks. The automation of network flow characterization with machine learning has received much attention in recent years. To this aim, raw network flows need to be transformed into structured and exploitable data. In this research work, we propose a method to encode raw network flows into robust latent representations exploitable in downstream tasks. First, raw network flows are transformed into graph-structured objects capturing their topological aspects (packet-wise transitional patterns) and features (used protocols, packets’ flags, etc.). Then, using self-supervised learning techniques like Graph Auto-Encoders and Anonymous Walk Embeddings, each network flow graph is encoded into a latent representation that encapsulates both the structure of the graph and the features of its nodes, while minimizing information loss. This results in semantically-rich and robust representation vectors which can be manipulated by machine learning algorithms to perform downstream network-related tasks. To evaluate our network flow embedding models, we use probing flows captured with two /20 network telescopes and labeled using reports originating from different sources. The experimental results show that the proposed network flow embedding method allows for reliable darknet probing activity classification. Furthermore, a comparison between our self-supervised approach and a fully-supervised graph convolutional network shows that, in situations with limited labeled data, the downstream classification model that uses the derived latent representations as inputs outperforms the fully-supervised graph convolutional network. There are many applications of this research work in cybersecurity, such as network flow clustering, attack detection and prediction, malware detection, vulnerability exploit analysis, and inference of attacker’s intentions.

Depth information can benefit various computer vision tasks on both images and videos. However, depth maps may suffer from invalid values in many pixels, and also large holes. To improve such data, we propose a joint self-supervised and reference-guided learning approach for depth inpainting. For the self-supervised learning strategy, we introduce an improved spatial convolutional sparse coding module in which total variation regularization is employed to enhance the structural information while preserving edge information. This module alternately learns a convolutional dictionary and sparse coding from a corrupted depth map. Then, both the learned convolutional dictionary and sparse coding are convolved to yield an initial depth map, which is effectively smoothed using local contextual information. The reference-guided learning part is inspired by the fact that adjacent pixels with close colors in the RGB image tend to have similar depth values. We thus construct a hierarchical joint bilateral filter module using the corresponding color image to fill in large holes. In summary, our approach integrates a convolutional sparse coding module to preserve local contextual information and a hierarchical joint bilateral filter module for filling using specific adjacent information. Experimental results show that the proposed approach works well for both invalid value restoration and large hole inpainting.

Three-dimensional (3D) LiDAR is widely used because it can accurately measure 360° 3D depth. One of the most widely used sensors to this end is VLP-16, a 16-channel LiDAR manufactured by Velodyne. This 16-channel LiDAR provides dense 3D depth data in the horizontal direction. However, in the vertical direction, it provides only sparse depth data owing to the LiDAR architecture. We propose an upsampling method to obtain a dense 3D depth map from a 16-channel LiDAR by applying an edge-aware filter to the LiDAR data. A camera- LiDAR sensor system composed of a 16-channel LiDAR and six vision cameras is used for this purpose. First, the LiDAR depth data are projected onto the camera image coordinates to generate a 2.5D depth map. Next, bilinear interpolation and a weighted median filter are applied to upsample the original depth map. Compared with the original LiDAR data, we can generate an extremely dense depth map by using the proposed method. The results of quantitative analysis show that the proposed method can dramatically increase the number of 3D depth points and reduce the depth reconstruction error.

This paper presents underwater 3D capture using a commercial depth camera. Previous underwater capture systems use ordinary cameras, and it is well-known that a calibration procedure is needed to handle refraction. The same is true for a depth camera being used underwater. We describe a calibration method that corrects the depth maps of refraction effects. Another challenge is that depth cameras use infrared light (IR) which is heavily attenuated in water. We demonstrate scanning is possible with commercial depth cameras for ranges up to 20 cm in water. The motivation for using a depth camera under water is the same as in air — it provides dense depth data and higher quality 3D reconstruction than multi-view stereo. Underwater 3D capture is being increasingly used in marine biology and oceanology; our approach offers exciting prospects for such applications. To the best of our knowledge, ours is the first approach that successfully demonstrates underwater 3D capture using low cost depth cameras like Intel RealSense. We describe a complete system, including protective housing for the depth camera which is suitable for handheld use by a diver. Our main contribution is an easy-to-use calibration method, which we evaluate on exemplar data as well as 3D reconstructions in a lab aquarium. We also present initial results of ocean deployment.

One of the key aspects of 3D visualization is computation of depth maps. Depth maps enables synthesis of 3D video from 2D video and use of multi-view displays. Depth maps can be acquired in several ways. One method is to measure the real 3D properties of the scene objects. Other methods rely on using two cameras and computing the correspondence for each pixel. Once a depth map is acquired for every frame, it can be used to construct its artificial stereo pair. There are many known methods for computing the optical flow between adjacent video frames. The drawback of these methods is that they require extensive computation power and are not very well suited to high quality real-time 3D rendering. One efficient method for computing depth maps is extraction of motion vector information from standard video encoders. In this paper we present methods to improve the 3D visualization quality acquired from compression CODECS by spatial/temporal and logical operations and manipulations. We show how an efficient real time implementation of spatial-temporal local order statistics such as median and local adaptive filtering in 3D-DCT domain can substantially improve the quality of depth maps and consequently 3D video while retaining real-time rendering. Real-time performance is achived by utilizing multi-core technology using standard parallelization algorithms and libraries (OpenMP, IPP).

Self-supervised learning for RGB-D object tracking

Camouflaged object detection (COD) faces unique challenges due to the extremely high visual similarity between objects and their surroundings, coupled with indistinct boundary features. While the introduction of depth information has provided new insights into addressing these challenges, existing methods still exhibit considerable limitations in depth data quality assessment and optimization. To address this issue, this paper proposes a depth screening and calibration (DSC) framework aimed at constructing a high-quality RGBD COD dataset. The framework first establishes a comprehensive evaluation metric that quantitatively assesses depth data generated by various monocular depth estimation (MDE) methods across multiple dimensions, including structural similarity, edge consistency, foreground smoothness, depth value utilization, and depth disparity between foreground and background. Based on these metrics, optimal depth maps are selected from those generated by multiple MDE methods for each image, forming an initial RGBD COD dataset. Subsequently, a Two-stage Depth Calibration (TDC) strategy is designed to calibrate the depth maps in the initial dataset through two consecutive phases: positive-negative sample discrimination and calibrated depth map generation, effectively enhancing the overall quality of depth maps. Experimental results on three benchmark datasets demonstrate that detection models trained with our high-quality depth data significantly outperform alternative approaches. This work provides a reliable data foundation for further exploring the role of depth information in improving COD performance.

Prospective 3D video transmission systems that would use compression of both multiview video and depth maps was dealt with in this paper. To get the optimal synthesized view quality with the transmission bit rate being constant, it firstly analyzed the overall behavior of combined coding and view synthesis, color and depth data with different coding qualities were combined for intermediate view synthesis. Then, individual coding results for color and depth data were evaluated in order to investigate, how well the synthesized view quality was at certain compression. Finally, a new joint coding method for multi-view video plus depth (MVD) was proposed by utilizing the optimal combinations of the quantization parameters of color videos and the corresponding depth maps. Simulation results demonstrated that the proposed method decreased the bit-rate by 12.72%~24.02% in comparison with JMVC and can be used to guide sequence-level texture/depth coding for 3D video compression. The proposed method has better performance in optimizing the tradeoff between bit-rate and visual quality.

Newborn patients in the neonatal intensive care unit (NICU) require continuous monitoring of vital signs. Non-contact patient monitoring is preferred in this setting, due to fragile condition of neonatal patients. Depth-based approaches for estimating the respiratory rate (RR) can operate effectively in conditions where an RGB-based method would typically fail, such as low-lighting or where a patient is covered with blankets. Many previously developed depth-based RR estimation techniques require careful camera placement with known geometry relative to the patient, or manual definition of a region of interest (ROI). We here present a framework for depth-based RR estimation where the camera position is arbitrary and the ROI is determined automatically and directly from the depth data. Camera placement is addressed through perspective transformation of the scene, which is accomplished by selecting a small number of registration points known to lie in the same plane. The chest ROI is determined automatically from examining the morphology of progressive depth slices in the corrected depth data. We demonstrate the effectiveness of this RR estimation pipeline using actual neonatal patient depth data collected from an RGB-D sensor. RR estimation accuracy is measured relative to gold standard RR captured from the bedside patient monitor. Perspective transformation is shown to be critical to effectively achieve automated ROI segmentation algorithm. Furthermore, the automated ROI segmentation algorithm is shown to improve both time- and frequency-domain based RR estimation accuracy. When combined, these pre-processing stages are shown to substantially improve the depth-based RR estimation pipeline, with a percentage of acceptable estimates (where the mean absolute error is less than 5 breaths per minute) increasing from 3.60% to 13.47% in the frequency domain and 6.12% to 8.97% in the time domain. Further development will focus on RR estimation from the perspective-corrected depth data and segmented ROI.

Stereoscopic images and video are synthesized from virtual views, which are produced from a real view and its depth map. However, this process involves a large amount of data which not only increases the transmission load, but also must be stored within the display devices which therefore must possess huge storage capacities. To solve these problems, the Joint Video Team has developed the 3D video coding (3DVC) standard to reduce the redundant data for multiview video and its depth map through the correlation between the real view and its corresponding depth map. When depth data in the 3DVC bitstream are lost during transmission, it causes the decoded depth map to be incomplete. Then the quality of the decoded depth map is reduced and the quality of the stereoscopic video will also be decreased. To increase the reconstructed quality of the damaged macro-block, this paper proposes an adaptive error concealment selection to determine the most suitable error concealment method based on fuzzy reasoning. The temporal and spatial correlations of the damaged macro-block which are defined by the motion vectors of neighboring blocks and their depth information are used for the adaptive selection. From the simulation result, the proposed algorithm can reduce the concealing time by at least 0.15–2.86 frame/sec and can improve quality by about 0.13–0.17 dB. The proposed algorithm is suitable for the 3DVC video transmission.