Biologically Inspired Region of Interest Selection for Lowbit-Rate Video Coding

In this paper we propose a novel saliency-based computational model for visual attention. This model processes both top-down (goal directed) and bottom-up information. Processing in the top-down channel creates the so called skin conspicuity map and emulates the visual search for human faces performed by humans. This is clearly a goal directed task but is generic enough to be context independent. Processing in the bottom-up information channel follows the principles set by Itti et al. but it deviates from them by computing the orientation, intensity and color conspicuity maps within a unified multi-resolution framework based on wavelet subband analysis. In particular, we apply a wavelet based approach for efficient computation of the topographic feature maps. Given that wavelets and multiresolution theory are naturally connected the usage of wavelet decomposition for mimicking the center surround process in humans is an obvious choice. However, our implementation goes further. We utilize the wavelet decomposition for inline computation of the features (such as orientation angles) that are used to create the topographic feature maps. The bottom-up topographic feature maps and the top-down skin conspicuity map are then combined through a sigmoid function to produce the final saliency map. A prototype of the proposed model was realized through the TMDSDMK642-0E DSP platform as an embedded system allowing real-time operation. For evaluation purposes, in terms of perceived visual quality and video compression improvement, a ROI-based video compression setup was followed. Extended experiments concerning both MPEG-1 as well as low bit-rate MPEG-4 video encoding were conducted showing significant improvement in video compression efficiency without perceived deterioration in visual quality.

In the latest years, videoconferencing has taken a fundamental role in interpersonal relations, both for personal and business purposes. Lossy video compression algorithms are the enabling technology for videoconferencing, as they reduce the bandwidth required for real-time video streaming. However, lossy video compression decreases the perceived visual quality. Thus, many techniques for reducing compression artifacts and improving video visual quality have been proposed in recent years. In this work, we propose a novel GAN-based method for compression artifacts reduction in videoconferencing. Given that, in this context, the speaker is typically in front of the camera and remains the same for the entire duration of the transmission, we can maintain a set of reference keyframes of the person from the higher-quality I-frames that are transmitted within the video stream and exploit them to guide the visual quality improvement; a novel aspect of this approach is the update policy that maintains and updates a compact and effective set of reference keyframes. First, we extract multi-scale features from the compressed and reference frames. Then, our architecture combines these features in a progressive manner according to facial landmarks. This allows the restoration of the high-frequency details lost after the video compression. Experiments show that the proposed approach improves visual quality and generates photo-realistic results even with high compression rates. Code and pre-trained networks are publicly available at https://github.com/LorenzoAgnolucci/Keyframes-GAN.

With the advent of overdriving technologies, the problems faced by computer vision are rectified with the working process like segmentation, compression, etc. The segmentation process is done for the video analysis. In this, based on the coherence between the regions involved in the video, the information is clustered and segregated for analyzing the required part from the video. The capability of video segmentation is to divide and abstract the required frames, recognize the activity, detect the object and enhance the video quality. Video compression is an out-bouncing technique to reduce the number of bits that zip the video file without loss to store and process in a compact space. It gets uplifted with maximum efficacy using artificial intelligence, machine learning and deep learning algorithms. The proposed model is designed for social distance prediction on the basis of the COVID-19 scenario. In the proposed model, a clipped video is taken and segmented into several numbers. With the segments, the region of interest is extracted, and it is applied to the convolutional neural network for further classification to indicate whether social distancing is maintained in that recorded area.

The demand for video through over-the-top (OTT) has been constantly increasing in recent years. During the COVID-19 pandemic, demand skyrocketed, hence leading to the need for better video compression. The human visual system (HVS) can quickly select visually important regions in its visual field. These regions are captured at high resolution, while other peripheral regions receive little attention. Saliency maps are a way to imitate the HVS attention mechanism. Recently, deep learning-based saliency models have achieved tremendous improvements. This article leverages state-of-the-art deep learning-based saliency models to improve video coding efficiency. First, a saliency-based rate control scheme is integrated in a high-efficiency video encoder (HEVC). Then, a saliency-guided preprocessing filtering step is introduced. Finally, the two approaches are combined. Objective and subjective evaluations show that it can lower the bitrate from 6% to almost 30% while maintaining the same visual quality.

Conventional rate control schemes focus on making output bit rate approach a target value and are deficient in ensuring a higher quality of ROI (Region-of-Interest) than others in a frame. In this paper, we propose a new scheme for H.264/AVC, aiming to allocate more bit resource for the encoding of ROI and still maintain the accuracy of the output bit rate. ROI-based rate control algorithms can find their specific advantages in video telephony and video surveillance applications. Our proposed scheme is based on the one implemented in H.264/AVC JM software, but enhanced with several features: ROI determination with saliency map, tunable quality factor for ROI, two-channel (ROI & non-ROI) rate control, and QP adjustment with constraints from temporal and spatial domains, as well as from ROI/non-ROI adjacency boundaries. Experiment results show both advantages in objective PSNR and subjective evaluations for ROI, while making the output bit rate accurate as before.

Canonical resting state networks (RSNs) can be obtained through independent component analysis (ICA). RSNs are reproducible across subjects but also present inter-individual differences, which can be used to individualize regions-of-interest (ROI) definition, thus making fMRI analyses more accurate. Unfortunately, no automatic tool for defining subject-specific ROIs exists, making the classification of ICAs as representatives of RSN time-consuming and largely dependent on visual inspection. Here, we present Personode, a user-friendly and open source MATLAB-based toolbox that semi-automatically performs the classification of RSN and allows for defining subject- and group-specific ROIs. To validate the applicability of our new approach and to assess potential improvements compared to previous approaches, we applied Personode to both task-related activation and resting-state data. Our analyses show that for task-related activation analyses, subject-specific spherical ROIs defined with Personode produced higher activity contrasts compared to ROIs derived from single-study and meta-analytic coordinates. We also show that subject-specific irregular ROIs defined with Personode improved ROI-to-ROI functional connectivity analyses.Hence, Personode might be a useful toolbox for ICA map classification into RSNs and group- as well as subject-specific ROI definitions, leading to improved analyses of task-related activation and functional connectivity.

The compressed domain video watermarking has been less expensive since complete decoding and then re-encoding is not required for watermark embedding. Handling synchronization error due to embedding is a challenging task, especially, due to motion compensation. In this paper, we propose an embedding scheme that can minimize desynchronization in the watermark extraction process at the decoder for embedding in compressed videos with better visual quality. Experimental results show the effectiveness of the proposed scheme.

This paper presents an evaluation of the latest MPEG-5 Part 2 Low Complexity Enhancement Video Coding (LCEVC) for video streaming applications using best effort protocols. LCEVC is a new video standard by MPEG, which enhances any base codec through an additional low bitrate stream, improving both video compression efficiency and and transmission. However, there is an interplay between packetization, packet loss visibility, choice of codec and video quality, which implies that prior studies with other codecs may be not as relevant. The contributions of this paper is, therefore in twofold: It evaluates the compression performance of LCEVC and then the impact of packet loss on its video quality when compared to H.264 and HEVC.The results from this evaluation suggest that, regarding compression, LCEVC outperformed its base codecs, overall in terms average encoding bitrate savings when using the constant rate factor (CRF) rate control. For example at a CRF of 19, the average encoding bitrate was reduced by 18.7% and 15.8% when compared with the base H.264 and HEVC codecs respectively. Furthermore, LCEVC produced better visual quality across the packet loss range compared to its base codecs and the quality only started to decrease once packet loss exceeded 0.8-1%, and decreases at a slower pace compared to its equivalent base codecs. This suggests that the LCEVC enhancement layer also provides error concealment. The results presented in this paper will be of interest to those considering the LCEVC standard and expected video quality in error-prone environments

To evaluate the impact of Compound Tropicamide (Trop) Eye Drops on the visual quality in juvenile pseudomyopia patients. Data from 111 juvenile pseudomyopia patients treated between September 2019 and September 2022 were reviewed in this retrospective study, including 50 cases treated with Atropine Sulfate Ophthalmic Gel (control group) and 61 with Compound Trop Eye Drops (research group). Clinical efficacy, amplitude of accommodation (AA), uncorrected visual acuity (UCVA), intraocular pressure (IOP), axial length (AL), symptom improvement, complications, and quality of life were compared. Risk factors for visual quality improvement were analyzed using Logistic regression. The research group demonstrated significantly higher treatment efficacy compared to the control group (P>0.05). Post-treatment, AA, IOP, and AL decreased notably in the research group compared to baseline and the control group (all P<0.05). UCVA and quality of life scores improved significantly in the research group (both P<0.05). Recovery indicators, including resolution of dizziness, alleviation of ocular soreness, and vision recovery, were significantly shorter in the research group (all P<0.05). Both groups had similar complication rates, including dry mouth, facial blushing, conjunctival congestion, and transient tachycardia (P>0.05). The research group reported superior post-treatment quality of life across all domains (all P<0.05). Logistic regression showed that gender, age, disease duration, family history of myopia, education level, AA, UCVA, and treatment method were not significant risk factors for visual quality improvement (all P>0.05). Compound Trop Eye Drops effectively enhances visual function and quality of life in juvenile pseudomyopia patients, with a safety profile comparable to low-dose atropine, offering potential clinical value.

Within the recent twenty years, the dental X-ray images have widely been employed in forensic odontology for human identification, particularly where mass disasters happen. In this paper, a novel method is proposed for the process of teeth segmentation and individual teeth isolation of Bitewing X-ray radiographs. The main objective of this study is to develop an automatic teeth segmentation approach that can be used in an Automated Dental Identification System (ADIS). The proposed method is based on separating teeth according to edge lines between crowns of teeth. It comprises four phases as image enhancement, edge detection by using wavelet transform, Region of Interest (ROI) definition, and morphological processing. Image enhancement in our case is done by image sharpening using a Butterworth high pass filter. Directional changes of the image and a blurred version of it are obtained by wavelet transform in the second phase. In ROI definition the upper and lower jaws are first separated using the integral intensity projection and then a region containing the desired edge lines are defined. In the final stage, some morphological operations are applied to isolate the teeth based on separating edge lines. The evaluation of the teeth segmentation is measured by isolating accuracy and visual inspection. Experimental results with 90.6% isolation accuracy of total 681 teeth illustrate that the proposed method is more efficient that the existing algorithms.

The increasing popularity of video consumption from mobile devices requires an effective video coding strategy. To overcome diverse communication networks, video services often need to maintain sustainable quality when the available bandwidth is limited. One of the strategy for a visually-optimised video adaptation is by implementing a region-of-interest (ROI) based scalability, whereby important regions can be encoded at a higher quality while maintaining sufficient quality for the rest of the frame. The result is an improved perceived quality at the same bit rate as normal encoding, which is particularly obvious at the range of lower bit rate. However, because of the difficulties of predicting region-of-interest (ROI) accurately, there is a limited research and development of ROI-based video coding for general videos. In this paper, the phase spectrum quaternion of Fourier Transform (PQFT) method is adopted to determine the ROI. To improve the results of ROI detection, the saliency map from the PQFT is augmented with maps created from high level knowledge of factors that are known to attract human attention. Hence, maps that locate faces and emphasise the centre of the screen are used in combination with the saliency map to determine the ROI. The contribution of this paper lies on the automatic ROI detection technique for coding a low bit rate videos which include the ROI prioritisation technique to give different level of encoding qualities for multiple ROIs, and the evaluation of the proposed automatic ROI detection that is shown to have a close performance to human ROI, based on the eye fixation data.

Region of interest (ROI) detection plays an important role in object detection. It needs to be accurate and fast in some applications like real time disaster victim detection systems. ROI can reduce time and search space in detecting objects. In this paper visual saliency map is used for ROI detection. In most literature, most of ROI detection models only concentrate on reducing false positive (detecting wrong objects as intended ones) rate rather than false negative (missing intended object). In disaster victim detection, missing disaster victims is more important than detecting other objects like victim. So, the proposed method also focuses on reducing false negative error rate in object detection. In the proposed system, local entropy feature is added in Graph Based Visual Saliency (GBVS) map in addition to colour, orientation and shape feature maps.

Low doses radiation in mammography results in low contrast images. In this paper we propose a method to enhance the contrast in mammography; it combines the modification of the coefficients of the Logarithmic Discrete Wavelet Transform using the Local Correlation method and Symmetric Logarithmic Image Processing model. Experimental results shown the better performance for the anomalies known as calcifications and masses. This paper also presents a methodology to select a combination of decomposition levels to be processed for good contrast improvement according to the values of measures based on region of interest. This procedure relies on Principal Components Analysis of the data. The experiments show that the chosen combination of levels can improve the contrast in mammograms, and that the regions of interest definition is an important factor to explain the poor contrast improvement of some anomalies.

One interesting feature of the new JPEG2000 image coding standard is support of region of interest (ROI) coding using the maximum shift (Maxshift) method, which allows for arbitrarily shaped ROI image compression without shape coding or explicitly transmitting any shape information to the decoder. The major disadvantage of the Maxshift method is that it cannot adjust the scaling value which determines the degree of relative importance between the ROI and the background wavelet coefficients. The bitplane-by-bitplane shift (BbBShift) method was introduced to support both arbitrary ROI shape and arbitrary scaling without shape coding. We propose a generalized BbBShift (GBbBShift) method, which delivers much more flexibility than both Maxshift and BbBShift for degree-of-interest adjustment of the ROI with insignificant effect on coding efficiency and computational complexity. Experiments show that it can provide significantly better visual quality than Maxshift at low bit rates. GBbBShift is not compliant with the current JPEG2000 definitions. In order to use it, a new ROI coding mode would need to be added to the standard.

Extracting salient regions plays an important role in computer vision tasks, e.g., object detection, recognition and video compression. Previous saliency detection study is mostly conducted on individual frames and tends to extract saliency with spatial cues. The development of various motion feature further extends the saliency concept to the motion saliency from videos. In contrast to image-based saliency extraction, video-based saliency extraction is more challenging due to the complicated distractors, e.g., the background dynamics and shadows. In this paper, we propose a novel saliency extraction method by fusing temporal and spatial cues. In specific, the long-term and short-term variations are comprehensively fused to extract the temporal cue, which is then utilized to establish the background guidance for generating the spatial cue. Herein, the long-term variations and spatial cues jointly highlight the contrast between objects and the background, which can solve the problem caused by shadows. The short-term variations contribute to the removal of background dynamics. Spatiotemporal cues are fully exploited to constrain the saliency extraction across frames. The saliency extraction performance of our method is demonstrated by comparing it to both unsupervised and supervised methods. Moreover, this novel saliency extraction model is applied in the video compression tasks, helping to accelerate the video compression task and achieve a larger PSNR value for the region of interest (ROI).