U-FIN: Unsupervised Feature Integration Approach for Salient Object Detection

Salient object detection in a real-time environment demands high accuracy with less computation time. It is a changeling task to investigate a saliency model which improves saliency detection accuracy as well as reduces computation time. In this paper, we propose a hybrid model that improves detection accuracy with low computational time. The input image is simplified and clustered at multiple scales using SLIC and K-means clustering algorithms respectively. Gaussian Mixture Model (GMM) is developed on various color components of the digital image at multiple scales. The parameters of GMM are learnt using Expectation Maximization (EM) algorithm. The spatial variance of each color component is determined using GMM parameters and hence object position is estimated. Further, spatial variance of color components and object position is exploited to compute saliency map at a scale level. Afterwards, all the saliency maps generated across various scales are linearly combined to produce the final saliency map. The performance of the proposed model is compared in terms of Precision, Recall, F-Measure, Area under the Curve (AUC), Receiver Operating Characteristics (ROC) and Mean Absolute Error (MAE). Extensive experiments on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K show that the proposed model outperforms or comparable against 11 state-of-the-art methods of the last decade. The key features of the proposed method are object completeness and efficiency in terms of computational time.

Hierarchical-model salient object detection based on manifold ranking

Salient object detection is a fundamental problem in both pattern recognition and image processing tasks. Previous salient object detection algorithms usually involve various features based on priors/assumptions about the properties of the objects. Inspired by the effectiveness of recently developed feature learning, we propose a novel deep salient object detection (DSOD) model using the deep residual network (ResNet 152-layers) for saliency computation. In particular, we model the image saliency from both local and global perspectives. In the local feature estimation stage, we detect local saliency by using a deep residual network (ResNet-L) which learns local region features to determine the saliency value of each pixel. In the global feature extraction stage, another deep residual network (ResNet-G) is trained to predict the saliency score of each image based on the global features. The final saliency map is generated by a conditional random field (CRF) to combining the local and global-level saliency map. Our DSOD model is capable of uniformly highlighting the objects-of-interest from complex background while well preserving object details. Quantitative and qualitative experiments on three benchmark datasets demonstrate that our DSOD method outperforms state-of-the-art methods in the salient object detection.

Even though there have been great advancements in computer vision tasks, the development of human visual attention models is still not well investigated. In day-to-day life, one can find ample applications of saliency detection in image and video processing. This paper presents an efficient visual saliency detection model based on Ripplet transform, which aims at detecting the salient region and achieving higher Receiver Operating Characteristics (ROC). Initially the feature maps are obtained from Ripplet transform in different scales and different directions of the image. The global and local saliency maps are computed based on the global probability density distribution and feature distribution of local areas, which are combined together to get the final saliency map. Ripplet-transform-based visual saliency detection is the novel approach carried out in this paper. Experimental results indicate that the proposed method based on Ripplet transformation can give excellent performance in terms of precision, recall, F measure and Mean Absolute Error (MAE), and is compared with 10 state-of-the-art methods on five benchmark datasets.

Event Abstract Back to Event Can state-of-the-art saliency systems model infant gazing behavior in tutoring situations? Vikram Tadmeri Narayan1*, Katrin S. Lohan1, Marko Tscherepanow1, Katharina Rohlfing1 and Britta Wrede1 1 Bielefeld University, Applied Informatics Group, CoR-Lab, Germany I. I NTRODUCTION The behavior for a humanoid robot is often modeled in accordance with human behavior. Current research suggests that analyzing infant behavior as a basis for designing the robot behavior can guide us to a natural robot interface. Based on this idea many researchers support saliency systems as a bottom-up inspired way to simulate infant-like gazing behavior. In the field of saliency systems many different approaches have proposed and quantified in terms of speed, quality and other technical issues. But so far, no one compared and quantified them in terms of natural infant tutor interaction. The question we would like to address in this paper is: Can state-of-the-art saliency systems model infant gazing behavior in tutoring situations? By addressing these issues we want to take a step towards an autonomous robot system, which could be used more natural interaction experiments in future. II. M ETHODS FOR A NALYZING We compared 7 different saliency systems with the gazing behavior of infants between the age of 8-11 months. The infant’s gazing behavior and the interaction structure of the tutoring situation was manually annotated. Based on this annotation we selected images as input for the saliency systems. The interaction data used in the course of this research is described in the following subsections. The employed saliency systems are briefly described in subsection II-B. A. Parent-child interaction Out of the well-known Bielefeld-motionese corpus [8], we focus on the youngest participants comprising 12 families of 8 to 11 months old children to analyze their gazing behavior, as the main feedback and controlling capabilities of these infants is based on the gazing behavior [13]. We selected the stacking cups task, because it was often analyzed [7],[12] and the interaction is very detailed analysed. We selected several pictures based on an action segmentation. The pictures represent the beginning and ending point of the stacking cups task. We have chosen this points because at these points the scene is changing. Setting: Parents were instructed to demonstrate a stacking cups task to an interaction partner. The first interaction partner was their infant, and another adult constituted the second. Fig. 1 illustrates the top-view of the experimental setup, and shows sample image frames of cameras which were set behind the parent and the interaction partner and focused on each of them. The stacking cups task was to sequentially pick up the green (a1), the yellow (a2), and the red (a3) cup and put them into the blue one on the white tray. B. Saliency Systems Frequency tuned saliency model [1] is perhaps the simplest of all the existing saliency systems. The absolute difference of each pixel to the image mean is accounted as its saliency value. They recommend to decompose a given input image into L*, a*, b* color space and perform the aforementioned operation on each of the color plane and fuse the results to compute the final saliency map. Graph-based visual saliency models [4] envisages the input image as a complete graph with each pixel as its nodes. A Weber’s [3] law based dissimilarity measure is employed to compute the dissimilarity between any two given pixels. A normalizing function is further employed to compute the final saliency map. Experiments carried out in [4], [10] have shown that the method has high performance in correlating with human eye gaze and also for object segmentation. The Multi resolution saliency map [5] is perhaps the most cited saliency system to date. The authors propose a computational model for the theoretical framework presented by Koch and Ulman [6] to model visual attention. An input image is analyzed with opponent color maps, pixels gradient and orientation at different scale spaces. A winner take all (WTA) network fuses these multiple maps into a single saliency map. Local steering kernel-based saliency [9] is a based on the center-surround paradigm, which is employed to compute the saliency of a selected region. They propose a novel local steering kernel which is employed to compute the similarity between a center region and the surrounding patches. The methodology is inherently robust to image brightness and contrast changes, has very few parameters that requires to be fined-tuned. Symmetry based Saliency [2] is compute rectangular regions of interest centered on a pixel and compute first order moments of features to calculate the saliency of the given pixel. The methodology is among the most best in terms of programming simplicity. The authors also claim the biological plausibility of the model. Random center-surround pattern based Saliency [11] is a methodology that employs a biologically plausible dissimilarity metric which is employed to compute the contrast between any two random pixels on the input image. The contrasts are updated and normalized to generate the final saliency map. The said methodology is shown to have state-of the-art performance in the task of salient region detection. For the Random rectangular regions of interest based saliency [10], the same authors as in [11] propose to compute frequent- tuned salient region detection [1], on random rectangular regions of an image for a large number of times and then sum them to compute the final saliency map. Such a formulation is shown to have excellent correlation with human eye-gaze and has good performance for the task of salient regions as shown in their experiments. The methodology also has only two parameters which requires fine tuning, which reduces the implementation complexity. III. DATA The beginning and concluding snapshots of 12 motionese videos were cropped, and were parsed into the seven aforementioned saliency systems. A 15 x 15 region centered on the maximally salient point was chosen as the focus of attention. see Fig. 1. Results: Results are summarized in Fig. 2. The percentages describes the matching accuracy of a given saliency system to the child’s eye gazing behavior. It seems that the methodology of Seo and Milanfar [9] has the best matching with the gazing behavior of a 8-11 month old child. Figure 1 Figure 2 Acknowledgements Tadmeri Narayan Vikram gratefully acknowledges the financial support from the EU FP7 Marie Curie ITN RobotDoc. Contract No. 235065. Katrin S. Lohan, Katharina Rohlfing, and Britta Wrede gratefully acknowledge the financial support from the FP7 European Project ITALK.

In this study, we propose a novel salient object detection strategy based on regional contrast and relative spatial compactness. Our algorithm consists of four basic steps. First, we learn color names offline using the probabilistic latent semantic analysis (PLSA) model to find the mapping between basic color names and pixel values. The color names can be used for image segmentation and region description. Second, image pixels are assigned to special color names according to their values, forming different color clusters. The saliency measure for every cluster is evaluated by its spatial compactness relative to other clusters rather than by the intra variance of the cluster alone. Third, every cluster is divided into local regions that are described with color name descriptors. The regional contrast is evaluated by computing the color distance between different regions in the entire image. Last, the final saliency map is constructed by incorporating the color cluster's spatial compactness measure and the corresponding regional contrast. Experiments show that our algorithm outperforms several existing salient object detection methods with higher precision and better recall rates when evaluated using public datasets.

A salient region is the most distinctive part of the image that captures human's attention. Saliency detection is a fundamental characteristic of the human visual system. Finding computational models which are able to detect salient regions is a challenging task for image processing and computer vision applications. Salient regions of various sizes can be detected from different scales. Therefore, selecting the best scales is an important issue. In this paper, an efficient multi-scale method to find salient regions is proposed. In order to include more features in evaluating saliency of a pixel, feature maps are generated using components of both the RGB and YUV color spaces. These features are combined into quaternions. Detecting salient regions of different sizes is addressed by utilizing a scale space analysis. Salient regions are detected by convolving the image amplitude spectrum with a low-pass Gaussian kernel of multiple scales. To incorporate more meaningful information, more than one scale is considered based on entropy criterion. The final saliency map is generated by normalizing the weighted saliency maps of these scales. Experiments are conducted on a dataset of natural images to evaluate the performance of the proposed method. Results show that the proposed method provides larger values of area under receiver operating characteristics curve, precision, recall and F-measure, in comparison to some of the state-of-the-art methods.

Microvascular invasion (MVI) is an important predictor of postoperative recurrence or poor outcomes of hepatocellular carcinoma (HCC). Radiomics is able to predict MVI in HCC preoperatively. This study aims to investigate the influence of different region of interest (ROI) sizes on CT-based radiomics model for MVI prediction in HCC. Patients with HCC with or without MVI confirmed by pathology and those who underwent preoperative plain or enhanced abdominal CT scans in the Third Xiangya Hospital of Central South University from January 2010 to December 2020 were retrospectively and consecutively included. According to the ratio of 7 to 3, the patients were randomly assigned into a training set and a validation set. Clinical data were collected from medical records, and radiomics features were extracted from the arterial phase (AP) and portal venous phase (PVP) of preoperatively acquired CT in all patients. Six different ROI sizes were employed. The original ROI (OROI) was manually delineated along the visible borders of the tumor layer-by-layer. The OROI was expanded out by 1-5 mm. The OROI was combined with 5 different peritumoral regions to generate the other 5 ROIs, named Plus1-Plus5. Feature extraction, dimension reduction, and model development were conducted in 6 different ROIs separately. Supporter vector machine (SVM) was used for model construction. Model performance was assessed via receiver operating characteristic (ROC) curve. A total of 172 HCC patients were included, in which 83 (48.3%) were MVI positive, and 89 (51.7%) were MVI negative. Three hundred and ninety-six features based on AP or PVP images were extracted from each ROI. After feature selection and dimension reduction, 4, 5, 15, 11, 6, and 3 features of OROI, Plus1, Plus2, Plus 3, Plus4, and Plus5 were selected for model construction, respectively. In the training set, the sensitivity, specificity, and area under the curve (AUC) of OROI were 0.759, 0.806, and 0.855, respectively. The AUC values of Plus2 (0.979) and Plus3 (0.954) were higher than that of OROI. The AUC values of Plus1 (0.802), Plus4 (0.792), and Plus5 (0.774) were not significantly different from those of OROI. In the validation set, the sensitivity, specificity, and AUC value of OROI were 0.640, 0.630, and 0.664, respectively. The AUC value of Plus3 was 0.903, which was higher than that of OROI. The AUC values of Plus1 (0.679), Plus2 (0.536), Plus4 (0.708), and Plus5 (0.757) were not significantly different from that of OROI (P>0.05). The size of ROI significantly inflluences on the performance of CT-based radiomics model for MVI prediction in HCC. Including appropriate area around the tumor into ROI could improve the predictive performance of the model, and 3 mm might be appropriate distance.

Saliency models provide heatmaps highlighting the probability of each pixel to attract human gaze. To define image's important regions, features maps are extracted. The rarity, surprise or contrast are computed leading to conspicuity maps, showing important regions of each feature map. The final saliency map is obtained by merging these maps. The fusion process is usually a linear combination of the maps where the coefficients show their importance. We propose a novel generic fusion mechanism based on 1) using a rarity-based attention module and 2) using neural networks to achieve the fusion. The first layer of the NN merges the weighted feature maps into a saliency map. The second layer takes into account the spatial information. The approach is compared to 8 models using 4 different comparison metrics on open state-of-the-art databases.

Machine learning-based receiver operating characteristic (ROC) curves for crisp and fuzzy classification of DNA microarrays in cancer research

Research Type: Level 3 - Retrospective cohort study, Case-control study, Meta-analysis of Level 3 studies Introduction/Purpose: The recently proposed Progressive Collapsing Foot Deformity (PCFD) classification system categorizes different deformity patterns into five classes (A, Hindfoot Valgus; B, Midfoot/Forefoot Abduction; C, Arch Collapse; D, Peritalar Subluxation; E, Ankle Valgus Instability). Currently, the decision-making process to determine the presence or absence of each deformity class relies on the surgeon's experience and interpretation of clinical and radiographic assessments, without established cut-off threshold values in the literature for PCFD measurements. This limits the reliability and reproducibility of the classification system. The goal of the current study was to establish cut-off threshold values for the most commonly used PCFD measurements assessing Class A to D deformities, using a large cohort of PCFD patients and controls. Methods: Two patient cohorts were included: a prospective cohort consisting of normally aligned and asymptomatic control volunteers (n=197 patients, 103 females/94 males, average age 41.7 years, average BMI 28.9) and a retrospective cohort of PCFD patients (n=321 patients, 136 females/185 males, average age 50.7 years, average BMI 29.8). All patients underwent Weight-Bearing Computed Tomography (WBCT) scans of the foot and ankle. Foot bone segmentation was performed semi-automatically using specialized software, followed by automatic measurements of various parameters including: Class A (Hindoot Moment Arm – HMA), Class B (Talonavicular Coverage Angle – TCA; Talus-First Metatarsal Angle Axial – TFMA-A), Class C (Talus-First Metatarsal Angle Sagittal – TFMA-S; Forefoot Arch Angle - FAA), Class D (Coverage Maps of the Subtalar Joint Middle-MF, Posterior Facets-PF; and Sinus Tarsi-ST). Receiver Operating Characteristic (ROC) curves, Youden's Indexes and Areas Under the Curve (AUC) were calculated for each measurement. P-values of 0.05 or less were considered significant. Results: The Cut-Off Threshold Values identified by the ROC curves using Youden’s Index for different PCFD measurements were as follows (all p-values < 0.0001): HMA of 13.9mm or more (AUC 85.3%) was considered diagnostic for Class A PCFD deformity. For Class B PCFD deformity, a TNCA of 38.7o or higher (AUC 84.4%) and TFMA-A of 20.3o or higher (AUC 82.1%) were considered diagnostic. For Class C PCFD deformity, a FAA of 8.7o or lower (AUC 83.9%) and TFMA-S of 18.7o or higher (AUC 82.9%) were diagnostic. For Class D Deformity, Coverage values for Middle Facet of 73.5% or lower (AUC 73.4%), Posterior Facet of 84.5% or lower (AUC 82.7%), and Sinus Tarsi of 25.7% or higher (AUC 84.4%) were diagnostic. Conclusion: In this diagnostic study involving 312 PCFD patients and 197 volunteer controls, we successfully determined the cut-off threshold values for key PCFD measurements related to Hindfoot Valgus (Class A), Midfoot/Forefoot Abduction (Class B), Arch Collapse (Class C), and Peritalar Subluxation (Class D) deformities. By utilizing a large cohort of patients and controls, we achieved high diagnostic accuracy for most measurements assessed (AUC of 80% or more) in identifying the presence or absence of these PCFD deformity classes. Our findings can potentially serve as normative threshold values to enhance the utilization, reliability, and reproducibility of the PCFD classification system. Figure 1 Scatter plots and ROC (Receiver Operating Characteristic) curves used to analyze parameters for the various deformity classifications in Progressive Collapsing Foot Deformity (PCFD). For the scatter plots, red indicates a normal control, green indicates an asymptomatic PCFD, and blue indicates a symptomatic PCFD.

Visual saliency detection plays a significant role in the fields of computer vision. In this paper, we introduce a novel saliency detection method based on weighted linear multiple kernel learning (WLMKL) framework, which is able to adaptively combine different contrast measurements in a supervised manner. As most influential factor is contrast operation in bottom-up visual saliency, an average weighted corner-surround contrast (AWCSC) is first designed to measure local visual saliency. Combined with common-used center-surrounding contrast (CESC) and global contrast (GC), three types of contrast operations are fed into our WLMKL framework to produce the final saliency map. We show that the assigned weights for each contrast feature maps are always normalized in our WLMKL formulation. In addition, the proposed approach benefits from the advantages of the contribution of each individual contrast feature maps, yielding more robust and accurate saliency maps. We evaluated our method for two main visual saliency detection tasks: human fixed eye prediction and salient object detection. The extensive experimental results show the effectiveness of the proposed model, and demonstrate the integration is superior than individual subcomponent.

To investigate the relation between obesity-related indices and mild cognitive impairment (MCI) in elderly patients with type 2 diabetes (T2D). A total of 597 eligible elderly patients with T2D were included in this retrospective study. All patients were divided into MCI group and normal cognitive group based on neuropsychological assessment. Twelve obesity-related indices were calculated, including body mass index (BMI), waist-hip ratio (WHR), waist-to-height ratio (WHtR), lipid accumulation product (LAP), body roundness index (BRI), conicity index (CI), visceral adiposity index (VAI), body adiposity index (BAI), abdominal volume index (AVI), a body shape index (ABSI), triglyceride glucose (TyG) index and cardiometabolic index (CMI). Multivariate logistic regression analysis, tests for trend and restricted cubic splines were used to assess the relationships between the tests for trend and MCI in elderly patients with T2D. Receiver operating characteristic (ROC) curves and areas under the curves (AUC) were used to assess the performance and predictive ability of the obesity-related indices for identifying MCI in elderly patients with T2D. Multivariate logistic regression showed that elevated BMI, WHR, WHtR, LAP, BRI, CI, VAI, AVI, TyG index, and CMI were associated with an increased risk of MCI in elderly T2D patients after adjusting for potential confounders (all P<0.05). In addition, TyG index, LAP, CMI, VAI, AVI, WHR, WHtR, BRI, and CI had negative correlations with Montreal Cognitive Assessment (MoCA) scores (all P<0.05). There was a significant linear trend between the levels of BMI (P for trend = 0.004, P for non-linearity = 0.637), WHR (P for trend = 0.006, P for non-linearity = 0.430), WHtR (P for trend <0.001, P for non-linearity = 0.452), BRI (P for trend <0.001, P for non-linearity = 0.252), AVI ( P for trend <0.001, P for non-linearity = 0.944), and TyG index (P for trend <0.001, P for non-linearity = 0.514) and risk of MCI in elderly patients with T2D after adjusting for potential confounders. There was a nonlinear association between LAP, VAI or CMI and risk of MCI in elderly patients with T2D (all P for non-linearity < 0.001). CMI had the greatest AUC (AUC = 0.682), followed by VAI (AUC = 0.679), TyG index (AUC = 0.673), LAP (AUC = 0.669), AVI (AUC = 0.580), WHtR and BRI (AUC = 0.575), BMI (AUC = 0.560), CI (AUC = 0.556), WHR (AUC = 0.554), BAI (AUC = 0.547), and ABSI (AUC = 0.536). Elevated obesity-related indices, particularly CMI, VAI, TyG index and LAP, which displayed the higher predictive power, were instrumental in forecasting and evaluating MCI in elderly T2D patients. These findings may provide clues for future studies exploring early diagnostic biomarkers and treatment of MCI in elderly T2D patients.

Objective:Blood-based biomarkers represent a scalable and accessible approach for the detection and monitoring of Alzheimer’s disease (AD). Plasma phosphorylated tau (p-tau) and neurofilament light (NfL) are validated biomarkers for the detection of tau and neurodegenerative brain changes in AD, respectively. There is now emphasis to expand beyond these markers to detect and provide insight into the pathophysiological processes of AD. To this end, a reactive astrocytic marker, namely plasma glial fibrillary acidic protein (GFAP), has been of interest. Yet, little is known about the relationship between plasma GFAP and AD. Here, we examined the association between plasma GFAP, diagnostic status, and neuropsychological test performance. Diagnostic accuracy of plasma GFAP was compared with plasma measures of p-tau181 and NfL.Participants and Methods:This sample included 567 participants from the Boston University (BU) Alzheimer’s Disease Research Center (ADRC) Longitudinal Clinical Core Registry, including individuals with normal cognition (n=234), mild cognitive impairment (MCI) (n=180), and AD dementia (n=153). The sample included all participants who had a blood draw. Participants completed a comprehensive neuropsychological battery (sample sizes across tests varied due to missingness). Diagnoses were adjudicated during multidisciplinary diagnostic consensus conferences. Plasma samples were analyzed using the Simoa platform. Binary logistic regression analyses tested the association between GFAP levels and diagnostic status (i.e., cognitively impaired due to AD versus unimpaired), controlling for age, sex, race, education, and APOE e4 status. Area under the curve (AUC) statistics from receiver operating characteristics (ROC) using predicted probabilities from binary logistic regression examined the ability of plasma GFAP to discriminate diagnostic groups compared with plasma p-tau181 and NfL. Linear regression models tested the association between plasma GFAP and neuropsychological test performance, accounting for the above covariates.Results:The mean (SD) age of the sample was 74.34 (7.54), 319 (56.3%) were female, 75 (13.2%) were Black, and 223 (39.3%) were APOE e4 carriers. Higher GFAP concentrations were associated with increased odds for having cognitive impairment (GFAP z-score transformed: OR=2.233, 95% CI [1.609, 3.099], p&lt;0.001; non-z-transformed: OR=1.004, 95% CI [1.002, 1.006], p&lt;0.001). ROC analyses, comprising of GFAP and the above covariates, showed plasma GFAP discriminated the cognitively impaired from unimpaired (AUC=0.75) and was similar, but slightly superior, to plasma p-tau181 (AUC=0.74) and plasma NfL (AUC=0.74). A joint panel of the plasma markers had greatest discrimination accuracy (AUC=0.76). Linear regression analyses showed that higher GFAP levels were associated with worse performance on neuropsychological tests assessing global cognition, attention, executive functioning, episodic memory, and language abilities (ps&lt;0.001) as well as higher CDR Sum of Boxes (p&lt;0.001).Conclusions:Higher plasma GFAP levels differentiated participants with cognitive impairment from those with normal cognition and were associated with worse performance on all neuropsychological tests assessed. GFAP had similar accuracy in detecting those with cognitive impairment compared with p-tau181 and NfL, however, a panel of all three biomarkers was optimal. These results support the utility of plasma GFAP in AD detection and suggest the pathological processes it represents might play an integral role in the pathogenesis of AD.

In recent years, many research works indicate that human's visual attention is very helpful in some research areas that are related to computer vision, such as object recognition, scene understanding and object-based image/video retrieval or annotation. This paper presents a visual attention model for natural scenes based on a dynamic feature combination strategy. The model can be divided into three parts, which are feature extraction, dynamic feature combination and salient objects detection. First, the saliency features of color, information entropy and salient boundary are extracted from an original colored image. After that, two different evaluation measurements are proposed for two different categories of feature maps defined in this dynamic combination strategy, which measures the contribution of each feature map to saliency and carries out a dynamic weighting of individual feature maps. Finally, salient objects are located from an integrated saliency map and a computational method is given to simulate the location shift of the real human visual attention. Experimental results show that this model is effective and robust for saliency detection in natural scenes, also similar to the real human visual attention mechanism.