<title>Performance evaluation of document layout analysis algorithms on the UW data set</title>

The importance of building an objective and accurate benchmarking platform for document layout analysis (DLA) algorithms has been realized by more and more researchers. In this paper, an experimental benchmarking framework for DLA algorithms is proposed. Firstly, a ground-truth database is built where test images and expected layout analysis results for corresponding image are stored. Secondly, the analysis result of a certain test image given by a DLA algorithm is evaluated based on the expected results in the ground-truth database, and a performance metric is given. It is also important to note that the reasonable layout analysis result for a given image might not be unique. Therefore, we implemented a flexible learning mechanism into our framework to get a more accurate performance metric. In the end, we apply our framework to the benchmarking of a document layout analysis algorithm and the experimental results are given.

In this work we study the impact of noise on the training of object detection\nnetworks for the medical domain, and how it can be mitigated by improving the\ntraining procedure. Annotating large medical datasets for training data-hungry\ndeep learning models is expensive and time consuming. Leveraging information\nthat is already collected in clinical practice, in the form of text reports,\nbookmarks or lesion measurements would substantially reduce this cost.\nObtaining precise lesion bounding boxes through automatic mining procedures,\nhowever, is difficult. We provide here a quantitative evaluation of the effect\nof bounding box coordinate noise on the performance of Faster R-CNN object\ndetection networks for breast mass detection. Varying degrees of noise are\nsimulated by randomly modifying the bounding boxes: in our experiments,\nbounding boxes could be enlarged up to six times the original size. The noise\nis injected in the CBIS-DDSM collection, a well curated public mammography\ndataset for which accurate lesion location is available. We show how, due to an\nimperfect matching between the ground truth and the network bounding box\nproposals, the noise is propagated during training and reduces the ability of\nthe network to correctly classify lesions from background. When using the\nstandard Intersection over Union criterion, the area under the FROC curve\ndecreases by up to 9%. A novel matching criterion is proposed to improve\ntolerance to noise.\n

An object detector based on convolutional neural network (CNN) has been widely used in the field of computer vision because of its simplicity and efficiency. The average accuracy of CNN model detection results in the object detector is greatly affected by the loss function. The precision of the localization algorithm in the loss function is the main factor affecting the result. Based on the complete intersection over union (CIoU) loss function, an improved penalty function is proposed to improve the localization accuracy. Specifically, the algorithm more comprehensively considers matching bounding boxes between prediction with ground truth, using the proportional relationship of the aspect ratio from both bounding boxes. Under the same aspect ratio of the two bounding boxes, the influence factors of the prediction box on localization accuracy were considered. In this way, the function of the penalty function is strengthened, and localization accuracy of the network model improved. This loss function is called Improved CIoU (ICIoU). Experiments on the Udacity, PASCAL VOC, and MS COCO datasets have demonstrated the effectiveness of ICIoU in improving localization accuracy of network models by using the one-stage object detector YOLOv4. Compared with CIoU, the proposed ICIoU improved average precision (AP) by 0.57% and AP75 by 0.12% on Udacity, AP by 0.26% and AP75 by 1.28% on PASCAL VOC, and AP by 0.06% and AP75 by 0.65% on MS COCO.

Automatic detection of weapons is significant for improving security and well being of individuals, nonetheless, it is a difficult task due to large variety of size, shape and appearance of weapons. View point variations and occlusion also are reasons which makes this task more difficult. Further, the current object detection algorithms process rectangular areas, however a slender and long rifle may really cover just a little portion of area and the rest may contain unessential details. To overcome these problem, we propose a CNN architecture for Orientation Aware Weapons Detection, which provides oriented bounding box with improved weapons detection performance. The proposed model provides orientation not only using angle as classification problem by dividing angle into eight classes but also angle as regression problem. For training our model for weapon detection a new dataset comprising of total 7801 weapons images is gathered from the web and then manually annotated with position oriented bounding boxes. Our dataset provides not only oriented bounding box as ground truth but also horizontal bounding box. We also provide our dataset in multiple formats of modern object detectors for further research in this area. The proposed model is evaluated on this dataset, and the comparative analysis with off-the shelf object detectors yields superior performance of proposed model, measured with standard evaluation strategies. The dataset and the model implementation are made publicly available at this link: https://bit.ly/2TyZICF .

R-trees can be used to store and query sets of point data in two or more dimensions. An easy way to construct and maintain R-trees for two-dimensional points, due to Kamel and Faloutsos, is to keep the points in the order in which they appear along the Hilbert curve. The R-tree will then store bounding boxes of points along contiguous sections of the curve, and the efficiency of the R-tree depends on the size of the bounding boxes---smaller is better. Since there are many different ways to generalize the Hilbert curve to higher dimensions, this raises the question which generalization results in the smallest bounding boxes. Familiar methods, such as the one by Butz, can result in curve sections whose bounding boxes are a factor $\Omega(2^{d/2})$ larger than the volume traversed by that section of the curve. Most of the volume bounded by such bounding boxes would not contain any data points. In this paper we present a new way of generalizing Hilbert's curve to higher dimensions, which results in much tighter bounding boxes: they have at most 4 times the volume of the part of the curve covered, independent of the number of dimensions. Moreover, we prove that a factor 4 is asymptotically optimal.

The shape initialization is a crucial step for face alignment. In the literature, many approaches use the ground truth points to compute the bounding box. However, it is not always possible to detect an accurate bounding box in real applications due to various adverse factors. In this work, an effective initialization approach for face alignment is proposed. Firstly a modified Deformable Part Models (DPM) is used to estimate the face pose and the bounding box to obtain an initial shape. Then by detecting the two pupils, the roll rotation of the face is measured to correct the initial shape. To further increase the robustness and accuracy of face alignment, multiple initial shapes for each face are generated, then each one is refined by a cascade regression-based approach and we can get multiple shape estimations. Finally a better final shape is obtained by fusing the multiple estimations via the structured SVM learning. Experiments on challenging datasets and comparison with the state-of-the-art methods validate our proposed method in unconstrained environment.

Automatic localization of pneumonia on chest X-rays (CXRs) is highly desirable both as an interpretive aid to the radiologist and for timely diagnosis of the disease. However, pneumonia's amorphous appearance on CXRs and complexity of normal anatomy in the chest present key challenges that hinder accurate localization. Existing studies in this area are either not optimized to preserve spatial information of abnormality or depend on expensive expert-annotated bounding boxes. We present a novel generative adversarial network (GAN)-based machine learning approach for this problem, which is weakly supervised (does not require any location annotations), was trained to retain spatial information, and can produce pixel-wise abnormality maps highlighting regions of abnormality (as opposed to bounding boxes around abnormality). Our method is based on the Wasserstein GAN framework and, to the best of our knowledge, the first application of GANs to this problem. Specifically, from an abnormal CXR as input, we generated the corresponding pseudo normal CXR image as output. The pseudo normal CXR is the "hypothetical" normal, if the same abnormal CXR were not to have any abnormalities. We surmise that the difference between the pseudo normal and the abnormal CXR highlights the pixels suspected to have pneumonia and hence is our output abnormality map. We trained our algorithm on an "unpaired" data set of abnormal and normal CXRs and did not require any location annotations such as bounding boxes/segmentations of abnormal regions. Furthermore, we incorporated additional prior knowledge/constraints into the model and showed that they help improve localization performance. We validated the model on a data set consisting of 14184 CXRs from the Radiological Society of North America pneumonia detectionchallenge. We evaluated our methods by comparing the generated abnormality maps with radiologist annotated bounding boxes using receiver operating characteristic (ROC) analysis, image similarity metrics such as normalized cross-correlation/mutual information, and abnormality detection rate.We also present visual examples of the abnormality maps, covering various scenarios of abnormality occurrence. Results demonstrate the ability to highlight regions of abnormality with the best method achieving an ROC area under the curve (AUC) of 0.77 and a detection rate of 85%.The GAN tended to perform better as prior knowledge/constraints were incorporated into themodel. We presented a novel GAN based approach for localizing pneumonia on CXRs that (1) does not require expensive hand annotated location ground truth; and (2) was trained to produce abnormality maps at the pixel level as opposed to bounding boxes. We demonstrated the efficacy of our methods via quantitative and qualitative results.

We present an approach MSIL-CRF that incorporates multiple instance learning (MIL) into conditional random fields (CRFs). It can generalize CRFs to work on training data with uncertain labels by the principle of MIL. In this work, it is applied to saving manual efforts on annotating training data for semantic segmentation. Specifically, we consider the setting in which the training dataset for semantic segmentation is a mixture of a few object segments and an abundant set of objects' bounding boxes. Our goal is to infer the unknown object segments enclosed by the bounding boxes so that they can serve as training data for semantic segmentation. To this end, we generate multiple segment hypotheses for each bounding box with the assumption that at least one hypothesis is close to the ground truth. By treating a bounding box as a bag with its segment hypotheses as structured instances, MSIL-CRF selects the most likely segment hypotheses by leveraging the knowledge derived from both the labeled and uncertain training data. The experimental results on the Pascal VOC segmentation task demonstrate that MSIL-CRF can provide effective alternatives to manually labeled segments for semantic segmentation.

The main goal of this work is to automatically segment colorectal tumors in 3D T2-weighted (T2w) MRI with reasonable accuracy. For such a purpose, a novel deep learning-based algorithm suited for volumetric colorectal tumor segmentation is proposed. The proposed CNN architecture, based on densely connected neural network, contains multiscale dense interconnectivity between layers of fine and coarse scales, thus leveraging multiscale contextual information in the network to get better flow of information throughout the network. Additionally, the 3D level-set algorithm was incorporated as a postprocessing task to refine contours of the network predicted segmentation. The method was assessed on T2-weighted 3D MRI of 43 patients diagnosed with locally advanced colorectal tumor (cT3/T4). Cross validation was performed in 100 rounds by partitioning the dataset into 30 volumes for training and 13 for testing. Three performance metrics were computed to assess the similarity between predicted segmentation and the ground truth (i.e., manual segmentation by an expert radiologist/oncologist), including Dice similarity coefficient (DSC), recall rate (RR), and average surface distance (ASD). The above performance metrics were computed in terms of mean and standard deviation (mean ± standard deviation). The DSC, RR, and ASD were 0.8406 ± 0.0191, 0.8513 ± 0.0201, and 2.6407 ± 2.7975 before postprocessing, and these performance metrics became 0.8585 ± 0.0184, 0.8719 ± 0.0195, and 2.5401 ± 2.402 after postprocessing, respectively. We compared our proposed method to other existing volumetric medical image segmentation baseline methods (particularly 3D U-net and DenseVoxNet) in our segmentation tasks. The experimental results reveal that the proposed method has achieved better performance in colorectal tumor segmentation in volumetric MRI than the other baseline techniques.

We introduce a new large-scale data set of video URLs with densely-sampled object bounding box annotations called YouTube-BoundingBoxes (YT-BB). The data set consists of approximately 380,000 video segments about 19s long, automatically selected to feature objects in natural settings without editing or post-processing, with a recording quality often akin to that of a hand-held cell phone camera. The objects represent a subset of the COCO [32] label set. All video segments were human-annotated with high-precision classification labels and bounding boxes at 1 frame per second. The use of a cascade of increasingly precise human annotations ensures a label accuracy above 95% for every class and tight bounding boxes. Finally, we train and evaluate well-known deep network architectures and report baseline figures for per-frame classification and localization. We also demonstrate how the temporal contiguity of video can potentially be used to improve such inferences. The data set can be found at https://research.google.com/youtube-bb. We hope the availability of such large curated corpus will spur new advances in video object detection and tracking.

Aerial imagery plays an important role in land-use planning, population analysis, precision agriculture, and unmanned aerial vehicle tasks. However, existing aerial image datasets generally suffer from the problem of inaccurate labeling, single ground truth type, and few category numbers. In this work, we implement a simulator that can simultaneously acquire diverse visual ground truth data in the virtual environment. Based on that, we collect a comprehensive Virtual AeriaL Image Dataset named VALID, consisting of 6690 high-resolution images, all annotated with panoptic segmentation on 30 categories, object detection with oriented bounding box, and binocular depth maps, collected in 6 different virtual scenes and 5 various ambient conditions (sunny, dusk, night, snow and fog). To our knowledge, VALID is the first aerial image dataset that can provide panoptic level segmentation and complete dense depth maps. We analyze the characteristics of VALID and evaluate state-of-the-art methods for multiple tasks to provide reference baselines. The experiment results demonstrate that VALID is well presented and challenging. The dataset is available at https://sites.google.com/view/valid-dataset/.

Cross-Project Defect Prediction (CPDP) as a means to focus quality assurance of software projects was under heavy investigation in recent years. However, within the current state-of-the-art it is unclear which of the many proposals performs best due to a lack of replication of results and diverse experiment setups that utilize different performance metrics and are based on different underlying data. Within this article [2, 3], we provide a benchmark for CPDP. Our benchmark replicates 24 CPDP approaches proposed by researchers between 2008 and 2015. Through our benchmark, we answer the following research questions: • RQ1: Which CPDP approaches perform best in terms of F-measure, G-measure, AUC, and MCC? • RQ2: Does any CPDP approach consistently fulfill the performance criteria for successful predictions postulated by Zimmermann et al. [4], i.e., have at least 0.75 recall, 0.75 precision, and 0.75 accuracy? • RQ3: What is the impact of using only larger products (> 100 instances) with a certain balance (at least 5% defective instances and at least 5% non-defective instances) on the benchmark results? • RQ4: What is the impact of using a relatively small subset of a larger data set on the benchmark results? We identified 5 public data sets, which contain defect data about 86 software products that we used to answer these research question. The advantage of using multiple data sets was that we could increase the number of software products and, thereby, increase the external validity of our results. Moreover, we wanted to use multiple performance criteria for the evaluation of the CPDP approaches. Therefore, RQ1 ranks approaches not just using a single criterion, but using the four performance metrics AUC, F-measure, G-measure, and MCC. Existing approaches for the ranking of statistically different approaches neither account for software products from different data sets, nor multiple performance metrics. Therefore, we defined a new approach for the combination of separate rankings for the performance criteria and data sets, into one common ranking. Figure 1 depicts the results for RQ1. The results show that an approach proposed by Camargo Cruz and Ochimizu [1] performs best and even outperforms cross-validation. Moreover, our results show that only 6 of the 24 approaches outperform one of our baselines, i.e., using all data for training without any transfer learning. Regarding RQ2, we determined that predictions only seldomly achieve a high performance of 0.75 recall, precision, and accuracy. The best CPDP approaches only fulfill the criterion for 4 of the 86 products, i.e., 4.6% of the time. Thus, CPDP still has not reached a point where the performance of the results is sufficient for the application in practice. RQ3 and RQ4 were used to see if results are affected by subsetting data, as is often done for defect prediction experiments. For RQ3, i.e., using a large subset, we determined no difference between using all data and using the subset. For RQ4, i.e., using a small subset of of data, we found that there are statistically signifcant differences in reported performances of up to 5%. Thus, the use of small subsets should be avoided.

Three dimensional point cloud data sets are easy to acquire and manipulate, but are often too large to process directly for embedded real-time applications. The spatial information in a point cloud can be represented in a variety of reduced forms, such as voxel grids, Gaussian mixture models, or spatial semantic structures. In this article, we show how a segmented point cloud can be represented as a spatial relationship graph using bounding boxes and triangular fuzzy numbers. This model is a lightweight encoding of the relative distance and direction between objects, and can be used to describe and query for particular spatial configurations using linguistic terms in a multicriteria framework. We show how this approach can be applied on a hand-segmented subset of the NPM3D data set with several illustrative examples. The work herein has useful applications in many applied domains, such as human-robot interaction with unmanned aerial systems.

Target tracking is an important part of many applications in computer vision. With continuous researching and developing, more and more advanced tracking algorithms have been put forward. Therefore, a benchmark and dataset used to evaluate the most advanced tracking algorithms are needed. One of the most widely used benchmarks is Online Tracking Benchmark (OTB). This benchmark has a clear statement of the video attributes, and each frame in the sequences is appropriately provided with the ground-truth annotations. There may be some errors due to the annotations done by manual operation for bounding box and thus we find and modify the annotations that have some errors. The error means the bounding box for tracking target in some frames is beyond boundaries of corresponding frames. Then we study the effect of these small errors on the tracking algorithm evaluation in the corresponding video sequences.

There is a growing demand for surveillance systems that can detect fall-down events because of the increased number of surveillance cameras being installed in many public indoor and outdoor locations. Fall-down event detection has been vigorously and extensively researched for safety purposes, particularly to monitor elderly peoples, patients, and toddlers. This computer vision detector has become more affordable with the development of high-speed computer networks and low-cost video cameras. This paper proposes moving object detection method based on human motion analysis for human fall-down events. The method comprises of three parts, which are preprocessing part to reduce image noises, motion detection part by using TV-L1 optical flow algorithm, and performance measure part. The last part will analyze the results of the object detection part in term of the bounding boxes, which are compared with the given ground truth. The proposed method is tested on Fall Down Detection (FDD) dataset and compared with Gunnar-Farneback optical flow by measuring intersection over union (IoU) of the output with respect to the ground truth bounding box. The experimental results show that the proposed method achieves an average IoU of 0.92524.