False alarm rate: a critical performance measure for face recognition

The development of tools for automated analysis of data collected for mine countermeasures (MCM) search, classify and map (SCM) has predominantly focused on automated target recognition for individual mines. As MCM target detection becomes more automated, detecting spatial patterns in the positioning of mines should follow. Natural restrictions on mine deployment such as depth and bottom contour as well as mine deployment methodology and protocol effectively create mine line patterns that are typical to coastal areas. We have developed a mine line detection algorithm that uses nonparametric spatial analysis to identify mine-like contacts (MLCs) that fit into mine line patterns dictated by standard mine laying protocol in coastal regions. In general, a set of MLCs reported from a mine detection sensor will consist of both actual mines and false alarms; the algorithm classifies every MLC in this set into one of three classes: fixed — the MLCψfits into a mine line pattern with fixed mine spacing; random -the MLCψfits into a mine line pattern with random mine spacing; or false alarm-ψ-the MLCψdoes not fit into a mine line pattern and is subsequently not considered a mine. The latest version of the algorithm utilizes nonparametric techniques and requires no knowledge of either the detection sensor's probability of detection (P d ) or probability of false alarm (P fa ). A nonparametric anomaly detector first identifies regions with unusually high MLC density for more detailed analysis. Suspected regions are subsequently analyzed for unusually regular patterns of MLCs. The probabilities that the observed target densities and target patterns could occur simply by chance are estimated by comparison with results from nearby areas using basic information about mine deployment. To ensure that a mine line's full extent is captured, the algorithm attempts to grow a line after it is detected. And a more detailed probability analysis distinguishes random mine lines from fixed-spaced mine lines. Extensive Matlab simulations characterize the algorithm's performance across a wide range of operating environments and simulated minefield configurations. In total, ten replications each of 1,367 different configurations have been evaluated. These configurations include three different water depths (10 ft., 6 ft. and 200 ft.), six false notional alarm rates (1, 2,5,10,15 and 20 false alarms per sq. NM), three false alarm distributions (uniform, sloped, and striped), three sensor probabilities of detections (0.4, 0.6 and 0.9), three target confidence scores (0.6,0.8 and 1.0) and four false alarm confidence scores (0.5, 0.6, 0.8 and 1.0). The range of simulation parameters was selected in part as an attempt to push the algorithm past its acceptable performance limits. However, the algorithm performed remarkably well even under the worst conditions. For MLC sets with false alarm rates as high as 20 per square nautical mile, the algorithm correctly classifies over 98% of the MLCs as mines and detects 97% of the mine lines. Furthermore, it reduces the false alarm rate from 20 to 0.2 per square nautical mile with a false mine line rate of 0.14 false lines per square nautical mile. The algorithm achieves even lower false alarm rates when the initial false alarm rate is less than 20. The automated mine line detection discovers many obscure lines that are virtually undetectable to the naked eye.

Development of an algorithm to discriminate between valid and false alarms in a loose-parts monitoring system

High-resolution space-borne synthetic aperture radar (SAR) images are important sources of information for marine monitoring and target recognition. For inshore port scenes, there is a wide variety of ship targets; moreover, as the scattering intensity of the land background is much higher than that of the sea surface, similar structures on land in SAR images can cause significant interference to the identification of ship targets, resulting in a high false alarm rate and false recognition rate. Therefore, the classification of ship targets in SAR images still requires professional interpretation and correction to eliminate possible false alarms and recognitions. In this paper, by imitating the basic structural requirements for manual discrimination of target categories, a space-borne SAR high-resolution inshore ship classification and correction method of “Pre-recognition – Fine Recognition – Matching Correction” is designed based on the YOLO recognition network. Taking three types of typical ship targets as examples, a high-resolution SAR inshore ship target dataset and a high-resolution SAR ship component architecture dataset are constructed to train the pre-recognition network and the fine recognition network respectively. The latter is based on the former to recognize the component composition of possible targets, and forms a component arrangement architecture vector. The final recognition results are obtained by matching and correcting the similarity distance with the three types of targets. This method can reduce the false alarm rate in ship target recognition in complex inshore port scenes to less than 10% and correct some false recognition results, thereby improving the accuracy of ship target identification and reducing the cost for manual correction.

Chapter 43 - Statistical considerations when making potential multiple comparisons

ObjectiveTo determine if reviewer experience impacts the ability to discriminate between human-written and ChatGPT-written abstracts.MethodsThirty reviewers (10 seniors, 10 juniors, and 10 residents) were asked to differentiate between 10 ChatGPT-written...

When old/new recognition memory is tested with equal numbers of studied and nonstudied items and no rewards or instructions that favour one response over the other, there is no obvious reason for response bias. In line with this, Canadian undergraduates have shown, on average, a neutral response bias when we tested them on recognition of common English words. By contrast, most subjects we have tested on recognition of richly detailed images have shown a conservative bias: they more often erred by missing a studied image than by judging a nonstudied image as studied. Here, in an effort to better understand these materials-based bias effects (MBBEs), we examined changes in hit and false alarm (FA) rates (and in sensitivity and bias) from the first to fourth quartile of a recognition memory test in eight experiments in which undergraduates studied words and/or images of paintings. Response bias for images tended to increase across quartiles, whereas bias for words showed no consistent pattern across quartiles. This pattern could be described as an increase in the MBBE over the course of the test, but the underlying patterns for hits and FAs are not easily reconciled with this interpretation. Hit rates decreased over the course of the test for both materials types, with that decline tending to be steeper for images than words. For words, FA rates tended to increase across quartiles, whereas for paintings FA rates did not increase across quartiles. We discuss implications of these findings for theoretical accounts of the MBBE.

The two primary measures of detection performance are probability of detection (PD) and false alarm rate (FAR). Since these are statistical measures, the large statistical uncertainties associated with measurements on small data sets impose a severe limitation on the interpretation and extrapolation of the test results. The main difficulty with the data collected from many of the landmine detection experiments is that the number of mine targets is small, and the clutter data is measured on small areas. As a result of analyzing data from several detection tests for both UXO and mine detection technologies, the Institute for Defense Analyses (IDA) has generated a number of suggestions to improve the results obtained from future demonstrations and tests. In the absence of a rigorous error analysis, we can estimate the uncertainty in a probability of detection measurement with a simple model. If detections can be described as a binomial process weighted by the true probability of detection, then uncertainty equals (root)N, where N is the number of mines detected, and uncertainty in PD (percent) equals (root)N/N X 100 for large N. A similar statistical uncertainty is encountered in the measurement of probability of false alarm. An opportunity for a false alarm can be defined by the amount of ground covered by the projected area of the target plus an allowable miss distance, say 1 m2. Thus, for each 1 m2 of ground that it passes over, the system has one opportunity to declare a false alarm. A test field that is 1 m by 20 m offers only 20 m2 of area and thus only 20 samples for probability of false alarm measurement. A site that is 4 m wide and 4 km long covers 16,000 m2. Thus, there will be a substantial increase in the clutter environment samples by the sensor and a concomitant improvement in false alarm measurements. Of course, false alarm density will be highly site dependent, so extrapolation of the result is still uncertain. Unfortunately, this limitation is insurmountable in single test and at a single site.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We developed and tested the Bayesian multiple comparison correction method for Bayesian voxelwise second-level fMRI analysis with R. The performance of the developed method was tested with simulation and real image datasets. First, we compared false alarm and hit rates, which were used as proxies for selectivity and sensitivity, respectively, between Bayesian and classical inference methods. For the comparison, we created simulated images, added noise to the created images, and analyzed the noise-added images while applying Bayesian and classical multiple comparison correction methods. Second, we analyzed five real image datasets to examine how our Bayesian method worked in realistic settings. When the performance assessment was conducted, the Bayesian correction method demonstrated good sensitivity (hit rate ≥ 75%) and acceptable selectivity (false alarm rate < 10%) when N ≥ 8. Furthermore, the Bayesian correction method showed better sensitivity compared with the classical correction method while maintaining the aforementioned acceptable selectivity.

A regression method for estimating performance in a rapid serial visual presentation target detection task

Vision-based pedestrian detection technology is widely used in security surveillance, such as transportation, prisons, and important places. Security monitoring requires pedestrian detection not only to be accurate and reliable, but also to meet the needs of real-time monitoring, and the false alarm rate is low. The existing pedestrian learning method based on deep learning has high reliability, but the timeliness cannot meet the monitoring requirements. Traditional pedestrian detection methods are time-sensitive, but false alarm rates tend to be higher. This paper proposes a self-learning pedestrian detection method, combined with motion detection, HOG feature extraction, and SVM classification for preliminary pedestrian detection. The false alarms caused during the pedestrian detection process are manually confirmed, and then the system automatically detects the false alarm image at a specific time. Self-learning, retraining the classifier, so as to continuously reduce the pedestrian detection false alarm rate and improve the pedestrian detection performance index through self-learning during the system operation, and the timeliness can meet the needs of video surveillance. Experiments show that the false alarm rate of this method is low, timeliness is high, and the recognition rate is also high.

Cyber-Physical Systems (CPS) tightly couple information technology with physical processes, which rises new vulnerabilities such as physical attacks that are beyond conventional cyber attacks. Attackers may non-invasively compromise sensors and spoof the controller to perform unsafe actions. This issue is even emphasized with the increasing autonomy in CPS. While this fact has motivated many defense mechanisms against sensor attacks, a clear vision on the timing and usability (or the false alarm rate) of attack detection still remains elusive. Existing works tend to pursue an unachievable goal of minimizing the detection delay and false alarm rate at the same time, while there is a clear trade-off between the two metrics. Instead, we argue that attack detection should bias different metrics when a system sits in different states. For example, if the system is close to unsafe states, reducing the detection delay is preferable to lowering the false alarm rate, and vice versa. To achieve this, we make the following contributions. In this paper, we propose a real-time adaptive sensor attack detection framework. The framework can dynamically adapt the detection delay and false alarm rate so as to meet a detection deadline and improve the usability according to different system status. The core component of this framework is an attack detector that identifies anomalies based on a CUSUM algorithm through monitoring the cumulative sum of difference (or residuals) between the nominal (predicted) and observed sensor values. We augment this algorithm with a drift parameter that can govern the detection delay and false alarm. The second component is a behavior predictor that estimates nominal sensor values fed to the core component for calculating the residuals. The predictor uses a deep learning model that is offline extracted from sensor data through leveraging convolutional neural network (CNN) and recurrent neural network (RNN). The model relies on little knowledge of the system (e.g., dynamics), but uncovers and exploits both the local and complex long-term dependencies in multivariate sequential sensor measurements. The third component is a drift adaptor that estimates a detection deadline and then determines the drift parameter fed to the detector component for adjusting the detection delay and false alarms. Finally, we implement the proposed framework and validate it using realistic sensor data of automotive CPS to demonstrate its efficiency and efficacy.

In this article, an improved negative selection algorithm (INSA) has been proposed to identify faulty sensor nodes in wireless sensor network (WSN) and then the faults are classified into soft permanent, soft intermittent, and soft transient fault using the support vector machine technique. The performance metrics such as fault detection accuracy, false alarm rate, false positive rate, diagnosis latency (DL), energy consumption, fault classification accuracy (FCA), and false classification rate (FCR) are used to evaluate the performance of the proposed INSA. The simulation result shows that the INSA gives better result as compared to the existing algorithms in terms of performance metrics. The fault classification performance is measured by FCA and FCR. It has also seen that the proposed algorithm gives less DL and consumes less energy than that of existing algorithms proposed by Mohapatra et al, Zhang et al, and Panda et al for WSN.

Study 1 (N=205) reveals that witnesses often provide vague descriptions. Witnesses leave out information such as sex and race that they certainly noticed (default values). Study 2 (N=89) weakly supports the claim (Luus & Wells, 1991) that correct identification rates from lineups are enhanced by selecting foils who fit the description of the criminal rather than foils who are highly similar to the suspect. Study 3 (N=210) indicates that false identification rates can be inflated by selecting lineup foils who fit vague descriptions of the criminal but otherwise differ from the suspect on default values.

Face recognition technology has recently become controversial over concerns about possible bias due to accuracy varying based on race or skin tone. We explore three important aspects of face recognition technology related to this controversy. Using two different deep convolutional neural network face matchers, we show that for a fixed decision threshold, the African-American image cohort has a higher false match rate (FMR), and the Caucasian cohort has a higher false nonmatch rate. We present an analysis of the impostor distribution designed to test the premise that darker skin tone causes a higher FMR, and find no clear evidence to support this premise. Finally, we explore how using face recognition for one-to-many identification can have a very low false-negative identification rate and still present concerns related to the false-positive identification rate. Both the ArcFace and VGGFace2 matchers and the MORPH dataset used in our experiments are available to the research community so that others should be able to reproduce or reanalyze our results.

In this letter, we propose a method to reduce the number of false alarms in a wavelength–resolution synthetic aperture radar (SAR) change detection scheme by using a convolutional neural network (CNN). The detection is performed in two steps: change analysis and object classification. A simple technique for wavelength–resolution SAR change detection is implemented to extract potential targets from the image of interest. A CNN is then used for classifying the change map detections as either a target or nontarget, further reducing the false alarm rate (FAR). The scheme is tested for the CARABAS-II data set, where only three false alarms over a testing area of 96 km <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> are reported while still sustaining a probability of detection above 96%. We also show that the network can still reduce the FAR even when the flight heading of the SAR system measurement campaign differs by up to 100° between the images used for training and test.