The consequences of statistical tests on proxy variables, an application to short-TE as an estimation for MEGA-edited GABA

Ranaviruses have been identified as the etiologic agent in many amphibian die-offs across the globe. Polymerase chain reaction (PCR) is commonly used to detect ranavirus infection in amphibian hosts, but the test results may vary between tissue samples obtained by lethal and non-lethal procedures. Testing liver samples for infection is a common lethal sampling technique to estimate ranavirus prevalence because the pathogen often targets this organ and the liver is easy to identify and collect. However, tail clips or swabs may be more practicable for ranavirus surveillance programs compared with collecting and euthanizing animals, especially for uncommon species. Using PCR results from liver samples for comparison, we defined false-positive test results as occurrences when a non-lethal technique indicated positive but the liver sample was negative. Similarly, we defined false-negative test results as occurrences when a non-lethal technique was negative but the liver sample was positive. Using these decision rules, we estimated false-negative and false-positive rates for tail clips and swabs. Our study was conducted in a controlled facility using American bullfrog Lithobates catesbeianus tadpoles; false-positive and false-negative rates were estimated after different periods of time following exposure to ranavirus. False-negative and false-positive rates were 20 and 6%, respectively, for tail samples, and 22 and 12%, respectively, for swabs. False-negative rates were constant over time, but false-positive rates decreased with post-exposure duration. Our results suggest that non-lethal sampling techniques can be useful for ranavirus surveillance, although the prevalence of infection may be underestimated when compared to results obtained with liver samples.

The Effect of Testing Reliability on Visual Field Sensitivity in Normal Eyes: The Singapore Chinese Eye Study

To evaluate the false-negative and false-positive error rates both in a screening and a non-screening population. A total of 4192 prostatic biopsies were reported in a 6-year period by 15 consultant histopathologists, two of whom had an interest in uropathology and were deemed to be specialists (J.O. and C.S.). All biopsies were reviewed prior to the multidisciplinary team (MDT) meeting. The overall false-negative rate was 1.7% (screening 2.1%, non-screening 1.5%). The overall false-positive rate was 0.5% (screening 0.9%, non-screening 0.4%). These error rates varied among pathologists, with the false-negative rate ranging from 0% to 9.3%, and the false-positive rate ranging from 0% to 3.8%. The false-negative rate was three times greater than the false-positive rate, showing that detection of significant pathology is far greater in the negative biopsies. More errors occurred in the screening population than in the non-screening population. The consultants making the most errors were non-specialists, but the specialists also made false-negative errors, suggesting that just using specialist reporting alone would not have eradicated errors.

An Evaluation of Conventional Culture, invA PCR, and the Real-Time PCR iQ-Check Kit as Detection Tools for Salmonella in Naturally Contaminated Premarket and Retail Turkey

The influence of fatigue and related factors on automated perimetric testing was evaluated in both eyes of sixteen normal observers and sixteen patients with early-to-moderate visual field loss using a Digilab 750 automated perimeter and a customized test procedure. False positive rate, false negative rate, and detection sensitivity at 5,10,15, and 20 degrees eccentricity were measured in 1.5-rain intervals throughout a 21-min visual field examination. Half of the normal observers and patients with visual field loss were given a briefrest(1.5-min) midway through each visual field exam to determine whether this would reduce fatigue effects. Our findings revealed that patients displayed considerably higher average false positive and false negative rates than normal observers. However, neither the patients nor normal observers demonstrated any consistent changes in false positive rate or false negative rate as a function of testing duration. In contrast, both normal observers and patients showed an average decrease in sensitivity as a function of increasing test duration with the magnitude of the time-dependent sensitivity loss becoming greater with increasing stimulus eccentricity. Patients demonstrated a larger time-dependent sensitivity loss than normal observers, averaging ~4dB at 20 degrees eccentricity. The introduction of a brief pause midway through the test procedure appeared to reduce the time-dependent sensitivity loss for the second half of the test procedure, especially for greater eccentricities.

Several different types of detectors, software, and hardware designs are employed in instruments used to monitor radioactive content of freight shipments in order to detect illicit trafficking of nuclear materials. The instruments can be container monitors, hand-held radiation detectors, mobile analysis systems, or fixed radiation portal monitors. However, within the various groupings (e.g., portal monitors), all instruments are expected to solve the same problem, that is, to detect and identify any radioactive material present according to the prescribed investigative procedure or CONOPS. The best way to compare the performance of different instruments is with a numerical score or Figure of Merit (FOM). The FOM must quantify the performance of the instrument with respect to true positives (TP), false positives (FP), and false negatives (FN). The minimization of FN for certain radionuclides (e.g., uranium and plutonium or other Special Nuclear Material (SNM)) is more important than the minimization of FN for non-threat nuclides (e.g., low level NORM). In a similar way, the minimization of FP for SNM is more important than falsely reporting the common NORM nuclides which represent no threat. System performance depends on the measurement detail (e.g., transit speed through a portal, measurement time in general, amount and distribution of NORM), therefore the test conditions should also be included in the statement of the FOM. The FOM is expected to vary significantly with the above measurement details. A FOM has been developed based on the number of true positives (TP), the number of false important positives (FIP), the number of FP, the number of true positives for SNM (TPSNM), and the number of false positives for SNM (FPSNM). This formula rates the overall performance with extra weight given to FP and FN for SNM. Examples will be shown for testing of portal monitors.

Relationships between age and visual field (VF) reliability indices were investigated using a large real-world dataset (42,421 VF data points from 11,525 eyes of 5930 subjects). All VFs tested and stored at Shimane University Hospital between 1988 and 2019 were exported. Correlations between age, mean deviation (MD), pattern standard deviation (PSD), and reliability indices including fixation losses (FLs), false negatives (FNs), and false positives (FPs) were analyzed. The mean ± standard deviation age was 65.0 ± 15.1 years; MD—−6.9 ± 8.1 decibels (dB); PSD—6.3 ± 4.6 dB; FL—8.6 ± 11.7%; FN—5.3 ± 8.3%; and FP—2.6 ± 5.0%. Univariate analyses showed strong associations between age and FNs (correlation coefficient, ρ = 0.20, p < 0.0001) and MD (ρ = −0.21, p < 0.0001). All FLs, FNs, and FPs were lowest during the third decade (20–29 years) of life. FLs were elevated consistently after that decade, and FNs were elevated sharply after the seventh decade. FPs were relatively stable after the fourth decade (30–39 years). Mixed-effect regression analyses in subjects 40 years and older showed that older age was associated with worse FLs (p < 0.0001) and FNs (p < 0.0001) but not FPs (p = 0.4126). Aging affects FLs and FNs with different modes but had minimal effects on FPs. Decreased VF sensitivity, deteriorated macular function, and technical difficulties with testing may be mechanisms of age-related changes in FLs and FNs.

This study aimed to compare the results of the IMO vifa 24plus(1-2) and HFA 24-2 visual field tests. We included 52 patients (104 eyes) with glaucoma who visited Chukyo Eye Clinic between June 2023 and March 2024. On the same day, the HFA 24-2 test using the Swedish Interactive Threshold Algorithm Standard and the IMO vifa 24plus(1-2) test using the Ambient Interactive Zippy Estimated strategy were performed. A Wilcoxon signed-rank test was used to compare fixation, false positives, false negatives, and total test time for both eyes between the HFA and IMO vifa tests. Only eyes with reliable test results were selected, and the visual field index (VFI), mean deviation (MD), and pattern standard deviation (PSD) were examined using Spearman's rank correlation coefficient and the Wilcoxon signed-rank test. The test times for the HFA 24-2 and IMO vifa 24plus(1-2) were 716.83±118.80 and 628.75±142.70 s, respectively, with the IMO vifa being significantly shorter (P<0.001). For fixation, the results were 15.85±16.57% and 11.09±15.20%, with significantly better fixation in the IMO vifa (P<0.0001). False positives and negatives were 4.52±5.19 / 4.14±6.85% and 4.97±7.99 / 5.47±7.86%, respectively, with no significant differences, though the IMO vifa showed a slightly higher trend (P=0.980, 0.056). In eyes with reliable results, the HFA 24-2 and IMO vifa 24plus(1-2) outcomes were as follows: MD (-6.45±7.23, -6.85±7.35, P=0.724), PSD (7.13±4.75, 7.49±4.87, P=0.061), VFI (80.86±21.61, 80.86±21.74, P=0.644). The Spearman rank correlation coefficients between the HFA 24-2 and IMO vifa 24plus(1-2) were MD: 0.938, PSD: 0.949, VFI: 0.932 (all P<0.001). The IMO vifa demonstrates a very high correlation with HFA and allows for a shorter examination time. Fixation errors are significantly improved compared to HFA, while there is no significant difference in false positives or false negatives.

How do we make the best choice in assays used to monitor HLA antibodies in renal recipients awaiting transplant? A comparison of flowpra beads with luminex single-antigen beads

MP32-18 EVALUATING THE DIAGNOSTIC ACCURACY OF BIPARAMETRIC MAGNETIC RESONANCE IMAGING IN MUSCLE-INVASIVE BLADDER CANCER

Recent advances in microarray technology have opened new ways for functional annotation of previously uncharacterised genes on a genomic scale. This has been demonstrated by unsupervised clustering of co-expressed genes and, more importantly, by supervised learning algorithms. Using prior knowledge, these algorithms can assign functional annotations based on more complex expression signatures found in existing functional classes. Previously, support vector machines (SVMs) and other machine-learning methods have been applied to a limited number of functional classes for this purpose. Here we present, for the first time, the comprehensive application of supervised neural networks (SNNs) for functional annotation. Our study is novel in that we report systematic results for ~100 classes in the Munich Information Center for Protein Sequences (MIPS) functional catalog. We found that only ~10% of these are learnable (based on the rate of false negatives). A closer analysis reveals that false positives (and negatives) in a machine-learning context are not necessarily "false" in a biological sense. We show that the high degree of interconnections among functional classes confounds the signatures that ought to be learned for a unique class. We term this the "Borges effect" and introduce two new numerical indices for its quantification. Our analysis indicates that classification systems with a lower Borges effect are better suitable for machine learning. Furthermore, we introduce a learning procedure for combining false positives with the original class. We show that in a few iterations this process converges to a gene set that is learnable with considerably low rates of false positives and negatives and contains genes that are biologically related to the original class, allowing for a coarse reconstruction of the interactions between associated biological pathways. We exemplify this methodology using the well-studied tricarboxylic acid cycle.

Testing for the presence of a condition may give an incorrect result Sensitivity and specificity of a test depend upon the chosen cut-off value ‘No difference’ usually means ‘these data do not suggest a difference’ The power of a test needs to be known to conclude that there is no difference Power calculations should be considered if the null hypothesis is not rejected Calvin Coolidge was a taciturn president of the United States, nicknamed ‘Silent Cal’. When told that he had died, the acerbic writer Dorothy Parker remarked, ‘How could they tell?’ Telling if something has not happened is a perennial problem in science, particularly so in laboratory experiments. Why should this be? In the case of Cal Coolidge, they probably could tell. But in science, random events make it hard to tell, with certainty. There are also different ways of ‘telling’. Mortality is a good categorical measure, an unequivocal endpoint. When diagnostic tests are used to define categories, we meet the concepts of ‘false positive’ and ‘false negative’. Here, we use the expression ‘false’ to indicate that the conclusion we drew from the test would be incorrect. In the example we use below, we have a test for ‘alive’. If we know the true condition, we can classify test results as true or false. Classifying a dead animal as alive would be a false positive. True and false are words usually applied to a diagnosis or similar categorical events but also crop up in other aspects of statistical inference. Categorical analysis and categorical logic (it's either there or it isn’t) are not the same as the frequentist logic (the likelihood is small) that is used in common statistical analysis, and this leads to a great deal of argument and misunderstanding. Our previous comment that statistics has an impoverished vocabulary rings true: in this instance, the concepts of false positive and false negative conclusions are applied in different ways to different procedures, causing confusion. Considering categories, imagine that a pool that has become overheated in the sun: some of the frogs in it have died. Our observations suggest that the frogs on the surface of the pool are still alive, and the ones at the bottom of the pool are dead. Is this correct? A random sample produces the data in Fig. 1. Not all the frogs on the surface are alive. If we used ‘on the surface’ to indicate being alive, we would falsely attribute life to 10 frogs. Since 107 frogs are dead, this is a false positive rate of 10/107 (which we only know if we have an unequivocal means of determining death). The positive predictive power of the test for life being present, if we were to look at the next frog on the surface, would be 80/90. Unfortunately, expressions such as sensitivity, false positive, and so on can be used in two different senses. Here we assume we already have an exact means to determine death, much in the way that some statistics theory will start by defining a population. Thus we consider that the false positive rate of our test for ‘alive’ is the proportion of dead frogs on the surface, expressed as a proportion of the frogs that really ARE dead. An alternative view would be if we were to consider the proportion of frogs on the surface that are later found to be dead, we would then be considering the outcome of the test. The diagnostic false positive rate would be 10/90. This is a diagnostic probability, and this form is less commonly used. Many diagnostic tests don't rely on categories like this: a continuous variable is often used to distinguish categories. Suppose we measure the blood lactate concentration in frogs that are still alive, and relate these concentrations to subsequent survival, we might obtain the results shown in Fig. 2. Plasma lactate values in frogs after overheating, related to survival The groups are different (P < 0.001, Mann–Whitney U test). The median value for survivors is 1.29, and for non-survivors, 1.86 (mmol l−1). Although the median concentrations are different, some frogs with high lactate concentrations can survive. Is there a particular lactate concentration that will distinguish survival from non-survival? The capacity to predict outcome (sensitivity and specificity) alters as we choose different cut-off values. Using a series of cut-off values we can plot sensitivity and (1 – specificity) for each chosen value (Fig. 3). This is known as a Receiver-Operator Characteristic Curve, or ROC curve, because one of its first applications was to find how operators of radar receivers distinguish ‘real’ echoes from background noise. In this case, we are looking for survivors using a lactate concentration that is less than the cut-off value we have chosen. A Receiver Operator Characteristic curve for the capacity of a lactate concentration below a chosen value to predict survival An effective diagnostic test should have a large area below the curve, which would show a combination of good sensitivity and specificity. Since tests vary, so does opinion about the area below the curve that indicates a good test. It should be substantially greater that 50%, which would be no better than chance alone, and the closer to 100% the better. The large overlap of individual values between the two groups in Fig. 2 suggests that lactate concentration is a poor indicator of survival, as there is no value that combines useful specificity and sensitivity. Any cut-off values we could choose would have too many false positive or false negative conclusions. Despite this, there is a statistically significant difference according to the Mann–Whitney test, showing that a significant difference doesn't mean the measure is the basis of a good diagnostic test. In other forms of statistical inference, an expression such as ‘false positive’ is less appropriate and is often not well defined, hidden in the logic of these tests. For example, the t test does not provide black or white conclusions: statistical inference is restricted to judging how large a part chance could play in what we observe. Frequentist statistical tests such as the t tests are measures of uncertainty. The usual laboratory experiment question ‘do these data show an effect?’ has to be worded ‘are these samples statistically different?’ As we know, the question then is re-worded ‘How likely are these results if the samples came from the same population?’ Let's say our test result is that P= 0.01, which would usually be taken to indicate there is an ‘effect’. This is not so: this P value is the probability that we conclude there is an effect, and we still could be wrong. Although small, a chance still exists that we would be wrong, about 1 in 100 times. This is known as the alpha error, the possibility that we might classify our result as showing an effect, when in fact there is none. In the long run, if we were to repeatedly sample two populations that actually were the same, then results like the ones we found would be found 1 in 100 times. This is sometimes called the ‘false positive’ rate. However, it's based on the premise that there is NO difference and is perhaps better called the ‘false conclusion’ rate. Reasoning for ‘no effect’ is even less certain, and also often misconstrued. If the P value is 0.12, then we judge that the results are quite possible, assuming the data are from the same population. If we did the same experiment again and again, then we might obtain data like this in almost 1 out of 8 experiments. Is this unlikely? We usually choose to decide this is quite likely, and thus do not reject the null hypothesis: if we rejected it, the chance of a false conclusion is too great. This ‘chance of a false conclusion’ is often misleadingly called a ‘false negative rate’. The hypothesis the test uses, which is the NULL hypothesis, is resoundingly negative; there are no positive results lurking here. Positive results come if we are persuaded, by an estimate of probability (the P value), that the null hypothesis is not tenable. The context of the question ‘is there an effect?’ is relevant. The consequence of the answer ‘no effect’ may be unimportant. However, in some cases the consequence of concluding that there is no difference, because a difference is not apparent, could be devastating. How often do we hear the claim ‘there is no evidence that X is toxic?’ to assert that X is not toxic? Immediately, the sceptic will ask how toxic, and how much evidence? The statistician may add another qualification: how much variation is present? A reasonable example (not a perfect analogy) is the radar operator: there is a greater certainty about a blip on the screen if the signal is large (accept the answer only if it's very likely), if you can keep looking at the screen (improve the sample size), and if there is not a lot of background noise (Fig. 4). In each of these examples of signal averaging, there is an identical signal with added random noise. The signal is more easily seen if the noise is reduced (vertical arrow). Signal detection is improved by taking more samples to allow more averaging (horizontal arrow). The final feature of signal detection is how certain we wish to be (from ‘just discernible’ to ‘barn door’) that the event is not random (a concept analogous to the P value). In this example, a signal is always present, but we just can't be sure in all the cases. This is equally so when statistics are applied to the null hypothesis. We have to consider the beta error: the possibility that we might classify our result as showing no effect where in fact there could be a difference. To demonstrate, consider two populations that are known to be different, as shown in Fig. 5. A, random samples from two different populations, low and high, where the population means differ by 0.1 mmol l−1. A t test finds the P values shown. B, a valuable further analysis that should be applied before erroneously assuming ‘no difference’ is to inspect the 90% confidence limits of the difference between the means. This indicates the size of the possible differences. Here, a small sample of 20 per group, and a small true difference between the groups, does not reject the null hypothesis, with a P value of 0.05. In this particular case, there is indeed a true difference in the populations from which these samples have been taken, but the power of the test applied has been inadequate. The power of a test is the chance of finding a defined difference: a power of 90% would mean that there was only a 10% chance of failing to find a difference of the defined magnitude. Thus power is (1 −β). The 10% false negative conclusion rate or beta error, is also known as a type II error. Some call this a ‘false negative’ but this could be misleading since there is never a ‘true’ negative here. The frequentist definition is that if the experiment were repeated over and over, the conclusion would be incorrect occasionally, and the rate is the beta error. For the example we have chosen, the relationships between power, sample size and the difference we seek to detect are shown in Fig. 6. Using the samples in Fig. 4, the power of the test (the chance of finding a specified difference between samples, given that it exists) is shown, where n= 20 and n= 40, for two different P values. The populations that provided the example data in Fig. 4 were different, by 0.1 mmol l−1, indicated by the vertical dotted line. If P is accepted at 0.05 and n= 40, the power of the test is only about 40%. The retrospective use of power calculations has been often criticized, but this is usually because the authors were seeking to reject a null hypothesis (they would have liked to find a difference). If a null hypothesis cannot be rejected, this is not sufficient evidence that there is NO difference. Claiming ‘no difference’ can be a serious error. If we wish to convince ourselves, and others, that there is no difference, we must be sure that the test was capable of rejecting a specifically defined difference, if it were present. If a small difference is considered important, and variation in the population is substantial, then a small sample will be inadequate. Put simply, the study is underpowered. Although there are software packages to calculate power, a simple approach for a simple two group comparison is to plot the 90% confidence limits of the difference between the means (Fig. 5B). This will show the likely range of differences that could be detected. The logical error (concluding that no significant difference is equivalent to the same) is frequently made worse when test results are compared, for example: the effect of substance A causes a significant difference, but substance B does not; thus A has more effect than B. As we explain above, several factors, not explicitly stated, could affect this result. It could well be that B has a greater, but more varied, effect. Because most tests are looking for a positive, many scientists fail to consider negatives adequately. In the old song ‘Accentuate the positive’ the accomplished song writer Johnny Mercer also cautioned to ‘eliminate the negative, don't mess with Mr In-Between’.

Many species on Earth can carry up to two different versions of a given gene, with each of these ‘alleles’ having only a 50/50 chance of being transmitted to the next generation via sexual reproduction. Certain ‘selfish’ sequences, however, can hijack this process and increase their probability of being passed on to an offspring. Known as transmission distortion, this phenomenon may result in alleles spreading through the population even if they are detrimental to fertility. Transmission distortion has been detected in many species such as flies, mice and some plants. It can take place at various stages during reproduction; for example, the selfish alleles may become overrepresented among eggs or sperm. However, scientists need to study a large number of offspring or reproductive cells to be able to detect whether an allele is inherited more often than expected. This has made it difficult to determine whether transmission distortion also happens in humans, and research so far has resulted in conflicting conclusions. A recently published dataset of human sperm from 25 donors offered Carioscia, Weaver et al. the opportunity to examine this question. Every volunteer had produced between 969 and 3377 sperm cells, each with about 1% of their genome sequenced. Carioscia, Weaver et al. developed a computational method, which they named rhapsodi, that allowed them to ‘fill in the gaps’ and infer missing regions of the genome for each cell. To do so, they relied on the fact that sperm cells from a given individual are highly related to one another. With this more complete data at hand, it became possible to look for evidence of transmission distortion by searching for alleles that were overrepresented in sperm from a given donor. No selfish sequence could be detected in any of the 25 individuals, suggesting that human sperm may not be subject to pervasive transmission distortion. Signatures of selfish alleles detected in previous human studies may have therefore not resulted from this mechanism taking place at the sperm level. Instead, transmission distortion in humans could primarily target eggs or happen at later stages (for instance, if embryos carrying the selfish allele have better chances of survival). The ‘rhapsodi’ method developed by Carioscia, Weaver et al. should allow other scientists to work with datasets for which large portions of the genetic information is missing. It may therefore become easier for researchers to track selfish alleles which are difficult to detect, and to examine bigger, more diverse samples which also include individuals with known fertility challenges.

Short‐term precipitation forecasts are critical to regional water management, particularly in the Western U.S. where atmospheric rivers can be predicted reliably days in advance. However, spatial error in these forecasts may reduce their utility when the costs of false positives and negatives differ greatly. Here we investigate whether deep learning methods can leverage spatial patterns in precipitation forecasts to (a) improve the skill of predicting the occurrence of precipitation events at lead times from 1 to 14 days, and (b) balance the tradeoff between the rate of false negatives and false positives by modifying the discrimination threshold of the classifiers. This approach is demonstrated for the Sacramento River Basin, California, using the Global Ensemble Forecast System (GEFS) v2 precipitation fields as input to convolutional neural network (CNN) and multi‐layer perceptron models. Results show that the deep learning models do not significantly improve the overall skill (F1 score) relative to the ensemble mean GEFS forecast with bias‐corrected threshold. However, additional analysis of the CNN models suggests they often correct missed predictions from GEFS by compensating for spatial error at longer lead times. Additionally, the deep learning models provide the ability to adjust the rate of false positives and negatives based on the ratio of costs. Finally, analysis of the network activations (saliency) indicates spatial patterns consistent with physical understanding of atmospheric river events in this region, lending additional confidence in the ability of the method to support water management applications.

The revised ICH E14 Question and Answer (R3) document issued in December 2015 enables pharmaceutical companies to use concentration-QTc (C-QTc) modeling as the primary analysis for assessing QTc prolongation risk of new drugs. A new approach by including the time effect into the current C-QTc model is introduced. Through a simulation study, we evaluated performances of different C-QTc modeling with different dependent variables, covariates, and covariance structures. This simulation study shows that C-QTc models with ΔQTc being dependent variable without time effect inflate false negative rate and that fitting C-QTc models with different dependent variables, covariates, and covariance structures impacts the control of false negative and false positive rates. Appropriate C-QTc modeling strategies with good control of false negative rate and false positive rate are recommended.