Hypothesis Testing Methods for Multivariate Multi-Occasion Intra-Individual Change

PURPOSE: Muscular and cardiorespiratory fitness are associated with adiposity and cardiovascular disease risk in youth. The degree to which these measures discriminate obesity is not lucid. This study investigated muscular and cardiorespiratory fitness capacities to discriminate obesity. METHODS: Participants (N = 210; 116 males) (9.7 ± 1.08 years; 138.6 ± 9.4 cm; 42.3 ± 14.4 kg) (84% Hispanic/Latino) muscular and cardiorespiratory fitness were assessed using the 20-meter Progressive Aerobic Cardiovascular Endurance Run (PACER) and 90o push-up test, respectively, per FITNESSGRAM® protocols. Weight status was classified using CDC Growth Charts. Non-obesity was coded as “0” and obesity was coded as “1.” A Receiver Operating Characteristics (ROC) curve analysis was performed to identify the True Positive Rate (TPR) (i.e., Sensitivity) and False Positive Rate (FPR) (i.e., 1 - Specificity) using SPSS. The area under the curve (AUC) indicated the degree to which each fitness measure distinguishes between the presence and absence of obesity. The measurement threshold with the greatest TPR-FPR distance was considered optimally discriminative. Statistical significance was set at P < .05. RESULTS: Of the sample, 43% had obesity and 55% did not. PACER was fairly discriminative of obesity in girls (AUC = .748; P = .000; 95% CI [.651, .845]) and boys (AUC = .755; P = .000; 95% CI [.665, .845]). 90o push-up was fairly discriminative in girls (AUC = .732; P = .000; 95% CI [.632, .831]) and boys (AUC = .789; P = .000; 95% CI [.703, .876]). PACER thresholds of 8 and 8 laps were optimally discriminative in girls (TPR = .714, FPRs = .352) and boys (TPR = .635, FPRs = .190), respectively. 90o push-up thresholds of 7 and 5 repetitions were optimally discriminative in girls (TPR = .714, FPRs = .333) and boys (TPR = .692, FPRs = .175), respectively. CONCLUSIONS: Based on the probability of classifying individuals with and without obesity as having obesity (i.e., TPRs and FPRs, respectively), PACER and 90o push-up appear similarly discriminative of obesity. Notably, the optimal PACER threshold to discriminate obesity was considerably lower than cut-off scores (23 for boys and 15 for girls) for HFZ classification in the same age group. Cardiorespiratory-related disease risk may persist in subpopulations of Hispanic/Latino youth without obesity.

Summary The mixed linear model is a popular method for analysing unbalanced repeated measurement data. The classical statistical tests for parameters in this model are based on asymptotic theory that is unreliable in the small samples that are often encountered in practice. For testing a given fixed effect parameter with a small sample, we develop and investigate refined likelihood ratio (LR) tests. The refinements considered are the Bartlett correction and use of the Cox–Reid adjusted likelihood; these are examined separately and in combination. We illustrate the various LR tests on an actual data set and compare them in two simulation studies. The conventional LR test yields type I error rates that are higher than nominal. The adjusted LR test yields rates that are lower than nominal, with absolute accuracy similar to that of the conventional LR test in the first simulation study and better in the second. The Bartlett correction substantially improves the accuracy of the type I error rates with either the conventional or the adjusted LR test. In many cases, error rates that are very close to nominal are achieved with the refined methods.

Classification requirements for real-world classification problems are often constrained by a given true positive or false positive rate to ensure that the classification error for the most important class is within a desired limit. For a sufficiently high true positive rate, this may result in the set-point being located somewhere in the flat portion of the ROC curve where the associated false positive rate is high. Any further classifier design will then attempt to reduce the false positive rate while maintaining the desired true positive rate is. We call this type of performance requirements for classifier design the constrained performance requirement. This type of performance requirements is different from the accuracy maximization requirement and thus requires different strategies for classifier design. This paper is concerned with designing classifier ensembles under such constrained performance requirements. Classifier ensembles are one of the most significant advances in pattern recognition/classification in recent years and have been actively studied by many researchers. However, not much attention has been given to designing ensembles to satisfy constrained performance requirements. This paper attempts to identify and address some of design related issues associated with the constrained performance requirement. Specifically, we present a design strategy for designing neural network ensembles to satisfy constrained performance requirements, which is illustrated by designing a real-world classification problem. The results are compared to those from conventional design method.

Malware detection engines are required a high True Positive Rate (TPR) and low False Positive rate (FPR). For example, to obtain the VB100 certification conducted by Virus Bulletin in the U.K., the detector's TPR must be 99.5% or higher, and FPR less than 0.01%. However, it is difficult for signature-based malware detection engines to detect zero-day or new variants of malware. For such malware, an approach based on machine learning is considered effective because the features of specimens are analyzed in a complex manner. Therefore, we created a malware detection model from surface analysis logs, and PE header dumps by machine learning using the FFRI Dataset 2018. Furthermore, we verified the accuracy under the constraint that the FPR is less than a certain value. As a result, we succeeded in creating a new model with high accuracy: when FPR is set to be less than 1%, TPR is 99.7%, when FPR is set to be less than 0.1%, TPR is 98.7%, and when FPR is set to be less than 0.01% and TPR is 94.5%. In addition, we revealed features with high contribution to malware detection in this model.

The identification of encrypted Instant Messaging (IM) channels between users is made difficult by the presence of variable and high levels of uncorrelated background traffic. In this paper, we propose a novel Cross-correlation Outlier Detector (CCOD) to identify communicating end-users in a large group of users. Our technique uses traffic flow traces between individual users and IM service provider's data center. We evaluate the CCOD on a data set of Yahoo! IM traffic traces with an average SNR of −6.11dB (data set includes ground truth). Results show that our technique provides 88% true positives (TP) rate, 3% false positives (FP) rate and 96% ROC area. Performance of the previous correlation-based schemes on the same data set was limited to 63% TP rate, 4% FP rate and 85% ROC area.

A Miniaturized Microfluidic Dielectric Sensor for Point-of-Care Assessment of Blood Coagulation

Objective To evaluate the performance of three antibody kits for novel coronavirus (SARS-CoV-2) and to investigate the feasibility and advantages of them in clinical application. Methods A total of 104 patients who were admitted to Guangzhou Eighth People's Hospital with COVID-19 from January to February 2020 were selected as research group. Fifty-one healthy subjects were selected during the same period as negative control group. Serum antibodies (IgM/IgG) against SARS-CoV-2 were detected using two kinds of colloidal gold kits (A and B kits) and one chemiluminescence kit (C kit). The positive rates of SARS-CoV-2 nucleic acid in different samples from patients with COVID-19 were retrospectively analyzed. Results The clinical sensitivity of A kit to detect SARS-CoV-2-specific IgM and IgG was 77.88% (81/104) and 65.38% (68/104), respectively, and the clinical specificity was 70.59% (36/51) and 100.00% (51/51). However, the false positive rate in IgM detection was as high as 29.41% (15/51). The sensitivity of B kit to test total antibodies to SARS-CoV-2 was 63.46% (66/104), and the clinical specificity was 94.12% (48/51). The clinical sensitivity of C kit to detect SARS-CoV-2-specific IgM and IgG were respectively 31.73% (33/104) and 64.42% (67/104), and the clinical specificity were both 98.04% (50/51). There was a moderate correlation between the detection results of two colloidal gold kits and the chemiluminescence kit with the Kappa values of 0.462 and 0.587 (Z=6.157, P<0.01; Z=7.345, P<0.01). C kit had the highest positive detection rate for IgG, and would be more reliable to be used for IgG detection in COVID-19 patients 14 d after onset. The total positive detection rate of nucleic acid in all types of samples was 63.46% (66/104). The highest positive detection rate was in throat swabs or sputum samples, followed by those in blood samples and anal swabs. No viral nucleic acid was detected in urine samples for the time being. Conclusions SARS-CoV-2-specific antibodies could be detected in the early or late stage of COVID-19. The method of antibody detection has the advantages of shorter detection time, simple operation and high biological safety, indicating that it could be used as a supplementary or auxiliary detection for the diagnosis of suspected COVID-19 cases with negative nucleic acid test results. The chemiluminescence kit has good sensitivity and specificity, and is well recommended for clinical laboratories. Key words: SARS-CoV-2; Chemiluminescence immunoassay; Colloidal gold immunoaasay; Nucleic acid detection; Performance evaluation; Clinical application

Introduction: Studies that examine the role of rare variants in both simple and complex disease are increasingly common. Though the usual approach of testing rare variants in aggregate sets is more powerful than testing individual variants, it is of interest to identify the variants that are plausible drivers of the association. We present a novel method for prioritization of rare variants after a significant aggregate test by quantifying the influence of the variant on the aggregate test of association. Methods: In addition to providing a measure used to rank variants, we use outlier detection methods to present the computationally efficient Rare Variant Influential Filtering Tool (RIFT) to identify a subset of variants that influence the disease association. We evaluated several outlier detection methods that vary based on the underlying variance measure: interquartile range (Tukey fences), median absolute deviation, and SD. We performed 1,000 simulations for 50 regions of size 3 kb and compared the true and false positive rates. We compared RIFT using the Inner Tukey to 2 existing methods: adaptive combination of p values (ADA) and a Bayesian hierarchical model (BeviMed). Finally, we applied this method to data from our targeted resequencing study in idiopathic pulmonary fibrosis (IPF). Results: All outlier detection methods observed higher sensitivity to detect uncommon variants (0.001 < minor allele frequency, MAF > 0.03) compared to very rare variants (MAF <0.001). For uncommon variants, RIFT had a lower median false positive rate compared to the ADA. ADA and RIFT had significantly higher true positive rates than that observed for BeviMed. When applied to 2 regions found previously associated with IPF including 100 rare variants, we identified 6 polymorphisms with the greatest evidence for influencing the association with IPF. Discussion: In summary, RIFT has a high true positive rate while maintaining a low false positive rate for identifying polymorphisms influencing rare variant association tests. This work provides an approach to obtain greater resolution of the rare variant signals within significant aggregate sets; this information can provide an objective measure to prioritize variants for follow-up experimental studies and insight into the biological pathways involved.

Utility of plasma cell-free DNA next-generation sequencing for diagnosis of infectious diseases in patients with hematological disorders

With the wide application of information technolo-gies in communication based train control (CB TC) systems, the information security risks it confronts to are ever increasing. It is very difficult to design an intrusion detection system (IDS) which can detect both known and unknown cyber attacks with high true positive rate (TPR) and low false positive rate (FPR), simultaneously. In this paper, we propose a hybrid intrusion detection system (RIDS) which is composed of a signature-based intrusion detection system (SIDS) and an anomaly-based intrusion detection system (AIDS). An improved stacking ensemble algorithm (ISEA) is proposed to enhance the TPR of the SIDS which uses weighted classification probabilities outputted by base-classifiers to generate meta-features for the meta-classifier. The FPR of the AIDS is significantly reduced through adopting the proposed false positive elimination strategy (FPES). In addition, a Bayesian optimization algorithm is used for hyper-parameters adjustment to optimize the overall performance of the RIDS. The proposed RIDS is evaluated using both a KDD99 dataset and a CBTC dataset collected on a hardware-in-the-Ioop simulation platform. For the KDD99 dataset, the TPR and FPR of the proposed RIDS are 98.6% and 1.3%, respectively. For the CBTC dataset, the TPR and FPR are 98.1% and 1.1 %, respectively. Based on the performance comparison among different RIDSs, it is can be concluded that the proposed RIDS is superior to the existing RIDS in both TPR and FPR.

If we can estimate the accuracy of our observations then we can estimate the true and false positive rates over a series of samples in high dimensional data mining problems. To date such issues have been largely neglected and previously no algorithm has been provided to facilitate the computations involved. In high dimensional data mining tasks, increasing sparsity leads to decreasing true positive rates. Estimating this effect allows the estimation of the true size of membership of a class or cluster allowing us to identify the top candidates for these false negatives, while tracking the likelihood of false positives. These estimates of true and false positive rates can also help researchers avoid unnecessary costs by collecting only the number of samples that are really needed. We propose an algorithm for these computations designated the Statistical Error Rate Algorithm (SERA) and give an example of its use.

As there is currently a marked increase in the use of both unidimensional (UCAT) and multidimensional computerized adaptive testing (MCAT) in psychological and health measurement, the main aim of the present study is to assess the incremental value of using MCAT rather than separate UCATs for each dimension. Simulations are based on empirical data that could be considered typical for health measurement: a large number of dimensions (4), strong correlations among dimensions (.77-.87), and polytomously scored response data. Both variable- (SE < .316, SE < .387) and fixed-length conditions (total test length of 12, 20, or 32 items) are studied. The item parameters and variance–covariance matrix Φ are estimated with the multidimensional graded response model (GRM). Outcome variables include computerized adaptive test (CAT) length, root mean square error (RMSE), and bias. Both simulated and empirical latent trait distributions are used to sample vectors of true scores. MCATs were generally more efficient (in terms of test length) and more accurate (in terms of RMSE) than their UCAT counterparts. Absolute average bias was highest for variable-length UCATs with termination rule SE < .387. Test length of variable-length MCATs was on average 20% to 25% shorter than test length across separate UCATs. This study showed that there are clear advantages of using MCAT rather than UCAT in a setting typical for health measurement.

BackgroundVenous thromboembolism (VTE) is a common complication of hospitalized trauma patients and has an adverse impact on patient outcomes. However, there is still a lack of appropriate tools for effectively predicting VTE for trauma patients. We try to verify the accuracy of the Caprini score for predicting VTE in trauma patients, and further improve the prediction through machine learning algorithms.MethodsWe retrospectively reviewed emergency trauma patients who were admitted to a trauma center in a tertiary hospital from September 2019 to March 2020. The data in the patient’s electronic health record (EHR) and the Caprini score were extracted, combined with multiple feature screening methods and the random forest (RF) algorithm to constructs the VTE prediction model, and compares the prediction performance of (1) using only Caprini score; (2) using EHR data to build a machine learning model; (3) using EHR data and Caprini score to build a machine learning model. True Positive Rate (TPR), False Positive Rate (FPR), Area Under Curve (AUC), accuracy, and precision were reported.ResultsThe Caprini score shows a good VTE prediction effect on the trauma hospitalized population when the cut-off point is 11 (TPR = 0.667, FPR = 0.227, AUC = 0.773), The best prediction model is LASSO+RF model combined with Caprini Score and other five features extracted from EHR data (TPR = 0.757, FPR = 0.290, AUC = 0.799).ConclusionThe Caprini score has good VTE prediction performance in trauma patients, and the use of machine learning methods can further improve the prediction performance.

Intelligent data analytics approaches are popular in landslide susceptibility mapping. This chapter develops a random forest (RF) approach for spatial modeling of landslide susceptibility. A total number of 78 landslide locations are identified using field survey, 55 of which are randomly selected to model landslide susceptibility and remaining 23 locations considered for model validation. Twelve predictor variables are selected: elevation, slope percentage, slope aspect, plan curvature, profile curvature, distance from roads, distance from streams, distance from faults, lithological formations, land use, soil type, and topographic wetness index (TWI) to create an RF model for landslide susceptibility mapping. The results of RF model are evaluated using efficiency (E), true positive rate (TPR), false positive rate (FPR), true skill statistic (TSS), and area under receiver operating characteristic curve (AUC) in training and validation steps. RF model registered excellent goodness-of-fit with AUC = 93.6%, E = 0.887, TSS = 0.776, TPR = 0.905, FPR = 0.129, and predictive performance with AUC = 90.7%, E = 0.777, TSS = 0.559, TPR = 0.809, FPR = 0.25. Intelligent data analytic method, therefore, has a significant promise in tackling challenges of landslide susceptibility mapping in large regions, which may not have sufficient geotechnical data to employ a physically based method.

Differential item functioning (DIF) refers to a difference in item performance between equally proficient members of two demographic groups. From an item response theory (IRT) perspective, DIF can be defined as a difference between groups in item response functions. The classic example of a DIF item is a mathematics question that contains sports jargon that is more likely to be understood by men than by women. An item of this kind would be expected to manifest DIF against women: They are less likely to give a correct response than men with equivalent math ability. In reality, the causes of DIF are often far more obscure. The recent book by Camilli and Shepard (1994) and the volume edited by Holland and Wainer (1993) provide an excellent background in the history, theory, and practice of DIF analysis. There are several reasons that DIF detection may be more important for computerized adaptive tests (CATs) than it is for nonadaptive tests. Because fewer items are administered in a CAT, each item response plays a more important role in the examinees’ test scores than it would in a nonadaptive testing format. Any flaw in an item, therefore, may be more consequential. Also, an item flaw can have major repercussions in a CAT because the sequence of items administered to the examinees depends in part on their responses to the flawed item. Finally, administration of a test by computer creates several potential sources of DIF which are not present in conventional tests, such as differential computer familiarity, facility, and anxiety, and differential preferences for computerized administration. Legg and Buhr (1992) and Schaeffer, Reese, Steffen, McKinley, and Mills (1993) both reported ethnic and gender group differences in some of these attributes. Powers and O’Neill (1993) reviewed the literature on this topic.