Chapter Eleven - Assessing the combined effects of multiple independent variables

This paper presents a review and critique of statistical null hypothesis testing in ecological studies in general, and wildlife studies in particular, and describes an alternative. Our review of Ecology and the Journal of Wildlife Management found the use of null hypothesis testing to be pervasive. The estimated number of P-values appearing within articles of Ecology exceeded 8,000 in 1991 and has exceeded 3,000 in each year since 1984, whereas the estimated number of P-values in the Journal of Wildlife Management exceeded 8,000 in 1997 and has exceeded 3,000 in each year since 1994. We estimated that 47% (SE = 3.9%) of the P-values in the Journal of Wildlife Management lacked estimates of means or effect sizes or even the sign of the difference in means or other parameters. We find that null hypothesis testing is uninformative when no estimates of means or effect size and their precision are given. Contrary to common dogma, tests of statistical null hypotheses have relatively little utility in science and are not a fundamental aspect of the scientific method. We recommend their use be reduced in favor of more informative approaches. Towards this objective, we describe a relatively new paradigm of data analysis based on Kullback-Leibler information. This paradigm is an extension of likelihood theory and, when used correctly, avoids many of the fundamental limitations and common misuses of null hypothesis testing. Information-theoretic methods focus on providing a strength of evidence for an a priori set of alternative hypotheses, rather than a statistical test of a null hypothesis. This paradigm allows the following types of evidence for the alternative hypotheses: the rank of each hypothesis, expressed as a model; an estimate of the formal likelihood of each model, given the data; a measure of precision that incorporates model selection uncertainty; and simple methods to allow the use of the set of alternative models in making, formal inference. We provide an example of the information-theoretic approach using data on the effect of lead on survival in spectacled eider ducks (Somateria fischeri). Regardless of the analysis paradigm used, we strongly recommend inferences based on a priori considerations be clearly separated from those resulting from some form of data dredging.

Chapter Seven - The debate over null hypothesis testing and wise use as the solution

The internet is no doubt the biggest and the most important tool of modern civilisation. But along with its numerous benefits, it also comes with its own set of risks, the most important of them being breaches in security and privacy. An anomaly-based Intrusion Detection System (IDS) is a type of security system that is used to detect and alert on unusual or abnormal behaviour that may indicate an attack or intrusion. Unlike signature-based IDS, which rely on known patterns of attack, anomaly-based IDS is designed to detect previously unseen or unknown attacks by identifying deviations from normal patterns of behaviour. Multiple linear regression is a statistical technique used to analyse the relationship between a dependent variable and multiple independent variables. In this technique, a linear equation is established between the dependent variable and multiple independent variables, with the aim of predicting the value of the dependent attribute for a given set of values of the independent attribute. In this paper, we collected a data set of 125974 entries and 42 attributes from Kaggle, pre-processed the data and used logistic regression to predict the dependent variable (called xAttack) using 25 independent variables, as we found a high correlation between the aforementioned variables The results are simulated using 10-fold cross validation, using various train test splits of the data set. The data has been split into 80-20,50-50, and 66-34. After testing the given data set in different train test splits, an accuracy of 92.73 was achieved.

The application of null hypothesis testing to psychological research has been much criticized (e.g., Bakan, 1966; Gigerenzer & Murray, 1987). I examine the specific relevance of these and other criticisms for research on sex differences. Four specific problems are identified: (1) drawing inferences about general properties that are attributed to all members of a population; (2) the distinction between the size of p and the size and theoretical importance of a difference; (3) the frequently unjustified assumption of normality; and (4) the semantic problems inherent in the language of interpretation. A few solutions are explored, and it is concluded that descriptions of the results of sex comparisons, as well as others, must reflect the data more accurately than they now do. A number of years ago, in an invited address that I gave at the annual meeting of the Canadian Psychological Association, I criticized the way in which standard null hypothesis statistics are used in research on sex differences. I argued that in just about every known comparison of the behaviour of females and males there is generally a substantial amount of overlap between the distributions of the dependent variable for the members of each sex (if the behaviour in question is physically possible for both). This is true even if a statistical test has led to the inference that the difference is significant (p .01, p Notwithstanding such extreme cases of overlap, the mere fact that any overlap occurs at all, I reasoned, renders logically false the descriptive statements that are made when differences are said to be significant. For example, to cite a currently contentious issue, suppose it is found that the average score obtained by boys on a math test is significantly higher than that obtained by girls. Since these scores invariably overlap, (see, e.g., Benbow, 1988, p. 219; Hyde et. al., 1990), then it is false to translate the statistical result into an affirmation which states that boys are better at math than are girls. I should perhaps add that this logic also applies when girls obtain higher average scores than boys, a result which has been observed in a larger variety of situations than seems to be commonly known (Kimball, 1989).During the question period, I was asked how it is possible to make a special case that null hypothesis testing is inappropriate for sex differences, when these tests were in general use for most other kinds of psychological data. My questioner clearly did not mean to imply that significance testing should be discarded altogether (or so I understood him) but rather, since these types of statistical procedures are the standard tools of contemporary experimental psychology, and therefore must be valid, then it could hardly be legitimate to make a special case for their irrelevance to sex differences. To my embarrassment, I was unable at that time to give a satisfactory reply, neither to the questioner nor to myself. On the one hand, I was convinced that my reasoning was correct as far as sex differences are concerned, yet, on the other hand, I was still so attached to significance tests as a general tool for psychological research that I was not yet ready to think that their application could be more generally problematical.Over the intervening years I have discovered a rather substantial literature that is critical of the ways in which null hypothesis testing has been integrated into the entire psychological research process. …

REX B. KLINE Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research Washington, DC: American Psychological Association, 2004, 336 pages (ISBN 1-59147-118-4, US$49.95 Hardcover) In 1999, a blue-ribbon task force assembled by the American Psychological Association published their findings with regards to the long-standing controversy pertaining to null hypothesis testing (NHST). The task force dictated effect sizes and confidence intervals be reported, and p values and dichotomous accept-reject decisions be given less weight. Editorial policies in a number of journals came to reflect the views of the task force as did a subsequent revision to the American Psychological Association Publication Manual. Rex B. Kline wrote Beyond Significance Testing. Reforming Data Analysis Methods in Behavioral Research as a follow-up to both the task force recommendations and the revision to the publication manual. Kline's 1998 book Principles and Practice of Structural Equation Modeling (Guilford Press) was well received and a second edition is being published this fall. In Beyond Significance Testing, Kline reviews the controversy regarding testing, offers methods for effect size and confidence interval estimation, and suggests some methodologies. There is an accompanying website that includes resources for instructors and students. Part I of the book is a review of fundamental concepts and the debate regarding testing. Part II provides statistics for effect size and confidence interval estimation for parametric and nonparametric two-group, oneway, and factorial designs. Part III examines metaanalysis, resampling, and Bayesian estimation procedures. In the first chapter, Kline provides a scholarly summary of the null hypothesis testing debate concluding with the APA task force findings and what Kline regards as ambiguous recommendations in the publication manual. Kline predicts the future will see a smaller role for traditional statistical testing (p values) in psychology. This change will take time and may not occur until the next generation of researchers are trained, but Kline anticipates the social sciences will then become more like the natural sciences in that we will report the directions and magnitudes of our effects, determine whether they replicate, and evaluate them for their theoretical, clinical, or practical significance (p. 15). Chapter 2 is a review of fundamental concepts of research design, including sampling and estimation, the logic of statistical testing, and t, F, and chi-square tests. The problems with statistical tests are revisited in Chapter 3. What follows is a long list of errors in interpretation of p values and conclusions made after null hypothesis testing. The emphasis on null hypothesis testing in psychology is also argued to inhibit advancement of the discipline. To be fair, Kline recognizes there is yet no magical alternative to statistical tests and that such tests are appropriate in some circumstances when applied correctly. Nonetheless, Kline envisions a future where effect sizes and confidence intervals are reported, substantive rather than statistical predominates, and NHST-Centric thinking has diminished. Part II covers effect size and confidence interval calculations. Chapter 4 is a presentation of parametric effect size indexes. Independent and dependent sample statistics are covered separately. The textbook's website has a supplementary chapter on twogroup multivariate designs. Group difference indexes such as d are distinguished from measures of association such as r. Case level analyses of group differences are also reviewed. Sections not relevant to a reader's needs can be skipped without loss of continuity. Interpretive guidelines for effect size magnitude and how one might be fooled by effect size estimation are sections that should not be passed over. …

FOUNDATIONS Philosophical and Historical Foundations Introduction Nature of Science Scientific Principles Scientific Method Scientific Hypotheses Logic Variability and Uncertainty in Investigations Science and Statistics Statistics and Biology Introduction to Probability Introduction: Models for Random Variables Classical Probability Conditional Probability Odds Combinatorial Analysis Bayes Rule Probability Density Functions Introduction Introductory Examples of pdfs Other Important Distributions Which pdf to Use? Reference Tables Parameters and Statistics Introduction Parameters Statistics OLS and ML Estimators Linear Transformations Bayesian Applications Interval Estimation: Sampling Distributions, Resampling Distributions, and Simulation Distributions Introduction Sampling Distributions Confidence Intervals Resampling Distributions Bayesian Applications: Simulation Distributions Hypothesis Testing Introduction Parametric Frequentist Null Hypothesis Testing Type I and Type II Errors Power Criticisms of Frequentist Null Hypothesis Testing Alternatives to Parametric Null Hypothesis Testing Alternatives to Null Hypothesis Testing Sampling Design and Experimental Design Introduction Some Terminology The Question Is: What Is the Question? Two Important Tenets: Randomization and Replication Sampling Design Experimental Design APPLICATIONS Correlation Introduction Pearson's Correlation Robust Correlation Comparisons of Correlation Procedures Regression Introduction Linear Regression Model General Linear Models Simple Linear Regression Multiple Regression Fitted and Predicted Values Confidence and Prediction Intervals Coefficient of Determination and Important Variants Power, Sample Size, and Effect Size Assumptions and Diagnostics for Linear Regression Transformation in the Context of Linear Models Fixing the Y-Intercept Weighted Least Squares Polynomial Regression Comparing Model Slopes Likelihood and General Linear Models Model Selection Robust Regression Model II Regression (X Not Fixed) Generalized Linear Models Nonlinear Models Smoother Approaches to Association and Regression Bayesian Approaches to Regression ANOVA Introduction One-Way ANOVA Inferences for Factor Levels ANOVA as a General Linear Model Random Effects Power, Sample Size, and Effect Size ANOVA Diagnostics and Assumptions Two-Way Factorial Design Randomized Block Design Nested Design Split-Plot Design Repeated Measures Design ANCOVA Unbalanced Designs Robust ANOVA Bayesian Approaches to ANOVA Tabular Analyses Introduction Probability Distributions for Tabular Analyses One-Way Formats Confidence Intervals for p Contingency Tables Two-Way Tables Ordinal Variables Power, Sample Size, and Effect Size Three-Way Tables Generalized Linear Models Appendix References Index A Summary and Exercises appear at the end of each chapter.

The Hubble parameter is a critical measurement in cosmology, which contains the most direct information of the cosmic expansion history. Since discrepancy is found between low redshift and high redshift estimations of Hubble constant, we are interested in whether that tension indicates dynamical dark energy. In this paper, we emphasize that the observed Hubble parameters at various redshifts, along with observed Hubble constant, can help us in probing the evolutional behavior of the mysterious dark energy. Null hypothesis tests are carried out with two diagnostic approaches. We find out that, according to the present measurements of Hubble parameters, rejection of constant dark energy is captured at 1σ level from null tests with and without the observed value of Hubble constant.

The power flows in the unified power flow controller (UPFC) embedded system is mainly regulated by the two variables containing the magnitude and the phase angle of the output series inserted voltage (OSIV) of UPFC. Different value combinations of the two variables can form multiple regulation modes of OSIV, and the regulation principles and efficiencies for power flows are distinct by different regulation modes. This paper dedicates to research the regulation principles of active and reactive power flow gradients (PFG) to multiple characteristic independent variables (CIVs) at several selected critical points (SCP) of the system in different operation conditions. The CIVs contains the magnitude of OSIV, the phase angle of OSIV, and the phase difference of the system. First, multiple power flow regulation modes of OSIV are designed, the mathematical models of the PFG to each CIV at each SCP are established, and the theoretical principles for the PFG to each CIV at each SCP are analyzed and compared. Next, four typical operation conditions of the system and four regulation scenarios are assumed and case studies for the PFG to each CIV at different SCP are carried out. The test results at each SCP are analyzed both in the two-dimensional planes and three-dimensional spaces. The regulation principles and efficiencies of PFG to each CIV at different SCP are compared with each other and summarized, which can offer useful references for practical engineering and applications of UPFC.

Multivariate regression is known as a multivariate extension of multiple regression, which explain/predict the variations in multiple dependent variables by multiple independent variables. Recently, various procedures for Sparse Multivariate Regression (SMR) have been proposed, in which a sparse regression coefficient matrix (having a number of zero elements) is obtained aiming to facilitate its interpretation. The procedures for SMR can be classified into the following two types; penalized and cardinality-constrained procedures. In them, the resulting number of zeros in the regression coefficient matrix is controlled/constrained by a prespecified penalty parameter or cardinality value. In this research, we propose another approach for SMR, referred to as Layered Multivariate Regression (LMR). In LMR, the regression coefficient matrix is assumed to be a sum of several sparse matrices, which is called layer. Therefore, the sparseness of the resulting coefficient matrix is controlled by how many layers are used. In LMR, k-th layer can be viewed as the coefficient matrix in the regression of a partial residual (i.e., the residual for all but k-th layer) on independent variables, and thus the variance explained by LMR gets closer to that for the unconstrained regression as the number of layers increases. We present an alternating least squares algorithm for LMR and a procedure for determining how many layers should be used. LMR is assessed in a simulation study and illustrated with a real data example. As an application of LMR, procedures for sparse estimation in some multivariate analysis techniques (e.g., principal component analysis) are also presented.

این مطالعه به منظور بررسی پیش بینی شاخص های عملکرد تولیدی در مرغان تخم گذار با استفاده از شبکه های عصبی مصنوعی و رگرسیون غیرخطی چندگانه انجام شد. بررسی بر روی اطلاعات چهار دوره متوالی پرورش در یک واحد پرورش مرغ تخم گذار صورت گرفت. روش های داده-کاوی شامل رگرسیون خطی و غیر خطی، شبکه عصبی پرسپترون سه لایه، شبکه عصبی پرسپترون چهار لایه و شبکه عصبی با تابع پایه ای شعاعی بود. در این مدل ها از متغیرهای سن گله، میزان خوراک مصرفی و فصل تولید به عنوان متغیر پیشگو و شاخص های عملکرد تولیدی شامل درصد تخم-گذاری، وزن توده ای تخم مرغ تولیدی و ضریب تبدیل غذایی به عنوان متغیر پاسخ استفاده شد. نتایج نهایی رگرسیون های خطی نشان داد که برای تمامی متغیرهای وابسته مورد مطالعه متغیر مستقل سن گله معنی دار می باشد. بنابراین رگرسیون غیر خطی شاخص های عملکرد تولیدی در مقابل سن برای مقایسه با شبکه های عصبی مختلف مورد بررسی قرار گرفت و برای مقایسه کلیه مدل ها از ضریب تعیین (R2) و میانگین قدر مطلق خطا (MAE) استفاده شد. نتایج نشان داد بین شبکه های عصبی مصنوعی مختلف مورد مطالعه، شبکه با تابع پایه ای شعاعی بهتر از سایر مدل های در پیش‌بینی شاخص های عملکرد تولیدی مرغان تخم گذار عمل می کند.

Null hypothesis testing has long since been the ‘go‐to analytic approach’ in quantitative second language (L2) research (Norris, 2015, p. 97). To many, however, years of reliance on this approach has resulted in a crisis of inference across the social and behavioral sciences (e.g., Rouder et al., 2016). As an alternative to the null hypothesis testing approach, many such experts recommend the Bayesian hypothesis testing approach. Adopting an open‐science framework, the present study (a) re‐evaluates the empirical findings of 418 t‐tests from published L2 research using Bayesian hypothesis testing, and (b) compares the Bayesian results with their conventional, null hypothesis testing counterparts as observed in the original reports. The results show that the Bayesian and the null hypothesis testing approaches generally arrive at similar inferential conclusions. However, considerable differences arise in the rejections of the null hypothesis. Notably, in 64.06% of cases when p‐values fell between .01 and .05 (i.e., evidence to reject the null), the Bayesian analysis found the evidence in the primary studies to be only at an ‘anecdotal’ level (i.e., insufficient evidence to reject the null). Practical implications, field‐wide recommendations, and an introduction to free online software (https://rnorouzian.shinyapps.io/bayesian-t-tests) for Bayesian hypothesis testing are discussed.

Multivariate analysis refers to statistical techniques that simultaneously look at three or more variables in relation to the subjects under investigation with the aim of identifying or clarifying the relationships between them. These techniques have been broadly classified as dependence techniques, which explore the relationship between one or more dependent variables and their independent predictors, and interdependence techniques, that make no such distinction but treat all variables equally in a search for underlying relationships. Multiple linear regression models a situation where a single numerical dependent variable is to be predicted from multiple numerical independent variables. Logistic regression is used when the outcome variable is dichotomous in nature. The log-linear technique models count type of data and can be used to analyze cross-tabulations where more than two variables are included. Analysis of covariance is an extension of analysis of variance (ANOVA), in which an additional independent variable of interest, the covariate, is brought into the analysis. It tries to examine whether a difference persists after “controlling” for the effect of the covariate that can impact the numerical dependent variable of interest. Multivariate analysis of variance (MANOVA) is a multivariate extension of ANOVA used when multiple numerical dependent variables have to be incorporated in the analysis. Interdependence techniques are more commonly applied to psychometrics, social sciences and market research. Exploratory factor analysis and principal component analysis are related techniques that seek to extract from a larger number of metric variables, a smaller number of composite factors or components, which are linearly related to the original variables. Cluster analysis aims to identify, in a large number of cases, relatively homogeneous groups called clusters, without prior information about the groups. The calculation intensive nature of multivariate analysis has so far precluded most researchers from using these techniques routinely. The situation is now changing with wider availability, and increasing sophistication of statistical software and researchers should no longer shy away from exploring the applications of multivariate methods to real-life data sets.

Experimental power comes from powerful theories — the real problem in null hypothesis testing

Multiple Linear Regression is a well-known technique used to experimentally investigate the relationship between one dependent variable and multiple independent variables. However, fitting this model has problems, for example when the sample size is large. Consequently, the results of traditional methods to estimate the model can be misleading. So, there have been proposed regularization or shrinkage techniques to estimate the model in this case. In this work, we have proposed a methodology to build a crime rate performance prediction model using multiple linear regression methods with regularization. Our methodology consists of three major steps: i) analyzing and preprocessing the dataset; ii) optimizing the model using k-fold cross-validation and hyperparameter tuning; iii) comparing the performance of different models using accuracy metrics. The obtained results show that the model built using lasso regression, outperforms the other constructed models.

In behavioral research, interest is often in examining the degree to which the effect of an independent variable X on an outcome Y is mediated by an intermediary or mediator variable M. This article illustrates how generalized estimating equations (GEE) modeling can be used to estimate the indirect or mediated effect, defined as the amount by which the regression coefficient of X on Y changes after adjusting for M. Advantages of this method are: (a) it applies to the class of generalized linear models, including linear, logistic, and Poisson regression as special cases; (b) it allows multiple independent variables and mediators in the same model; and (c) asymptotically valid standard errors and confidence intervals are obtained using standard software. This methodology is compared with the bootstrap, another general methodology that can be applied to the same broad class of models, and is evaluated using simulation in both linear and logistic regression scenarios. The methods are utilized to examine the degree to which the effect of low birthweight status on internalizing symptoms at age 20 is mediated through IQ at age 8.