Information‐Theoretic Scores for Bayesian Model Selection and Similarity Analysis: Concept and Application to a Groundwater Problem

: Calculating the amount of information about a random function contained in another random function has important uses in communication theory. An expression for the mutual information for continuous time random processes has been given by Gelfand and Yaglom, Chiang, and Perez by generalizing Shannon's result in a natural way. Under a condition of absolute continuity of measures the continuous time expression has the same form as Shannon's result. For two Gaussian processes Gelfand and Yaglom express the mutual information in terms of a mean square estimation error. We generalize this result to diffusion processes and express the solution in a different form which is more naturally related to a corresponding filtering problem. We also use these results to calculate some information rates.

Objective The Bayesian model plays an important role in diagnostic test evaluation in the absence of the gold standard, which used the external prior distribution of a parameter combined with sample data to yield the posterior distribution of the test characteristics. However, the correlation between diagnostic tests has always been a problem that cannot be ignored in the Bayesian model evaluation. This study will discuss how different Bayesian model, correlation scenarios, and prior distribution affect the outcome. MethodsThe data analyzed in this study was gathered during studies of patients presenting to the Nanjing Chest Hospital with suspected tuberculosis. The diagnostic character of T-SPOT.Tb and KD38 tuberculosis antibody test were evaluated in different Bayesian model, and discharge diagnosis as a gold standard was used to verify the model results in the end. Result The comparison of four models under the conditional independence situation found that Bayesian probabilistic constraint model was consistent with the Conditional Covariance Bayesian model. The results were mainly affected by prior information. The sensitivity and specificity of the two tests in Conditional Covariance Bayesian model in prior constraint situation were considerably higher than the Bayesian probabilistic constraint model in prior constraint situation. The results of the four models under the conditional dependence situation were similar to the conditional independence situation; pD was also negative with no prior constraint situation in both model Bayesian probabilistic constraint model and Conditional Covariance Bayesian model. The Deviance Information Criterion of Bayesian probabilistic constraint model was close to model Conditional Covariance Bayesian model, but pD of Conditional Covariance Bayesian model in Prior constraint situation (pD=2.40) was higher than the Bayesian probabilistic constraint model in Prior constraint situation (pD=1.66). Conclusion The result of Conditional Covariance Bayesian model in prior constraint with conditional independence situation was closest to the result of gold standard evaluation in our data. Both of the two Bayesian methods are the feasible way for the evaluation of diagnostic test in the absence of the gold standard diagnostic. Prior source, priority number, and conditional dependencies should be considered in the method selection, the accuracy of posterior estimation mainly depending on the prior distribution.

For pt. I see ibid., vol. 19, p. 2045 (1993). The information entropy of the atomic nucleus is calculated from the wavefunctions of the resonance states. The transition from low- to high-level density is traced as a function of the coupling strength between the discrete nuclear states and the environment of decay channels. In the critical region of the coupling strength where a redistribution inside the nucleus takes place, information entropy in relation to the discrete states of the closed system is created. Beyond the critical value, a few relevant short-lived modes exist: together with long-lived noise. This result is in full correspondence to the maximum information entropy principle of synergetics formulated by Haken (1992). The noise is characterized by disorder expressed by a large information entropy while the relevant modes have a high order and take, correspondingly, a small part of the information entropy of the whole system. The entropy excess accompanying the formation of the new order is used inside the system for creation of noise. Further, the noise is not structureless and the corresponding information entropy is smaller than its maximal value.

This article considers the issues of dynamics of road safety provision systems in three multiscaled systems (Russia, Siberian Federal Okrug or SFO and separate regions of SFO). The indicator of relative information entropy Hn of road safety provision process was taken as the characteristic of system orderliness. The ideology, methodic and results of quantitative assessment of road safety provision systems information entropy H were considered. Time series of modification of relative information entropy Hn of road safety provision systems in 2004.. .2018 in Russia, Siberian Federal Okrug and individual subjects of SFD were presented. Special attention was paid to consideration of the re-searched process in the Republic of Tuva. This region is characterized by nonstandard, different from general trend dynamics of information entropy H of road safety provision system. The general conclusion of the article is that during last 15 years information entropy of road safety provision systems significantly decreased, i.e. orderliness of formation process of road traffic participants safety greatly increased. However, results achieved in this sphere by Siberian regions are still far from best world results.

Quantitative palaeoclimatic reconstructions based on biological fossils are a major source of information on long-term climatic variability. Such reconstructions typically use some kind of a modern calibration data set describing the variation of the studied biological group in present-day climate space. Here, we explore the effect of calibration data set selection on palaeoclimatic reconstructions, by creating alternate calibration data sets via stratified random sampling to reconstruct mean July temperature ( Tjul) for four fossil pollen sequences from northern Europe. We show that palaeoclimatic reconstructions using methods based on taxon-response models can be highly sensitive to the calibration data set used. In particular, the absolute reconstructed temperatures show great sensitivity to calibration data selection, which suggests that the absolute values of palaeoclimatic reconstructions may not be robust. By contrast, we find the relative shapes of the reconstructed curves to be more robust to calibration data selection because taxa tend to occupy similar relative locations along the sampled gradient regardless of calibration data set location. Based on this robustness of relative palaeoclimate curves, we suggest a debiasing procedure in which palaeoclimate values are estimated by fixing the relative curve with the modern observed value, thus correcting biases resulting from calibration data selection.

Development of Fourier transform mid-infrared calibrations to predict acetone, β-hydroxybutyrate, and citrate contents in bovine milk through a European dairy network

This white paper lays out a vision of research and development in the field of artificial intelligence for the next decade (and beyond). Its denouement is a cyber-physical ecosystem of natural and synthetic sense-making, in which humans are integral participants—what we call “shared intelligence.” This vision is premised on active inference, a formulation of adaptive behavior that can be read as a physics of intelligence, and which inherits from the physics of self-organization. In this context, we understand intelligence as the capacity to accumulate evidence for a generative model of one’s sensed world—also known as self-evidencing. Formally, this corresponds to maximizing (Bayesian) model evidence, via belief updating over several scales, that is, inference, learning, and model selection. Operationally, this self-evidencing can be realized via (variational) message passing or belief propagation on a factor graph. Crucially, active inference foregrounds an existential imperative of intelligent systems; namely, curiosity or the resolution of uncertainty. This same imperative underwrites belief sharing in ensembles of agents, in which certain aspects (i.e., factors) of each agent’s generative world model provide a common ground or frame of reference. Active inference plays a foundational role in this ecology of belief sharing—leading to a formal account of collective intelligence that rests on shared narratives and goals. We also consider the kinds of communication protocols that must be developed to enable such an ecosystem of intelligences and motivate the development of a shared hyper-spatial modeling language and transaction protocol, as a first—and key—step towards such an ecology.

Amidst certainty, efficiency can improve sustainability by reducing resource consumption. However, flexibility is needed to be able to survive when uncertainty increases. Apropos, sustainable production cannot persist in the long-term without having both flexibility and efficiency. Referring to cognitive science to inform the development of production systems is well established. However, recent research in cognitive science encompassing flexibility and efficiency in brain functioning have not been considered previously. In particular, research by others that encompasses information (I), information entropy (H), relative entropy (D), transfer entropy (TE), and brain entropy. By contrast, in this paper, flexibility and efficiency for persistent sustainable production is analyzed in relation to these information theory applications in cognitive science and is quantified in terms of information. Thus, this paper is consistent with the established practice of referring to cognitive science to inform the development of production systems. However, it is novel in addressing the need to combine flexibility and efficiency for persistent sustainability in terms of cognitive functioning as modelled with information theory.

The current concordance model of cosmology is dominated by two mysterious ingredients: dark matter and dark energy. In this paper, we explore the possibility that, in fact, there exist two dark-energy components: the cosmological constant Λ, with equation-of-state parameter wΛ=−1, and a 'missing matter' component X with wX=−2/3, which we introduce here to allow the evolution of the universal scale factor as a function of conformal time to exhibit a symmetry that relates the big bang to the future conformal singularity, such as in Penrose's conformal cyclic cosmology. Using recent cosmological observations, we constrain the present-day energy density of missing matter to be ΩX,0=−0.034 ± 0.075. This is consistent with the standard ΛCDM model, but constraints on the energy densities of all the components are considerably broadened by the introduction of missing matter; significant relative probability exists even for ΩX,0 ∼ 0.1, and so the presence of a missing matter component cannot be ruled out. As a result, a Bayesian model selection analysis only slightly disfavours its introduction by 1.1 log-units of evidence. Foregoing our symmetry requirement on the conformal time evolution of the universe, we extend our analysis by allowing wX to be a free parameter. For this more generic 'double dark energy' model, we find wX = −1.01 ± 0.16 and ΩX,0 = −0.10 ± 0.56, which is again consistent with the standard ΛCDM model, although once more the posterior distributions are sufficiently broad that the existence of a second dark-energy component cannot be ruled out. The model including the second dark energy component also has an equivalent Bayesian evidence to ΛCDM, within the estimation error, and is indistuingishable according to the Jeffreys guideline.

Editorial| July 2023 Anesthesia and the Merry-go-round of Information in the Brain Jamie W. Sleigh, M.D.; Jamie W. Sleigh, M.D. 1Department of Anaesthesiology, Waikato Clinical Campus, Faculty of Medical and Health Sciences, University of Auckland, Hamilton, New Zealand. Search for other works by this author on: This Site PubMed Google Scholar Logan Voss, Ph.D. Logan Voss, Ph.D. 2Department of Anaesthesiology, Waikato Clinical Campus, Faculty of Medical and Health Sciences, University of Auckland, Hamilton, New Zealand. Search for other works by this author on: This Site PubMed Google Scholar Author and Article Information This editorial accompanies the article on p. 49. Accepted for publication March 30, 2023. Address correspondence to Dr. Sleigh: jamie.sleigh@waikatodhb.health.nz Anesthesiology July 2023, Vol. 139, 4–5. https://doi.org/10.1097/ALN.0000000000004571 Connected Content Article: Measures of Information Content during Anesthesia and Emergence in the Caenorhabditis elegans Nervous System Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Share Icon Share Facebook Twitter LinkedIn Email Cite Icon Cite Get Permissions Search Site Citation Jamie W. Sleigh, Logan Voss; Anesthesia and the Merry-go-round of Information in the Brain. Anesthesiology 2023; 139:4–5 doi: https://doi.org/10.1097/ALN.0000000000004571 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAll PublicationsAnesthesiology Search Advanced Search Topics: brain It is widely believed that anesthesia is a syndrome caused by drug-induced disruption of coordination within and between neuronal networks. The underlying teleological assumption is that normal function of the central nervous system requires precise electrical and chemical communication between neurons, and communication implies flow of “information.” In this issue of Anesthesiology is an article by Chang et al.1 in which isoflurane was given to a nematode (Caenorhabditis elegans), and the resultant real-time changes in activity of about half of its 302 neurons were visualized using high-resolution calcium fluorescence imaging. In this study, the authors focused on how anesthesia alters interneuronal coupling by measuring the entropy of the calcium transients in the imaged neurons. The meaning of the word “entropy” is intuitively difficult. It is often used as a synonym for disorder, uncertainty, or unpredictability. To take the extreme examples: if a neuron has no activity (a... You do not currently have access to this content.

In this paper, we present a novel method for computing the relative entropy as well as the expected relative entropy using an MCMC chain. The relative entropy from information theory can be used to quantify differences in posterior distributions of a pair of experiments. In cosmology, the relative entropy has been proposed as an interesting tool for model selection, experiment design, forecasting and measuring information gain from subsequent experiments. In contrast to Gaussian distributions, these quantities are not generally available analytically and one needs to use numerical methods to estimate them which are certainly computationally expensive. We propose a method and provide its python package to estimate the relative entropy as well as expected relative entropy from a posterior sample. We consider the linear Gaussian model to check the accuracy of our code. Our results indicate that the relative error is below $0.2\%$ for sample size larger than $10^5$ in the linear Gaussian model. In addition, we study the robustness of our code in estimating the expected relative entropy in this model.

<p>Rainfall timing is a key parameter that farmers rely on to match the cropping season with the time window over which seasonal precipitation provides adequate soil moisture to meet plant growth demand. The unpredictability of rainfall timing affects the selection of an optimal growing season, and hence crop production in regions where rainfed agriculture (RFA) is practiced. In this study, we (a) map rainfall timing, and its interannual variability and changes over RFA areas across Ethiopia for the period 1981-2010, and (b) explore the impact of variability in rainfall timing on cereal crop production in the period 1995-2010.</p><p>For the mapping of rainfall timing, we used the quasi-global CHIRPS precipitation dataset over Ethiopia. We use information entropy on monthly rainfall to define the rainfall seasonality metrics, i.e. the relative entropy and dimensionless seasonality index, and map them in space. For rainfall timing attributes, we determine the onset, cessation, and length of the wet season from LOESS-smoothed cumulative pentad rainfall anomalies for each hydrological year. Changes in seasonality metrics and rainfall timing attributes are analyzed using non-parametric methods. We show that high seasonality (unimodal rainfall regime) is located in the northern part of the Ethiopian RFA area where high annual rainfall and high relative entropy are coincident, and where the onset of the rainfall season varies between mid-April to late-June and cessation occurs between mid-September to late-October. Low seasonality in the southern part of the Ethiopian RFA area shows low relative entropy regardless of the annual rainfall total. We observed a considerable interannual variability both in seasonality and rainfall timing over the study period, especially in the onset and length of the wet season. The length of the wet season and magnitude of seasonal rainfall are predominantly controlled by the timing of rainfall onset.</p><p>For the impacts of rainfall timing on crop production, we used cereal crop production data from the Central Statistical Agency of Ethiopia for the period 1995-2010 in 45 administrative zones. We carried out a parametric correlation analysis between rainfall timing and rescaled and de-trended crop production anomalies. We observe that anomalies in seasonal cereal crop production in RFA areas are significantly correlated with anomalies in rainfall onset (negatively) and the length of the wet season (positively), with a regional average production loss of 3% per pentad of late rainfall onset, and 2.7% per pentad of shorter length of the wet season. Seasonal rainfall is less strongly correlated with cereal crop production anomalies compared to the rainfall onset. These results show that the interannual variability in rainfall timing (start of the rainy season) even under present climate has strong impacts on crop yields in RFA areas in Ethiopia, and this may be exacerbated in a future climate.</p>

A new process was developed at Ames Research Center that automatically generates a permitted and a recommended math model for the analysis of strain–gage balance calibration data. The accuracy of these math models was investigated using two calibration data sets that were provided by the German– Dutch Wind Tunnels. These calibration data sets were obtained in a balance calibration machine. The first calibration data set was generated using the traditional OFAT method. The second data set was obtained using the MDOE approach. A single data reduction matrix was computed for the OFAT data set using the permitted math model. Two data reduction matrices were calculated for the MDOE data set using the permitted and the recommended math model. Measured and fitted loads at check points were compared in order to assess the accuracy of the three data reduction matrices. This comparison showed that the data reduction matrix of the recommended math model, i.e., of the math model that uses only the most significant terms of each gage, is as accurate as the data reduction matrix of the much larger permitted math model. In addition, it was demonstrated that, for the given balance calibration task, the two data reduction matrices of the MDOE data set are as accurate as the data reduction matrix of the much larger OFAT data set.

A new approach to the problem of multimodality medical image registration is proposed, using a basic concept from information theory, mutual information (MI), or relative entropy, as a new matching criterion. The method presented in this paper applies MI to measure the statistical dependence or information redundancy between the image intensities of corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned. Maximization of MI is a very general and powerful criterion, because no assumptions are made regarding the nature of this dependence and no limiting constraints are imposed on the image content of the modalities involved. The accuracy of the MI criterion is validated for rigid body registration of computed tomography (CT), magnetic resonance (MR), and photon emission tomography (PET) images by comparison with the stereotactic registration solution, while robustness is evaluated with respect to implementation issues, such as interpolation and optimization, and image content, including partial overlap and image degradation. Our results demonstrate that subvoxel accuracy with respect to the stereotactic reference solution can be achieved completely automatically and without any prior segmentation, feature extraction, or other preprocessing steps which makes this method very well suited for clinical applications.

This paper investigates the evolution of information entropy (IE) in Hermite–Gaussian correlated Schell-model (HGcSM) beams propagating through a gradient-index (GRIN) fiber using Shannon information theory. Our results reveal that the IE of such beams evolves periodically, with the beam order significantly influencing its initial distribution. Compared with traditional Gaussian Schell-model beams, HGcSM beams exhibit more complex IE dynamics, characterized by periodically emerging low-entropy regions whose IE decreases with increasing beam order. Furthermore, the fiber’s central refractive index and core radius strongly affect the evolution period and fluctuation amplitude of IE. These findings provide a theoretical basis for optimizing partially coherent beams in optical fiber applications.