Expyriment: A Python library for cognitive and neuroscientific experiments

Since the coining of the term phylodynamics, the use of phylogenies to understand infectious disease dynamics has steadily increased. As methods for phylodynamics and genomic epidemiology have proliferated and grown more computationally expensive, the epidemiological information they extract has also evolved to better complement what can be learned through traditional epidemiological data. However, for genomic epidemiology to continue to grow, and for the accumulating number of pathogen genetic sequences to fulfill their potential widespread utility, the extraction of epidemiological information from phylogenies needs to be simpler and more efficient. Summary statistics provide a straightforward way of extracting information from a phylogenetic tree, but the relationship between these statistics and epidemiological quantities needs to be better understood. In this work we address this need via simulation. Using two different benchmark scenarios, we evaluate 74 tree summary statistics and their relationship to epidemiological quantities. In addition to evaluating the epidemiological information that can be inferred from each summary statistic, we also assess the computational cost of each statistic. This helps us optimize the selection of summary statistics for specific applications. Our study offers guidelines on essential considerations for designing or choosing summary statistics. The evaluated set of summary statistics, along with additional helpful functions for phylogenetic analysis, is accessible through an open-source Python library. Our research not only illuminates the main characteristics of many tree summary statistics but also provides valuable computational tools for real-world epidemiological analyses. These contributions aim to enhance our understanding of disease spread dynamics and advance the broader utilization of genomic epidemiology in public health efforts.Author SummaryOur study focuses on the use of phylogenetic analysis to get valuable epidemiological insights. We conducted a simulation study to evaluate 74 phylogenetic summary statistics and their relationship to epidemiological quantities, shedding light on the potential of each of these statistics to quantify different characteristics of disease spread dynamics. Additionally, we assessed the computational cost of each statistic. This gives us additional information when selecting a statistic for a particular application. Our research is available through an open-source Python library. This work helps us enhance our understanding of phylogenetic tree structures and contributes to the broader application of genomic epidemiology in public health initiatives.

World Wide Web is an extremely large collection of information, i.e. beyond our imagination. It provides enough information according to user's need. Web is rising dreadfully as approximately 70 million pages are added daily. Knowledge Discovery on web data is referred as Web Mining. Web Structure Mining based on the analysis of patterns from hyperlink structure in the web. Like as Data Mining, Web Mining has four stages i.e. Data Collection, Preprocessing, Knowledge Discovery and Knowledge Analysis. This paper based on the first two stages Data collection and Preprocessing. Data collection is to collect the data required for analysis. Data preprocessing is considered as an important stage of Web Structure mining because of data available on web is unstructured, heterogeneous and noisy.

Hierarchically clustered heatmaps are a popular visualization technique to display high-dimensional biological datasets as figures in research publications. Most tools developed to visualize hierarchically clustered heatmaps generate static images. Clustergrammer is a web-based matrix visualization tool built, using the D3 JavaScript library, to enable experimental and computational-biologists to easily generate highly interactive heatmaps. Clustergrammer implements many interactive features including: zooming, panning, filtering, sorting, reordering, dimensionality reduction, gene set enrichment analysis, gene description text, and shareable persistent links. Clustergrammer can be used to visualize and analyze a diverse set of high-dimensional biological data including: gene expression, protein-protein interactions, and CyTOF single-cell proteomics data. Clustergrammer provides interactive visualizations for several web-tools including: Enrichr, GEN3VA, GEO2Enrichr, Harmonizome, and L1000CDS2. In addition, Clustergrammer can be embedded within IPython Jupyter Notebooks or as a standalone open-source JavaScript or Python library.

The lack of long-term and consistent meteorological observations limits the application of land-surface simulators (e.g., of phenomena in hydrology, the cryosphere, ecology) at remote locations. For example, most permafrost areas are remote and lacking consistent meteorological time series, models that describe permafrost change over time cannot be driven for comparison with observations or for impact studies. Reanalysis-derived time series are valuable because they are available with global coverage, for a long time period, and for a broad set of physically consistent variables. Multiple reanalyses can be used to provide estimates of uncertainty. Practically, however, this data is difficult to use for several reasons: grid-scale reanalyses must be downscaled and interpolated horizontally (and vertically within the atmospheric column for mountains regions) to the site‑scale, differences in variables, units, and delivery between reanalyses must be reconciled, and large volumes of data need to be handled. Globsim is an open-source python library (available via GitHub) that was developed to handle these challenges and to facilitate a simulation workflow that takes advantage of the multiple reanalysis products available today. It outputs sub-daily meteorological time series that resemble meteorological stations for any location on the planet. Since the release of the first version of Globsim, we have improved usability, refactored code for maintainability and speed, and fixed a number of bugs. We also added support for ERA5 ensemble data, and added more sophisticated heuristic downscaling algorithms, including TOPOscale for elevation-adjusted radiative fluxes. We use Globsim as a core tool in a multi-model permafrost simulation workflow and, as a future step, we intend to use it as part of a debiasing routine to make predictions of permafrost using climate scenarios. We expect this tool to be broadly applicable to climate change impact modelers and other scientists using climate driven simulations working in (remote) locations that lack meteorological data of sufficient quality and duration for their application.

We present ichor, an open-source Python library that simplifies data management in computational chemistry and streamlines machine learning force field development. Ichor implements many easily extensible file management tools, in addition to a lazy file reading system, allowing efficient management of hundreds of thousands of computational chemistry files. Data from calculations can be readily stored into databases for easy sharing and post-processing. Raw data can be directly processed by ichor to create machine learning-ready datasets. In addition to powerful data-related capabilities, ichor provides interfaces to popular workload management software employed by High Performance Computing clusters, making for effortless submission of thousands of separate calculations with only a single line of Python code. Furthermore, a simple-to-use command line interface has been implemented through a series of menu systems to further increase accessibility and efficiency of common important ichor tasks. Finally, ichor implements general tools for visualization and analysis of datasets and tools for measuring machine-learning model quality both on test set data and in simulations. With the current functionalities, ichor can serve as an end-to-end data procurement, data management, and analysis solution for machine-learning force-field development.

IMDLIB: An open-source library for retrieval, processing and spatiotemporal exploratory assessments of gridded meteorological observation datasets over India

In an age of both big data and increasing strain on water resources, sound management decisions often rely on numerical models. Numerical models provide a physics-based framework for assimilating and making sense of information that by itself only provides a limited description of the hydrologic system. Often, numerical models are the best option for quantifying even intuitively obvious connections between human activities and water resource impacts. However, despite many recent advances in model data assimilation and uncertainty quantification, the process of constructing numerical models remains laborious, expensive, and opaque, often precluding their use in decision making. Modflow-setup aims to provide rapid and consistent construction of MODFLOW groundwater models through robust and repeatable automation. Common model construction tasks are distilled in an open-source, online code base that is tested and extensible through collaborative version control. Input to Modflow-setup consists of a single configuration file that summarizes the workflow for building a model, including source data, construction options, and output packages. Source data providing model structure and parameter information including shapefiles, rasters, NetCDF files, tables, and other (geolocated) sources to MODFLOW models are read in and mapped to the model discretization, using Flopy and other general open-source scientific Python libraries. In a few minutes, an external array-based MODFLOW model amenable to parameter estimation and uncertainty quantification is produced. This paper describes the core functionality of Modflow-setup, including a worked example of a MODFLOW 6 model for evaluating pumping impacts to a lake in central Wisconsin, United States.

In the era of digital media, the ability to accurately identify music from audio input has gained significant importance in areas such as entertainment, copyright enforcement, and media content management. This project presents a robust and efficient system for song recognition based on audio fingerprinting techniques. By capturing real-time audio through a microphone and converting it into a unique digital signature using SHA1 hashing, the system enables accurate and fast identification of songs from a reference database. The implementation leverages open-source tools and Python libraries to ensure scalability, portability, and cost-effectiveness. The methodology involves real-time audio capture using PyAudio, preprocessing and normalization with PyDub, and feature extraction for fingerprint generation. These fingerprints are then matched against a SQLite database containing known song signatures. The system is designed to handle short audio clips and operate reliably even in the presence of background noise or distortions. This enhances its usability in real-world environments, making it a practical tool for mobile apps, music streaming platforms, and media monitoring services

An optimization methodology has been developed for the aluminum alloy wheel casting process. The methodology is focused on improving the timing of cooling processes in a die to achieve improved casting quality. This methodology utilizes (1) a casting process model, which was developed within the commercial finite element package, ABAQUS™—ABAQUS is a trademark of Dassault Systems; (2) a Python-based results extraction procedure; and (3) a numerical optimization module from the open-source Python library, Scipy. To achieve optimal casting quality, a set of constraints have been defined to ensure directional solidification, and an objective function, based on the solidification cooling rates, has been defined to either maximize, or target a specific, cooling rate. The methodology has been applied to a series of casting and die geometries with different cooling system configurations, including a 2-D axisymmetric wheel and die assembly generated from a full-scale prototype wheel. The results show that, with properly defined constraint and objective functions, solidification conditions can be improved and optimal cooling conditions can be achieved leading to process productivity and product quality improvements.

Current interpretability methods for Large Language Models (LLMs) operate on a fundamental yet flawed assumption: that subword tokens represent independent semantic units.We prove that this assumption creates a fidelity bottleneck in Morphologically Rich Languages (MRLs), where semantic meaning is densely encoded in sub-token morphemes.We term this phenomenon the Tokenization-Morphology Misalignment (TMM).To resolve TMM, we introduce MAFEX (Morpheme-Aligned Faithful Explanations), a theoretically grounded framework that redefines feature attribution as a linear projection from the computational (token) basis to the linguistic (morpheme) basis.We evaluate our method on a diverse suite of Turkish LLMs, including BERTurk, BERTurk-Sentiment, Cosmos-BERT, and Kumru-2B.On our embedded benchmark (N = 20), MAFEX achieves an average F1@1 of 91.25% compared to 13.75% for standard token-level baselines (IG, SHAP, DeepLIFT), representing a +77.5% absolute improvement, establishing it as the new standard for faithful multilingual interpretability.* The MAFEX framework and evaluation suite is available as a comprehensive, open-source Python library at https: //github.com/anilyagiz/mafex(pip install mafex) to facilitate reproducible multilingual XAI research.

Land surface temperature (LST) is a key variable for monitoring and evaluating global long-term climate change. However, existing satellite-based twice-daily LST products only date back to 2000, which makes it difficult to obtain robust long-term temperature variations. In this study, we developed the first global historical twice-daily LST dataset (GT-LST), with a spatial resolution of 0.05°, using Advanced Very High Resolution Radiometer (AVHRR) Level-1b Global Area Coverage (GAC) data from 1981 to 2005. The GT-LST product was generated using four main processes: (1) GAC data reading, calibration, and pre-processing using open-source Python libraries; (2) cloud detection using the AVHRR-Phase I algorithm; (3) land surface emissivity estimation using an improved method considering annual land cover changes; and (4) LST retrieval based on a nonlinear generalized split-window algorithm. Validation with in situ measurements from Surface Radiation Budget (SURFRAD) sites showed that the overall root-mean-square errors of GT-LST varied from 2.0 K to 3.9 K, and nighttime LSTs were typically better than daytime LSTs. Inter-comparison with a common LST product (i.e., MYD11A1) revealed that the overall root-mean-square-difference (RMSD) was approximately 3.2 K, a positive bias was obtained for GT-LST, and relatively large RMSDs were obtained during the daytime, spring and summer. Furthermore, we compared our newly generated dataset with a global AVHRR daytime LST product at the selected measurements of SURFRAD sites (i.e., measurements of these two satellite datasets were valid), which revealed similar accuracies for the two datasets. However, GT-LST can additionally provide nighttime LST, which can be combined with daytime observations estimating relatively accurate monthly mean LST under all-sky conditions, with RMSE of 4.1 K. Finally, we compared GT-LST with a regional twice-daily AVHRR LST product over continental Africa in different seasons, with RMSDs ranging from 2.1 to 4.3 K. Considering these advantages, the proposed dataset provides a better data source for a range of research applications. GT-LST is freely available at https://doi.org/10.5281/zenodo.7113080 (1981–2000) (Li et al., 2022a) and https://doi.org/10.5281/zenodo.7134158 (2001–2005) (Li et al., 2022b).

Abstract. Land surface temperature (LST) is a key variable for monitoring and evaluating global long-term climate change. However, existing satellite-based twice-daily LST products only date back to 2000, which makes it difficult to obtain robust long-term temperature variations. In this study, we developed the first global historical twice-daily LST dataset (GT-LST), with a spatial resolution of 0.05∘, using Advanced Very High Resolution Radiometer (AVHRR) Level-1b Global Area Coverage (GAC) data from 1981 to 2021. The GT-LST product was generated using four main processes: (1) GAC data reading, calibration, and preprocessing using open-source Python libraries; (2) cloud detection using the AVHRR-Phase I algorithm; (3) land surface emissivity estimation using an improved method considering annual land cover changes; (4) LST retrieval based on a nonlinear generalized split-window algorithm. Validation with in situ measurements from Surface Radiation Budget (SURFRAD) sites and Baseline Surface Radiation Network sites showed that the overall root-mean-square errors (RMSEs) of GT-LST varied from 1.6 to 4.0 K, and nighttime LSTs were typically better than daytime LSTs. Intercomparison with the Moderate Resolution Imaging Spectroradiometer LST products (MYD11A1 and MYD21A1) revealed that the overall root-mean-square difference (RMSD) was approximately 3.0 K. Compared with MYD11A1 LST, GT-LST was overestimated, and relatively large RMSDs were obtained during the daytime, spring, and summer, whereas the significantly smaller positive bias was obtained between GT-LST and MYD21A1 LST. Furthermore, we compared our newly generated dataset with a global AVHRR daytime LST product at the selected measurements of SURFRAD sites (i.e., measurements of these two satellite datasets were valid), which revealed similar accuracies for the two datasets. However, GT-LST can additionally provide nighttime LST, which can be combined with daytime observations estimating relatively accurate monthly mean LST, with an RMSE of 2.7 K. Finally, we compared GT-LST with a regional twice-daily AVHRR LST product over continental Africa in different seasons, with RMSDs ranging from 2.1 to 4.3 K. Considering these advantages, the proposed dataset provides a better data source for a range of research applications. GT-LST is freely available at https://doi.org/10.5281/zenodo.7113080 (1981–2000) (Li et al., 2022a), https://doi.org/10.5281/zenodo.7134158 (2001–2005) (Li et al., 2022b), and https://doi.org/10.5281/zenodo.7813607 (2006–2021) (J. H. Li et al., 2023).

Cloud2FEM: A finite element mesh generator based on point clouds of existing/historical structures

MOCAT-pySSEM: An open-source Python library and user interface for orbital debris and source sink environmental modeling

BackgroundClinical natural language processing (cNLP) systems are of crucial importance due to their increasing capability in extracting clinically important information from free text contained in electronic health records (EHRs). The conversion of a nonstructured representation of a patient’s clinical history into a structured format enables medical doctors to generate clinical knowledge at a level that was not possible before. Finally, the interpretation of the insights gained provided by cNLP systems has a great potential in driving decisions about clinical practice. However, carrying out robust evaluations of those cNLP systems is a complex task that is hindered by a lack of standard guidance on how to systematically approach them.ObjectiveOur objective was to offer natural language processing (NLP) experts a methodology for the evaluation of cNLP systems to assist them in carrying out this task. By following the proposed phases, the robustness and representativeness of the performance metrics of their own cNLP systems can be assured.MethodsThe proposed evaluation methodology comprised five phases: (1) the definition of the target population, (2) the statistical document collection, (3) the design of the annotation guidelines and annotation project, (4) the external annotations, and (5) the cNLP system performance evaluation. We presented the application of all phases to evaluate the performance of a cNLP system called “EHRead Technology” (developed by Savana, an international medical company), applied in a study on patients with asthma. As part of the evaluation methodology, we introduced the Sample Size Calculator for Evaluations (SLiCE), a software tool that calculates the number of documents needed to achieve a statistically useful and resourceful gold standard.ResultsThe application of the proposed evaluation methodology on a real use-case study of patients with asthma revealed the benefit of the different phases for cNLP system evaluations. By using SLiCE to adjust the number of documents needed, a meaningful and resourceful gold standard was created. In the presented use-case, using as little as 519 EHRs, it was possible to evaluate the performance of the cNLP system and obtain performance metrics for the primary variable within the expected CIs.ConclusionsWe showed that our evaluation methodology can offer guidance to NLP experts on how to approach the evaluation of their cNLP systems. By following the five phases, NLP experts can assure the robustness of their evaluation and avoid unnecessary investment of human and financial resources. Besides the theoretical guidance, we offer SLiCE as an easy-to-use, open-source Python library.