An information and statistical analysis pipeline for microbial metagenomic sequencing data

MotivationBacteriophages are viruses infecting bacteria. Being key players in microbial communities, they can regulate the composition/function of microbiome by infecting their bacterial hosts and mediating gene transfer. Recently, metagenomic sequencing, which can sequence all genetic materials from various microbiome, has become a popular means for new phage discovery. However, accurate and comprehensive detection of phages from the metagenomic data remains difficult. High diversity/abundance, and limited reference genomes pose major challenges for recruiting phage fragments from metagenomic data. Existing alignment-based or learning-based models have either low recall or precision on metagenomic data.ResultsIn this work, we adopt the state-of-the-art language model, Transformer, to conduct contextual embedding for phage contigs. By constructing a protein-cluster vocabulary, we can feed both the protein composition and the proteins’ positions from each contig into the Transformer. The Transformer can learn the protein organization and associations using the self-attention mechanism and predicts the label for test contigs. We rigorously tested our developed tool named PhaMer on multiple datasets with increasing difficulty, including quality RefSeq genomes, short contigs, simulated metagenomic data, mock metagenomic data and the public IMG/VR dataset. All the experimental results show that PhaMer outperforms the state-of-the-art tools. In the real metagenomic data experiment, PhaMer improves the F1-score of phage detection by 27%.

Microorganisms are everywhere. Recent studies showed that the mixture of microbes or the microbiome on the human body plays important roles in human physiology and diseases. Metagenomic sequencing is a key technology for studying microbiomes. It produces massive amounts of data in the form of short sequencing reads. A single metagenomic sample can contain 10 7 to 10 8 reads of about 100-nucleotide (nt) length each in a typical shotgun metagenomic sequencing study. They contain rich information about microbiomes and their functions, but reading out those information from the huge highly fragmented data has multiple challenges for mathematical models, bioinformatics methods, and computer algorithms. In this paper, we review the basic bioinformatics tasks and existing methods in processing and analyzing metagenomic data, and discuss remaining open challenges and practical observations. The aim of the paper is to provide readers a whole picture of metagenomic data processing and analysis, and a reference and perspective to start with for computational scientists who are interested in this exciting field.

As more than 90% of microbial community could not be isolated and cultivated, the metagenomic methods have been commonly used to analyze the microbial community as a whole. With the fast acumination of metagenomic samples, it is now intriguing to find simple biomarkers, especially functional biomarkers, which could distinguish different metagenomic samples. Next-generation sequencing techniques have enabled the detection of very accurate gene-presence (abundance) values in metagenomic studies. And the presence/absence or different abundance values for a set of genes could be used as appropriate biomarker for identification of the corresponding microbial community's phenotype. However, it is not yet clear how to select such a set of genes (features), and how accurate would it be for such a set of selected genes on prediction of microbial community's phenotype. In this study, we have evaluated different machine learning methods, including feature selection methods and classification methods, for selection of biomarkers that could distinguish different samples. Then we proposed a machine learning framework, which could discover biomarkers for different microbial communities from the mining of metagenomic data. Given a set of features (genes) and their presence values in multiple samples, we first selected discriminative features as candidate by feature selection, and then selected the feature sets with low error rate and classification accuracies as biomarkers by classification method. We have selected whole genome sequencing data from simulation, public domain and in-house metagenomic data generation facilities. We tested the framework on prediction and evaluation of the biomarkers. Results have shown that the framework could select functional biomarkers with very high accuracy. Therefore, this framework would be a suitable tool to discover functional biomarkers to distinguish different microbial communities.

BackgroundMicrobial communities of the mouse gut have been extensively studied; however, their functional roles and regulation are yet to be elucidated. Metagenomic and metatranscriptomic analyses may allow us a comprehensive profiling of bacterial composition and functions of the complex gut microbiota. The present study aimed to investigate the active functions of the microbial communities in the murine cecum by analyzing both metagenomic and metatranscriptomic data on specific bacterial species within the microbial communities, in addition to the whole microbiome.ResultsBacterial composition of the healthy mouse gut microbiome was profiled using the following three different approaches: 16S rRNA-based profiling based on amplicon and shotgun sequencing data, and genome-based profiling based on shotgun sequencing data. Consistently, Bacteroidetes, Firmicutes, and Deferribacteres emerged as the major phyla. Based on NCBI taxonomy, Muribaculaceae, Lachnospiraceae, and Deferribacteraceae were the predominant families identified in each phylum. The genes for carbohydrate metabolism were upregulated in Muribaculaceae, while genes for cofactors and vitamin metabolism and amino acid metabolism were upregulated in Deferribacteraceae. The genes for translation were commonly enhanced in all three families. Notably, combined analysis of metagenomic and metatranscriptomic sequencing data revealed that the functions of translation and metabolism were largely upregulated in all three families in the mouse gut environment. The ratio of the genes in the metagenome and their expression in the metatranscriptome indicated higher expression of carbohydrate metabolism in Muribaculum, Duncaniella, and Mucispirillum.ConclusionsWe demonstrated a fundamental methodology for linking genomic and transcriptomic datasets to examine functional activities of specific bacterial species in a complicated microbial environment. We investigated the normal flora of the mouse gut using three different approaches and identified Muribaculaceae, Lachnospiraceae, and Deferribacteraceae as the predominant families. The functional distribution of these families was reflected in the entire microbiome. By comparing the metagenomic and metatranscriptomic data, we found that the expression rates differed for different functional categories in the mouse gut environment. Application of these methods to track microbial transcription in individuals over time, or before and after administration of a specific stimulus will significantly facilitate future development of diagnostics and treatments.

Energy supply is characterized by its diversity, including traditional energy, such as fossil fuels, nuclear power, as well as renewable energy, such as solar, hydroelectric, geothermal, biomass, and wind energy. It involves a complex network system composed of energy generation, energy transformation, energy transportation, and energy consumption. The network does provide the great flexibility for energy transformation and transportation; meanwhile, it presents a complex task for conducting agile energy dispatching when extreme events have caused local energy shortages that need to be restored timely. One of the useful methodologies to solve such a problem is data mining and analysis. Their main objective is to take advantage of inherent tolerance of the imprecision and uncertainty to obtain tractability, robustness, and low solution-cost. The applications and developments of data mining and analysis have amazingly evolved in the last two decades. Many of these applications can be found in the field of renewable energy and energy efficiency where data mining and analysis techniques are showing a great potential to solve the problems that arise in this area. In this chapter, data mining and analysis techniques are briefly introduced. Then the implementation procedures are presented to demonstrate the application of curve fitting for renewable energy network design and optimization, which has the capability to handle the restoration during extreme and emergency situations with uncertain parameters.

With the advance of new sequencing technologies producing massive short reads data, metagenomics is rapidly growing, especially in the fields of environmental biology and medical science. The metagenomic data are not only high dimensional with large number of features and limited number of samples but also complex with a large number of zeros and skewed distribution. Efficient computational and statistical tools are needed to deal with these unique characteristics of metagenomic sequencing data. In metagenomic studies, one main objective is to assess whether and how multiple microbial communities differ under various environmental conditions. We propose a two-stage statistical procedure for selecting informative features and identifying differentially abundant features between two or more groups of microbial communities. In the functional analysis of metagenomes, the features may refer to the pathways, subsystems, functional roles and so on. In the first stage of the proposed procedure, the informative features are selected using elastic net as reducing the dimension of metagenomic data. In the second stage, the differentially abundant features are detected using generalized linear models with a negative binomial distribution. Compared with other available methods, the proposed approach demonstrates better performance for most of the comprehensive simulation studies. The new method is also applied to two real metagenomic datasets related to human health. Our findings are consistent with those in previous reports. R code and two example datasets are available at http://cals.arizona.edu/∼anling/software.htm. Supplementary file is available at Bioinformatics online.

Microbial diversity is typically characterized by clustering ribosomal RNA (SSU-rRNA) sequences into operational taxonomic units (OTUs). Targeted sequencing of environmental SSU-rRNA markers via PCR may fail to detect OTUs due to biases in priming and amplification. Analysis of shotgun sequenced environmental DNA, known as metagenomics, avoids amplification bias but generates fragmentary, non-overlapping sequence reads that cannot be clustered by existing OTU-finding methods. To circumvent these limitations, we developed PhylOTU, a computational workflow that identifies OTUs from metagenomic SSU-rRNA sequence data through the use of phylogenetic principles and probabilistic sequence profiles. Using simulated metagenomic data, we quantified the accuracy with which PhylOTU clusters reads into OTUs. Comparisons of PCR and shotgun sequenced SSU-rRNA markers derived from the global open ocean revealed that while PCR libraries identify more OTUs per sequenced residue, metagenomic libraries recover a greater taxonomic diversity of OTUs. In addition, we discover novel species, genera and families in the metagenomic libraries, including OTUs from phyla missed by analysis of PCR sequences. Taken together, these results suggest that PhylOTU enables characterization of part of the biosphere currently hidden from PCR-based surveys of diversity?

Here, we apply an alignment-free method (kPal) that leverages on k-mer frequencies to classify reads within a single long read metagenomic dataset. In addition to a series of simulated metagenomic datasets, we generated sequencing data from a bioreactor microbiome using the PacBio RSII single-molecule real-time sequencing platform. We show that distances obtained after the comparison of k-mer profiles can reveal relationships between reads within a single metagenome, leading to a clustering per species.

BackgroundAnaerobic digestion has been widely applied to treat the waste activated sludge from biological wastewater treatment and produce methane for biofuel, which has been one of the most efficient solutions to both energy crisis and environmental pollution challenges. Anaerobic digestion sludge contains highly complex microbial communities, which play crucial roles in sludge treatment. However, traditional approaches based on 16S rRNA amplification or fluorescent in situ hybridization cannot completely reveal the whole microbial community structure due to the extremely high complexity of the involved communities. In this sense, the next-generation high-throughput sequencing provides a powerful tool for dissecting microbial community structure and methane-producing pathways in anaerobic digestion.ResultsIn this work, the metagenomic sequencing was used to characterize microbial community structure of the anaerobic digestion sludge from a full-scale municipal wastewater treatment plant. Over 3.0 gigabases of metagenomic sequence data were generated with the Illumina HiSeq 2000 platform. Taxonomic analysis by MG-RAST server indicated that overall bacteria were dominant (~93%) whereas a considerable abundance of archaea (~6%) were also detected in the anaerobic digestion sludge. The most abundant bacterial populations were found to be Proteobacteria, Firmicutes, Bacteroidetes, and Actinobacteria. Key microorganisms and related pathways involved in methanogenesis were further revealed. The dominant proliferation of Methanosaeta and Methanosarcina, together with the functional affiliation of enzymes-encoding genes (acetate kinase (AckA), phosphate acetyltransferase (PTA), and acetyl-CoA synthetase (ACSS)), suggested that the acetoclastic methanogenesis is the dominant methanogenesis pathway in the full-scale anaerobic digester.ConclusionsIn short, the metagenomic sequencing study of this work successfully dissected the detail microbial community structure and the dominated methane-producing pathways of a full-scale anaerobic digester. The knowledge garnered would facilitate to develop more efficient full-scale anaerobic digestion systems to achieve high-rate waste sludge treatment and methane production.Electronic supplementary materialThe online version of this article (doi:10.1186/s12934-015-0218-4) contains supplementary material, which is available to authorized users.

Customer Relationship Management (CRM) refers to the methodologies and tools that help businesses manage customer relationships in an organized way. Data Mining is the process that uses a variety of data analysis and modeling techniques to discover patterns and relationships in data that may be used to make accurate predictions. It can help you to select the right prospects on whom to focus. The essence of the information technology revolution and, in particular, the World Wide Web is the opportunity to build better relationships with customers than has been previously possible in the offline world. For instance, as part of their CRM strategy, a business might use a database of customer information to help construct a customer satisfaction survey, or decide which new product their customers might be interested in.

BackgroundNext-Generation-Sequencing (NGS) enables detection of microorganisms present in biological and other matrices of various origin and nature, allowing not only the identification of known phyla and strains but also the discovery of novel ones. The large amount of metagenomic shotgun data produced by NGS require comprehensive and user-friendly pipelines for data analysis, that speed up the bioinformatics steps, relieving the users from the need to manually perform complex and time-consuming tasks.ResultsWe describe here HOME-BIO (sHOtgun MEtagenomic analysis of BIOlogical entities), an exhaustive pipeline for metagenomics data analysis, comprising three independent analytical modules designed for an inclusive analysis of large NGS datasets.ConclusionsHOME-BIO is a powerful and easy-to-use tool that can be run also by users with limited computational expertise. It allows in-depth analyses by removing low-complexity/ problematic reads, integrating the analytical steps that lead to a comprehensive taxonomy profile of each sample by querying different source databases, and it is customizable according to specific users’ needs.

Databases of the marine metagenomics

Cyanobacteria bloom is a serious public health threat and a global challenge. Literature on the bloom prediction and forecasting has been accumulating and the emphasis appears to have been on the relation between the blooms and environmental factors, whilst the complexity of the bloom mechanism makes it difficult to reach adequate output of the models. Rapid development of next generation sequencing techniques provides a way in which comprehensive and quick examination of the microbial community can be achieved, especially for the bloom community structure. This facilitates using of merely the sequence data along with the machine learning techniques to predict and forecast the bloom occurrence. But there has been rare report on this theme in the literature. In this case study, machine learning approaches were applied with the metagenomic data as the only input (rather than with environmental data) to predict the Cyanobacteria blooms. k-NN classification, SVM classification and k-means clustering were applied and their efficiencies were evaluated using relevant indices. Feature selection was performed and the yielded sub datasets were worked on seriatim. In the predicting experiment with k-NN approach, the final year's data among the 8 years OTU time series were used as target data and various combination of the preceding years' data were used as predictor data; the output came with the best values of 1.00 and 100% for the evaluation indices F1 score and sensitivity, specificity, precision, and accuracy, for the 7 preceding years' predictor input, among the experiment results. This case study demonstrated the feasibility of using machine learning approaches in the Cyanobacteria bloom prediction with only metagenomic sequence data, and the importance of feature selection processing in obtaining better output of the machine learning approaches. The metagenomic data based machine learning approaches are efficient, economic, and faster, possessing the advantage and potential for being adopted as a promising means in the bloom prediction practice.

Chapter 4 - Computational and Statistical Considerations in the Analysis of Metagenomic Data

BackgroundNext-generation sequencing (NGS) has greatly facilitated metagenomic analysis but also raised new challenges for metagenomic DNA sequence assembly, owing to its high-throughput nature and extremely short reads generated by sequencers such as Illumina. To date, how to generate a high-quality draft assembly for metagenomic sequencing projects has not been fully addressed.ResultsWe conducted a comprehensive assessment on state-of-the-art de novo assemblers and revealed that the performance of each assembler depends critically on the sequencing depth. To address this problem, we developed a pipeline named InteMAP to integrate three assemblers, ABySS, IDBA-UD and CABOG, which were found to complement each other in assembling metagenomic sequences. Making a decision of which assembling approaches to use according to the sequencing coverage estimation algorithm for each short read, the pipeline presents an automatic platform suitable to assemble real metagenomic NGS data with uneven coverage distribution of sequencing depth. By comparing the performance of InteMAP with current assemblers on both synthetic and real NGS metagenomic data, we demonstrated that InteMAP achieves better performance with a longer total contig length and higher contiguity, and contains more genes than others.ConclusionsWe developed a de novo pipeline, named InteMAP, that integrates existing tools for metagenomics assembly. The pipeline outperforms previous assembly methods on metagenomic assembly by providing a longer total contig length, a higher contiguity and covering more genes. InteMAP, therefore, could potentially be a useful tool for the research community of metagenomics.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-015-0686-x) contains supplementary material, which is available to authorized users.