An energy-aware bioinformatics application for assembling short reads in high performance computing systems

Contigs assembled from the second generation sequencing short reads may contain misassemblies, and thus complicate downstream analysis or even lead to incorrect analysis results. Fortunately, with more and more sequenced species available, it becomes possible to use the reference genome of a closely related species to detect misassemblies. In addition, long reads of the third generation sequencing technology have been more and more widely used, and can also help detect misassemblies. Here, we introduce ReMILO, a reference assisted misassembly detection algorithm that uses both short reads and PacBio SMRT long reads. ReMILO aligns the initial short reads to both the contigs and reference genome, and then constructs a novel data structure called red-black multipositional de Bruijn graph to detect misassemblies. In addition, ReMILO also aligns the contigs to long reads and find their differences from the long reads to detect more misassemblies. In our performance test on short read assemblies of human chromosome 14 data, ReMILO can detect 41.8-77.9% extensive misassemblies and 33.6-54.5% local misassemblies. On hybrid short and long read assemblies of S.pastorianus data, ReMILO can also detect 60.6-70.9% extensive misassemblies and 28.6-54.0% local misassemblies. The ReMILO software can be downloaded for free under Artistic License 2.0 from this site: https://github.com/songc001/remilo. baoe@bjtu.edu.cn. Supplementary data are available at Bioinformatics online.

De novo DNA sequence assembly is very important in genome sequence analysis. In this paper, we investigated two of the major approaches for de novo DNA sequence assembly of very short reads: overlap-layout-consensus (OLC) and Eulerian path. From that investigation, we developed a new assembly technique by combining the OLC and the Eulerian path methods in a hierarchical process. The contigs yielded by these two approaches were treated as reads and were assembled again to yield longer contigs. We tested our approach using three real very-short-read datasets generated by an Illumina Genome Analyzer and four simulated very-short-read datasets that contained sequencing errors. The sequencing errors were modeled based on Illumina's sequencing technology. As a result, our combined approach yielded longer contigs than those of Edena (OLC) and Velvet (Eulerian path) in various coverage depths and was comparable to SOAPdenovo, in terms of N50 size and maximum contig lengths. The assembly results were also validated by comparing contigs that were produced by assemblers with their reference sequence from an NCBI database. The results show that our approach produces more accurate results than Velvet, Edena, and SOAPdenovo alone. This comparison indicates that our approach is a viable way to assemble very short reads from next generation sequencers.

Next generation sequencing (NGS) technologies have enabled cheap, large-scale, and high-throughput production of short DNA sequence reads and thereby have promoted the explosive growth of data volume. Unfortunately, the produced reads are short and prone to contain errors that are incurred during sequencing cycles. Both large data volume and sequencing errors have complicated the mapping of NGS reads onto the reference genome and have motivated the development of various aligners for very short reads, typically less than 100 base pairs (bps) in length. As read length continues to increase, propelled by advances in NGS technologies, these longer reads tend to have higher sequencing error rates and more true mutations (including substitutions, insertions, or deletions) to the genome. Such new characteristics make inefficient the aligners, which are optimized for very short reads and support only ungapped alignments or gapped alignments with very limited number of gaps (typically one gap), and thereby call for new aligners with fully gapped alignment supported. In this chapter, we present the CUSHAW software suite for NGS read alignment, which is open-source and consists of three individual aligners: CUSHAW, CUSHAW2, and CUSHAW3. This suite offers parallel and efficient NGS read alignments to large genomes, such as the human genome, by harnessing multi-core CPUs or compute unified device architecture (CUDA)-enabled graphics processing units (GPUs). Moreover, it has the capability to align both base-space and color-space reads and is consistently shown to be one of the best alignment tools through our performance evaluations.

Motivation and Objectives The output of a high throughput next generation sequencing (NGS) machine is a collection of short reads, which have to be properly assembled in order to reconstruct the original DNA sequence of the analyzed organism (Metzker, 2010; Earl, 2011). The DNA sequence assembly process is based on aligning and merging these reads for effectively reconstructing the real primary structure of the DNA sample sequence or reference genome. The use of NGS machines results in much larger sets of reads to be assembled, posing new problems for computer scientists and bioinformaticians. In particular, a relevant issue is related with the trade-off between precision of the assembly process and its computational time, stating the need for faster methods that can keep pace with the speed and volume of reads that are generated with NGS. An important step in DNA assembly is the identification of a subset of read pairs that have a high probability of being aligned sequentially in the reconstruction. Such step is often referred to as filtering, and amounts in selecting a significantly smaller subset of the initial set of read pairs (whose dimension is quadratic in the number of initial reads) that can be then processed by an alignment algorithm, usually quite time consuming. The desired effect of filtering is then to quickly filter out from the candidate set of read pairs those that would not provide a good alignment in the following phase. The computation cost of filtering should then be balanced by the speed-up obtained when a smaller set of read pairs is considered for alignment. In this work we propose and test the use of alignment free distances to evaluate the similarity between two short reads as a technique for filtering good read pairs to be assembled. The method operates in constant time in the string length and is tested in its ability to emulate, with a proper level of precision, much more time consuming methods to evaluate the similarity between short DNA sequences, such as the established Needleman-Wunsch edit distance (Needleman, Wunsch, and Christian, 1970), often used in the final step of the assembly procedure. These preliminary experiments show the efficacy of this approach for filtering the promising read pairs - eligible candidates to successfully assemble the entire genome of a given organism. Therefore, the alignment free reads filtering may significantly accelerate the assembly process without a substantial loss in accuracy of the DNA sample sequence reconstruction. Methods ODNA sequence assembly The DNA sequence assembly process is based on the alignment and merging of reads (stretch of sequences) in order to reconstruct the original primary structure of the DNA sample sequences. Given a set of sequences S = {s1, s2, …, sn}, where s ∈ S is a fragment of the primary structure of DNA (read) (e.g. s = {ATTCGA...CTGACT}), assembly is in charge of building the longest sequence from the set S where each pair of consequent reads obey certain similarity conditions. DNA read pairs filtering and Alignment Free Distance This step identifies the promising read pairs in order to reduce the amount of input data given to the real assembly algorithm. We adopted a very quick measure of the similarity between two reads, Alignment Free (AF) based distance (Vinga and Almeida, 2003). AF computes the similarity of two strings based only on the dictionary of their substrings, irrespective of their relative position. As a dictionary we considered the set of 4-mer (sequences composed of 4 different nucleotides) and then built a profile for each read composed by the relative frequencies of each 4-mer in the read. The Euclidean distance between the profiles of two reads was taken as an inverse measure of the similarity of the two reads and thus as an indication that the two reads formed a promising pair to be considered in the assembly phase. AF filtering was then used defining a proper threshold on the AF distance and discarding all the pairs that exhibited a AF distance above the threshold. Computational complexity of AF distance is a constant linearly bounded by the number of k-mers adopted and the length of the strings to be compared. Comparing with other distances: Needleman-Wunsch and “Bowtie” distance Along with AF we considered the well-established Needleman-Wunsch edit distance (NW) and compared them in their ability to identify significant pairs. This comparison was based on the computation of a sort of perfect distance computed after an alignment over an already known sequence has been performed. Such distance, referred to as Bowtie distance (BT), was obtained as follows: a. a large number of reads coming from a known sequence were considered; b. hese reads were aligned over the known sequence using the standard Bowtie algorithm (Langmead et al, 2009); c. any two reads received a maximum BT distance if their alignment did not intersect over the reference sequence, else they received a distance inversely proportional to their intersection over the sequence (e.g., they would have BT distance equal to 0 if they were aligned one on top (or inside) of the other by the Bowtie algorithm). By construction we assumed BT distance to be the reference distance, e.g., the distance that expressed the best possible alignments - being based on the knowledge of the reference sequence - and tested the correlation of AF and NW with BT; moreover, we verified the ability of AF and NW to predict that a given read pair had BT distance above or below a given threshold. Results and Discussion For our test we considered the E.Coli genome and a set of reads from this genome reads obtained by Roche 454 sequencing machine. Reads have average length of ~235 nucleotides and standard deviation of approx. 10 (the large majority of them having length in the interval 225-245). Reads were aligned with the reference sequence with Bowtie and then 100,000 were sampled at random according to their alignment along the sequence. Reads were considered both forward and reversed, giving rise to a total of 200,0002 read pairs. All 620,798 read pairs with BT distance < 1 were considered for the experiments; then, out of the remaining pairs, 233,099 were sampled at random. A total of 853,897 read pairs composed the working data set. For all these reads, NW distance and AF distance over the 4-mer were computed. AF, NW and BT distances were all normalized between 0 (maximal similarity) and 1 (maximal dissimilarity). The first interesting results was that the correlation between distances showed that AF approximates BT somehow better than NW: we obtained a correlation coefficient of 0.761 for AF and BT, compared with a smaller 0.706 when NW and BT were considered (coherently, correlation between AF and NW is 0,721). The second interesting results was obtained when we compared the ability of AF and NW to predict whether BT was above or below a given threshold. We defined a threshold predictor for a given function F2 based on function F1 and on a given pair a1, a2 as follows: if (F1 < a1) then predict (F2 < a2), else predict (F2 ≥ a2). To a given pair (a1, a2), we associated the measure of True Positive rate (TP) (percentage of cases where (F1 < a1) and (F2 < a2) and of True Negative rate (TN) (percentage of cases where (F1 ≥ a1) and (F2 ≥ a2); analogously we defined False Positive rate (FP) and False Negative rate (FN). For each (a1, a2) with both values ranging from 0 to 1, we then computed, with step 0.05, the positive and negative error rates taking AF as a predictor of BT and NW as a predictor of BT. Part of the results are summarized in the charts of Figure 1, that show for 3 different levels of a2 (0.1, 0.2, and 0.3) the precision of the predictors (y-axis) when the value of a1 is changed (x-axis), both when AF is used as a predictor of BT (charts in the first row) and when NW is used as a predictor of BT (charts in second row). Similar results are obtained also for other levels of a2, here omitted for brevity. The curves bring to light very clearly how AF is a very good threshold predictor for BT for the considered data; despite its light computational complexity, it appears to perform significantly better than the more complex NW edit distance when its ability to support a threshold predictor is considered. Acknowledgements The authors are partially supported by the FLAGSHIP ”InterOmics” project (PB.P05) funded by the Italian MIUR and CNR institutions, and by the cooperative programme 2010–2012 between the National Research Council of Italy (CNR) and the Polish Academy of Sciences (PAN). References Earl D et al (2011); Assemblathon 1: A competitive assessment of de novo short read assembly methods; Genome Research, 21 Langmead B, Trapnell C, et al (2009); Ultrafast and memory-efficient alignment of short DNA sequences to the human genome; Genome Biology, 10:R25 Metzker ML (2010); Sequencing technologies — the next generation; Nat Rev Genet., 11(1) Needleman SB, Wunsch CD. (1970). A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology 48(3): 443–53 Vinga S, Almeida J (2003); Alignment-free sequence comparison—a review, Bioinformatics 19 (4), 513-523 Note: Figures and tables are available in PDF version only.

BackgroundNext generation sequencing technologies have greatly advanced many research areas of the biomedical sciences through their capability to generate massive amounts of genetic information at unprecedented rates. The advent of next generation sequencing has led to the development of numerous computational tools to analyze and assemble the millions to billions of short sequencing reads produced by these technologies. While these tools filled an important gap, current approaches for storing, processing, and analyzing short read datasets generally have remained simple and lack the complexity needed to efficiently model the produced reads and assemble them correctly.ResultsPreviously, we presented an overlap graph coarsening scheme for modeling read overlap relationships on multiple levels. Most current read assembly and analysis approaches use a single graph or set of clusters to represent the relationships among a read dataset. Instead, we use a series of graphs to represent the reads and their overlap relationships across a spectrum of information granularity. At each information level our algorithm is capable of generating clusters of reads from the reduced graph, forming an integrated graph modeling and clustering approach for read analysis and assembly. Previously we applied our algorithm to simulated and real 454 datasets to assess its ability to efficiently model and cluster next generation sequencing data. In this paper we extend our algorithm to large simulated and real Illumina datasets to demonstrate that our algorithm is practical for both sequencing technologies.ConclusionsOur overlap graph theoretic algorithm is able to model next generation sequencing reads at various levels of granularity through the process of graph coarsening. Additionally, our model allows for efficient representation of the read overlap relationships, is scalable for large datasets, and is practical for both Illumina and 454 sequencing technologies.

Next-generation de novo short reads assemblers typically use the following strategy: (1) assemble unpaired reads using heuristics leading to contigs; (2) order contigs from paired reads information to produce scaffolds. We propose to unify these two steps by introducing localized assembly: direct construction of scaffolds from reads. To this end, the paired string graph structure is introduced, along with a formal framework for building scaffolds as paths of reads. This framework leads to the design of a novel greedy algorithm for memory-efficient, parallel assembly of paired reads. A prototype implementation of the algorithm has been developed and applied to the assembly of simulated and experimental short reads. Our experiments show that our methods yields longer scaffolds than recent assemblers, and is capable of assembling diploid genomes significantly better than other greedy methods.

Antibiotic resistance has escalated as a significant problem of broad public health significance. Regular surveillance of antibiotic resistance genes (ARGs) in microbes and metagenomes from human, animal and environmental sources is vital to understanding ARGs’ epidemiology and foreseeing the emergence of new antibiotic resistance determinants. Whole-genome sequencing (WGS)-based identification of the microbial ARGs using antibiotic resistance databases and in silico prediction tools can significantly expedite the monitoring and characterization of ARGs in various niches. The major hindrance to the annotation of ARGs from WGS data is that most genome databases contain fragmented genes/genomes (due to incomplete assembly). Herein, we describe an insilicoBacterial Antibiotic Resistance scan (BacARscan) (http://proteininformatics.org/mkumar/bacarscan/) that can detect, predict and characterize ARGs in -omics datasets, including short sequencing, reads, and fragmented contigs. Benchmarking on an independent non-redundant dataset revealed that the performance of BacARscan was better than other existing methods, with nearly 92% Precision and 95% F-measure on a combined dataset of ARG and non-ARG proteins. One of the most notable improvements of BacARscan over other ARG annotation methods is its ability to work on genomes and short-reads sequence libraries with equal efficiency and without any requirement for assembly of short reads. Thus, BacARscan can help monitor the prevalence and diversity of ARGs in microbial populations and metagenomic samples from animal, human, and environmental settings. The authors intend to constantly update the current version of BacARscan as and when new ARGs are discovered. Executable versions, source codes, sequences used for development and usage instructions are available at (http://www.proteininformatics.org/mkumar/bacarscan/downloads.html) and GitHub repository (https://github.com/mkubiophysics/BacARscan).

Ticks secrete proteins in their saliva that change over the course of feeding to modulate the host inflammation, immune responses, haemostasis or may cause paralysis. RNA next generation sequencing technologies can reveal the complex dynamics of tick salivary glands as generated from various tick life stages and/or males and females. The current study represents 15,115 Illumina sequenced contigs of the salivary gland transcriptome from male and female Rhipicephalus evertsi evertsi ticks of early, mid and late feeding stages from 1320 separate assemblies using three short read assemblers. The housekeeping functional class contributed to the majority of the composition of the transcriptome (80%) but with lower expression (51%), while the secretory protein functional class represented only 14% of the transcriptome but 46% of the total coverage. Six percent had an unknown status contributing 3% of the overall expression in the salivary glands. Platelet aggregation inhibitors, blood clotting inhibitors and immune-modulators orthologous to the ancestral tick lineages were confirmed in the transcriptome and their differential expression during feeding in both genders observed. This transcriptome contributes data of importance to salivary gland biology and blood feeding physiology of non-model organisms.

Since the advent of Next Generation Sequencing (NGS) technology, the amount of data from whole genome sequencing has been rising fast. In turn, the availability of these resources led to the tapping of whole new research fields in molecular and cellular biology, producing even more data. On the other hand, the available computational power is only increasing linearly. In recent years though, special-purpose high-performance devices started to become prevalent in today’s scientific data centers, namely graphics processing units (GPUs) and, to a lesser extent, field-programmable gate arrays (FPGAs). Driven by the need for performance, developers started porting regular applications to GPU frameworks and FPGA configurations to exploit the special operations only these devices may perform in a timely manner. However, applications using both accelerator technologies are still rare. Major challenges in joint GPU/FPGA application development include the required deep knowledge of associated programming paradigms and the efficient communication both types of devices. In this work, two algorithms from bioinformatics are implemented on a custom hybrid-parallel hardware architecture and a highly concurrent software platform. It is shown that such a solution is not only possible to develop but also its ability to outperform implementations on similar- sized GPU or FPGA clusters in terms of both performance and energy consumption. Both algorithms analyze case/control data from genome- wide association studies to find interactions between two or three genes with different methods. Especially in the latter case, the newly available calculation power and method enables analyses of large data sets for the first time without occupying whole data centers for weeks. The success of the hybrid-parallel architecture proposal led to the development of a high- end array of FPGA/GPU accelerator pairs to provide even better runtimes and more possibilities.

Metagenomics enables direct investigation of the gene content and potential functions of gut bacteria without isolation and culture. However, metagenome-assembled genomes are often incomplete and have low contiguity due to challenges in assembling repeated genomic elements. Long-read sequencing approaches have successfully yielded circular bacterial genomes directly from metagenomes, but these approaches require high DNA input and can have high error rates. Illumina has recently launched the Illumina Complete Long Read (ICLR) assay, a new approach for generating kilobase-scale reads with low DNA input requirements and high accuracy. Here, we evaluate the performance of ICLR sequencing for gut metagenomics for the first time. We sequenced a microbial mock community and 10 human gut microbiome samples with standard, shotgun 2 × 150 paired-end sequencing, ICLR sequencing, and nanopore long-read sequencing and compared performance in read lengths, assembly contiguity, and bin quality. We find that ICLR human metagenomic assemblies have higher N50 (119.5 ± 24.8 kilobases) than short read assemblies (9.9 ± 4.5 kilobases; P = 0.002), and comparable N50 to nanopore assemblies (91.0 ± 43.8 kilobases; P = 0.32). Additionally, we find that ICLR draft microbial genomes are more complete (94.0% ± 20.6%) than nanopore draft genomes (85.9% ± 23.0%; P ≤ 0.001), and that nanopore draft genomes have truncated gene lengths (924.6 ± 114.7 base pairs) relative to ICLR genomes (954.6 ± 71.5 base pairs; P ≤ 0.001). Overall, we find that ICLR sequencing is a promising method for the accurate assembly of microbial genomes from gut metagenomes.IMPORTANCEMetagenomic sequencing allows scientists to directly measure the genome content and structure of microbes residing in complex microbial communities. Traditional short-read metagenomic sequencing methods often yield fragmented genomes, whereas advanced long-read sequencing methods improve genome assembly quality but often suffer from high error rates and are logistically limited due to high input requirements. A new method, the Illumina Complete Long Read (ICLR) assay, is capable of generating highly accurate kilobase-scale sequencing reads with minimal input material. To evaluate the utility of ICLR in metagenomic contexts, we applied short-read, long-read, and ICLR methods to simple and complex microbial communities. We found that ICLR outperforms short-read methods and yields comparable metagenomic assemblies to standard long-read approaches while requiring less input material. Overall, ICLR represents an additional option for assembling complete genomes from complex metagenomes.

Genome Walking (GW) comprises a number of PCR-based methods for the identification of nucleotide sequences flanking known regions. The different methods have been used for several purposes: from de novo sequencing, useful for the identification of unknown regions, to the characterization of insertion sites for viruses and transposons. In the latter cases Genome Walking methods have been recently boosted by coupling to Next Generation Sequencing technologies. This review will focus on the development of several protocols for the application of Next Generation Sequencing (NGS) technologies to GW, which have been developed in the course of analysis of insertional libraries. These analyses find broad application in protocols for functional genomics and gene therapy. Thanks to the application of NGS technologies, the original vision of GW as a procedure for walking along an unknown genome is now changing into the possibility of observing the parallel marching of hundreds of thousands of primers across the borders of inserted DNA molecules in host genomes.

Short-read DNA sequencing instruments can yield over 1012 bases per run, typically composed of reads 150 bases long. Despite this high throughput, de novo assembly algorithms have difficulty reconstructing contiguous genome sequences using short reads due to both repetitive and difficult-to-sequence regions in these genomes. Some of the short read assembly challenges are mitigated by scaffolding assembled sequences using paired-end reads. However, unresolved sequences in these scaffolds appear as “gaps”. Here, we introduce GapPredict – An implementation of a proof of concept that uses a character-level language model to predict unresolved nucleotides in scaffold gaps. We benchmarked GapPredict against the state-of-the-art gap-filling tool Sealer, and observed that the former can fill 65.6% of the sampled gaps that were left unfilled by the latter with high similarity to the reference genome, demonstrating the practical utility of deep learning approaches to the gap-filling problem in genome assembly.

Nontuberculous mycobacteria occasionally harbour clustered tRNA genes, referred to as a tRNA array unit, which is considered a putative antidefense system within their genomes. However, the precise genomic location of these tRNA array units remains unclear. To address this, we sequenced the complete genomes of 5 Mycobacterium avium strains carrying a tRNA array unit using a hybrid assembly of long and short reads followed by manual curation. The assemblies indicated that each strain harbours 3 to 5 extrachromosomal elements. In all genomes, the tRNA array unit was found on a linear contig exceeding 300 kb. Pulse-field gel electrophoresis (PFGE) and sodium dodecyl sulphate-PFGE revealed that the strains harbour linear plasmids corresponding to these large contigs with protein-capped termini. These linear plasmids encode a hybrid type VII/type IV secretion system but lack relaxase genes, which are typically present in mycobacterial circular plasmids. Additionally, they contain approximately 415 bp inverted repeats at the termini. Sequences of related plasmids were identified exclusively in the genomes of M. avium isolates from Japan available in public databases, suggesting a possible Asian origin. This study provides the first experimental evidence that M. avium harbours giant invertron-type linear plasmids carrying a tRNA array unit.

BackgroundAmong next generation sequence technologies, platforms such as Illumina and SOLiD produce short reads but with higher coverage and lower cost per sequenced nucleotide than 454 or Sanger. A challenge now is to develop efficient strategies to use short-read length platforms for de novo assembly and marker development. The scope of this study was to develop a de novo assembly of carrot ESTs from multiple genotypes using the Illumina platform, and to identify polymorphisms.ResultsA de novo assembly of transcriptome sequence from four genetic backgrounds produced 58,751 contigs and singletons. Over 50% of these assembled sequences were annotated allowing detection of transposable elements and new carrot anthocyanin genes. Presence of multiple genetic backgrounds in our assembly allowed the identification of 114 computationally polymorphic SSRs, and 20,058 SNPs at a depth of coverage of 20× or more. Polymorphisms were predominantly between inbred lines except for the cultivated x wild RIL pool which had high intra-sample polymorphism. About 90% and 88% of tested SSR and SNP primers amplified a product, of which 70% and 46%, respectively, were of the expected size. Out of verified SSR and SNP markers 84% and 82% were polymorphic. About 25% of SNPs genotyped were polymorphic in two diverse mapping populations.ConclusionsThis study confirmed the potential of short read platforms for de novo EST assembly and identification of genetic polymorphisms in carrot. In addition we produced the first large-scale transcriptome of carrot, a species lacking genomic resources.

Structural variations (SVs) are differences in genomic regions that are larger than 1 kilobase-pair (Kbp) between individuals, and can arise from errant DNA repair mechanisms, whole genome duplications, and transposable element activity across the genome. Recent advances, optimizations, and cost reductions in next generation sequencing technologies have facilitated the exponential increase in the amount of available short read genomic data. Here we present SaVor, a flexible, reproducible SV calling workflow that accepts single or multi-lane short-read paired-end Illumina sequence data, or BAM files as input to generate a consensus SV call-set based on user-provided merge parameters. We tested SaVor on 1,165 Arabidopsis thaliana whole genome sequences and benchmarked its performance on a set of SVs derived from the same accessions using Lumpy. Intersection calls i.e. SVs supported by 3 SV callers showed the highest precision (>0.91) while union calls supported by at least 1 caller showed the highest recall (>0.88). We found that the former suffers from decreased recall (<0.51) and the latter decreased precision (<0.57). Depending on the merge strategy, trade-offs in recall and precision need to be considered for downstream analyses of SV call-sets from short-read data. SaVor is an open-source Snakemake pipeline and is available on GitHub at https://github.com/ChabbyTMD/SaVor