DREAM-Stellar: Parallel and space efficient exact local alignment

This chapter guides through the development of a pairwise sequence alignment tool that implements global, ends-free global, and local alignment. It focuses on the algorithms needed to implement the tool. However, because Python is a commonly used programming language for bioinformatics, the chapter encourages you to apply the Python concepts to implement your pairwise sequence alignment tool. The chapter begins by analysing the output report of an existing local sequence alignment tool, EMBOSS Water, to familiarize ourselves with its inputs, outputs, and functionality. It then looks at an overview of simple pairwise alignment (SPA), and introduces the concept of algorithms–taking a look at different ways to express algorithms. Towards the end, the chapter explains the longest common subsequence (LCS) algorithm and how it can be extended to implement local and global pairwise alignment. It then assesses the complexity of algorithms–that is, how much memory and time they require.

Non-coding RNAs (ncRNAs) show a unique evolutionary process in which the substitutions of distant bases are correlated in order to conserve the secondary structure of the ncRNA molecule. Therefore, the multiple alignment method for the detection of ncRNAs should take into account both the primary sequence and the secondary structure. Recently, there has been intense focus on multiple alignment investigations for the detection of ncRNAs; however, most of the proposed methods are designed for global multiple alignments. For this reason, these methods are not appropriate to identify locally conserved ncRNAs among genomic sequences. A more efficient local multiple alignment method for the detection of ncRNAs is required. We propose a new local multiple alignment method for the detection of ncRNAs. This method uses a local multiple alignment construction procedure inspired by ProDA, which is a local multiple aligner program for protein sequences with repeated and shuffled elements. To align sequences based on secondary structure information, we propose a new alignment model which incorporates secondary structure features. We define the conditional probability of an alignment via a conditional random field and use a gamma-centroid estimator to align sequences. The locally aligned subsequences are clustered into blocks of approximately globally alignable subsequences between pairwise alignments. Finally, these blocks are multiply aligned via MXSCARNA. In benchmark experiments, we demonstrate the high ability of the implemented software, SCARNA_LM, for local multiple alignment for the detection of ncRNAs. The C++ source code for SCARNA_LM and its experimental datasets are available at http://www.ncrna.org/software/scarna_lm/download. Supplementary data are available at Bioinformatics online.

With the development of new sequencing techniques, whole genomes of many species have become available. This huge amount of data gives rise to new opportunities and challenges. These new sequences provide valuable information on relationships among species, e.g. genome recombination and conservation. One of the principal ways to investigate such information is multiple sequence alignment (MSA). Currently, there is large amount of MSA data on the internet, such as the UCSC genome database, but how to effectively use this information to solve classical and new problems is still an area lacking of exploration. In this thesis, we explored how to use this information in four problems, i.e. sequence orthology search problem, multiple alignment improvement problem, short read mapping problem, and genome rearrangement inference problem. For the first problem, we developed a EM algorithm to iteratively align a query with a multiple alignment database with the information from a phylogeny relating the query species and the species in the multiple alignment. We also infer the query's location in the phylogeny. We showed that by doing alignment and phylogeny inference together, we can improve the accuracies for both problems. For the second problem, we developed an optimization algorithm to iteratively refine the multiple alignment quality. Experiment results showed our algorithm is very stable in term of resulting alignments. The results showed that our method is more accurate than existing methods, i.e. Mafft, Clustal-O, and Mavid, on test data from three sets of species from the UCSC genome database. For the third problem, we developed a model, PhyMap, to align a read to a multiple alignment allowing mismatches and indels. PhyMap computes local alignments of a query sequence against a fixed multiple-genome alignment of closely related species. PhyMap uses a known phylogenetic tree on the species in the multiple alignment to improve the quality of its computed alignments while also estimating the placement of the query on this tree. Both theoretical computation and experiment results show that our model can differentiate between orthologous and paralogous alignments better than other popular short read mapping tools (BWA, BOWTIE and BLAST). For the fourth problem, we gave a simple genome recombination model which can express insertions, deletions, inversions, translocations and inverted translocations on aligned genome segments. We also developed an MCMC algorithm to infer the order of the query segments. We proved that using any Euclidian metrics to measure distance between two sequence orders in the tree optimization goal function will lead to a degenerated solution where the inferred order will be the order of one of the leaf nodes. We also gave a graph-based formulation of the problem which can represent the probability distribution of the order of the query sequences.

MotivationMost methods for pairwise and multiple genome alignment use fast local homology search tools to identify anchor points, i.e. high-scoring local alignments of the input sequences. Sequence segments between those anchor points are then aligned with slower, more sensitive methods. Finding suitable anchor points is therefore crucial for genome sequence comparison; speed and sensitivity of genome alignment depend on the underlying anchoring methods.ResultsIn this article, we use filtered spaced word matches to generate anchor points for genome alignment. For a given binary pattern representing match and don’t-care positions, we first search for spaced-word matches, i.e. ungapped local pairwise alignments with matching nucleotides at the match positions of the pattern and possible mismatches at the don’t-care positions. Those spaced-word matches that have similarity scores above some threshold value are then extended using a standard X-drop algorithm; the resulting local alignments are used as anchor points. To evaluate this approach, we used the popular multiple-genome-alignment pipeline Mugsy and replaced the exact word matches that Mugsy uses as anchor points with our spaced-word-based anchor points. For closely related genome sequences, the two anchoring procedures lead to multiple alignments of similar quality. For distantly related genomes, however, alignments calculated with our filtered-spaced-word matches are superior to alignments produced with the original Mugsy program where exact word matches are used to find anchor points.Availability and implementationhttp://spacedanchor.gobics.deSupplementary informationSupplementary data are available at Bioinformatics online.

The Smith-Waterman (SW) algorithm is a typical technique for local sequence alignment in computational biology. However, the SW algorithm does not consider the local behaviours of the amino acids, which may result in loss of some useful information. Inspired by the success of Markov Edit Distance (MED) method, this paper therefore proposes a novel Markov pairwise protein sequence alignment (MPPSA) method that takes the local context dependencies into consideration. The numerical results have shown its superiority to the SW for pairwise protein sequence comparison.

Motivations. A central problem in biological network analysis is the Local Network Alignment. The aim is to detect conserved subnetworks or complexes of proteins, across two or more species, which are involved in processes or functions. This allows to predict either new interactions or the functions of unknown proteins. Since the problem of finding conserved subnetworks in a set of networks is related to subgraph isomorphism, which is known to be NP-Hard, several heuristics have been proposed. These include PathBlast [1] and MaWISh [2] for pairwise local alignment and NetworkBlast-M [3] and Graemlin [4] for the multiple case. Although NetworkBlast-M has been proved to be the most efficient and accurate method, it is able to find significant conserved complexes composed by no more than 15 proteins. Here we introduce GASOLINE (Greedy And Stochastic algorithm for Optimal Local alignment of Interaction Networks) an algorithm based on Gibbs Sampling [5] in connection to a seed-extend approach to search for significant conserved complexes of any size. Methods. The algorithm consists of two main phases. In the first phase, we look for ortholog proteins across the networks. We call these proteins seeds of the suboptimal pattern. In the second phase, called iterative phase, we extend each seed, by adding one adjacent node. Here, through a stochastic process based on Gibbs Sampling, we choose a node among a set of randomly picked adjacent ones. The chosen nodes will be those that maximize similarity among the N extended seeds. We repeat the iterative phase until we obtain a set of N conserved subgraphs each of size W. These N subgraphs represent our final alignment. The topological density and the conservation of a complex are measured through an Index of Density and Structural Conservation (hereafter IDSC). The IDSC score ranges from 0 to 1 and it is dynamically computed during the iterative phase. The iteration will terminate when IDSC is above a fixed threshold. This allows the removal of parameter W producing a set of highly dense and conserved complexes of different sizes. Results. GASOLINE has been tested on 10 microbial PPI networks, taken from Graemlin [4] and on 6 eukaryotic PPI networks, taken from STRING database [6]. The number of proteins in microbial networks ranges from 1,000 to 7,000 with the amount of interactions ranging from 13,000 to 230,000, whereas the size of eukaryotic networks ranges from 6,000 to 12,500 proteins and from 26,000 to 166,000 edges. 2000 of execution of GASOLINE have been performed. As output we consider the best distinct (not overlapping) complexes, with respect to size andIDSC score. In our experiments we selected complexes of at least 5 proteins with IDSC score > 0.7. All tests have been performed on an Intel Core i5-2500 3.30Ghz CPU with 4 GB RAM. The complexes computed by the algorithm have been then validated by annotating their proteins with GO categories. For the microbial networks, annotations have been downloaded from DAVID [7,8], while eukaryotic networks proteins have been annotated using BioDBNet [9]. Significant conserved categories have been obtained by computing a p-value (< 0.0001), based on hypergeometric distribution. A GO category has been considered conserved when it resulted significant in at least N-1 species, where N is the number of aligned networks. The executions of GASOLINE on microbial networks revealed the existence of a big conserved complex of 40 proteins, forming the large and small subunits of ribosome, with IDSC equals to 0.755 (see Tab. 1). As for the 6 eukaryotic networks, 15 conserved complexes have been found by GASOLINE with IDSC greater than 0.75. They are listed in Tab. 2, with their IDSC and the number of GO categories enriched.

In this paper, we investigate the feasibility of computing DNA patterns on FPGAs by utilising high-performance and parallel computing. In particular, the report analyses and contrasts the FSM -based core implementation of the well-known Smith-Waterman pair-wise sequence alignment algorithm with the dynamic and heuristic alternatives that are currently on the market. In this research project, we investigate the use of parallel and high-performance computing to determine how DNA patterns can be computed on FPGAs. In particular, the report details the findings of a study that compared the FSM -based core implementation of the well-known Smith-Waterman pair-wise sequence alignment algorithm with the algorithm's current dynamic and heuristic programming methods. The study was carried out by comparing these three aspects of the algorithm. The traditional methods of sequence alignment that are software-implied are not adequate to meet the actual data rate requirements. A strategy that is based on the use of hardware makes it possible to process multiple new databases in parallel, which enables a high degree of scalability. A finite state machine serves as the primary data processor in our demonstration of how to organize the sequences of amino acids in a protein (FSM). We also investigate the effectiveness of bit-based sequence alignment algorithms and present the inner stage pipelined FPGA architecture for implementing sequence alignment. Both of these are presented in this paper. Synchronized controllers are utilized in the programming techniques of the algorithm for the purpose of parallel sequence alignment. Inference is the foundation of traditional software-based methods for sequence alignment, but these methods are unable to keep up with the required data rates. A strategy that is based on hardware allows for the processing of parallel tasks with a large number of new databases and provides excellent scalability. Our explanation of the classification of protein sequences is based on a Finite State Machine, which serves as the primary processing unit (FSM). In addition to this, we investigate how well bit-based sequence alignment algorithms work and present the inner stage pipelined Field Programmable Gate Array architecture that can be used to implement sequence alignment. Synchronized controllers are utilized in the process of performing sequence alignment in parallel. The architecture as a whole was made to be scalable and fast, with a small number of computations and the ability to perform parallel processing in hardware using field-programmable gate arrays to compare bits in patterns. Finally, we use FPGA hardware synthesis to show off impressive performance and low resource consumption.

SummaryIn this paper, we present GSWABE, a graphics processing unit (GPU)‐accelerated pairwise sequence alignment algorithm for a collection of short DNA sequences. This algorithm supports all‐to‐all pairwise global, semi‐global and local alignment, and retrieves optimal alignments on Compute Unified Device Architecture (CUDA)‐enabled GPUs. All of the three alignment types are based on dynamic programming and share almost the same computational pattern. Thus, we have investigated a general tile‐based approach to facilitating fast alignment by deeply exploring the powerful compute capability of CUDA‐enabled GPUs. The performance of GSWABE has been evaluated on a Kepler‐based Tesla K40 GPU using a variety of short DNA sequence datasets. The results show that our algorithm can yield a performance of up to 59.1 billions cell updates per second (GCUPS), 58.5 GCUPS and 50.3 GCUPS for global, semi‐global and local alignment, respectively. Furthermore, on the same system GSWABE runs up to 156.0 times faster than the Streaming SIMD Extensions (SSE)‐based SSW library and up to 102.4 times faster than the CUDA‐based MSA‐CUDA (the first stage) in terms of local alignment. Compared with the CUDA‐based gpu‐pairAlign, GSWABE demonstrates stable and consistent speedups with a maximum speedup of 11.2, 10.7, and 10.6 for global, semi‐global, and local alignment, respectively. Copyright © 2014 John Wiley & Sons, Ltd.

The content of the 13th Mutation Detection meeting (Leiden, April 2015) is summarized in this report. Topics discussed at the meeting included current challenges of clinical NGS, advances in bioinformatics, data quality control, single cell analysis and RNA sequencing, among others. Social, ethical and regulatory challenges of genomic data handling and data sharing were the focus of an expert panel debate. The 14th International Symposium on Variants in the Genome will take place in Santiago de Compostela, June 5-8, 2017. http://isv.variome.org.

The importance of genomic predictors for clinical outcome of hematological malignancies

Recent advances in High-Throughput Sequencing (HTS) technology have greatly facilitated the researches in bioinformatics field. With the ultra-high sequencing speed and improved base-calling accuracy, Illumina Genome Analyzer is currently the most widely used platform in the field. To use the raw reads generated from the sequencing machine, the 3’ adapter sequence attached to the real read in the process of ligation needs to be correctly trimmed. This is often done by some inhouse scripts or different packages with various parameters. They either use the Smith-Waterman algorithm or search for an exact match of the 3’ adapter sequence. In this report, I investigated methodologies as well as the strengths and weaknesses of five representative mainstream adapter trimming tools in order to suggest a direction for other researchers. Furthermore, four sets of detailed analysis were performed to evaluate the performances of these tools. I demonstrated that my adapter trimming method is flexible, accurate and efficient for Next Generation Sequencing (NGS) analysis. Type of Report: MS Project Report Department of Computer Science & Engineering Washington University in St. Louis Campus Box 1045 St. Louis, MO 63130 ph: (314) 935-6160 Master’s Project Report LCScanner: An Efficient and Accurate Trimming Tool for Illumina Next Generation Sequencing Reads by Xiang Zhou Master of Engineering in Computer Science Washington University in St. Louis, 2012 Research Advisor: Professor Weixiong Zhang

The emergence of bioinformatics has led to many new discoveries in living organisms. These discoveries would not have been possible without the developments made in the sequence alignment techniques. Many sequence alignment algorithms were developed to make the alignment process fast and accurate. However, the more precise algorithms take longer than their less precise counterparts. Researchers came with innovative approaches to combat the time consuming constraint. Their aim was to speed up the computational process by using more efficient implementations of the algorithms using state-of-the-art hardware platforms. Smith Waterman (SW) algorithm, being the most accurate in the alignment process, has been implemented on various high performance computing platforms for the same purpose. However, the intrinsic structure of the algorithm has got little attention. In this paper, we present a novel structure of the SW algorithm that takes less number of cycles at the cost of utilizing a minimal amount of extra hardware resources as compared to its existing form. The newly proposed architecture achieves up to 25% performance gain.

The POLYFIT rigid-body algorithm for automated global pairwise and multiple protein structural alignment is presented. Smith-Waterman local alignment is used to establish a set of seed equivalences that are extended using Needleman-Wunsch dynamic programming techniques. Structural and functional interaction constraints provided by evolution are encoded as one-dimensional residue physical environment strings for alignment of highly structurally overlapped protein pairs. Local structure alignment of more distantly related pairs is carried out using rigid-body conformational matching of 15-residue fragments, with allowance made for less stringent conformational matching of metal-ion and small molecule ligand-contact, disulphide bridge, and cis-peptide correspondences. Protein structural plasticity is accommodated through the stepped adjustment of a single empirical distance parameter value in the calculation of the Smith-Waterman dynamic programming matrix. Structural overlap is used both as a measure of similarity and to assess alignment quality. Pairwise alignment accuracy has been benchmarked against that of 10 widely used aligners on the Sippl and Wiederstein set of difficult pairwise structure alignment problems, and more extensively against that of Matt, SALIGN, and MUSTANG in pairwise and multiple structural alignments of protein domains with low shared sequence identity in the SCOP-ASTRAL 40% compendium. The results demonstrate the advantages of POLYFIT over other aligners in the efficient and robust identification of matching seed residue positions in distantly related protein targets and in the generation of longer structurally overlapped alignment lengths. Superposition-based application areas include comparative modeling and protein and ligand design. POLYFIT is available on the Web server at http://polyfit.insa-toulouse.fr.

The Smith-Waterman algorithm is used in Bio-informatics to perform pairwise local alignment between a query sequence and a subject sequence. We present a GPU based parallel version of this algorithm that is able to perform pair-wise alignment faster than previous algorithms. In particular, it parallelizes each alignment, rather than relying on parallelism across multiple pair alignments, which many other proposed GPU algorithms do. As a result it scales better. We further extend our algorithm to work efficiently on a cluster of GPUs. At a high level, our approach subdivides the iterative computation of elements of a matrix among blocks of processors such that each block can simply recompute the data it needs instead of waiting for other processors to compute them. Sometimes this may lead to excessive recomputation, however. We evaluate these cases and employ a hybrid approach, recomputing only limited data and communicating the rest. Our algorithm is also extended to produce not only the best but all ‘best K’ alignments. Our results on SSCA#1 benchmark show that our method is upto 5-24 times faster than previous method.

BackgroundPrograms based on hash tables and Burrows-Wheeler are very fast for mapping short reads to genomes but have low accuracy in the presence of mismatches and gaps. Such reads can be aligned accurately with the Smith-Waterman algorithm but it can take hours and days to map millions of reads even for bacteria genomes.ResultsWe introduce a GPU program called MaxSSmap with the aim of achieving comparable accuracy to Smith-Waterman but with faster runtimes. Similar to most programs MaxSSmap identifies a local region of the genome followed by exact alignment. Instead of using hash tables or Burrows-Wheeler in the first part, MaxSSmap calculates maximum scoring subsequence score between the read and disjoint fragments of the genome in parallel on a GPU and selects the highest scoring fragment for exact alignment. We evaluate MaxSSmap’s accuracy and runtime when mapping simulated Illumina E.coli and human chromosome one reads of different lengths and 10% to 30% mismatches with gaps to the E.coli genome and human chromosome one. We also demonstrate applications on real data by mapping ancient horse DNA reads to modern genomes and unmapped paired reads from NA12878 in 1000 genomes.ConclusionsWe show that MaxSSmap attains comparable high accuracy and low error to fast Smith-Waterman programs yet has much lower runtimes. We show that MaxSSmap can map reads rejected by BWA and NextGenMap with high accuracy and low error much faster than if Smith-Waterman were used. On short read lengths of 36 and 51 both MaxSSmap and Smith-Waterman have lower accuracy compared to at higher lengths. On real data MaxSSmap produces many alignments with high score and mapping quality that are not given by NextGenMap and BWA. The MaxSSmap source code in CUDA and OpenCL is freely available from http://www.cs.njit.edu/usman/MaxSSmap.Electronic supplementary materialThe online version of this article (doi:10.1186/1471-2164-15-969) contains supplementary material, which is available to authorized users.