A Hitchhiker Guide to Structural Variant Calling: A Comprehensive Benchmark Through Different Sequencing Technologies

Structural variants (SVs) are genomic rearrangements that involve at least 50 nucleotides and are known to have a serious impact on human health. While prior short-read sequencing technologies have often proved inadequate for a comprehensive assessment of structural variation, more recent long reads from Oxford Nanopore Technologies have already been proven invaluable for the discovery of large SVs and hold the potential to facilitate the resolution of the full SV spectrum. With many long-read sequencing studies to follow, it is crucial to assess factors affecting current SV calling pipelines for nanopore sequencing data. In this brief research report, we evaluate and compare the performances of five long-read SV callers across four long-read aligners using both real and synthetic nanopore datasets. In particular, we focus on the effects of read alignment, sequencing coverage, and variant allele depth on the detection and genotyping of SVs of different types and size ranges and provide insights into precision and recall of SV callsets generated by integrating the various long-read aligners and SV callers. The computational pipeline we propose is publicly available at https://github.com/davidebolo1993/EViNCe and can be adjusted to further evaluate future nanopore sequencing datasets.

Recent benchmarks of structural variant (SV) detection tools revealed that the majority of human genome structural variations (SVs), especially the medium-range (50-10,000 bp) SVs cannot be resolved with short-read sequencing, but long-read SV callers achieve great results on the same datasets. While improvements have been made, high-coverage long-read sequencing is associated with higher costs and input DNA requirements. To decrease the cost one can lower the sequence coverage, but the current long-read SV callers perform poorly with coverage below 10×. Synthetic long-read (SLR) technologies hold great potential for structural variant (SV) detection, although utilizing their long-range information for events smaller than 50 kbp has been challenging. In this work, we propose a hybrid novel integrated alignment- and local-assembly-based algorithm, Blackbird, that uses SLR together with low-coverage long reads to improve SV detection and assembly. Without the need for a computationally expensive whole genome assembly, Blackbird uses a sliding window approach and barcode information encoded in SLR to accurately assemble small segments and use long reads for an improved gap closing and contig assembly. We evaluated Blackbird on simulated and real human genome datasets. Using the HG002 GIAB benchmark set, we demonstrated that in hybrid mode, Blackbird demonstrated results comparable to state-of-the-art long-read tools, while using less long-read coverage. Blackbird requires only 5× coverage to achieve F1 scores (0.835 and 0.808 for deletions and insertions) similar to PBSV (0.856 and 0.812) and Sniffles2 (0.839 and 0.804) using 10× Pacbio Hi-Fi long-read coverage.

BackgroundThe human genome contains variants ranging in size from small single nucleotide polymorphisms (SNPs) to large structural variants (SVs). High-quality benchmark small variant calls for the pilot National Institute of Standards and Technology (NIST) Reference Material (NA12878) have been developed by the Genome in a Bottle Consortium, but no similar high-quality benchmark SV calls exist for this genome. Since SV callers output highly discordant results, we developed methods to combine multiple forms of evidence from multiple sequencing technologies to classify candidate SVs into likely true or false positives. Our method (svclassify) calculates annotations from one or more aligned bam files from many high-throughput sequencing technologies, and then builds a one-class model using these annotations to classify candidate SVs as likely true or false positives.ResultsWe first used pedigree analysis to develop a set of high-confidence breakpoint-resolved large deletions. We then used svclassify to cluster and classify these deletions as well as a set of high-confidence deletions from the 1000 Genomes Project and a set of breakpoint-resolved complex insertions from Spiral Genetics. We find that likely SVs cluster separately from likely non-SVs based on our annotations, and that the SVs cluster into different types of deletions. We then developed a supervised one-class classification method that uses a training set of random non-SV regions to determine whether candidate SVs have abnormal annotations different from most of the genome. To test this classification method, we use our pedigree-based breakpoint-resolved SVs, SVs validated by the 1000 Genomes Project, and assembly-based breakpoint-resolved insertions, along with semi-automated visualization using svviz.ConclusionsWe find that candidate SVs with high scores from multiple technologies have high concordance with PCR validation and an orthogonal consensus method MetaSV (99.7 % concordant), and candidate SVs with low scores are questionable. We distribute a set of 2676 high-confidence deletions and 68 high-confidence insertions with high svclassify scores from these call sets for benchmarking SV callers. We expect these methods to be particularly useful for establishing high-confidence SV calls for benchmark samples that have been characterized by multiple technologies.Electronic supplementary materialThe online version of this article (doi:10.1186/s12864-016-2366-2) contains supplementary material, which is available to authorized users.

Structural variants (SVs) are one of the significant types of DNA mutations and are typically defined as larger-than-50-bp genomic alterations that include insertions, deletions, duplications, inversions, and translocations. These modifications can profoundly impact the phenotypic characteristics and contribute to disorders like cancer, response to treatment, and infections. Four long-read aligners and five SV callers have been evaluated using three Oxford Nanopore NGS human genome datasets in terms of precision, recall, and F1-score statistical metrics, depth of coverage, and speed of analysis. The best SV caller regarding recall, precision, and F1-score when matched with different aligners at different coverage levels tend to vary depending on the dataset and the specific SV types being analyzed. However, based on our findings, Sniffles and CuteSV tend to perform well across different aligners and coverage levels, followed by SVIM, PBSV, and SVDSS in the last place. The CuteSV caller has the highest average F1-score (82.51%) and recall (78.50%), and Sniffles has the highest average precision value (94.33%). Minimap2 as an aligner and Sniffles as an SV caller act as a strong base for the pipeline of SV calling because of their high speed and reasonable accomplishment. PBSV has a lower average F1-score, precision, and recall and may generate more false positives and overlook some actual SVs. Our results are valuable in the comprehensive evaluation of popular SV callers and aligners as they provide insight into the performance of several long-read aligners and SV callers and serve as a reference for researchers in selecting the most suitable tools for SV detection.

It has recently been shown that structural variants (SV) can have a higher impact on gene expression variation compared to single nucleotide variants (SNV) in different plant species. Additionally, SV were associated with phenotypic variation in several crops. However, compared to the established SV detection based on short-read sequencing, less approaches were described for linked-read based SV calling. We therefore evaluated the performance of six linked-read SV callers compared to an established short-read SV caller based on simulated linked-reads in tetraploid potato. The objectives of our study were to i) compare the performance of SV callers based on linked-read sequencing to short-read sequencing, ii) examine the influence of SV type, SV length, haplotype incidence (HI), as well as sequencing coverage on the SV calling performance in the tetraploid potato genome, and iii) evaluate the accuracy of detecting insertions by linked-read compared to short-read sequencing. We observed high break point resolutions (BPR) detecting short SV and slightly lower BPR for large SV. Our observations highlighted the importance of short-read signals provided by Manta and LinkedSV to detect short SV. Manta and NAIBR performed well for detecting larger deletions, inversions, and duplications. Detected large SV were weakly influenced by the HI. Furthermore, we illustrated that large insertions can be assembled by Novel-X. Our results suggest the usage of the short-read and linked-read SV callers Manta, NAIBR, LinkedSV, and Novel-X based on at least 90x linked-read sequencing coverage to ensure the detection of a broad range of SV in the tetraploid potato genome.

The complexities of cancer genomes are becoming more easily interpreted due to advancements in sequencing technologies and improved bioinformatic analysis. Structural variants (SVs) represent an important subset of somatic events in tumors. While the detection of SVs has been markedly improved by the development of long-read sequencing, somatic variant identification and annotation remain challenging. We hypothesized that the use of a completed human reference genome (CHM13-T2T) would improve somatic SV calling. Our findings in a tumor-normal matched benchmark sample and three patient samples show that the CHM13-T2T improves SV detection accuracy compared to GRCh38 with a notable reduction in false-positive calls, and thus supports improved prioritization. We also overcame the lack of annotation resources for CHM13-T2T by lifting over CHM13-T2T-aligned reads to the GRCh38 genome, therefore combining both improved alignment and advanced annotations. In this process, we assessed the current SV benchmark set for COLO829/COLO829BL across four replicates sequenced at different centers with different long-read technologies. We discovered instability of this cell line across these replicates; 346 SVs (1.13%) were only discoverable in a single replicate. We identify 54 somatic SVs, which appear to be stable as they are consistently present across the four replicates. As such, we propose this consensus set as an updated benchmark for somatic SV calling and include both GRCh38 and CHM13-T2T coordinates in our benchmark. Our work demonstrates new approaches to optimize somatic SV detection in cancer with potential improvements in other genetic diseases.

Simple SummaryThe wide variety of next-generation sequencing technologies requires thorough evaluation and understanding of their advantages and shortcomings of these different approaches prior to their implementation in a precision medicine setting. Here, we compared the performance of two DNA sequencing methods, whole-exome and linked-read exome sequencing, to detect large structural variants (SVs) and short variants in eight multiple myeloma (MM) patient cases. For three patient cases, matched tumor-normal samples were sequenced with both methods to compare somatic SVs and short variants. The methods’ clinical relevance was also evaluated, and their sensitivity and specificity to detect MM-specific cytogenetic alterations and other short variants were measured. Thus, this study systematically demonstrates and evaluates the performance of whole-exome and linked-read exome sequencing technologies for detecting genetic alterations to aid in selecting the optimal method for clinical application.Linked-read sequencing was developed to aid the detection of large structural variants (SVs) from short-read sequencing efforts. We performed a systematic evaluation to determine if linked-read exome sequencing provides more comprehensive and clinically relevant information than whole-exome sequencing (WES) when applied to the same set of multiple myeloma patient samples. We report that linked-read sequencing detected a higher number of SVs (n = 18,455) than WES (n = 4065). However, linked-read predictions were dominated by inversions (92.4%), leading to poor detection of other types of SVs. In contrast, WES detected 56.3% deletions, 32.6% insertions, 6.7% translocations, 3.3% duplications and 1.2% inversions. Surprisingly, the quantitative performance assessment suggested a higher performance for WES (AUC = 0.791) compared to linked-read sequencing (AUC = 0.766) for detecting clinically validated cytogenetic alterations. We also found that linked-read sequencing detected more short variants (n = 704) compared to WES (n = 109). WES detected somatic mutations in all MM-related genes while linked-read sequencing failed to detect certain mutations. The comparison of somatic mutations detected using linked-read, WES and RNA-seq revealed that WES and RNA-seq detected more mutations than linked-read sequencing. These data indicate that WES outperforms and is more efficient than linked-read sequencing for detecting clinically relevant SVs and MM-specific short variants.

Circular consensus sequencing reads are promising for the comprehensive detection of structural variants (SVs). However, alignment-based SV calling pipelines are computationally intensive due to the generation of complete read-alignments and its post-processing. Herein, we propose a SKeleton-based analysis toolkit for Structural Variation detection (SKSV). Benchmarks on real and simulated datasets demonstrate that SKSV has an order of magnitude of faster speed than state-of-the-art SV calling approaches; moreover, it achieves higher F1 scores for various types of SVs. SKSV is available from https://github.com/ydLiu-HIT/SKSV. Supplementary data are available at Bioinformatics online.

Structural variants (SVs) are increasingly recognized as key contributors to human diseases. However, our understanding of SVs in health and disease is limited, mainly due to their structural complexity and variable length in individuals, as well as limitations inherent to the available genomic technologies and reference genome used. To systematically evaluate SVs across human whole-genome samples using hg38/GRCh38 and gapless T2T-CHM13 references, we introduced an innovative multiplatform approach, LongReadChecker (LoReC), which advances SV comparison and annotation based on distance variance, intersection, gene overlap, and the closest SV in the clinical database. Comparison of the performance in detecting SVs from public and our own whole-genome datasets from short-read sequencing (SRS), available long-read sequencing (LRS) platforms, and optical genome mapping (OGM) revealed that most SVs detected by SRS were confirmed by LRS, but LRS can identify twice as many SVs (25,000 SVs/genome) with greater read mapping accuracy. Our LongReadChecker (LoReC) analysis further highlights the utility of the T2T-CHM13 reference in SV detection, as 20% more deletions and 20% less insertions were detected compared with hg38/GRCh38, which was particularly evident in long-read datasets. Since 80% of the SVs detected by LRS/SRS are smaller than 0.5 kbp, OGM did not detect them. Our study revealed that introducing distance variance, intersection, gene overlap, and the closest SV in the clinical database may help compare and annotate SVs in diagnostics. Our data showed that LRS, together with T2T-CHM13 gapless sequences, can improve the diagnostics of patients with human diseases when SRS fails to identify the cause.

BackgroundLung cancer is a heterogeneous disease and the primary cause of cancer-related mortality worldwide. Somatic mutations, including large structural variants, are important biomarkers in lung cancer for selecting targeted therapy. Genomic studies in lung cancer have been conducted using short-read sequencing. Emerging long-read sequencing technologies are a promising alternative to study somatic structural variants, however there is no current consensus on how to process data and call somatic events. In this study, we preformed whole genome sequencing of lung cancer and matched non-tumour samples using long and short read sequencing to comprehensively benchmark three sequence aligners and seven structural variant callers comprised of generic callers (SVIM, Sniffles2, DELLY in generic mode and cuteSV) and somatic callers (Severus, SAVANA, nanomonsv and DELLY in somatic modes).ResultsDifferent combinations of aligners and variant callers influenced somatic structural variant detection. The choice of caller had a significant influence on somatic structural variant detection in terms of variant type, size, sensitivity, and accuracy. The performance of each variant caller was assessed by comparing to somatic structural variants identified by short-read sequencing. When compared to somatic structural variants detected with short-read sequencing, more events were detected with long-read sequencing. The mean recall of somatic variant events identified by long-read sequencing was higher for the somatic callers (72%) than generic callers (53%). Among the somatic callers when using the minimap2 aligner, SAVANA and Severus achieved the highest recall at 79.5% and 79.25% respectively, followed by nanomonsv with a recall of 72.5%.ConclusionLong-read sequencing can identify somatic structural variants in clincal samples. The longer reads have the potential to improve our understanding of cancer development and inform personalized cancer treatment.

The last two decades have seen tremendous advancements in tools like CRISPR, single-molecule sequencing technologies, optical genome mapping, single-cell omics techniques, etc. Genomic analyses of multiple individuals become important to address medical questions and advances in genomic technologies and methods directly contribute to improved outcomes in disease research and healthcare. Genome-wide association studies conducted in the past inform us about the limitations of technologies in analyzing factors contributing to human traits and diseases using only short-read amplicon sequencing technologies, especially in repetitive genomic stretches where the likelihood of structural variations is higher. While short-read sequencing is a highly evolved technology, the discovery of causative variants with it alone is impeded by its read length and depending on the assay, is limited to gene coding regions, and less penetrative variants are often missed. In a recent report, it was found that only about one in three known genetic disorders in children have been diagnosed successfully with short-read sequencing (SRS, including whole exome and whole genome assays). On the other hand, building evidence that variants occurring in extragenic regions are found to be driving mutations for diseases shows that there is a need to capture both intergenic and extragenic information. Long-read platforms including optical genome mapping and Pacific Biosciences (PacBio) or Oxford Nanopore Technology (ONT) sequencers have advantages like improved contiguity, long-range haplotype phasing, and therefore, identification of structural variants associated with diseases, especially where alleles extend beyond typical read length. They are better suited to capture large structural variations, their genotypes, and copy number alterations but some practical challenges, like lack of base-level resolution for optical mapping, high error rates, cost per sample, low throughput, and resource-heavy informatics analysis for long-read sequencing, prevent routine adoption long-read platforms. Targeted interrogation of the genome in lieu of the whole genome alleviates some of the above challenges. CRISPR-Cas9-based targeting methods offer programmability and thus are being increasingly used for long-read platforms. This thesis discusses novel applications of CRISPR-Cas9 nickase-based experimental methods for targeted interrogation of genomic features on long-read sequencing and optical mapping platforms. First, the linked-pair sequencing strategy enabled by CRISPR-Cas9n chemistry is described. Here, orderly fragmentation of DNA molecules is performed using CRISPR-Cas9n and pairs of sgRNA in such a way that produced fragments share identical termini. This strategy is capable of greater multiplicity and can assemble critical genetic loci effectively. The validity and efficacy of the method for sequencing were first demonstrated on a lambda phage model and Haemophilus influenzae model, and later for sequencing a cancer panel in the human genome containing 100 full-length gene sequences. When the designed linker sequences contained heterozygous genetic variants, long haplotypes could be established. These whole-gene haplotypes enable the study of genetic variants at non-coding regulatory elements and the detection of any allele-specific effects. The use of Cas9n to introduce nicks allowed for preferential ligation of sequencing adapters resulting in significant enrichment of target regions. Second, multicolor whole genome labeling for optical mapping technique, enabled by CRISPR-Cas9n, is described. Here, Cas9n is used to make directed nicks at target sites to incorporate fluorophores. This technique was used in the detection and quantification of known biomarkers (telomeres and D4Z4 units) and in finding the locations of LINE-1 insertions across the genome With Cas9-mediated nick-labeling, it is possible to target and fluorescently label any 20mer or the combination of multiple 20mers across the whole genome, especially in repetitive regions lacking DLE motifs. Custom maps can be generated to enable precise detection of breakpoints and interrogate the repetitive sequences; this enables more in-depth analysis of SVs than was previously possible. Application of this method in the interrogation of exogenic DNA integrations in the host genome for gene therapy validation and safety assessment is also demonstrated. Finally, additional applications of Cas9n chemistry using both long-read sequencing and optical genome mapping platforms together are demonstrated. Multicolor Cas9n-mediated labeling chemistry is leveraged to create high-density optical maps with an increased resolution for use in structural variant analysis, calling breakpoints, and scaffolding genome assemblies. Such high-density optical maps can also be used to study interactions of Cas9 with target DNA.

We sequenced the genome of the Yoruban reference individual NA19240 on the long-read sequencing platform Oxford Nanopore PromethION for evaluation and benchmarking of recently published aligners and germline structural variant calling tools, as well as a comparison with the performance of structural variant calling from short-read sequencing data. The structural variant caller Sniffles after NGMLR or minimap2 alignment provides the most accurate results, but additional confidence or sensitivity can be obtained by a combination of multiple variant callers. Sensitive and fast results can be obtained by minimap2 for alignment and a combination of Sniffles and SVIM for variant identification. We describe a scalable workflow for identification, annotation, and characterization of tens of thousands of structural variants from long-read genome sequencing of an individual or population. By discussing the results of this well-characterized reference individual, we provide an approximation of what can be expected in future long-read sequencing studies aiming for structural variant identification.

Objectives and Rationale: Structural variants (SV) including large copy number aberrations (CNV), translocations, inversions, and large insertions and deletions (INDEL) play a critical role in tumorigenesis and progression. In fact, we now know that tumors can be categorized according to the size of mutations harbored. In several cancers, including ovarian and breast cancer, the large structural mutations, rather than single site mutations, play a dominate role in tumor etiology. Therefore, it's critical to implement reliable algorithm for the detection of structural variants in DNA sequencing data. Mate-pair sequencing is a protocol specifically implemented for detection of the whole-genome level structural variants. It requires less sequencing depth therefor is cost effective, and enables the detection of CNVs, translocations, and inversions simultaneously. However, so far there has been no reliable bioinformatics pipeline for the analyses of the mate-pair sequencing data. Methods: Our novel algorithm, the SnowShoes-SV, is an exhaustive search algorithm designed specifically for mate-pair DNA sequencing data analyses. It calls the SVs based on disconcordant read pairs. The false SVs are filtered according to the following criteria: (i) the number of the supporting read pairs; (ii) the lack of reads from control data that implicate SV at the same region; (iii) the mappability and uniqueness of the region based on data from the ENCODE project; (iv) consistencies of the mapping orientations of the supporting read pairs; (v) the similar sizes between sequencing library and the two end read clusters. Results: Using a set of samples previously genotyped by aCGH, the SnowShoes-SV successfully detected all known CNVs and other SVs. It also identified copy number neutral translocations and inversions previously not identified by aCGH. In addition, the algorithm nominated novel SVs which are to be validated by PCR. Conclusions: SnowShoes-SV is a highly sensitive and specific algorithm for SV detection from the mate-pair DNA sequencing data. Citation Format: Yan W Asmann, Chen Wang, Brian M Necela, Xianfeng Chen, Jean-Pierre A Kocher, Matthew J Maurer, Thomas M Habermann, Susan L Slager, Andrew L Feldman, Anne J Novak, James R Cerhan, Edith A Perez, E Aubrey Thompson. An exhaustive algorithm for detecting copy number aberrations and large structural variants in whole-genome mate-pair sequencing data [abstract]. In: Proceedings of the Thirty-Seventh Annual CTRC-AACR San Antonio Breast Cancer Symposium: 2014 Dec 9-13; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2015;75(9 Suppl):Abstract nr P6-07-03.

Genomic variation shared by members of the same species that are longer than a single nucleotide are commonly called structural variants (SVs). Though relatively rare, they represent an increasingly important class of variation as SVs have been associated with diseases and susceptibility to some types of cancer. Common approaches to SV detection require the sequencing and mapping of fragments from a test genome to a high-quality reference genome. Candidate SVs correspond to fragments with discordant mapped configurations, but because errors in the sequencing and mapping will also create discordant arrangements, many of these predictions will be false. When sequencing coverage is low, distinguishing true SVs from errors is even more complicated. In recent work, we have developed SV detection methods that simultaneously consider the genomes of closely related individuals – parents and children. Our approaches control false positive SVs by requiring children inherit all SVs in their genome from a parent. However, in doing so, our models may have missed true novel variants acquired by the child. In this work, we generalize our previous approaches to allow the child to carry novel variants but enforce sparsity through an l 1 penalty (since novel SVs in the child should be rare). We present results on both simulated genomes as well as two-sequenced parent-child trios from the 1000 Genomes Project.

BackgroundSince 2009, numerous tools have been developed to detect structural variants using short read technologies. Insertions >50 bp are one of the hardest type to discover and are drastically underrepresented in gold standard variant callsets. The advent of long read technologies has completely changed the situation. In 2019, two independent cross technologies studies have published the most complete variant callsets with sequence resolved insertions in human individuals. Among the reported insertions, only 17 to 28% could be discovered with short-read based tools.ResultsIn this work, we performed an in-depth analysis of these unprecedented insertion callsets in order to investigate the causes of such failures. We have first established a precise classification of insertion variants according to four layers of characterization: the nature and size of the inserted sequence, the genomic context of the insertion site and the breakpoint junction complexity. Because these levels are intertwined, we then used simulations to characterize the impact of each complexity factor on the recall of several structural variant callers. We showed that most reported insertions exhibited characteristics that may interfere with their discovery: 63% were tandem repeat expansions, 38% contained homology larger than 10 bp within their breakpoint junctions and 70% were located in simple repeats. Consequently, the recall of short-read based variant callers was significantly lower for such insertions (6% for tandem repeats vs 56% for mobile element insertions). Simulations showed that the most impacting factor was the insertion type rather than the genomic context, with various difficulties being handled differently among the tested structural variant callers, and they highlighted the lack of sequence resolution for most insertion calls.ConclusionsOur results explain the low recall by pointing out several difficulty factors among the observed insertion features and provide avenues for improving SV caller algorithms and their combinations.