Title: A Reference-Free Algorithm for Computational Normalization of Shotgun Sequencing Data
C. Titus Brown, Adina Howe, Qingpeng Zhang, Alexis B. Pyrkosz, and Timothy H. Brom
Deep shotgun sequencing and analysis of genomes, transcriptomes, amplified single-cell genomes, and metagenomes has enabled investigation of a wide range of organisms and ecosystems. However, sampling variation in short-read data sets and high sequencing error rates of modern sequencers present many new computational challenges in data interpretation. These challenges have led to the development of new classes of mapping tools and de novo assemblers. These algorithms are challenged by the continued improvement in sequencing throughput. We here describe digital normalization, a single-pass computational algorithm that systematizes coverage in shotgun sequencing data sets, thereby decreasing sampling variation, discarding redundant data, and removing the majority of errors. Digital normalization substantially reduces the size of shotgun data sets and decreases the memory and time requirements for de novo sequence assembly, all without significantly impacting content of the generated contigs. We apply digital normalization to the assembly of microbial genomic data, amplified single-cell genomic data, and transcriptomic data. Our implementation is freely available for use and modification.Online resources and data:
- A tutorial for running khmer on microbial genomes and eukaryotic transcriptomes.
- Git repository for khmer: github.com/ged-lab/khmer/tree/2012-paper-diginorm
- Git repository for paper & data analysis pipeline: github.com/ged-lab/2012-paper-diginorm
- Instructions on running the paper analysis pipeline & reproducing the paper
- HTML view of the ipython notebook containing code and scripts to reproduce the figures in the paper. (See the pipeline notes for a runnable version.)
- Data required to run the pipeline (.tar.gz, 7.9gb)
- Assembled microbial genomes and eukaryotic transcriptomes (.tar.gz, 110 mb)