But while politicians have been dithering over the increasingly unpopular Keystone XL, international tar sands development and other carbon-intensive pipeline projects have been charging ahead.
The exome analysis pipeline is perhaps the most time-intensive pipeline with three long processes read alignment, duplicate removal, and variant calling.
The platform supporting the JRC GMO-Amplicons functions comprises three servers: a server dedicated to the database (PostgreSQL, http://www.postgresql.org/) storing all DNA sequences and related information, a server hosting a Ruby on Rails web application (http://rubyonrails.org/) that acts as a user interface, and a high-performance computer that regularly runs a compute-intensive pipeline.
However, HPC environments have traditionally been used for tightly coupled applications and present several challenges for managing data-intensive pipelines.
Additionally, our work provides a framework and valuable experiences for future developments and deployments of data-intensive pipelines from other scientific domains on HPC systems.
Programmatic interaction with the database by custom clients, downloading bulk data from Reaxys using scripts, executing batch queries and integrating Reaxys into server-based data-intensive pipelining or workflow applications are possible through the Reaxys APIs.
For data-intensive pipelines, web service approaches can be limited by network performance, service availability and I/O compatibility between providers.
Protein domain analysis of unannotated genes was pursued using a computationally intensive bioinformatics pipeline called PURE (Prediction of Unassigned Regions) (15).
We formulate and categorize the data-intensive computing pipeline mapping problems into six classes with two optimization objectives, i.e. minimum end-to-end delay and maximum frame rate, and three network constraints, i.e. no, contiguous, and arbitrary node reuse.
Although GBS methods have some advantages for genome-wide SNP discovery and genotyping, especially for species for which a reference genome has not been established, they also have limitations, which include the requirements for complicated DNA library preparation procedures and intensive bioinformatics pipelines.
