Documentation

Converting a command to an iobio url

We tried to keep iobio urls as close to their command line counter-parts as possible. This was done to ensure that all commands could be expressed as iobio urls and to make it as easy as possible to convert a command to an iobio url.

A single webservice

In its simplest a command can be converted to an iobio url in 3 steps. (1) Replacing the program name with the webservice url (e.g. freebayes with http://freebayes.iobio.io). (2) Adding the options and arguments to the 'cmd' url parameter (e.g. http://freebayes.iobio.io?cmd=insert-options-and-arguments). (3) Url encoding that string. Here's an example.

// command line
// tabix -h http://s3.amazonaws.com/1000genomes/release/20101123/interim_phase1_release/ALL.chr2.phase1.projectConsensus.genotypes.vcf.gz 2:4000000-4050000
// this can be broken into two parts
var program = 'tabix';
var command = '-h http://s3.amazonaws.com/1000genomes/release/20101123/interim_phase1_release/ALL.chr2.phase1.projectConsensus.genotypes.vcf.gz 2:4000000-4050000';
// now create the iobio url by replacing the program with the iobio webservice
var url = "http://tabix.iobio.io?cmd=" + command;
// can't have spaces in url so urlEncode and we're done
url = encodeURI(url);
// > http://tabix.iobio.io/?cmd=-h%20http://s3.amazonaws.com/1000genomes/release/20101123/interim_phase1_release/ALL.chr2.phase1.projectConsensus.genotypes.vcf.gz%202:4000000-4050000

Optional url parameters

iobio supports several optional url parameters in addition to the required 'cmd' url parameter.

• protocol = websocket | http - this determines how data is transfered between iobio webservices for urls with multiple webservices
• binary = false | true. At the moment we are using socket.io under the covers to manage the websocket connections. One draw back of socket.io is it's lack of support of binary data. Because of this we have to explicitly tell iobio when we are passing binary data.
• event = string - this allows a custom event to be created when emitting a url request to an iobio webservice via websockets. The default event is results. This allows for the responses of multiple requests to the same webservice to be handled be different callbacks
• format = string - many webservices allow for it's response format to be determined. For example tabix.iobio.io supports json. Example *parseByLine = false | true - by default data is streamed in non-constant sized chunks. However, sometimes it is desirable for the data to be sent line by line. This is useful when using some response formats such as json.

Multiple webservices

Building on this single webservice approach, multiple iobio webservices can be strung together to create a pipeline with the results from each webservice becoming the streaming input of the next webservice. This is done by replacing a file in an iobio url with another iobio url that produces the same data type as the file.

For example, say we would like to merge a region of two bam files and then filter them for proper pairs. First lets create the iobio url for merging two bam files. We are going to use the bammerger webservice which efficiently merges bams.

var bamsUrl = "http://bammerger.iobio.io/?binary=true&cmd=11:10108473-10188473%20http://s3.amazonaws.com/1000genomes/data/NA06984/alignment/NA06984.chrom11.ILLUMINA.bwa.CEU.low_coverage.20111114.bam%20http://s3.amazonaws.com/1000genomes/data/NA06985/alignment/NA06985.chrom11.ILLUMINA.bwa.CEU.low_coverage.20111114.bam";

To filter for proper pairs on the command line would look something like this

bamtools filter -isProperPair true -in my.bam

So we want to convert the above to an iobio url using another iobio url to produce the bam data instead of my.bam. In keeping with using the command line as a guide, this is done by replacing my.bam with the bamsUrl, but with one extra step. To keep the url parameters associated with the correct iobio url (either the bamMerger url or the bamtools filter url), we need to further encode the internal iobio url with encodeURIComponent.

var btools = "http://bamtools.iobio.io";
var filterUrl = btools + "?binary=true&cmd=filter -isProperPair true -in " + encodeURIComponent(bamsUrl);
filterUrl = encodeURI(filterUrl);

And here's the filtered url

And if you want to add more, you just keep on going. For example if we wanted to make sure that we only have proper pairs left in our bam stream we could check it with bamtools stats like so.

 encodeURI("http://bamtools.iobio.io?cmd=stats -in " + encodeURIComponent(filterUrl)");

stats url