[CANNOLI-98] Adding container builder. #107
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Test PASSed. |
|
Test PASSed. |
heuermh
force-pushed
the
container-builder
branch
from
57b1460 to
5e71cf9
Compare
|
Test PASSed. |
|
It's not every day that you break the scalac compiler: |
|
Test FAILed. Build result: FAILUREGitHub pull request #107 of commit d7520ab automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-03 (centos spark-test) in workspace /home/jenkins/workspace/cannoli-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/cannoli.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/cannoli-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/cannoli.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/cannoli.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/cannoli.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/cannoli.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/cannoli.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/cannoli.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/107/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 75a33fc # timeout=10Checking out Revision 75a33fc (origin/pr/107/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 75a33fcd887fa654a9e9631ca5d91e993f30c3e0First time build. Skipping changelog.Triggering cannoli-prb ? 2.7.3,2.11,2.2.1,centoscannoli-prb ? 2.7.3,2.11,2.2.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Test FAILed. |
heuermh
force-pushed
the
container-builder
branch
from
d7520ab to
31dfefd
Compare
heuermh
changed the title
|
Test PASSed. |
|
Test PASSed. |
|
@fnothaft I'm trying to figure out when paths need to be absolute and when they need to be URIs and when those URIs need to be canonicalized. The current implementation in Bwa.scala seems to be incomplete, in that For files on the local file system, this looks ok to me, except that the mount directory With regards to #34 and #50, how do we get an absolute path for mounting to Docker or Singularity? I don't know if you've noticed, but the Hadoop FS API is terrible. ;) What schemes can we support via the @jpdna How do these Singularity commands look? |
|
After hacking a way to the absolute path for mounting to Docker or Singularity, I run into various issues. Apparently the path to the file on local disk to be accessed after mounting its parent directory to Docker must also be absolute With an absolute file there aren't any errors but the result is an empty file Using Docker with FWIW, things work fine without using Docker |
|
Test PASSed. |
|
Test PASSed. |
|
After a lot of fiddling around, including this diff to adam the absolute paths are looking ok, perhaps there is still a permissions problem? Or perhaps, if running in local mode is never to work, if |
After uninstalling Docker Toolbox on Mac, installing Docker for Mac, and adding |
heuermh
force-pushed
the
container-builder
branch
from
4981699 to
4fb9bb5
Compare
|
Test PASSed. |
|
Test PASSed. |
|
Test PASSed. |
|
Test PASSed. |
|
Still left todo before rebasing:
|
|
Test PASSed. |
|
Test PASSed. |
|
Test PASSed. |
|
Test PASSed. |
|
Test PASSed. |
|
In testing the but when turning on the singularity option it fails with: @heuermh are you able to run |
I don't have access to Singularity, so I've generated the command only from documentation. You may want to check if using an absolute path for Note I see the same thing when using Docker This works fine |
|
@jpdna Please see if the most recent commit helps; though I only made changes for |
|
Test PASSed. |
|
Test PASSed. |
|
Test PASSed. |
|
@jpdna I'd like to merge this today because the build is failing on git head. Can you file new issues if something is off with the Singularity support? |
heuermh
force-pushed
the
container-builder
branch
from
b8c3c0b to
a982855
Compare
|
Test PASSed. |
This was referenced Mar 22, 2018
Closed
sounds good, I'll run through all the tools again in Singularity and create issues if need be. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #98, #114, #67, #34.