Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

remove signpost from sheepdog deps #291

Merged
merged 8 commits into from
Jun 18, 2019
Merged

remove signpost from sheepdog deps #291

merged 8 commits into from
Jun 18, 2019

Conversation

vpsx
Copy link
Contributor

@vpsx vpsx commented Jun 14, 2019 •
edited
Loading

Remove signpost from sheepdog dependencies and code

Breaking Changes

  • looks for INDEX_CLIENT instead of SIGNPOST in app.config and gets INDEX_CLIENT_HOST env var instead of SIGNPOST_HOST--you must therefore use the latest cloud automation or manually recreate your sheepdog secrets
  • removes the USE_SIGNPOST backwards-compatibility workarounds and testing, since GDC forked sheepdog and signpost is dead

Improvements

  • stops importing and using signpost, and stops calling index client signpost where index client is already being used instead of signpost
  • renames various vbbs and fns to not be signpost-specific, eg "signpost_obj" to "file_record"

Dependency updates

  • remove signpost and signpostclient from requirements

Deployment changes

  • look for INDEX_CLIENT instead of SIGNPOST in app.config
  • get INDEX_CLIENT_HOST env var instead of SIGNPOST_HOST

@PlanXCyborg
Copy link

Jenkins Build 4 : time taken 48 min
Check the https://jenkins.planx-pla.net/job/Performance%20tests%20for%20uc-cdis/job/sheepdog/job/PR-291/4/display/redirect

Test results:
Passed: 109, Failed: 0, Skipped: 0

Test Time (PR) Time (master) Diff
@ExportPerformanceTests @performance: Export all nodes of type: aliquot 0.495 0.331 0.16
@ExportPerformanceTests @performance: Export all nodes of type: case 0.381 0.403 -0.02
@ExportPerformanceTests @performance: Export all nodes of type: experiment 0.497 0.371 0.13
@ExportPerformanceTests @performance: Export all nodes of type: program 0.189 0.059 0.13
@ExportPerformanceTests @performance: Export all nodes of type: project 0.119 0.086 0.03
@ExportPerformanceTests @performance: Export all nodes of type: read_group 0.7 0.661 0.04
@ExportPerformanceTests @performance: Export all nodes of type: read_group_qc 0.288 0.099 0.19
@ExportPerformanceTests @performance: Export all nodes of type: sample 0.702 0.451 0.25
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: aliquot 0.386 0.149 0.24
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: case 0.492 0.222 0.27
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: experiment 0.198 0.295 -0.10
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group 0.195 0.064 0.13
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: read_group_qc 0.14 0.061 0.08
@ExportPerformanceTests @performance: Exporting a record by ID on nodes of type: sample 0.295 0.112 0.18
@QueryPerformanceTests @performance: Executing bottomUp query # 0 0.767 0.533 0.23
@QueryPerformanceTests @performance: Executing bottomUp query # 1 0.548 0.505 0.04
@QueryPerformanceTests @performance: Executing bottomUp query # 3 0.431 0.409 0.02
@QueryPerformanceTests @performance: Executing bottomUp query # 4 0.305 0.323 -0.02
@QueryPerformanceTests @performance: Executing bottomUp query # 5 0.225 0.3 -0.08
@QueryPerformanceTests @performance: Executing bottomUp query # 6 0.176 0.212 -0.04
@QueryPerformanceTests @performance: Executing bottomUp query # 7 0.061 0.095 -0.03
@QueryPerformanceTests @performance: Executing topDown query # 0 0.049 0.126 -0.08
@QueryPerformanceTests @performance: Executing topDown query # 1 0.067 0.174 -0.11
@QueryPerformanceTests @performance: Executing topDown query # 2 0.099 0.091 0.01
@QueryPerformanceTests @performance: Executing topDown query # 5 0.376 0.326 0.05
@QueryPerformanceTests @performance: Executing topDown query # 6 0.523 0.42 0.10
@QueryPerformanceTests @performance: Executing topDown query # 7 0.631 0.556 0.07
@SubmissionPerformanceTests @performance: Submission acknowledgement 10 0.401 0.355 0.05
@SubmissionPerformanceTests @performance: Submission acknowledgement 100 1.995 2.829 -0.83
@SubmissionPerformanceTests @performance: Submission acknowledgement 1000 15.279 22.426 -7.15
@SubmissionPerformanceTests @performance: Submission aligned_reads_index 10 1.827 2.222 -0.39
@SubmissionPerformanceTests @performance: Submission aligned_reads_index 100 18.476 21.918 -3.44
@SubmissionPerformanceTests @performance: Submission aliquot 10 0.261 0.326 -0.06
@SubmissionPerformanceTests @performance: Submission aliquot 100 1.861 2.416 -0.55
@SubmissionPerformanceTests @performance: Submission aliquot 1000 18.296 23.388 -5.09
@SubmissionPerformanceTests @performance: Submission case 10 0.26 0.335 -0.08
@SubmissionPerformanceTests @performance: Submission case 100 2.04 2.22 -0.18
@SubmissionPerformanceTests @performance: Submission case 1000 18.401 23.724 -5.32
@SubmissionPerformanceTests @performance: Submission clinical_test 10 0.377 0.344 0.03
@SubmissionPerformanceTests @performance: Submission clinical_test 100 2.3 3.257 -0.96
@SubmissionPerformanceTests @performance: Submission clinical_test 1000 19.155 25.618 -6.46
@SubmissionPerformanceTests @performance: Submission core_metadata_collection 10 0.372 0.745 -0.37
@SubmissionPerformanceTests @performance: Submission core_metadata_collection 100 1.908 2.322 -0.41
@SubmissionPerformanceTests @performance: Submission core_metadata_collection 1000 18.298 25.615 -7.32
@SubmissionPerformanceTests @performance: Submission demographic 10 0.25 0.344 -0.09
@SubmissionPerformanceTests @performance: Submission demographic 100 2.101 3.493 -1.39
@SubmissionPerformanceTests @performance: Submission demographic 1000 17.563 26.838 -9.27
@SubmissionPerformanceTests @performance: Submission diagnosis 10 0.293 0.407 -0.11
@SubmissionPerformanceTests @performance: Submission diagnosis 100 2.094 2.668 -0.57
@SubmissionPerformanceTests @performance: Submission diagnosis 1000 22.406 29.934 -7.53
@SubmissionPerformanceTests @performance: Submission experiment 10 0.397 0.537 -0.14
@SubmissionPerformanceTests @performance: Submission experiment 100 1.808 2.21 -0.40
@SubmissionPerformanceTests @performance: Submission experiment 1000 18.497 24.453 -5.96
@SubmissionPerformanceTests @performance: Submission experimental_metadata 10 1.86 2.469 -0.61
@SubmissionPerformanceTests @performance: Submission experimental_metadata 100 18.297 21.445 -3.15
@SubmissionPerformanceTests @performance: Submission exposure 10 0.277 0.583 -0.31
@SubmissionPerformanceTests @performance: Submission exposure 100 1.798 2.262 -0.46
@SubmissionPerformanceTests @performance: Submission exposure 1000 17.806 27.176 -9.37
@SubmissionPerformanceTests @performance: Submission family_history 10 0.314 0.354 -0.04
@SubmissionPerformanceTests @performance: Submission family_history 100 1.799 2.939 -1.14
@SubmissionPerformanceTests @performance: Submission family_history 1000 16.762 26.332 -9.57
@SubmissionPerformanceTests @performance: Submission keyword 10 0.387 0.315 0.07
@SubmissionPerformanceTests @performance: Submission keyword 100 2.182 2.771 -0.59
@SubmissionPerformanceTests @performance: Submission keyword 1000 15.759 24.932 -9.17
@SubmissionPerformanceTests @performance: Submission publication 10 0.283 0.359 -0.08
@SubmissionPerformanceTests @performance: Submission publication 100 1.793 2.571 -0.78
@SubmissionPerformanceTests @performance: Submission publication 1000 15.016 20.664 -5.65
@SubmissionPerformanceTests @performance: Submission read_group 10 0.327 0.465 -0.14
@SubmissionPerformanceTests @performance: Submission read_group 100 2.101 2.615 -0.51
@SubmissionPerformanceTests @performance: Submission read_group 1000 19.903 28.736 -8.83
@SubmissionPerformanceTests @performance: Submission read_group_qc 10 0.376 0.515 -0.14
@SubmissionPerformanceTests @performance: Submission read_group_qc 100 2.914 4.912 -2.00
@SubmissionPerformanceTests @performance: Submission read_group_qc 1000 21.714 26.836 -5.12
@SubmissionPerformanceTests @performance: Submission sample 10 0.28 0.384 -0.10
@SubmissionPerformanceTests @performance: Submission sample 100 1.997 2.45 -0.45
@SubmissionPerformanceTests @performance: Submission sample 1000 20.61 25.34 -4.73
@SubmissionPerformanceTests @performance: Submission slide 10 0.328 0.42 -0.09
@SubmissionPerformanceTests @performance: Submission slide 100 2.027 2.838 -0.81
@SubmissionPerformanceTests @performance: Submission slide 1000 18.399 28.692 -10.29
@SubmissionPerformanceTests @performance: Submission slide_count 10 0.428 0.548 -0.12
@SubmissionPerformanceTests @performance: Submission slide_count 100 1.906 2.355 -0.45
@SubmissionPerformanceTests @performance: Submission slide_count 1000 17.788 23.82 -6.03
@SubmissionPerformanceTests @performance: Submission slide_image 10 1.843 2.294 -0.45
@SubmissionPerformanceTests @performance: Submission slide_image 100 19.11 20.71 -1.60
@SubmissionPerformanceTests @performance: Submission submitted_aligned_reads 10 1.891 2.459 -0.57
@SubmissionPerformanceTests @performance: Submission submitted_aligned_reads 100 18.411 20.744 -2.33
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 10 1.986 2.26 -0.27
@SubmissionPerformanceTests @performance: Submission submitted_copy_number 100 17.9 22.116 -4.22
@SubmissionPerformanceTests @performance: Submission submitted_methylation 10 2.168 2.422 -0.25
@SubmissionPerformanceTests @performance: Submission submitted_methylation 100 18.077 20.613 -2.54
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 10 1.958 2.448 -0.49
@SubmissionPerformanceTests @performance: Submission submitted_somatic_mutation 100 18.079 22.445 -4.37
@SubmissionPerformanceTests @performance: Submission submitted_unaligned_reads 10 1.975 2.579 -0.60
@SubmissionPerformanceTests @performance: Submission submitted_unaligned_reads 100 18.723 21.117 -2.39
@SubmissionPerformanceTests @performance: Submission treatment 10 0.45 0.318 0.13
@SubmissionPerformanceTests @performance: Submission treatment 100 2.311 23.095 -20.78

Avantol13
Avantol13 approved these changes Jun 18, 2019
View reviewed changes
Copy link
Contributor

@Avantol13 Avantol13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! 🎉

@vpsx vpsx merged commit d13676f into master Jun 18, 2019
@vpsx vpsx deleted the chore/rm-signpost branch June 18, 2019 19:19
This was referenced Jan 27, 2020
Merged
Merged
This was referenced Feb 10, 2020
Merged
Merged
Merged
Merged
Closed
@PlanXCyborg PlanXCyborg mentioned this pull request Mar 17, 2020
Merged
This was referenced Mar 31, 2020
Merged
Closed
Merged
@PlanXCyborg PlanXCyborg mentioned this pull request Oct 21, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Avantol13 Avantol13 Avantol13 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vpsx @PlanXCyborg @Avantol13