Skip to content

Topic/unify job tracking #5418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 60 commits into
base: master
Choose a base branch
from
Open

Topic/unify job tracking #5418

wants to merge 60 commits into from

Conversation

ryan-preble
Copy link
Contributor

Description

Creates CXGN::Job and sgn_people.sp_job. The module and table track background job submissions and display their info on the user profile page. It is a wrapper for CXGN::Tools::Run, but can also be used alongside it for recording purposes. Tool compatibility and solGS analyses are now recorded with CXGN::Job, with downloads, uploads, BLAST, reports, and other background job submissions to be recorded in the future.

to get this to work, you must also checkout topic/unify_job_tracking on cxgn-corelibs and run the dbpatch 00192/AddJobsTable.pm.

#5346

Checklist

  • Refactoring only
  • Documentation only
  • Fixture update only
  • Bug fix
    • The relevant issue has been closed.
    • Further work is required.
  • New feature
    • Relevant tests have been created and run.
    • Data was added to the fixture
      • Data was added via a patch in /t/data/fixture/patches/.
    • User-Facing Change
      • The user manual in /docs has been updated.
    • Any new Perl has been documented using perldoc.
    • Any new JavaScript has been documented using JSDoc.
    • Any new legacy JavaScript has been moved from /js to /js/source/legacy.

…unctions to AJAX/Job.pm. Make slurm config changes to sgn.conf
@isaak
Copy link
Member

isaak commented Apr 14, 2025

checked out the branches on both sgn and cxgn-corelibs, run the db patch.

When I tried to submit pca anaysis job, I am getting:

"Error occured submitting the job. Please contact the developers. Hint: run_analysis failed. Please try re-running the analysis and wait for it to finish. Error storing job in database!DBIx::Class::Schema::source(): Can't find source for SpJob at /home/production/cxgn/sgn/bin/../lib/CXGN/Job.pm line 546 "

Also, on my profile page, I am getting:
Error fetching submitted jobs

@ryan-preble
Copy link
Contributor Author

checked out the branches on both sgn and cxgn-corelibs, run the db patch.

When I tried to submit pca anaysis job, I am getting:

"Error occured submitting the job. Please contact the developers. Hint: run_analysis failed. Please try re-running the analysis and wait for it to finish. Error storing job in database!DBIx::Class::Schema::source(): Can't find source for SpJob at /home/production/cxgn/sgn/bin/../lib/CXGN/Job.pm line 546 "

Also, on my profile page, I am getting: Error fetching submitted jobs

Did you start the docker after switching to the cxgn-corelibs branch? The docker will not include changes to that repo if it was already running when you made the switch. If that still does not do it, then there is a chance I will need to redo the dbicdump

@lukasmueller
Copy link
Member

Cosmetic suggestion: The 'Dismiss failed and timed out jobs" and "Dismiss jobs older than" buttons appear stuck together on my browser. There should be slight space around them. Better yet, for a more compact presentation, the buttons could be in one line, including the pull down menu ("one week")

@lukasmueller
Copy link
Member

nothing appeared in the table when I ran different analyses. What is a good analysis to check this?

@ryan-preble
Copy link
Contributor Author

nothing appeared in the table when I ran different analyses. What is a good analysis to check this?

I tested using kinship and clustering - those ones appeared to work fine

@ryan-preble
Copy link
Contributor Author

Cosmetic suggestion: The 'Dismiss failed and timed out jobs" and "Dismiss jobs older than" buttons appear stuck together on my browser. There should be slight space around them. Better yet, for a more compact presentation, the buttons could be in one line, including the pull down menu ("one week")

Could you attach a screenshot? For me, the dropdown menu appears in-line with the button

Copy link
Member

@isaak isaak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-- Works great with PCA and Cluster analyses.

  1. solGS and kinship submissions are breaking:
    Error occured submitting the job. Please contact the developers. Hint: run_analysis failed. Please try re-running the analysis and wait for it to finish. Attribute (job_type) does not pass the type constraint because: Validation failed for '__ANON__' with value "". Value must be equal to "download", "genomic_prediction", "genotypic_analysis", "phenotypic_analysis", "report", "sequence_analysis", "tool_compatibility", or "upload". at /home/production/cxgn/local-lib/lib/perl5/x86_64-linux-gnu-thread-multi/Moose/Object.pm line 24 Moose::Object::new('CXGN::Job', 'HASH(0x587e464f5b80)') called at /home/production/cxgn/sgn/bin/../lib/SGN/Controller/solGS/AsyncJob.pm line 807 SGN::Controller::solGS::AsyncJob::submit_job_cluster('SGN::Controller::solGS::AsyncJob=HASH(0x587e3ff43110)', 'SGN=HASH(0x587e4653f370)', 'HASH(0x587e45bfb950)') called at /home/production/cxgn/sgn/bin/../lib/SGN/Controller/solGS/AsyncJob.pm line 525 SGN::Controller::solGS::AsyncJob::run_async('SGN::Controller::solGS::AsyncJob=HASH(0x587e3ff43110)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/sgn/bin/../lib/SGN/Controller/solGS/Kinship.pm line 242 SGN::Controller::solGS::Kinship::run_kinship('SGN::Controller::solGS::Kinship=HASH(0x587e3ff1d808)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/sgn/bin/../lib/SGN/Controller/solGS/AnalysisQueue.pm line 1144 SGN::Controller::solGS::AnalysisQueue::run_kinship_analysis('SGN::Controller::solGS::AnalysisQueue=HASH(0x587e3ff9bf88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/sgn/bin/../lib/SGN/Controller/solGS/AnalysisQueue.pm line 1009 eval {...} at /home/production/cxgn/sgn/bin/../lib/SGN/Controller/solGS/AnalysisQueue.pm line 996 SGN::Controller::solGS::AnalysisQueue::run_analysis('SGN::Controller::solGS::AnalysisQueue=HASH(0x587e3ff9bf88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/sgn/bin/../lib/SGN/Controller/solGS/AnalysisQueue.pm line 70 SGN::Controller::solGS::AnalysisQueue::run_saved_analysis('SGN::Controller::solGS::AnalysisQueue=HASH(0x587e3ff9bf88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Action.pm line 358 Catalyst::Action::execute('Catalyst::Action=HASH(0x587e45759638)', 'SGN::Controller::solGS::AnalysisQueue=HASH(0x587e3ff9bf88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst.pm line 2060 eval {...} at /home/production/cxgn/local-lib/lib/perl5/Catalyst.pm line 2060 Catalyst::execute('SGN=HASH(0x587e4653f370)', 'SGN::Controller::solGS::AnalysisQueue', 'Catalyst::Action=HASH(0x587e45759638)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Action.pm line 353 Catalyst::Action::dispatch('Catalyst::Action=HASH(0x587e45759638)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Controller.pm line 170 Catalyst::Controller::_ACTION('SGN::Controller::solGS::AnalysisQueue=HASH(0x587e3ff9bf88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Action.pm line 358 Catalyst::Action::execute('Catalyst::Action=HASH(0x587e45758e88)', 'SGN::Controller::solGS::AnalysisQueue=HASH(0x587e3ff9bf88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst.pm line 2060 eval {...} at /home/production/cxgn/local-lib/lib/perl5/Catalyst.pm line 2060 Catalyst::execute('SGN=HASH(0x587e4653f370)', 'SGN::Controller::solGS::AnalysisQueue', 'Catalyst::Action=HASH(0x587e45758e88)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Action.pm line 353 Catalyst::Action::dispatch('Catalyst::Action=HASH(0x587e45758e88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Dispatcher.pm line 261 Catalyst::Dispatcher::_do_forward('Catalyst::Dispatcher=HASH(0x587e1fec7ba0)', 'forward', 'SGN=HASH(0x587e4653f370)', '_ACTION') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Dispatcher.pm line 241 Catalyst::Dispatcher::forward('Catalyst::Dispatcher=HASH(0x587e1fec7ba0)', 'SGN=HASH(0x587e4653f370)', '_ACTION') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst.pm line 482 Catalyst::forward('SGN=HASH(0x587e4653f370)', '_ACTION') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Controller.pm line 134 Catalyst::Controller::_DISPATCH('SGN::Controller::solGS::AnalysisQueue=HASH(0x587e3ff9bf88)', 'SGN=HASH(0x587e4653f370)') called at /home/production/cxgn/local-lib/lib/perl5/Catalyst/Action.pm line 358

  2. In the submitted jobs table, the 'type' column shows the data type (e.g. 'genotypic_analysis'). Instead, it should be the type of analysis (e.g. 'cluster_analysis').

  3. For readability, the precision in start and end times could be enough up to the minutes.

@ryan-preble
Copy link
Contributor Author

-- Works great with PCA and Cluster analyses.

  1. solGS and kinship submissions are breaking:

  2. In the submitted jobs table, the 'type' column shows the data type (e.g. 'genotypic_analysis'). Instead, it should be the type of analysis (e.g. 'cluster_analysis').

  3. For readability, the precision in start and end times could be enough up to the minutes.

For point 1, I put in a catch to make sure a valid type is put in no matter what. For point 2, the broader terms of phenotypic_analysis and genotypic_analysis were specifically requested by Lukas instead of things like cluster_analysis or kinship_analysis. And I will implement point 3 shortly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants