Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add experiment tracking using mlflow #234

Merged
merged 24 commits into from
Nov 13, 2020
Merged

Conversation

wouterjdb
Copy link
Collaborator

@wouterjdb wouterjdb commented Nov 5, 2020

This PR adds the possibility to run multiple flownet configuration files and do experiment tracking using mlflow.


Contributor checklist

  • 🎉 This PR closes Tracking of flownet runs #232, DeprecationWarning: Specifying whether element are required directly is deprecated #233
  • 📜 I have broken down my PR into the following tasks:
    • Add mlflow dependency
    • Add hyperopt dependency
      • Expose choice between random search and optimization
    • Add hyperparamter running mode in arg parse
    • Parse configuration file with ranges and choices
      • Initial parsing
      • Add flexibility for user
    • Generate mlflow experiment
      • Run flownet
      • Generate metrics and store in mlflow
  • 🤖 I have added tests, or extended existing tests, to cover any new features or bugs fixed in this PR.
  • 📖 I have considered adding a new entry in CHANGELOG.md.
  • 📚 I have considered updating the documentation.

@wouterjdb wouterjdb added the enhancement New feature or request label Nov 5, 2020
@wouterjdb wouterjdb self-assigned this Nov 5, 2020
@wouterjdb
Copy link
Collaborator Author

Tested now with the Egg CI Model. The workflow is working using hyperopt and visualization/tracking with MLFlow:

image

I track however a metric of "1" at the moment. Still a todo. Let's look at this tomorrow @edubarrosTNO.

@wouterjdb
Copy link
Collaborator Author

Added metrics and parameters. Exposed hyperopt options in config. image

@wouterjdb wouterjdb marked this pull request as ready for review November 12, 2020 13:16
Copy link
Collaborator

@olelod olelod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work. LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tracking of flownet runs
2 participants