-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Addressing PR comments #528
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## preview #528 +/- ##
==========================================
Coverage ? 90.70%
==========================================
Files ? 66
Lines ? 4636
Branches ? 0
==========================================
Hits ? 4205
Misses ? 431
Partials ? 0
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
I'd really like if we could avoid changing the templates if the directory structure changes. If we can't do that, all that's left are a couple nitpicks on docstrings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Super nice work and incredible amount of code, I can definitely tell a lot of hard work went into this! Here is what I came up with:
Documentation:
I have some suggestions below of where to include information on preview.
- Make sure the new Experiment.preview is added to doc/api/smartsim_api.rst
- Add preview to this chart here
- Add preview as an Example step here
Preview templates:
- I would add the path for Ensemble, Model and Orchestrator to the previews! It is possible that a user specifies their own path and therefore it is not located in the exp_path. The path arguments for Orch and Ensemble have been pushed to develop.
- I notice that in an Experiment with multiple databases, the db_identifier for the standalone dbs are given, even if one is not provided. A colo db identifier is only shown if provided even though the identifier would be default: orchestrator -> should we show the colo db_identifier name always like for std?
Possible Bug 1:
I am running the two Model script below:
model = exp.create_model("hello_world", settings, path="./")
model_2 = exp.create_model("hello", settings, path="./")
model_2.colocate_db_tcp()
exp.preview(model, model_2)
exp.start(model, model_2)
The preview output is:
== Models ==
= Model Name: hello_world =
Colocated:
Connection Type: TCP
TCP/IP Port(s):
6379
Client Configuration:
Database Backend: redis
Connection Type: TCP
TCP/IP Port(s):
6379
Type: Colocated
= Model Name: hello =
Colocated:
Connection Type: TCP
TCP/IP Port(s):
6379
Client Configuration:
Database Backend: redis
Connection Type: TCP
TCP/IP Port(s):
6379
Type: Colocated
The preview suggests that both of the Models are colocated rather than the 1? Am I misunderstanding?
Document with more suggestions find here
… replicas and perm strat to ensemble, fiixed error with missing torch scripts; spacing error
…nges in tests, verbosity checking, pylint suppressing in experiment.py, deleting unncessary .dat file
…ssing PR comments