Train overhaul #373
Merged
Train overhaul #373
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
use hidden attributes (with an _) in the attrs classes so we can have abstract properties and allow architectures to take them as input redefine dims on architectures
We want to make runs more easily usable. Making changes is easier when only one class needs to be modified. Also makes it easier to learn and use the framework.
this allows the modelzoo save to describe the output channels
…s a `torch.nn.Sequential`
…ific layers and freeze the rest
Make parent weights directory if it doesn't exist. Save dummy file if tracing fails to prevent excessive trace attempts
Add support for loading scripted models via paths Add support for freezing layers
unzip first, then open. I think this should be done automatically by bioimageio.core
This should go in `dacapo_toolbox`
- consolidate `Trainer` and `TrainerConfig` classes - move snapshots/num_workers/batch_size etc. to `RunConfig` - make interface a simple `torch.utils.data.IterableDataset`. - update `train.py` and `validate.py` to use overhauled `Trainer` classes
dummy run wasn't doing anything reasonable
This allows you to run tests with `-n auto` flag to start up workers and significantly speed up testing
|
@pattonw can you please check this bug dacapo/experiments/__init__.py:2: in <module>
from .run import Run # noqa
dacapo/experiments/run.py:1: in <module>
from .run_config import RunConfig
dacapo/experiments/run_config.py:7: in <module>
from funlib.persistence import prepare_ds, open_ds
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/funlib/persistence/__init__.py:1: in <module>
from .arrays import Array, open_ds, prepare_ds, open_ome_ds, prepare_ome_ds # noqa
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/funlib/persistence/arrays/__init__.py:3: in <module>
from .ome_datasets import prepare_ome_ds, open_ome_ds # noqa
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/funlib/persistence/arrays/ome_datasets.py:5: in <module>
from iohub.ngff import AxisMeta, TransformationMeta, open_ome_zarr
E ImportError: cannot import name 'AxisMeta' from 'iohub.ngff' (/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/iohub/ngff/__init__.py) |
|
Oh I see what happened. I fixed them locally and thought the fix would get pushed, but this is a bug in |
|
tests are successfull. @pattonw do i merge ? |
|
Black is just failing due to not being able to modify the pull request since it's coming from my fork |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overhaul
Trainers.Removed
Trainerclass and all subclasses, we now only haveTrainerConfigand its subclasses.breaking api change
Moved much of the functionality (parallelization, snapshot saving, batching etc.) from the
TrainerConfigto theRunConfig.This includes arguments such as
num_workers,snapshot_interval,batch_size. The only thing that must be updated is moving these arguments from theTrainerConfigto theRunConfig.