Best practice when the model and datamodule have conflicting hyperparameters?

[Erotemic]

I'm getting an error when trying to use my lightning module and datamodule together:

MisconfigurationException: Error while merging hparams: the keys {'channels'} are present in both the LightningModule's and LightningDataModule's hparams.

This is because they both take "channels" as an argument. The value of this argument is a string-code indicating the input modalities that will be fed to the network.

For instance:

• "r|g|b" indicates that there will be one input stream with 3 channels named r, g, and b.
• "r|g|b,dx|dy" indicates that there will be two input images a 3-channel rgb image and a 2-channel disparity image.
• "B1|B2|B3|B4|B5" indicates a fancy 5-channel hyper-spectral image.

From a design perspective this makes sense that both the datamodule and lightning model will take this parameter. The datamodule needs to know which channels from its available pool it's torch datasets will produce, and the lightning module needs to know what the number of input modalities and the bands within those those modalities.

Typically when I construct a training session I will create my datamodule and then I will pass the datamodule.channels attribute as a constructor kwarg to the module.

However, at inference time, when I load a the model, I will construct a datamodule based on model.channels, so I ensure that it can produce what the model expects.

I suppose in the meantime, I'll just change the name of one of the parameters so they don't conflict, but in general what is the best practice to avoid this issue? Is this a shortcoming of lightning? Or could I be doing something better?

[carmocca]

I don't think there's a solution for this, considering that hparams is a flat dict.

Some repro code:

from pytorch_lightning import Trainer
from tests.helpers import BoringDataModule, BoringModel


class BoringDataModule(BoringDataModule):
    def __init__(self, x: int):
        super().__init__()
        self.save_hyperparameters()


class BoringModel(BoringModel):
    def __init__(self, x: int):
        super().__init__()
        self.save_hyperparameters()


def run():
    x = 5
    dm = BoringDataModule(x)
    model = BoringModel(dm.hparams.x)
    trainer = Trainer(fast_dev_run=True)
    trainer.fit(model, dm)


if __name__ == "__main__":
    run()

@kaushikb11 do you have any thoughts?

[Erotemic]

My feeling as well. I don't think nesting is the best option. But what about some sort of nesting where any leaf node that is unique among all nodes is aliased by the name of the leaf node itself? That hparams would be flat until a conflict arose, but conflicts could be resolve by specifying a path (which would also have to be a be a viable way to reference any item in hparams).

It sounds nice and fancy, but maybe a bit too fancy. I do like simple flat dicts.