Using LightningDataModule with custom data #19724
Unanswered
asusdisciple
asked this question in
Lightning Trainer API: Trainer, LightningModule, LightningDataModule
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
So at the moment I am working a lot with my own custom data. Of course I transferred that data into the classic pytorch dataset, but since I am using Lightning I wondered if there should be a way to use your custom data inside the LightningDataModule without relying on Pytorch Datasets.
Why?
Because for custom data I now have one custom class to create the pytorch dataset, which is then fed into my LightningDataModule to be used for training, testing etc. . This makes two classes for a task which could be handled by one. You could argue I would be able to introduce my pytorch datasetclass in the lightningdatamodule, however with a more complex structure, this makes it necessary to reimplement len() , getitem() and init() as well.
Is there a better workflow for this? Currently my datasets usually consist of a datafolder and .csv files with labels which have to be read in.
Beta Was this translation helpful? Give feedback.
All reactions