Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update utils.py to handle exception when pytorch_lightning >= 1.9 #213

Closed

Conversation

DevTae
Copy link

@DevTae DevTae commented Oct 21, 2023

What does this PR do?

I started openspeech as follow:

python3 ./openspeech_cli/hydra_train.py dataset=ksponspeech dataset.dataset_path=/workspace/data dataset.manifest_file_path=/workspace/data/transcripts.txt tokenizer=kspon_character model=conformer_lstm audio=fbank lr_scheduler=warmup_reduce_lr_on_plateau trainer=gpu criterion=cross_entropy

This error occured directly:

ImportError: cannot import name 'LightningLoggerBase' from 'pytorch_lightning.loggers' (/root/anaconda3/envs/mirae/lib/python3.8/site-packages/pytorch_lightning/loggers/__init__.py)

Fixes #212 Import error of pytorch_lightning LightningLoggerBase

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Was this discussed/approved via a Github issue? Please add a link
    to it if that's the case.
  • Did you write any new necessary tests?
    • I did testing on my environment. After applying this code, the error is removed.

Cannot import name 'LightningLoggerBase' from 'pytorch_lightning.loggers' #962 <-- this issue shows information that LightingLoggerBase transfers into Logger when version begins >=1.9

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag

@upskyy
Copy link
Member

upskyy commented Oct 22, 2023

@DevTae After modifying the code, does learning proceed well? The current code is written based on pytorch-lightning==1.4.0, so full testing is required when upgrading the lightning version.

@DevTae
Copy link
Author

DevTae commented Oct 22, 2023

@upskyy After modifying code, the learning proceed has problem as like below.

Traceback (most recent call last):
  File "./openspeech_cli/hydra_train.py", line 53, in hydra_main
    trainer.fit(model, data_module)
  File "/root/anaconda3/envs/temp/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 608, in fit
    call._call_and_handle_interrupt(
  File "/root/anaconda3/envs/temp/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 36, in _call_and_handle_interrupt
    return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
  File "/root/anaconda3/envs/temp/lib/python3.8/site-packages/pytorch_lightning/strategies/launchers/multiprocessing.py", line 113, in launch
    mp.start_processes(
  File "/root/anaconda3/envs/temp/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 189, in start_processes
    process.start()
  File "/root/anaconda3/envs/temp/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/root/anaconda3/envs/temp/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/root/anaconda3/envs/temp/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/root/anaconda3/envs/temp/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/root/anaconda3/envs/temp/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/root/anaconda3/envs/temp/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'dict_keys' object

So, I changed pytorch lightning version into 1.6.0. Then, the error is resolved, and training process performed well. (conformer_lstm)

I think this pr is unnecessary for now, so I close this pr.

However, It is necessary to announce to download Pytorch Lightning 1.4 , ..., 1.6, not above 1.9.0.

Thank you for your comment!

@DevTae DevTae closed this Oct 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Import error of pytorch_lightning LightningLoggerBase
2 participants