Tensorboard fails to log found lr when auto_lr_find is enabled.

[LeeJZh]

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

1. run job with --auto_lr_find enabled
2. note the found lr
3. open tensorboard hparams tab
4. note the logged lr
5. they are different, the logged lr is the default lr mannully assigned

Code sample

Expected behavior

the logged lr is the found lr

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/tests/collect_env_details.py
# For security purposes, please check the contents of collect_env_details.py before running it.
python collect_env_details.py

• PyTorch Version (e.g., 1.0): 1.6.0
• OS (e.g., Linux): ubuntu 16.04
• How you installed PyTorch (conda, pip, source): pip
• Build command you used (if compiling from source):
• Python version: 3.7.7
• CUDA/cuDNN version: 10.2
• GPU models and configuration: V100
• Any other relevant information:

Additional context

[awaelchli]

Hi, thanks for reporting this. I noticed you did not specify the PL version. Could you check that you are on 0.9, because we recently fixed a bug regarding the lr_find setting the learning rate attribute on hparams. #2821

[LeeJZh]

Hi, thanks for reporting this. I noticed you did not specify the PL version. Could you check that you are on 0.9, because we recently fixed a bug regarding the lr_find setting the learning rate attribute on hparams. #2821

yes pl version 0.9

[ddrevicky]

I've looked at this and this actually has nothing to do with TensorBoard or any other logger. PR #3293 added a tune() method which extracted the learning rate finder out of fit(). Looking at it, it seems that William intended it to be used separately from fit().

Anyway, learning rate finder is not called in fit() at all so the learning rate the user sets is used (and logged by TensorBoard). So learning rate finder doesn't work now with fit() and also auto_scale_batch_size does not since it was also extracted to the tune() method.

[vedal]

I had to both set Trainer(..., auto_lr_find=True) and call trainer.tune(model, datamodule=datamodule) explicitly to make this work before calling trainer.fit() in version 1.2.8