Skip to content

Commit

Permalink
[DOC] fix gradient_clip value in tutorials to ensure reproducible o…
Browse files Browse the repository at this point in the history
…utputs similar to the committed cell output (#1750)

Update gradient_clip=0.01 to gradient_clip=0.1.

### Description

I've been following along with the Interpretable forecasting with
N-Beats tutorial found here
https://pytorch-forecasting.readthedocs.io/en/latest/tutorials/ar.html,
copy-pasting the code exactly, and I am getting different results
compared to what is shown on the documentation page.

The first discrepancy between my results and the tutorial is when using
lightning.pytorch.tuner.Tuner to find the optimal learning rate. The
suggested learning rate and resulting plot are different.

Perhaps more of a problem, however, is that my model fails to learn
(training loss doesn't decrease), even when increasing the epochs. When
I run the code from the Evaluate Results section, all of the resulting
plots show a flat predicted curve.

I believe I've found the problem. Testing on my side confirms. 

Code cells [6] and [14] have the parameter gradient_clip=0.01, which is
the source of the problem. The N-HiTS tutorial uses gradient_clip=0.1,
so I tried that out, which fixes the problem. The outputs don't match
the tutorial outputs exactly, but they're much closer than what
gradient_clip=0.01 yields.
  • Loading branch information
gbilleyPeco authored Feb 17, 2025
1 parent 15df813 commit 654b94d
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/source/tutorials/ar.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@
],
"source": [
"pl.seed_everything(42)\n",
"trainer = pl.Trainer(accelerator=\"auto\", gradient_clip_val=0.01)\n",
"trainer = pl.Trainer(accelerator=\"auto\", gradient_clip_val=0.1)\n",
"net = NBeats.from_dataset(training, learning_rate=3e-2, weight_decay=1e-2, widths=[32, 512], backcast_loss_ratio=0.1)"
]
},
Expand Down Expand Up @@ -448,7 +448,7 @@
" max_epochs=3,\n",
" accelerator=\"auto\",\n",
" enable_model_summary=True,\n",
" gradient_clip_val=0.01,\n",
" gradient_clip_val=0.1,\n",
" callbacks=[early_stop_callback],\n",
" limit_train_batches=150,\n",
")\n",
Expand Down

0 comments on commit 654b94d

Please sign in to comment.