Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax torch compatibility to <3 #277

Merged
merged 2 commits into from
Feb 6, 2025

Conversation

lostella
Copy link
Contributor

@lostella lostella commented Feb 6, 2025

Issue #, if available: Solves #273

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@lostella
Copy link
Contributor Author

lostella commented Feb 6, 2025

chronos-t5-small evaluation before/after (df1/df2) the change, using https://github.com/amazon-science/chronos-forecasting/blob/39515ff0fcdae55bbbf546d90193dbe54b201556/scripts/evaluation/evaluate.py:

                                    MASE                  WQL
                                  before      after    before     after
dataset
ETTh                            0.851675   0.777327  0.086678  0.077777
ETTm                            0.682543   0.692318  0.060765  0.060481
dominick                        0.810877   0.807912  0.336810  0.335697
ercot                           0.564880   0.566693  0.015548  0.016285
exchange_rate                   1.814346   2.182487  0.014492  0.013811
m4_quarterly                    1.241533   1.236777  0.083838  0.083788
m4_yearly                       3.738750   3.748101  0.138451  0.138995
m5                              0.936871   0.936890  0.589607  0.589155
monash_australian_electricity   1.224115   1.215008  0.069514  0.072899
monash_car_parts                0.891751   0.887449  1.031499  1.031947
monash_cif_2016                 1.018794   0.994168  0.014633  0.014659
monash_covid_deaths            42.298997  42.342608  0.063395  0.062814
monash_fred_md                  0.474216   0.466764  0.014867  0.016123
monash_hospital                 0.709815   0.710685  0.057047  0.057027
monash_m1_monthly               1.172304   1.166303  0.137990  0.133907
monash_m1_quarterly             1.807783   1.794061  0.113234  0.109248
monash_m1_yearly                4.739968   4.813232  0.173074  0.173566
monash_m3_monthly               0.885658   0.889282  0.099853  0.100100
monash_m3_quarterly             1.278908   1.265200  0.080940  0.079545
monash_m3_yearly                3.382470   3.405070  0.157364  0.158634
monash_nn5_weekly               0.927740   0.932640  0.089639  0.089669
monash_tourism_monthly          1.925118   1.913863  0.109440  0.111581
monash_tourism_quarterly        1.762345   1.756119  0.068624  0.069391
monash_tourism_yearly           3.987690   3.972676  0.199605  0.209664
monash_traffic                  0.820422   0.824421  0.257119  0.258683
monash_weather                  0.855063   0.852889  0.147970  0.147772
nn5                             0.613079   0.618412  0.167714  0.168748

@lostella
Copy link
Contributor Author

lostella commented Feb 6, 2025

same eval before/after for chronos-bolt-small:

                                    MASE                  WQL
                                   after     before     after    before
dataset
ETTh                            0.792522   0.792522  0.075907  0.075907
ETTm                            0.620962   0.620962  0.050562  0.050562
dominick                        0.870632   0.870613  0.348112  0.348111
ercot                           0.756286   0.756286  0.025961  0.025961
exchange_rate                   1.774835   1.774835  0.011364  0.011364
m4_quarterly                    1.247807   1.247814  0.078089  0.078088
m4_yearly                       3.692647   3.692560  0.127727  0.127726
m5                              0.919538   0.919544  0.566841  0.566843
monash_australian_electricity   0.812842   0.812842  0.041510  0.041510
monash_car_parts                0.858489   0.858457  1.007510  1.007469
monash_cif_2016                 1.018187   1.018247  0.015839  0.015820
monash_covid_deaths            36.469423  36.467596  0.042717  0.042738
monash_fred_md                  0.613132   0.613286  0.037195  0.037304
monash_hospital                 0.695497   0.695449  0.058112  0.058120
monash_m1_monthly               1.127816   1.127762  0.133628  0.133566
monash_m1_quarterly             1.836011   1.835614  0.093653  0.093630
monash_m1_yearly                5.098022   5.098146  0.156718  0.156699
monash_m3_monthly               0.868643   0.868513  0.093960  0.093966
monash_m3_quarterly             1.326916   1.326910  0.076910  0.076910
monash_m3_yearly                3.410465   3.409933  0.154751  0.154764
monash_nn5_weekly               0.926574   0.926651  0.084514  0.084528
monash_tourism_monthly          1.610520   1.610673  0.093679  0.093623
monash_tourism_quarterly        1.835963   1.835782  0.067251  0.067343
monash_tourism_yearly           3.896325   3.896310  0.167642  0.167661
monash_traffic                  0.859796   0.859851  0.251720  0.251738
monash_weather                  0.802058   0.802041  0.132585  0.132586
nn5                             0.583269   0.583305  0.150665  0.150668

@lostella lostella requested a review from abdulfatir February 6, 2025 10:29
pyproject.toml Show resolved Hide resolved
@lostella lostella changed the title Allow torch 2.6 Relax torch compatibility to <3 Feb 6, 2025
Copy link
Contributor

@abdulfatir abdulfatir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@lostella lostella merged commit 75803f2 into amazon-science:main Feb 6, 2025
6 checks passed
@lostella lostella deleted the bump-torch-2.6 branch February 6, 2025 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants