Standardized Streamflow Index and Standardized Groundwater level Index #1877

LamAdr · 2024-08-08T19:30:40Z

Pull Request Checklist:

This PR addresses an already opened issue (for bug fixes / features)
- This PR fixes New standardized indices #1444
Tests for the changes have been added (for bug fixes / features)
- (If applicable) Documentation has been added / updated (for bug fixes / features)
CHANGELOG.rst has been updated (with summary of main changes)
- Link to issue (:issue:number) and pull request (:pull:number) has been added

What kind of change does this PR introduce?

2 new indices/indicators : SSI and SGI

Does this PR introduce a breaking change?

No

Other information:

SSI will use GEV or log-logistic (scipy does not support tweedie)
SGI will use log-normal, gamma or GEV

tests/test_indices.py

xclim/indicators/land/_streamflow.py

tests/test_hydrology.py

xclim/indicators/land/_streamflow.py

xclim/indices/_hydrology.py

xclim/indices/stats.py

Co-authored-by: Trevor James Smith <[email protected]>

LamAdr · 2024-08-23T20:44:24Z

Where things stand
SSI is completed
SGI

needs tests (xclim-testdata lacks groundwater data. Maybe we can get some here.)
I am not sure about the APP for lognorm

Zeitsperre · 2024-08-23T21:06:47Z

Just an FYI, but some tests are expected to fail until #1889 is merged!

for more information, see https://pre-commit.ci

github-actions · 2025-02-13T06:09:57Z

Warning

This Pull Request is coming from a fork and must be manually tagged approved in order to perform additional testing.

coxipi · 2025-02-13T17:18:34Z

I am not sure about the APP for lognorm

It was exactly the MLE with saddle point estimators, good job

coxipi · 2025-02-13T17:28:41Z

I can't seem to resolve the coveralls issue
still no groundwater testdata
I can't be the reviewer on this anymore

Zeitsperre · 2025-02-13T17:36:22Z

@coxipi

Ignore the Coveralls problem for now (I think their service is coming back online gradually). I've removed that as a required check for the time being.

I think we could safely add gwl data to the xclim-testdata repository. Something no bigger than 250 KB would be fine.

For reviewing, I don't mind jumping in, but I think @huard would be better.

coxipi

.

coxipi · 2025-02-13T06:10:46Z

src/xclim/indices/_hydrology.py

+    --------
+    >>> from datetime import datetime
+    >>> from xclim.indices import standardized_groundwater_index
+    >>> ds = xr.open_dataset(path_to_gwl_file)


this would need a xclim-testdata

same in the tests, I just used false data

coxipi · 2025-02-13T18:07:09Z

src/xclim/core/indicator.py

+
+
+class StandardizedIndexes(ResamplingIndicator):
+    """Resampling but flexible inputs indicators."""
+
+    src_freq = ["D", "MS"]
+    context = "hydro"


@aulemahal any opposition to this reorginazation? Previously I defined it in _precip, but now it's also needed in _hydrology

src/xclim/indices/_agro.py

for more information, see https://pre-commit.ci

huard

I just want to make sure this PR is not impeding the use of PWM for genextreme and gamma.

huard · 2025-02-26T16:30:22Z

src/xclim/indices/_hydrology.py

+        Resampling frequency. A monthly or daily frequency is expected. Option `None` assumes that desired resampling
+        has already been applied input dataset and will skip the resampling step.


Suggested change

Resampling frequency. A monthly or daily frequency is expected. Option `None` assumes that desired resampling

has already been applied input dataset and will skip the resampling step.

Resampling frequency. A monthly or daily frequency is expected. Option `None` assumes that the desired resampling

has already been applied to the input dataset and will skip the resampling step.

huard · 2025-02-26T16:31:39Z

src/xclim/indices/_hydrology.py

+        uses a deterministic function that does not involve any optimization.
+        `PWM` should be used with a `lmoments3` distribution.
+    fitkwargs : dict, optional
+        Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certains parameters (`floc`, `fscale`).


Suggested change

Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certains parameters (`floc`, `fscale`).

Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certain parameters (`floc`, `fscale`).

huard · 2025-02-26T16:35:42Z

src/xclim/indices/_hydrology.py

+        Resampling frequency. A monthly or daily frequency is expected. Option `None` assumes that desired resampling
+        has already been applied input dataset and will skip the resampling step.


Suggested change

Resampling frequency. A monthly or daily frequency is expected. Option `None` assumes that desired resampling

has already been applied input dataset and will skip the resampling step.

Resampling frequency. A monthly or daily frequency is expected. Option `None` assumes that the desired resampling

has already been applied to the input dataset and will skip the resampling step.

huard · 2025-02-26T16:36:08Z

src/xclim/indices/_hydrology.py

+        The approximate method uses a deterministic function that does not involve any optimization.
+        `PWM` should be used with a `lmoments3` distribution.
+    fitkwargs : dict, optional
+        Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certains parameters (`floc`, `fscale`).


Suggested change

Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certains parameters (`floc`, `fscale`).

Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certain parameters (`floc`, `fscale`).

huard · 2025-02-26T16:38:49Z

src/xclim/indices/stats.py

+    # "WPM" method doesn't seem to work for gamma or pearson3
+    dist_and_methods = {
+        "gamma": ["ML", "APP"],
+        "fisk": ["ML", "APP"],
+        "genextreme": ["ML", "APP"],
+        "lognorm": ["ML", "APP"],
+    }


Do you mean PWM ? Why add genextreme and lognorm here ?

huard · 2025-02-26T16:41:03Z

tests/test_hydrology.py

+        out1 = land.standardized_streamflow_index(
+            q,
+            freq="MS",
+            window=1,
+            dist="genextreme",
+            method="APP",
+            fitkwargs={"floc": 0},
+        )
+        out2 = land.standardized_streamflow_index(
+            qMM,
+            freq="MS",
+            window=1,
+            dist="genextreme",
+            method="APP",
+            fitkwargs={"floc": 0},
+        )
+        np.testing.assert_array_almost_equal(out1, out2, 3)


Not sure I understand what is checked here, beyond unit changes.

LamAdr added 2 commits August 8, 2024 09:58

SSI

d8e2415

Merge branch 'main' of https://github.com/Ouranosinc/xclim into SSI_SGI

15a86d9

github-actions bot added the indicators Climate indices and indicators label Aug 8, 2024

LamAdr marked this pull request as draft August 9, 2024 13:38

LamAdr changed the title ~~Standardised Streamflow Index and Standardised Groundwater level Index~~ Standardized Streamflow Index and Standardized Groundwater level Index Aug 9, 2024

test streamflow

51f01ed

LamAdr commented Aug 12, 2024

View reviewed changes

tests/test_indices.py Outdated Show resolved Hide resolved

LamAdr added 3 commits August 12, 2024 16:21

metadata, translation

5c9a11d

ssi reference

3e2e840

ssi test against self

59bfcfb

github-actions bot added the docs Improvements to documenation label Aug 13, 2024

LamAdr commented Aug 13, 2024

View reviewed changes

xclim/indicators/land/_streamflow.py Outdated Show resolved Hide resolved

Zeitsperre reviewed Aug 13, 2024

View reviewed changes

LamAdr and others added 8 commits August 13, 2024 11:55

Update xclim/indices/_hydrology.py

ee91fcc

Co-authored-by: Trevor James Smith <[email protected]>

Update xclim/indices/_hydrology.py

dad3889

Co-authored-by: Trevor James Smith <[email protected]>

Update xclim/indices/_hydrology.py

fe4f552

Co-authored-by: Trevor James Smith <[email protected]>

Update xclim/indices/stats.py

fe501a8

Co-authored-by: Trevor James Smith <[email protected]>

Merge branch 'Ouranosinc:main' into SSI_SGI

f2fde02

sgi

fc180f6

lognorm APP

610962b

pre-commit

97ee37a

Zeitsperre added 2 commits August 28, 2024 21:01

Merge branch 'main' into SSI_SGI

f10e18c

Merge branch 'main' into SSI_SGI

a0a4309

Zeitsperre assigned coxipi Feb 11, 2025

Zeitsperre added this to the v0.56.0 milestone Feb 11, 2025

coxipi and others added 3 commits February 12, 2025 09:51

Merge remote-tracking branch 'upstream/main' into SSI_SGI

16ebcec

[pre-commit.ci] auto fixes from pre-commit.com hooks

0613501

for more information, see https://pre-commit.ci

lots of fixes, add sgi test

88da7b1

coxipi added 2 commits February 13, 2025 01:03

Merge branch 'SSI_SGI' of https://github.com/LamAdr/xclim into SSI_SGI

ef299fb

Merge remote-tracking branch 'upstream/main' into SSI_SGI

b452332

coxipi marked this pull request as ready for review February 13, 2025 06:09

coxipi added 2 commits February 13, 2025 12:19

better doc

712b099

add gwl variable

0c2f6bc

Zeitsperre requested review from huard and Zeitsperre February 13, 2025 17:36

coxipi added 2 commits February 13, 2025 13:00

remove artefact

b2214fb

StandardizedIndexes in indicators.py

a73eee5

coxipi reviewed Feb 13, 2025

View reviewed changes

coxipi and others added 9 commits February 13, 2025 13:08

revert uneeded change

77aabcc

Merge branch 'main' into SSI_SGI

b7aee4a

Merge branch 'main' into SSI_SGI

d009fce

specify realm

c6c16c3

Merge branch 'SSI_SGI' of https://github.com/LamAdr/xclim into SSI_SGI

43d9be3

Merge branch 'main' into SSI_SGI

388dfe9

[pre-commit.ci] auto fixes from pre-commit.com hooks

ad59317

for more information, see https://pre-commit.ci

fix docstring length

3c5728a

Merge branch 'main' into SSI_SGI

4af17bd

huard approved these changes Feb 26, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Standardized Streamflow Index and Standardized Groundwater level Index #1877

Standardized Streamflow Index and Standardized Groundwater level Index #1877

LamAdr commented Aug 8, 2024 •

edited

Loading

LamAdr commented Aug 23, 2024

Zeitsperre commented Aug 23, 2024

github-actions bot commented Feb 13, 2025

coxipi commented Feb 13, 2025

coxipi commented Feb 13, 2025

Zeitsperre commented Feb 13, 2025

coxipi left a comment

coxipi Feb 13, 2025

coxipi Feb 13, 2025

huard left a comment

huard Feb 26, 2025

huard Feb 26, 2025

huard Feb 26, 2025

huard Feb 26, 2025

huard Feb 26, 2025

huard Feb 26, 2025

		Resampling frequency. A monthly or daily frequency is expected. Option `None` assumes that desired resampling
		has already been applied input dataset and will skip the resampling step.

	Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certains parameters (`floc`, `fscale`).
	Kwargs passed to ``xclim.indices.stats.fit`` used to impose values of certain parameters (`floc`, `fscale`).

Standardized Streamflow Index and Standardized Groundwater level Index #1877

Are you sure you want to change the base?

Standardized Streamflow Index and Standardized Groundwater level Index #1877

Conversation

LamAdr commented Aug 8, 2024 • edited Loading

Pull Request Checklist:

What kind of change does this PR introduce?

Does this PR introduce a breaking change?

Other information:

LamAdr commented Aug 23, 2024

Zeitsperre commented Aug 23, 2024

github-actions bot commented Feb 13, 2025

coxipi commented Feb 13, 2025

coxipi commented Feb 13, 2025

Zeitsperre commented Feb 13, 2025

coxipi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

huard left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LamAdr commented Aug 8, 2024 •

edited

Loading