Add VIIRS Fire Temperature rgb #728

adybbroe · 2019-04-18T14:55:59Z

Adding the Fire Temperature RGB for VIIRS

Tests added
Tests passed
Passes git diff origin/master -- "*py" | flake8 --diff
Fully documented

Bring my fork up to date

Signed-off-by: Adam Dybbroe <[email protected]>

adybbroe · 2019-04-18T14:56:57Z

djhoese · 2019-04-18T14:57:17Z

satpy/etc/enhancements/generic.yaml

@@ -79,6 +79,22 @@ enhancements:
    - name: gamma
      method: *gammafun
      kwargs: {gamma: 1.8}
+  fire_temperature_default:


Is this enhancement specific to VIIRS? Maybe we should put it in enhancements/viirs.yaml?

I guess the same enhancement can be applied to any sensor with a 1.6, 2.2 and 3.7 micron channel....

I will continue on Tuesday, possibly tuning it further. But now Easter holiday. So, @djhoese
don't merge yet. Just wanted to add it before closing down for today.

And here the one from http://rammb.cira.colostate.edu/projects/alaska/blog/index.php/tag/fire-temperature-rgb/

After some further tuning...

...after a bit of further tuning with @mraspaud using a bit of gamma we are even closer. This is my final bet!

coveralls · 2019-04-18T15:15:41Z

Coverage increased (+0.7%) to 80.755% when pulling f310d60 on adybbroe:fire-temperature-rgb into 1849b17 on pytroll:master.

Signed-off-by: Adam Dybbroe <[email protected]>

codecov · 2019-04-23T22:27:43Z

Codecov Report

Merging #728 into master will increase coverage by 0.69%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #728      +/-   ##
==========================================
+ Coverage   80.02%   80.71%   +0.69%     
==========================================
  Files         147      149       +2     
  Lines       21498    21836     +338     
==========================================
+ Hits        17203    17626     +423     
+ Misses       4295     4210      -85

Impacted Files	Coverage Δ
satpy/readers/geocat.py	`87.5% <0%> (-4.23%)`	⬇️
satpy/node.py	`91.42% <0%> (-2.86%)`	⬇️
satpy/readers/goes_imager_nc.py	`63.88% <0%> (-1.86%)`	⬇️
satpy/writers/geotiff.py	`41.66% <0%> (-0.36%)`	⬇️
satpy/readers/grib.py	`79.86% <0%> (-0.29%)`	⬇️
satpy/tests/test_yaml_reader.py	`98.51% <0%> (-0.26%)`	⬇️
satpy/writers/__init__.py	`85.21% <0%> (ø)`	⬆️
satpy/tests/reader_tests/test_electrol_hrit.py	`96.46% <0%> (ø)`
satpy/tests/test_config.py	`89.28% <0%> (ø)`
satpy/tests/test_scene.py	`99.44% <0%> (+0.02%)`	⬆️
... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1849b17...f310d60. Read the comment docs.

Signed-off-by: Adam Dybbroe <[email protected]>

…g the one we guessed Signed-off-by: Adam Dybbroe <[email protected]>

adybbroe · 2019-05-05T21:00:26Z

Below I show the results of the Alaska July 2015 fires for the four CIRA recipes and the one I tuned to look like the one shown on the CIRA web site.
First the original VIIRS:

Here the one where all bands use the reflectance - using the 3.9 reflective part, removing the emissive part:

Here the AWIPS version:

Here the EUMETSAT one:

And here the test/smhi version:

…YAML configurations Signed-off-by: Adam Dybbroe <[email protected]>

adybbroe · 2019-05-06T10:22:31Z

@djhoese and @mraspaud I added some documentation, please could you take what you feel is necessary and reorganize if needed?
I think as you both indicated it is preferable to have to specify only one composite recipe (when they are the same) and then couple the enhancement in another way. So, on default the recipe use one enhancement, if the user wants the "same" RGB with another enhancement it can be specified in the call. Like:
scene.load([fire_temperature], enhancement_recipe="awips")
?

djhoese · 2019-05-06T13:26:08Z

doc/source/overview.rst

+enhancements, see below. For more details on how to add your own composite
+recipe to the configuration see :doc:`composites` and :doc:`enhancements`.
+
+Satpy also makes it possible to create your own custom composites


I'm not sure this is the right place for this. This is the overview section and I don't think it should have real code in it or talk about the details of YAML files. You also mention creating your own composite, but the rest of the existing paragraph talks again about making your own custom composites. The Scene is missing a reader parameter.

djhoese

I'll have to see if I can find time this week to re-organize some of the compositor/enhancement documentation you started. It would make the PCW easier for people unfamiliar with Satpy.

djhoese · 2019-05-06T13:31:03Z

doc/source/composites.rst

+
+
+In the composite YAML file it os then necessary to map the right enhancement to
+the right composite using the `name` attribute::


I'm worried about this too because it over simplifies the logic of enhancers and mentions matching based on name but doesn't say anything about standard_name even though it is also specified.

This is also being talked about in detail in the "Built-in composites" when maybe it should be put in a custom compositor section and this section can add a link to it.

djhoese · 2019-05-06T13:37:32Z

@adybbroe Could you give me a summary of what I'm supposed to be seeing in your images? First, what is "original VIIRS" (your first image)? I know from being on the email chain that you were given two recipes so I understand what those are. Is there any reason to keep the SMHI version now that we have the "official" recipe?

adybbroe · 2019-05-06T20:48:38Z

To answer your question @djhoese I repeat what I wrote on Slack (for completeness):
The “original viirs” is from Curtis Seaman’s slides. So it is the one he thinks works best. But awips doesn’t have access to radiances he said, therefore the awips one. Then he listed a Eumetsat one, thought it was appropriate to add that as well. I think time will have to tell what works best when and where.

They are all supposed to show fires obviously, but background temperatures might have influence on how each one of them perform.

Signed-off-by: Adam Dybbroe <[email protected]>

mraspaud · 2019-05-07T09:11:18Z

I'm pasting here the documentation @adybbroe wrote, so that they can be taken up in another PR while this one can be merged.

In composites.rst:

RGB composites and associated enhancements
------------------------------------------

The default RGB composites available are all configured in YAML configuration
files, as mentioned above. However, the composite YAML configuration files
doesn't specify how the final image is enhanced (stretched). This is specified
in the YAML configuration files for the enhancements (e.g. `generic.yaml`).  It
is possible to have several recipes for the same combination of channels. This
is for instace the case for the fire temperature RGB, where different agencies
use slightly different enhancements for the same combination VIIRS bands.

Here are two different enhancements used at EUMETSAT and NOAA::

  fire_temperature_awips:
    standard_name: fire_temperature
    name: fire_temperature_awips
    operations:
    - name: stretch
      method: *stretchfun
      kwargs:
        stretch: crude
        min_stretch: [273.0, 0, 0]
        max_stretch: [333.0, 100., 75.]
    - name: gamma
      method: *gammafun
      kwargs: {gamma: [0.4, 1.0, 1.0]}
  fire_temperature_eumetsat:
    standard_name: fire_temperature
    name: fire_temperature_eumetsat
    operations:
    - name: stretch
      method: *stretchfun
      kwargs:
        stretch: crude
        min_stretch: [273.0, 0, 0]
        max_stretch: [350.0, 60., 60.]
    - name: gamma
      method: *gammafun
      kwargs: {gamma: [1.0, 1.0, 1.0]}


In the composite YAML file it os then necessary to map the right enhancement to
the right composite using the `name` attribute::

  fire_temperature_awips:
    # CIRA: EUMETSAT
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - name: M12
    - name: M11
    - name: M10
    standard_name: fire_temperature
    name: fire_temperature_awips
    
  fire_temperature_eumetsat:
    # CIRA: AWIPS
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - name: M12
    - name: M11
    - name: M10
    standard_name: fire_temperature
    name: fire_temperature_eumetsat

In overview.rst:

Compositing
===========

Many users of satellite imagery combine multiple sensor channels to bring
out certain features of the data. This includes using one dataset to enhance
another, combining 3 or more datasets in to an RGB image, or any other
combination of datasets. Satpy comes with a lot of common composite
combinations built-in and allows the user to request them like any other
dataset.

The list of default RGB composites available depends obviously on the
sensor. For instance the AVHRR sensor does not allow to generate a true color
RGB, whereas that is possible with VIIRS and MODIS. It is straightforward to
see what default possibilities are available:

    >>> from satpy.scene import Scene
    >>> scene = Scene(filenames=myfiles)
    >>> print(scene.available_composite_names())

These default RGB composites have recipes that are specified in the
`visir.yaml` and/or in the config file specific for the given sensor
(e.g. `viirs.yaml` or `seviri.yaml`). The composites may use different
enhancements, see below. For more details on how to add your own composite
recipe to the configuration see :doc:`composites` and :doc:`enhancements`.

Satpy also makes it possible to create your own custom composites
and have Satpy treat them like any other dataset. See :doc:`composites`
for more information.

…ncement YAML configurations" This reverts commit 4d6e88e.

codecov · 2019-05-07T09:29:08Z

Codecov Report

Merging #728 into master will increase coverage by 0.69%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #728      +/-   ##
==========================================
+ Coverage   80.02%   80.71%   +0.69%     
==========================================
  Files         147      149       +2     
  Lines       21498    21836     +338     
==========================================
+ Hits        17203    17626     +423     
+ Misses       4295     4210      -85

Impacted Files	Coverage Δ
satpy/readers/geocat.py	`87.5% <0%> (-4.23%)`	⬇️
satpy/node.py	`91.42% <0%> (-2.86%)`	⬇️
satpy/readers/goes_imager_nc.py	`63.88% <0%> (-1.86%)`	⬇️
satpy/writers/geotiff.py	`41.66% <0%> (-0.36%)`	⬇️
satpy/readers/grib.py	`79.86% <0%> (-0.29%)`	⬇️
satpy/tests/test_yaml_reader.py	`98.51% <0%> (-0.26%)`	⬇️
satpy/writers/__init__.py	`85.21% <0%> (ø)`	⬆️
satpy/tests/reader_tests/test_electrol_hrit.py	`96.46% <0%> (ø)`
satpy/tests/test_config.py	`89.28% <0%> (ø)`
satpy/tests/test_scene.py	`99.44% <0%> (+0.02%)`	⬆️
... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1849b17...f310d60. Read the comment docs.

mraspaud

LGTM

adybbroe and others added 3 commits April 11, 2019 20:47

Merge pull request #8 from pytroll/master

c10e2ea

Bring my fork up to date

Merge pull request #9 from pytroll/master

722df5c

Bring my fork up to date

Add fire temperature RGB

3e88e89

Signed-off-by: Adam Dybbroe <[email protected]>

adybbroe closed this Apr 18, 2019

djhoese reviewed Apr 18, 2019

View reviewed changes

adybbroe reopened this Apr 18, 2019

adybbroe self-assigned this Apr 18, 2019

adybbroe added component:compositors component:enhancements labels Apr 18, 2019

Tune stretch interval so that RGB imagery fits the CIRA examples

bae0796

Signed-off-by: Adam Dybbroe <[email protected]>

adybbroe added 2 commits April 24, 2019 13:21

Add a bit of gamma stretching to the RGB layers

f882b93

Signed-off-by: Adam Dybbroe <[email protected]>

Implementing four of the recipes from CIRA (curtis Seaman) and keepin…

481d80d

…g the one we guessed Signed-off-by: Adam Dybbroe <[email protected]>

adybbroe requested review from mraspaud and pnuu as code owners May 5, 2019 20:55

Try explain how RGB recipes are made using composite and enhancement …

4d6e88e

…YAML configurations Signed-off-by: Adam Dybbroe <[email protected]>

djhoese reviewed May 6, 2019

View reviewed changes

adybbroe and others added 3 commits May 6, 2019 22:51

Remove SMHI/test version of the fire temperature RGB

e5a29ec

Signed-off-by: Adam Dybbroe <[email protected]>

Add system tests

ddf9352

Cleanup yaml files

c3fb04f

Revert "Try explain how RGB recipes are made using composite and enha…

f310d60

…ncement YAML configurations" This reverts commit 4d6e88e.

mraspaud approved these changes May 7, 2019

View reviewed changes

mraspaud merged commit eaadb17 into pytroll:master May 7, 2019

djhoese added the enhancement code enhancements, features, improvements label May 10, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add VIIRS Fire Temperature rgb #728

Add VIIRS Fire Temperature rgb #728

adybbroe commented Apr 18, 2019 •

edited by mraspaud

Loading

adybbroe commented Apr 18, 2019

djhoese Apr 18, 2019

adybbroe Apr 18, 2019

adybbroe Apr 18, 2019

adybbroe Apr 23, 2019

adybbroe Apr 23, 2019

adybbroe Apr 24, 2019

coveralls commented Apr 18, 2019 •

edited

Loading

codecov bot commented Apr 23, 2019 •

edited

Loading

adybbroe commented May 5, 2019

adybbroe commented May 6, 2019

djhoese May 6, 2019

djhoese left a comment

djhoese May 6, 2019

djhoese commented May 6, 2019

adybbroe commented May 6, 2019

mraspaud commented May 7, 2019

codecov bot commented May 7, 2019

mraspaud left a comment



		In the composite YAML file it os then necessary to map the right enhancement to
		the right composite using the `name` attribute::

Add VIIRS Fire Temperature rgb #728

Add VIIRS Fire Temperature rgb #728

Conversation

adybbroe commented Apr 18, 2019 • edited by mraspaud Loading

adybbroe commented Apr 18, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Apr 18, 2019 • edited Loading

codecov bot commented Apr 23, 2019 • edited Loading

Codecov Report

adybbroe commented May 5, 2019

adybbroe commented May 6, 2019

Choose a reason for hiding this comment

djhoese left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djhoese commented May 6, 2019

adybbroe commented May 6, 2019

mraspaud commented May 7, 2019

codecov bot commented May 7, 2019

Codecov Report

mraspaud left a comment

Choose a reason for hiding this comment

adybbroe commented Apr 18, 2019 •

edited by mraspaud

Loading

coveralls commented Apr 18, 2019 •

edited

Loading

codecov bot commented Apr 23, 2019 •

edited

Loading