Record what the values from MODIS cloud mask represent

[AlysonR]

After loading in MODIS cloud masks, satpy returns arrays with various values (0, 1, 2, 3...). It would be nice to know what each of these correspond to. I'm guessing its related to confidence of whether cloud/no cloud for each pixel, but I'm not certain.

[djhoese]

@AlysonR We typically return whatever information is in the file. There should (or might) be a flag_meanings attribute as is standard/common for CF compliant NetCDF files. Since these are HDF4 files I wouldn't be surprised if the information is missing. That means it is up to the person who writes the reader to provide that information, if known.

If you look up the specific product online, can you find the meanings of the categories? We could add that information to the reader if we can find it.

[AlysonR]

You return a 2030 x 1354 array of ints. I can't find anywhere online that would explain that these are. Could the person who created the reader add documentation on what these mean ASAP?

[djhoese]

What variable (dataset) are you loading from the reader specifically (what you are passing to scn.load)?

[djhoese]

And is this MOD35 or MOD06?

[AlysonR]

from satpy import Scene
import numpy as np
import glob
import sys

def get_cloud_mask(cloud_mask_dir, level_1_filename):
cloud_mask_filename = glob.glob(cloud_mask_dir + '' + level_1_filename.split('.A')[1][:12] + '')[0]
print cloud_mask_filename
swath = Scene(reader = 'modis_l2', filenames = [cloud_mask_filename, level_1_filename])
swath.load(['cloud_mask'], resolution = 1000)
return cloud_mask

[AlysonR]

this is MOD35

[djhoese]

I google "MOD35 cloud mask" which lead to this page (https://modis-atmosphere.gsfc.nasa.gov/products/cloud-mask/file-spec) which lead to https://modis-atmosphere.gsfc.nasa.gov/sites/default/files/ModAtmo/MYD35_L2.C6.CDL.fs which is the HDF4 header information for the files. You can see for cloud_mask it is a complicated bit mask. I could be wrong but I'm pretty sure the cloud_mask variable in Satpy is the first byte of this HDF variable. @LTMeyer Might know more.

What are you working on that this information is needed "ASAP"? Satpy is created and contributed to by volunteers. We provide what we can for what we need. Please understand that Satpy's contributors have their own jobs and Satpy is something we use to get our work done. We can't always respond in a timely manner or provide software that meets every one of your needs.

[djhoese]

Actually rereading the python code, I think cloud_mask should be the "Unobstructed FOV Quality Flag" turned in to an integer (0=Cloudy, 1=Uncertain, 2=Probably Clear, 3=Confident Clear). Do you see values greater than 3?

[AlysonR]

We are seeing values greater than 3, which is why we realized it was probably representing a byte and not a flag as we thought. It does sound like Unobstructed FOV Quality Flag is what we're looking for rather than the cloud mask values.

[djhoese]

Hm, the MOD35 data I have shows a minimum of 0 and a maximum of 3. I ran the below after loading the data, what do you get?

print(scn['cloud_mask'].min().values)
print(scn['cloud_mask'].max().values)

Edit: This was resolution=1000

[djhoese]

The 250m resolution version seems wrong:

The 1000m version:

[djhoese]

Ok based on what @BENR0 said on slack (still verifying with other people), the 250m product is supposed to be 0 and 1 and the 1000m product is 0-3. If you are absolutely sure you are seeing values above 3 could you send me the MOD35 file somehow?

[AlysonR]

I'm also getting just 0 and 3. I think that it's returning Unobstructed FOV Value as you described. Sorry my colleague was apparently getting 9 from a different SatPy line call.