Add plot method and data.py to NAIP

[nilsleh]

Since RasterDatasets should have their own plot method per #253, this PR adds a plot method as well as a data.py file to the NAIP dataset.

[calebrob6]

It looks like this is missing the data.py file, and the tests are not passing with the new test NAIP data

[calebrob6]

Maybe we can cut this down to 1PB with some optimizations?

[adamjstewart]

Must be something from with the transform that's causing the file size to blow up to PBs?

[adamjstewart]

We seem to still be allocating a ~1 PB numpy array somewhere, causing the tests to fail.

[adamjstewart]

Has anyone figured out what is going on with this giant numpy array?

[nilsleh]

Has anyone figured out what is going on with this giant numpy array?

Could it have to do with the size of the bounds that is passed to index the dataset?

[adamjstewart]

That has to be it but I don't see anything fishy about the data. Maybe try printing the data length and dataset.bounds?

[adamjstewart]

P.S. To see the output of a print statement in a test file you'll need to use the -s flag:

$ pytest -s tests/datasets/test_geo.py::TestRasterDataset::test_getitem_single_file

[nilsleh]

That has to be it but I don't see anything fishy about the data. Maybe try printing the data length and dataset.bounds?

when running pytest on my ubuntu machine, I get:
BoundingBox(minx=0.0, maxx=64.0, miny=-63.0, maxy=1.0, mint=1541285999.999999, maxt=1559771999.999999) torch.Size([4, 64, 64])

[adamjstewart]

It's not the test_naip.py tests that are failing, it's the test_geo.py tests.

[nilsleh]

it's the test_geo.py tests.

If I uncomment, the hardcorded crs of crs = CRS.from_epsg(3005) in the test_geo.py file, then the array is normal sized and the tests pass.

[adamjstewart]

The new test failure is because our NAIP data and Chesapeake data have no overlap.

[nilsleh]

The new test failure is because our NAIP data and Chesapeake data have no overlap.

TJust for my understanding: The error I am seeing is that for the IntersectionDataset of naip & chesapeake, the Bounding box is invalid: 'minx=1.7976931348623157e+308' > 'maxx=-1.7976931348623157e+308'. Is that what happens when two datasets have no overlap?

I am not sure how to best proceed. If one now tunes the NAIP data.py to work with the Chesapeake bounds specifically, there might come a new test along that also needs to overlap with NAIP but then can't. What would you suggest to do?

[adamjstewart]

Correct, there's a bug in rtree where an empty index actually has invalid bounds. That's the error you're seeing. If you print the length of the dataset, you'll find that it's zero.

I think it's fine to make sure that the bounding box for NAIP and Chesapeake match. The actual location of these datasets on earth doesn't matter, they just need to overlap. I don't envision a test being added that asserts that these particular datasets don't overlap.