Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uavsar tutorial sections #63

Merged
merged 61 commits into from
Jul 1, 2022
Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
184717f
starting tutorial
jacktarricone Jun 22, 2022
4202ced
remove large photo
jacktarricone Jun 22, 2022
1726090
updatding
jacktarricone Jun 22, 2022
77f23ef
tutorial intro
ZachHoppinen Jun 23, 2022
276bb54
work in progress
jacktarricone Jun 23, 2022
2914b0b
Merge branch 'main' of https://github.com/snowex-hackweek/website2022…
jacktarricone Jun 23, 2022
95a421b
xarray
jacktarricone Jun 23, 2022
f16ffb4
add hls data
jacktarricone Jun 23, 2022
e3a2e45
adding optical plotting
jacktarricone Jun 24, 2022
acf0e6d
notebook work
ZachHoppinen Jun 25, 2022
dc55702
staged changes
ZachHoppinen Jun 25, 2022
6c36f38
Merge branch 'jack_uavsar' of https://github.com/snowex-hackweek/webs…
ZachHoppinen Jun 25, 2022
a2d21e3
tutorial skeleton
ZachHoppinen Jun 25, 2022
5d45fd3
tutorial change
ZachHoppinen Jun 25, 2022
1a4a838
tutorial work
ZachHoppinen Jun 26, 2022
ee84cc2
spell checking
ZachHoppinen Jun 26, 2022
adc4614
cleaned notebooks
ZachHoppinen Jun 26, 2022
07ce7ca
spell checkign
ZachHoppinen Jun 26, 2022
9064ad0
mores spellcheck
ZachHoppinen Jun 26, 2022
a3cc4be
accidently had output in this notebook somehow?
ZachHoppinen Jun 26, 2022
dcbff6b
removed empty cells
ZachHoppinen Jun 26, 2022
c1bca22
table of contents addition
ZachHoppinen Jun 27, 2022
4f978ad
fixed path to tutorial in _toc.yml
ZachHoppinen Jun 27, 2022
46b4aaf
removed ipynb from toc
ZachHoppinen Jun 27, 2022
bdb799b
adding captions to images to see if that is the error.
ZachHoppinen Jun 27, 2022
b3cc640
testing new packages
ZachHoppinen Jun 27, 2022
bf41339
Merge branch 'main' of https://github.com/snowex-hackweek/website2022…
ZachHoppinen Jun 27, 2022
330e256
comment out long running cell
ZachHoppinen Jun 27, 2022
c96af0f
remove all rasterio openings of local files
ZachHoppinen Jun 27, 2022
63b11d1
fixed github url
ZachHoppinen Jun 27, 2022
f16c9a7
Merge branch 'main' of https://github.com/snowex-hackweek/website2022…
ZachHoppinen Jun 29, 2022
7f19762
subsetted tif uploads
ZachHoppinen Jun 29, 2022
b0da299
setting up accessing imagery notebook
ZachHoppinen Jun 29, 2022
1419497
tutorial branch
ZachHoppinen Jun 29, 2022
1966533
adding to motivation section
ZachHoppinen Jun 29, 2022
76ae2ab
table of contents sections
ZachHoppinen Jun 29, 2022
e991e64
cleaned notebooks
ZachHoppinen Jun 29, 2022
7ffb926
ran notebook cells in the database
ZachHoppinen Jun 29, 2022
5b31841
remove empty notebook cell
ZachHoppinen Jun 29, 2022
d866dc8
add index.md to toc file
ZachHoppinen Jun 29, 2022
681a8b3
changed title of tutorial
ZachHoppinen Jun 29, 2022
971313e
plot iamges
ZachHoppinen Jun 29, 2022
d4e7e0e
swe inversion notebook
ZachHoppinen Jun 29, 2022
b70277a
swe inversion
ZachHoppinen Jun 29, 2022
fff8cea
removed clipped images to git clone isntead
ZachHoppinen Jun 29, 2022
0e39936
tutorial download to tmp
ZachHoppinen Jun 29, 2022
2ee9647
clean notebooks
ZachHoppinen Jun 29, 2022
10363c9
spellcheck
ZachHoppinen Jun 29, 2022
8b27e3e
better name for git data repo
ZachHoppinen Jun 29, 2022
23d78a0
Merge branch 'main' of https://github.com/snowex-hackweek/website2022…
ZachHoppinen Jun 30, 2022
1e24635
reverted database tutorial I had accidently cleaned
ZachHoppinen Jun 30, 2022
f44c3e2
one more to revert
ZachHoppinen Jun 30, 2022
d3f8e17
working on insar data types
ZachHoppinen Jun 30, 2022
dab2635
inline plotting and spellcheck
ZachHoppinen Jun 30, 2022
3e5d67b
elevation violin plot
ZachHoppinen Jun 30, 2022
f8d02f9
coherence vs unwrapped phase comparison
ZachHoppinen Jul 1, 2022
acfe185
cleaned noteboos
ZachHoppinen Jul 1, 2022
539f1cb
Merge branch 'main' of https://github.com/snowex-hackweek/website2022…
ZachHoppinen Jul 1, 2022
7bffe31
trying to merge with final pushes
ZachHoppinen Jul 1, 2022
2878085
index headers
ZachHoppinen Jul 1, 2022
0cf5fd9
header level changes
ZachHoppinen Jul 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions book/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,12 @@ parts:
- file: tutorials/database/5_plot_raster_example
- file: tutorials/database/6_exporting_data
- file: tutorials/database/7_wrap_up
- file: tutorials/uavsar/index
title: Uavsar
sections:
- file: tutorials/uavsar/1_accessing_imagery
- file: tutorials/uavsar/2_types_of_uavsar_images
- file: tutorials/uavsar/3_interferometric_swe_inversion
- caption: Projects
chapters:
- file: projects/list_of_projects
Expand Down
Binary file added book/img/SnowEx_map.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/img/uavsar/conceptual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/img/uavsar/multicolor_gm.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/img/uavsar/plane.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/img/uavsar/sthelens.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/img/uavsar/vertex_example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 1 addition & 2 deletions book/tutorials/database/1_getting_started_example.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -227,8 +227,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
3 changes: 1 addition & 2 deletions book/tutorials/database/2_database_structure.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
10 changes: 2 additions & 8 deletions book/tutorials/database/3_forming_queries.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,7 @@
},
{
"cell_type": "markdown",
"metadata": {
"tags": [
"nbsphinx-gallery",
"nbsphinx-thumbnail"
]
},
"metadata": {},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why the metadata changed in all the database notebooks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think when I was cleaning notebooks I ran the clean *.ipynb command in the wrong directory. I reverted all these files back to the commit before I did that.

"source": [
"**Questions**\n",
"* What happens if we changed the number in the limit? What will we get back?\n",
Expand Down Expand Up @@ -224,8 +219,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
17 changes: 3 additions & 14 deletions book/tutorials/database/4_get_spiral_example.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -103,12 +103,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"nbsphinx-gallery",
"nbsphinx-thumbnail"
]
},
"metadata": {},
"outputs": [],
"source": [
"# Get the Matplotlib Axes object from the dataframe object, color the points by snow depth value\n",
Expand Down Expand Up @@ -146,12 +141,7 @@
},
{
"cell_type": "markdown",
"metadata": {
"tags": [
"nbsphinx-gallery",
"nbsphinx-thumbnail"
]
},
"metadata": {},
"source": [
"**Try This**:\n",
"\n",
Expand Down Expand Up @@ -196,8 +186,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
3 changes: 1 addition & 2 deletions book/tutorials/database/5_plot_raster_example.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -273,8 +273,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
3 changes: 1 addition & 2 deletions book/tutorials/database/6_exporting_data.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
3 changes: 1 addition & 2 deletions book/tutorials/database/7_wrap_up.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
217 changes: 217 additions & 0 deletions book/tutorials/uavsar/1_accessing_imagery.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# UAVSAR\n",
"\n",
":::{figure-md} sthelens\n",
"<img src=\"../../img/uavsar/sthelens.jpeg\" alt=\"uavsar scene\" width=\"800px\">\n",
"\n",
"Uavsar polarimetry imagery of Mt. St Helens. Source: Jones et al. 2008\n",
":::\n",
"\n",
"*Developers: \\\n",
"Jack Tarricone, University of Nevada, Reno \\\n",
"Zach Keskinen, Boise State University*\n",
"\n",
"*Other contributors: \\\n",
"Ross Palomaki, Montana State University\\\n",
"Naheem Adebisi, Boise State University*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## What is UAVSAR?\n",
"\n",
"[UAVSAR](https://uavsar.jpl.nasa.gov/education/what-is-uavsar.html) is a low frequency plane-based synthetic aperature radar. UAVSAR stands for \"Uninhabited Aerial Vehicle Synthetic Aperature Radar\". It captures imagery using a L-band radar. This low frequency means it can penetrate into and through clouds, vegetation, and snow.\n",
"\n",
"| frequency (cm) | resolution (rng x azi m) | Swath Width (km) | Polarizations | Launch date |\n",
"| - | - | - | - | - |\n",
"| L-band 23| 1.8 x 5.5 | 16 | VV, VH, HV, HH | 2007 |\n",
"\n",
"### NASA SnowEx 2020 and 2021 UAVSAR Campaings\n",
"\n",
"During the winter of 2020 and 2021, NASA conducted an L-band InSAR timeseries across the Western US with the goal of tracking changes in SWE. Field teams in 13 different locations in 2020, and in 6 locations in 2021, deployed on the date of the flight to perform calibration and validation observations.\n",
"\n",
":::{figure-md} UAVSAR-map\n",
"<img src=\"../../img/SnowEx_map.jpg\" alt=\"uavsar map\" width=\"800px\">\n",
"\n",
"Map of the UAVSAR flight locations for NASA SnowEx. Note that the Montana site (Central Agricultral Research Center) is not on this map. Source: Chris Hiemstra\n",
":::\n",
"\n",
"---\n",
"\n",
"The site locations from the above map along with the [Uavsar defined campaign name](https://api.daac.asf.alaska.edu/services/utils/mission_list) and currently processed pairs of InSAR images for each site. Note that the image pair count may contain multiple versions of the same image and may increase as more pairs of images are processed by JPL. Also note that the Lowman campaign name is the wrong state when searching.\n",
"\n",
"| Site Location | Campaign Name | Image Pairs |\n",
"| - | - | - |\n",
"| Grand Mesa | Grand Mesa, CO | 13 |\n",
"| Boise River Basin | Lowman, CO | 17 |\n",
"| Frazier Experimental Forest | Fraser, CO | 16 |\n",
"| Senator Beck Basin | Ironton, CO | 9 |\n",
"| East River | Peeler Peak, CO | 4 |\n",
"| Cameron Pass | Rocky Mountains NP, CO | 15 |\n",
"| Reynold Creek | Silver City, ID | 1 |\n",
"| Central Agricultral Research Center | Utica, MT | 2 |\n",
"| Little Cottonwoody Canyon | Salt Lake City, UT | 21 |\n",
"| Jemez River | Los Alamos, NM | 3 |\n",
"| American River Basin | Eldorado National Forest, CA | 4 |\n",
"| Sagehen Creek | Donner Memorial State Park, CA | 4 |\n",
"| Lakes Basin | Sierra National Forest, CA | 3 |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Why would I use UAVSAR?\n",
"\n",
"UAVSAR works with low frequency radar waves. This means it is unaffected by clouds, maintains coherence, a measure of radar image quality, over long periods, and a time series was captured over 13 sites as part of the winter of 2019-2020 and 2020-2021. Additionally the uavsar is awesome."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Accessing Uavsar Images\n",
"\n",
"Uavsar imagery can be downloaded from both the [JPL](https://uavsar.jpl.nasa.gov/cgi-bin/data.pl) and [Alaska Satellite Facility](https://search.asf.alaska.edu/#/?dataset=UAVSAR). However both provide the imagery in a binary format that is not readily usable or readable by GIS software or python libraries. \n",
"\n",
"## Data Download and Conversion with ```uavsar_pytools```\n",
"```uavsar_pytools``` ([github](https://github.com/SnowEx/uavsar_pytools))is a Python package developed out of work started at SnowEx Hackweek 2021. It nativiely downloads, formats, and converts this data in analysis ready rasters projected in WSG-84 Lat/Lon (maybe more information here). The data traditionally comes in a binary format, which is not injestible by traditional geospatial analysis software (Python, R, QGIS, ArcGIS). It can download and convert either individual images - `UavsarScene` or entire collections of images - `UavsarCollection`.\n",
"\n",
"### Netrc Authorization\n",
"\n",
"In order to download uavsar images you will need a [netrc file](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html) that contains your earthdata username and password. If you need to register for a NASA earthdata account use this [link](https://urs.earthdata.nasa.gov/). A netrc file is a hidden file, it won't appear in the your file explorer, that is in your home directory and that programs can access to get the appropriate usernames and passwords. While you won't need to use this for this demonstration uavsar_pytools has a tool to create this netrc file on a local computer. You only need to create this file once and then it should be permanently stored on your computer."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# ## Creating .netrc file with Earthdata login information\n",
"# from uavsar_pytools.uavsar_tools import create_netrc\n",
"\n",
"# # This will prompt you for your username and password and save this\n",
"# # information into a .netrc file in your home directory. You only need to run\n",
"# # this command once per computer. Then it will be saved.\n",
"# create_netrc()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Downloading and converting a single Uavsar interferogram scene\n",
"\n",
"You can find urls for Uavsar images at the [ASF vertex website](https://search.asf.alaska.edu/#/?dataset=UAVSAR). Make sure to change the platform to Uavsar and you may also want to filter to ground projected interferograms.\n",
"\n",
":::{figure-md} vertexexample\n",
"<img src=\"../../img/uavsar/vertex_example.png\" alt=\"vertex example\" width=\"800px\">\n",
"\n",
"Example of getting the uavsar URL from vertex.\n",
":::\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"try:\n",
" from uavsar_pytools import UavsarScene\n",
"except ModuleNotFoundError:\n",
" print('Install uavsar_pytools with `pip install uavsar_pytools`')\n",
"\n",
"## This is the directory you want to download and convert the images in.\n",
"work_dir = '/tmp/uavsar_data'\n",
"\n",
"## This is a url you want to download. Can be obtained from vertex\n",
"url = 'https://datapool.asf.alaska.edu/INTERFEROMETRY_GRD/UA/\\\n",
"lowman_23205_21009-004_21012-000_0007d_s01_L090_01_int_grd.zip'\n",
"\n",
"## clean = True will delete the binary and zip files leaving only the tiffs\n",
"scene = UavsarScene(url = url, work_dir=work_dir, clean= True)\n",
"\n",
"## After running url_to_tiffs() you will download the zip file, unzip the binary \n",
"## files, and convert them to geotiffs in the directory with the scene name in\n",
"## the work directory. It also generate a .csv pandas dictionary of metadata.\n",
"# scene.url_to_tiffs()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Downloading and converting a full Uavsar collection\n",
"\n",
"If you want to download and convert an entire Uavsar collection for a larger analysis you can use `UavsarCollection`. The collection names for the SnowEx campaign are listed in the table in the introduction. The `UavsarCollection` only downloads interferometric image pairs, but will soon be able to select between InSAR pairs and PolSAR images."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from uavsar_pytools import UavsarCollection\n",
"## Collection name, the SnowEx Collection names are listed above. These are case \n",
"## and space sensitive.\n",
"collection_name = 'Grand Mesa, CO'\n",
"\n",
"## Directory to save collection into. This will be filled with directory with \n",
"## scene names and tiffs inside of them.\n",
"out_dir = '/tmp/collection_ex/'\n",
"\n",
"## This is optional, but you will generally want to at least limit the date\n",
"## range between 2019 and today.\n",
"date_range = ('2019-11-01', 'today')\n",
"\n",
"# Keywords: to download incidence angles with each image use `inc = True`\n",
"# For only certain pols use `pols = ['VV','HV']`\n",
"collection = UavsarCollection(collection = collection_name, work_dir = out_dir, dates = date_range)\n",
"\n",
"## You can use this to check how many image pairs have at least one image in\n",
"## the date range.\n",
"collection.find_urls()\n",
"\n",
"## When you are ready to download all the images run:\n",
"# collection.collection_to_tiffs()\n",
"## This will take a long time and a lot of space, ~1-5 gB and 10 minutes per \n",
"## image pair depending on which scene, so run it if you have the space and time."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10.5 ('hackweek')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
},
"vscode": {
"interpreter": {
"hash": "485426853efb45fbcac7604c62a774e4eca2d162588db5a3cdf8fcb52b528869"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading