All NaN Values - Landsat 8
-
- Posts: 10
- Joined: Tue Jun 09, 2020 4:42 pm
- company / institution: University of Maine
- Location: Orono, ME
All NaN Values - Landsat 8
Hello, I am trying to process Landsat-8 data. Polymer has been successful with some files such as this:
My basic run script looks like this:
...
run_atm_corr(
Level1_OLI(fileLoc),
Level2_NETCDF(outdir=outLoc, ext=(fileName + '.nc'),
overwrite=True), multiprocessing=-1
)
Thanks!
But has given me large files of only NaN values for other files such as these:
I realize some of the files have some cloud cover and a large amount of landmass, but there are clearly a decent amount of unobscured ocean pixels in all of them. Any idea what I might be doing wrong? I have tried re-downloading and re-installing both POLYMER and l8angles, and deleting the ancillary data in ANCILLARY?METEO on each run so it would have to get that data for each file.My basic run script looks like this:
...
run_atm_corr(
Level1_OLI(fileLoc),
Level2_NETCDF(outdir=outLoc, ext=(fileName + '.nc'),
overwrite=True), multiprocessing=-1
)
Thanks!
You do not have the required permissions to view the files attached to this post.
-
- Posts: 37
- Joined: Tue Jun 30, 2020 3:56 am
- company / institution: China University of Geosciences
- Location: China
Re: All NaN Values - Landsat 8
Dear GHesketh4
Hello,Can you further talk about the l8angles installation code?
Thank you very much
Best regards,
ding
Hello,Can you further talk about the l8angles installation code?
Thank you very much
Best regards,
ding
- fsteinmetz
- Site Admin
- Posts: 306
- Joined: Fri Sep 07, 2018 1:34 pm
- company / institution: Hygeos
- Location: Lille, France
- Contact:
Re: All NaN Values - Landsat 8
Dear GHesketh4,
I tried the same file but did not get a file full of NaNs. Water reflectance is recovered over the glint area. Which version of Polymer did you use ? ding: please open a new thread if you have a precise question that is unrelated to the current thread, thanks.
I tried the same file but did not get a file full of NaNs. Water reflectance is recovered over the glint area. Which version of Polymer did you use ? ding: please open a new thread if you have a precise question that is unrelated to the current thread, thanks.
You do not have the required permissions to view the files attached to this post.
-
- Posts: 10
- Joined: Tue Jun 09, 2020 4:42 pm
- company / institution: University of Maine
- Location: Orono, ME
Re: All NaN Values - Landsat 8
I am using Polymer 4.13 on a MAC. Would you recommend I download an earlier version? Would you be able to post your ancilliary files and l8angles files for the 20200615 Landsat image so I could check to see if I downloaded either of those incorrectly somehow?
Thanks!
Thanks!
- fsteinmetz
- Site Admin
- Posts: 306
- Joined: Fri Sep 07, 2018 1:34 pm
- company / institution: Hygeos
- Location: Lille, France
- Contact:
Re: All NaN Values - Landsat 8
Hello,
Please find my input and output files for this scene at the following address :
https://we.tl/t-JxBXtlrjxf
Cheers,
François
Please find my input and output files for this scene at the following address :
https://we.tl/t-JxBXtlrjxf
Cheers,
François
-
- Posts: 10
- Joined: Tue Jun 09, 2020 4:42 pm
- company / institution: University of Maine
- Location: Orono, ME
Re: All NaN Values - Landsat 8
Thank you! The folder you provided me did not work, but it did teach me a lot. Our Landsat 8 files and l8angle files are the same as far as I can tell ( I used diff -rq from the command line, which displayed no differences). The two sets of solar and sensor HDR files match but I don't know how to directly compare .img files if that is important to check.
I did a three-way comparison of Polymer, using:
1. Results of entirely my own files
2. My Landsat-8 imagery with your l8angle sensor and solar files
3. Results using only your posted files
Importantly, I was not generate your polymer_result.nc using my own version of Polymer even with option 3.
I also found all of the files swap the latitude and longitude in the resulting netcdf? For the 20200615 file for the Landsat Path 11 Row 29 file, the latitude range is ~ 43.5 to 45.5 and the longitude range is ~ -67 to -70. I don't know if this could be causing the root problem; however, the 20160823 file from the same location also has reverse lat-lon values but has proper non-NaN values in the expected areas.
Also importantly, your polymer_result.nc file does have the correct latitude and longitude values. Any advice? Re-running anaconda-deps.sh says everything has installed successfully, so I don't believe it's a problem with my installation of the osgeo/gdal package as I originally suspected.
I did a three-way comparison of Polymer, using:
1. Results of entirely my own files
2. My Landsat-8 imagery with your l8angle sensor and solar files
3. Results using only your posted files
Importantly, I was not generate your polymer_result.nc using my own version of Polymer even with option 3.
I also found all of the files swap the latitude and longitude in the resulting netcdf? For the 20200615 file for the Landsat Path 11 Row 29 file, the latitude range is ~ 43.5 to 45.5 and the longitude range is ~ -67 to -70. I don't know if this could be causing the root problem; however, the 20160823 file from the same location also has reverse lat-lon values but has proper non-NaN values in the expected areas.
Also importantly, your polymer_result.nc file does have the correct latitude and longitude values. Any advice? Re-running anaconda-deps.sh says everything has installed successfully, so I don't believe it's a problem with my installation of the osgeo/gdal package as I originally suspected.
You do not have the required permissions to view the files attached to this post.
-
- Posts: 10
- Joined: Tue Jun 09, 2020 4:42 pm
- company / institution: University of Maine
- Location: Orono, ME
Re: All NaN Values - Landsat 8
As a secondary update, my co-worker with a Linux setup of Polymer tried the two files and got the same failing results as me for the 20200615 file, with the lat-lon arrays swapped for both files as well.
- fsteinmetz
- Site Admin
- Posts: 306
- Joined: Fri Sep 07, 2018 1:34 pm
- company / institution: Hygeos
- Location: Lille, France
- Contact:
Re: All NaN Values - Landsat 8
Hello,
It seems indeed that there are cases where lat and lon are swapped. I'm investigating and will keep you updated !
Cheers,
François
It seems indeed that there are cases where lat and lon are swapped. I'm investigating and will keep you updated !
Cheers,
François
- fsteinmetz
- Site Admin
- Posts: 306
- Joined: Fri Sep 07, 2018 1:34 pm
- company / institution: Hygeos
- Location: Lille, France
- Contact:
Re: All NaN Values - Landsat 8
Ok I finally got it. The problem was that due to a swapping of the order of lat/lon in gdal v3.
A description of the issue is found here https://github.com/OSGeo/gdal/issues/1546
I am attaching an updated version of level1_landsat8.py which should fix the issue. Could you check ?
Though it is not quite clear to me why this would have caused issues for one file, and not for the other.
A description of the issue is found here https://github.com/OSGeo/gdal/issues/1546
I am attaching an updated version of level1_landsat8.py which should fix the issue. Could you check ?
Though it is not quite clear to me why this would have caused issues for one file, and not for the other.
You do not have the required permissions to view the files attached to this post.
-
- Posts: 10
- Joined: Tue Jun 09, 2020 4:42 pm
- company / institution: University of Maine
- Location: Orono, ME
Re: All NaN Values - Landsat 8
That did the trick! Both the 2016 and 2020 file now process correctly. Thanks for the speedy assistance.