Hi François,
I am receiving this error in regards to ancillary.py when trying to run my L8 imagery:
(polymer) [sbaratta@katahdin polymer-v4.16]$ python runPolymer.py
Traceback (most recent call last):
File "runPolymer.py", line 19, in <module>
Level1_OLI('/home/sbaratta/Downloads/polymer-v4.16/dur'),
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/level1_landsat8.py", line 69, in __init__
self.ancillary = Ancillary_NASA()
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/ancillary.py", line 181, in __init__
assert isdir(directory), 'Directory {} does not exist. Please create it, by default it will be automatically populated with ancillary data. Please see help for class Ancillary_NASA for more details.'.format(directory)
AssertionError: Directory ANCILLARY/METEO/ does not exist. Please create it, by default it will be automatically populated with ancillary data. Please see help for class Ancillary_NASA for more details.
And the links within the comments on ancillary.py (https://oceancolor.gsfc.nasa.gov/...) aren't working on my end. Any help would be greatly appreciated!
Best,
Sydney
L8 ancillary error
-
- Posts: 11
- Joined: Tue Aug 08, 2023 6:40 pm
- company / institution: University of Maine
- Location: Orono
-
- Posts: 11
- Joined: Tue Aug 08, 2023 6:40 pm
- company / institution: University of Maine
- Location: Orono
Re: L8 ancillary error
Just an update and new error that has arisen:
I created the directory ANCILLARY/METEO/ in my polymer v4.16 directory, along with a .netrc file that includes my Earthdata login. However, I receive a value error ("ValueError: A value (1.1496814974406155) in x_new is above the interpolation range's maximum value (0.4000000059604645)") after connecting to Earthdata and trying to run the imagery.
Here is the full error:
Reading coordinates from /home/sbaratta/Downloads/polymer-v4.16/LC08_L1TP_001017_20230618_20230623_02_T1/LC08_L1TP_001017_20230618_20230623_02_T1_B1.TIF
/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/osgeo/gdal.py:287: FutureWarning: Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default.
warnings.warn(
Image size is 8201x8271
Starting processing at 2023-08-26 16:33:06.610714
Initializing output file "/home/sbaratta/Downloads/polymer-v4.16/LC08_L1TP_001017_20230618_20230623_02_T1test.nc"
Processing block: size (500, 400), offset (0, 0)
Processing block: size (500, 400), offset (0, 400)
Processing block: size (500, 400), offset (0, 800)
Processing block: size (500, 400), offset (0, 1200)
Processing block: size (500, 400), offset (0, 1600)
Processing block: size (500, 400), offset (0, 2000)
Processing block: size (500, 400), offset (0, 2400)
Processing block: size (500, 400), offset (0, 2800)
Processing block: size (500, 400), offset (0, 3200)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/main.py", line 413, in process_block
c.cloudmask(block)
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/main.py", line 289, in cloudmask
block.Rnir[ok] = block.Rtoa_gc[:,:,inir_block][ok] - self.mlut['Rmol'][
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/luts.py", line 399, in __getitem__
keys = k.index(self.axes)
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/luts.py", line 1110, in index
res = interp1d(axis, np.arange(len(axis)),
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/scipy/interpolate/_polyint.py", line 80, in __call__
y = self._evaluate(x)
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/scipy/interpolate/_interpolate.py", line 752, in _evaluate
below_bounds, above_bounds = self._check_bounds(x_new)
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/scipy/interpolate/_interpolate.py", line 786, in _check_bounds
raise ValueError("A value ({}) in x_new is above "
ValueError: A value (1.1496814974406155) in x_new is above the interpolation range's maximum value (0.4000000059604645).
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "runPolymer.py", line 18, in <module>
run_atm_corr(
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/main.py", line 523, in run_atm_corr
for block in block_iter:
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/multiprocessing/pool.py", line 868, in next
raise value
ValueError: A value (1.1496814974406155) in x_new is above the interpolation range's maximum value (0.4000000059604645).
Processing block: size (500, 400), offset (0, 3600)
Thanks,
Sydney
I created the directory ANCILLARY/METEO/ in my polymer v4.16 directory, along with a .netrc file that includes my Earthdata login. However, I receive a value error ("ValueError: A value (1.1496814974406155) in x_new is above the interpolation range's maximum value (0.4000000059604645)") after connecting to Earthdata and trying to run the imagery.
Here is the full error:
Reading coordinates from /home/sbaratta/Downloads/polymer-v4.16/LC08_L1TP_001017_20230618_20230623_02_T1/LC08_L1TP_001017_20230618_20230623_02_T1_B1.TIF
/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/osgeo/gdal.py:287: FutureWarning: Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default.
warnings.warn(
Image size is 8201x8271
Starting processing at 2023-08-26 16:33:06.610714
Initializing output file "/home/sbaratta/Downloads/polymer-v4.16/LC08_L1TP_001017_20230618_20230623_02_T1test.nc"
Processing block: size (500, 400), offset (0, 0)
Processing block: size (500, 400), offset (0, 400)
Processing block: size (500, 400), offset (0, 800)
Processing block: size (500, 400), offset (0, 1200)
Processing block: size (500, 400), offset (0, 1600)
Processing block: size (500, 400), offset (0, 2000)
Processing block: size (500, 400), offset (0, 2400)
Processing block: size (500, 400), offset (0, 2800)
Processing block: size (500, 400), offset (0, 3200)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/main.py", line 413, in process_block
c.cloudmask(block)
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/main.py", line 289, in cloudmask
block.Rnir[ok] = block.Rtoa_gc[:,:,inir_block][ok] - self.mlut['Rmol'][
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/luts.py", line 399, in __getitem__
keys = k.index(self.axes)
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/luts.py", line 1110, in index
res = interp1d(axis, np.arange(len(axis)),
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/scipy/interpolate/_polyint.py", line 80, in __call__
y = self._evaluate(x)
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/scipy/interpolate/_interpolate.py", line 752, in _evaluate
below_bounds, above_bounds = self._check_bounds(x_new)
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/site-packages/scipy/interpolate/_interpolate.py", line 786, in _check_bounds
raise ValueError("A value ({}) in x_new is above "
ValueError: A value (1.1496814974406155) in x_new is above the interpolation range's maximum value (0.4000000059604645).
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "runPolymer.py", line 18, in <module>
run_atm_corr(
File "/home/sbaratta/Downloads/polymer-v4.16/polymer/main.py", line 523, in run_atm_corr
for block in block_iter:
File "/opt/ohpc/pub/anaconda3/users/sbaratta/2023.07/envs/polymer/lib/python3.8/multiprocessing/pool.py", line 868, in next
raise value
ValueError: A value (1.1496814974406155) in x_new is above the interpolation range's maximum value (0.4000000059604645).
Processing block: size (500, 400), offset (0, 3600)
Thanks,
Sydney
-
- Posts: 25
- Joined: Mon Mar 07, 2022 9:54 am
- company / institution: Finnish Environment Institute
- Location: Helsinki, Finland
Re: L8 ancillary error
Are you using version 4.16.1 instead of 4.16? It has fixed an issue related to ancillary data, which manifested itself as an index overflow in LUT table lookup.
viewtopic.php?f=5&t=210
viewtopic.php?f=5&t=210
-
- Posts: 11
- Joined: Tue Aug 08, 2023 6:40 pm
- company / institution: University of Maine
- Location: Orono
Re: L8 ancillary error
I am using 4.16, should I instead try 4.16.1? Also, what version of python should I be using with this?
Best,
Sydney
Best,
Sydney
-
- Posts: 11
- Joined: Tue Aug 08, 2023 6:40 pm
- company / institution: University of Maine
- Location: Orono
Re: L8 ancillary error
Another update and question about 4.16.1 installation -
I tried v4.16.1 with python v3.8, and am receiving this error when trying to run my imagery:
(polymer) [sbaratta@katahdin polymer-v4.16.1]$ python runPolymer.py
Traceback (most recent call last):
File "runPolymer.py", line 1, in <module>
from polymer.main import run_atm_corr
File "/home/sbaratta/Downloads/polymer-v4.16.1/polymer/main.py", line 16, in <module>
from polymer.polymer_main import PolymerMinimizer
ModuleNotFoundError: No module named 'polymer.polymer_main'
Do you know what I need to do that is different from 4.16?
I tried v4.16.1 with python v3.8, and am receiving this error when trying to run my imagery:
(polymer) [sbaratta@katahdin polymer-v4.16.1]$ python runPolymer.py
Traceback (most recent call last):
File "runPolymer.py", line 1, in <module>
from polymer.main import run_atm_corr
File "/home/sbaratta/Downloads/polymer-v4.16.1/polymer/main.py", line 16, in <module>
from polymer.polymer_main import PolymerMinimizer
ModuleNotFoundError: No module named 'polymer.polymer_main'
Do you know what I need to do that is different from 4.16?
-
- Posts: 25
- Joined: Mon Mar 07, 2022 9:54 am
- company / institution: Finnish Environment Institute
- Location: Helsinki, Finland
Re: L8 ancillary error
Did you compile Polymer? Quoting from the README.md:sydney.baratta wrote: ↑Tue Aug 29, 2023 3:42 pmAnother update and question about 4.16.1 installation -
I tried v4.16.1 with python v3.8, and am receiving this error when trying to run my imagery:
(polymer) [sbaratta@katahdin polymer-v4.16.1]$ python runPolymer.py
Traceback (most recent call last):
File "runPolymer.py", line 1, in <module>
from polymer.main import run_atm_corr
File "/home/sbaratta/Downloads/polymer-v4.16.1/polymer/main.py", line 16, in <module>
from polymer.polymer_main import PolymerMinimizer
ModuleNotFoundError: No module named 'polymer.polymer_main'
Do you know what I need to do that is different from 4.16?
### 1.3 Compilation
The pyx files are cython files which need to be converted to C, then compiled.
A makefile is provided, so just type:
```
$ make
```
NOTE: the command `make all` will download the auxiliary files and proceed to the compilation.