Polymer running on Sentinel-2 images

dfrgttn
Posts: 2
Joined: Sat Jun 22, 2019 10:30 am
company / institution: Vilnius University
Location: Lithuania

Polymer running on Sentinel-2 images

Post by dfrgttn »

Hello!

I installed anaconda, created an environment and installed required dependencies.
I am trying to run Polymer on Sentinel-2 images, however, this does not work to me:

Code: Select all

./polymer_cli.py S2A_MSIL1C_20170502T093041_N0205_R136_T35UMB_20170502T093358.SAFE
I received an error message:

Code: Select all

Traceback (most recent call last):
  File "./polymer_cli.py", line 11, in <module>
    from polymer.main import run_atm_corr, Level1, Level2
  File "/home/dalia/Documents/polymer-v4.12/polymer/main.py", line 18, in <module>
    from polymer.polymer_main import PolymerMinimizer
ImportError: /home/dalia/Documents/polymer-v4.12/polymer/polymer_main.so: undefined symbol: _Py_ZeroStruct
Then I tried to run through python:

Code: Select all

Level1_MSI('S2A_MSIL1C_20170502T093041_N0205_R136_T35UMB_20170502T093358.SAFE')
I received an error about a line 122 in file level1_msi.py as there are two xmls in current S2 products. So I removed this line and changed the line 123 to

Code: Select all

xmlfile = xmlfiles[1]
But then I get errors about the line 125 where Sensing time should be retrieved and I tried inserting "Product_start_time" that is found in the MTD_MSIL1C.xml file.

Could you help me? Or maybe you have old formatted metadata file?
I use linux 18.04.3 LTS and Polymer 4.12.

Kind regards,
Dalia
User avatar
fsteinmetz
Site Admin
Posts: 314
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Polymer running on Sentinel-2 images

Post by fsteinmetz »

Dear Dalia,

It seems that you may be using different versions of python for the compilation and the execution. Please make sure you are using python3 for both and use the following commands :
  • python --version (to verify that you are using python 3)
  • make rebuild
  • python polymer_cli.py (Just in case ./polymer_cli.py would use another interpreter for some reason)
If this still fails, could you send the output of conda list ?

Kind regards,
François
dfrgttn
Posts: 2
Joined: Sat Jun 22, 2019 10:30 am
company / institution: Vilnius University
Location: Lithuania

Re: Polymer running on Sentinel-2 images

Post by dfrgttn »

Thank you, François, for an answer.
I managed to run Polymer in python. It runs well and produces a result, but I get a warning like this:

Code: Select all

FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
Should I be worried?
I use python 3.8.0.

Best regards,
Dalia
User avatar
fsteinmetz
Site Admin
Posts: 314
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Polymer running on Sentinel-2 images

Post by fsteinmetz »

Good news !
No, this is just a warning, you can safely ignore it.
Cheers,
François
jcardoso
Posts: 12
Joined: Fri Feb 08, 2019 10:17 am
company / institution: National Institute for Space Research
Location: Brazil

Re: Polymer running on Sentinel-2 images

Post by jcardoso »

Dear François,

I am trying to run the MSI processing using Polymer-v4.12, Ubuntu 18.04.

I am running the following command for one MSI tile:

Code: Select all

./polymer_cli.py /home/jcardoso/msi_L1/S2B_MSIL1C_20190421T105039_N0207_R051_T31UES_20190421T125156.SAFE/GRANULE/L1C_T31UES_A011088_20190421T105033/ /home/jcardoso/msi_L2/S2B_MSIL1C_20190421T105039_N0207_R051_T31UES_POLYMER.nc
I also ran using the -fmt option, I wrote python and python3 before the polymer_cli.py, I added the full path to the polymer_cli.py file. I created the polymer environment using Anaconda and also manually. I used the python3.8 and the python3.7. Nothing solved my issue.

The error message is:
Starting processing at 2020-03-18 10:46:56.256747
Initialize MSI projection EPSG:32631
/home/jcardoso/miniconda3/envs/poly/lib/python3.7/site-packages/pyproj/crs/crs.py:280: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable ... -in-proj-6
projstring = _prepare_from_string(projparams)
Traceback (most recent call last):
File "./polymer_cli.py", line 35, in <module>
Level2(filename=args.output_file, fmt=args.fmt))
File "/home/jcardoso/polymer-v4.12/polymer/main.py", line 496, in run_atm_corr
with level2 as l2, level1 as l1:
File "/home/jcardoso/polymer-v4.12/polymer/level1.py", line 106, in __enter__
self.level1 = L1(self.filename, **self.kwargs)
File "/home/jcardoso/polymer-v4.12/polymer/level1_msi.py", line 151, in __init__
self.init_ancillary()
File "/home/jcardoso/polymer-v4.12/polymer/level1_msi.py", line 180, in init_ancillary
self.ozone = self.ancillary.get('ozone', self.date)
File "/home/jcardoso/polymer-v4.12/polymer/ancillary.py", line 278, in get
return self.read(param, res)
File "/home/jcardoso/polymer-v4.12/polymer/ancillary.py", line 190, in read
hdf = SD(filename)
File "/home/jcardoso/miniconda3/envs/poly/lib/python3.7/site-packages/pyhdf/SD.py", line 1444, in __init__
_checkErr('SD', id, "cannot open %s" % path)
File "/home/jcardoso/miniconda3/envs/poly/lib/python3.7/site-packages/pyhdf/error.py", line 23, in _checkErr
raise HDF4Error(err)
pyhdf.error.HDF4Error: SD (15): File is supported, must be either hdf, cdf, netcdf
Do you have any suggestion?
Thank you in advance for the help.
PS: was running with the Polymer-v4.11.

Joao Santos.
jcardoso
Posts: 12
Joined: Fri Feb 08, 2019 10:17 am
company / institution: National Institute for Space Research
Location: Brazil

Re: Polymer running on Sentinel-2 images

Post by jcardoso »

Dear François,

I am able to run the MSI tiles using the ERA5 ancillary data as suggested in the post "Sanity check for ancillary data". I was not able to use the NASA ancillary data by changing the line inside the ancillary.py file, but it's fine.

Sorry for didn't see the topic with the same question before written a new post.

Regards,
Joao Santos.
User avatar
fsteinmetz
Site Admin
Posts: 314
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Polymer running on Sentinel-2 images

Post by fsteinmetz »

Dear Joao,
Good to see that you could fix it !
There will be a fix in next version.
Cheers,
François
lanzhiqishi
Posts: 45
Joined: Fri May 22, 2020 7:00 pm
company / institution: university of maine
Location: maine

Re: Polymer running on Sentinel-2 images

Post by lanzhiqishi »

run example.py for MSI


code:
run_atm_corr(
Level1_MSI('/Users/Binbin/Documents/sentinel-2/S2B20180807/GRANULE/L1C_T50RQQ_A007408_20180807T024116/',
resolution='20' # if not provided, use the 60m resolution
),
Level2_NETCDF('20180807_POLYMER.nc') # by default, output in the same folder as input
)

error result:
Traceback (most recent call last):
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/example copy.py", line 4, in <module>
from polymer.main import run_atm_corr, Level1, Level2
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/polymer/main.py", line 18, in <module>
from polymer.polymer_main import PolymerMinimizer
ModuleNotFoundError: No module named 'polymer.polymer_main'



Could you help me
User avatar
fsteinmetz
Site Admin
Posts: 314
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Polymer running on Sentinel-2 images

Post by fsteinmetz »

Please make sure that you have properly compiled the code. The module polymer_main is written in cython, so it needs to be compiled.
lanzhiqishi
Posts: 45
Joined: Fri May 22, 2020 7:00 pm
company / institution: university of maine
Location: maine

Re: Polymer running on Sentinel-2 images

Post by lanzhiqishi »

Dear François,

i could not know how to complied. just use conda install the dependencies and ploymer. and now follow the "ReadME" file and your suggestion, and download all auxdata from the website and "make". after that, run the code

error result:
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/example copy.py", line 4, in <module>
from polymer.main import run_atm_corr, Level1, Level2
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/polymer/main.py", line 8, in <module>
from polymer.luts import read_mlut_hdf, Idx
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/polymer/luts.py", line 20, in <module>
from scipy.interpolate import interp1d
ModuleNotFoundError: No module named 'scipy'


Thank you


Best


Binbin
Post Reply