Page 1 of 2

Polymer running on Sentinel-2 images

Posted: Wed Dec 18, 2019 3:38 pm
by dfrgttn
Hello!

I installed anaconda, created an environment and installed required dependencies.
I am trying to run Polymer on Sentinel-2 images, however, this does not work to me:

Code: Select all

./polymer_cli.py S2A_MSIL1C_20170502T093041_N0205_R136_T35UMB_20170502T093358.SAFE
I received an error message:

Code: Select all

Traceback (most recent call last):
  File "./polymer_cli.py", line 11, in <module>
    from polymer.main import run_atm_corr, Level1, Level2
  File "/home/dalia/Documents/polymer-v4.12/polymer/main.py", line 18, in <module>
    from polymer.polymer_main import PolymerMinimizer
ImportError: /home/dalia/Documents/polymer-v4.12/polymer/polymer_main.so: undefined symbol: _Py_ZeroStruct
Then I tried to run through python:

Code: Select all

Level1_MSI('S2A_MSIL1C_20170502T093041_N0205_R136_T35UMB_20170502T093358.SAFE')
I received an error about a line 122 in file level1_msi.py as there are two xmls in current S2 products. So I removed this line and changed the line 123 to

Code: Select all

xmlfile = xmlfiles[1]
But then I get errors about the line 125 where Sensing time should be retrieved and I tried inserting "Product_start_time" that is found in the MTD_MSIL1C.xml file.

Could you help me? Or maybe you have old formatted metadata file?
I use linux 18.04.3 LTS and Polymer 4.12.

Kind regards,
Dalia

Re: Polymer running on Sentinel-2 images

Posted: Fri Dec 20, 2019 8:14 pm
by fsteinmetz
Dear Dalia,

It seems that you may be using different versions of python for the compilation and the execution. Please make sure you are using python3 for both and use the following commands :
  • python --version (to verify that you are using python 3)
  • make rebuild
  • python polymer_cli.py (Just in case ./polymer_cli.py would use another interpreter for some reason)
If this still fails, could you send the output of conda list ?

Kind regards,
François

Re: Polymer running on Sentinel-2 images

Posted: Mon Dec 30, 2019 10:03 am
by dfrgttn
Thank you, François, for an answer.
I managed to run Polymer in python. It runs well and produces a result, but I get a warning like this:

Code: Select all

FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
Should I be worried?
I use python 3.8.0.

Best regards,
Dalia

Re: Polymer running on Sentinel-2 images

Posted: Mon Dec 30, 2019 10:22 pm
by fsteinmetz
Good news !
No, this is just a warning, you can safely ignore it.
Cheers,
François

Re: Polymer running on Sentinel-2 images

Posted: Wed Mar 18, 2020 11:18 am
by jcardoso
Dear François,

I am trying to run the MSI processing using Polymer-v4.12, Ubuntu 18.04.

I am running the following command for one MSI tile:

Code: Select all

./polymer_cli.py /home/jcardoso/msi_L1/S2B_MSIL1C_20190421T105039_N0207_R051_T31UES_20190421T125156.SAFE/GRANULE/L1C_T31UES_A011088_20190421T105033/ /home/jcardoso/msi_L2/S2B_MSIL1C_20190421T105039_N0207_R051_T31UES_POLYMER.nc
I also ran using the -fmt option, I wrote python and python3 before the polymer_cli.py, I added the full path to the polymer_cli.py file. I created the polymer environment using Anaconda and also manually. I used the python3.8 and the python3.7. Nothing solved my issue.

The error message is:
Starting processing at 2020-03-18 10:46:56.256747
Initialize MSI projection EPSG:32631
/home/jcardoso/miniconda3/envs/poly/lib/python3.7/site-packages/pyproj/crs/crs.py:280: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable ... -in-proj-6
projstring = _prepare_from_string(projparams)
Traceback (most recent call last):
File "./polymer_cli.py", line 35, in <module>
Level2(filename=args.output_file, fmt=args.fmt))
File "/home/jcardoso/polymer-v4.12/polymer/main.py", line 496, in run_atm_corr
with level2 as l2, level1 as l1:
File "/home/jcardoso/polymer-v4.12/polymer/level1.py", line 106, in __enter__
self.level1 = L1(self.filename, **self.kwargs)
File "/home/jcardoso/polymer-v4.12/polymer/level1_msi.py", line 151, in __init__
self.init_ancillary()
File "/home/jcardoso/polymer-v4.12/polymer/level1_msi.py", line 180, in init_ancillary
self.ozone = self.ancillary.get('ozone', self.date)
File "/home/jcardoso/polymer-v4.12/polymer/ancillary.py", line 278, in get
return self.read(param, res)
File "/home/jcardoso/polymer-v4.12/polymer/ancillary.py", line 190, in read
hdf = SD(filename)
File "/home/jcardoso/miniconda3/envs/poly/lib/python3.7/site-packages/pyhdf/SD.py", line 1444, in __init__
_checkErr('SD', id, "cannot open %s" % path)
File "/home/jcardoso/miniconda3/envs/poly/lib/python3.7/site-packages/pyhdf/error.py", line 23, in _checkErr
raise HDF4Error(err)
pyhdf.error.HDF4Error: SD (15): File is supported, must be either hdf, cdf, netcdf
Do you have any suggestion?
Thank you in advance for the help.
PS: was running with the Polymer-v4.11.

Joao Santos.

Re: Polymer running on Sentinel-2 images

Posted: Thu Mar 19, 2020 4:40 pm
by jcardoso
Dear François,

I am able to run the MSI tiles using the ERA5 ancillary data as suggested in the post "Sanity check for ancillary data". I was not able to use the NASA ancillary data by changing the line inside the ancillary.py file, but it's fine.

Sorry for didn't see the topic with the same question before written a new post.

Regards,
Joao Santos.

Re: Polymer running on Sentinel-2 images

Posted: Thu Mar 19, 2020 9:23 pm
by fsteinmetz
Dear Joao,
Good to see that you could fix it !
There will be a fix in next version.
Cheers,
François

Re: Polymer running on Sentinel-2 images

Posted: Mon Jun 22, 2020 7:16 pm
by lanzhiqishi
run example.py for MSI


code:
run_atm_corr(
Level1_MSI('/Users/Binbin/Documents/sentinel-2/S2B20180807/GRANULE/L1C_T50RQQ_A007408_20180807T024116/',
resolution='20' # if not provided, use the 60m resolution
),
Level2_NETCDF('20180807_POLYMER.nc') # by default, output in the same folder as input
)

error result:
Traceback (most recent call last):
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/example copy.py", line 4, in <module>
from polymer.main import run_atm_corr, Level1, Level2
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/polymer/main.py", line 18, in <module>
from polymer.polymer_main import PolymerMinimizer
ModuleNotFoundError: No module named 'polymer.polymer_main'



Could you help me

Re: Polymer running on Sentinel-2 images

Posted: Tue Jun 23, 2020 8:35 am
by fsteinmetz
Please make sure that you have properly compiled the code. The module polymer_main is written in cython, so it needs to be compiled.

Re: Polymer running on Sentinel-2 images

Posted: Tue Jun 23, 2020 8:52 pm
by lanzhiqishi
Dear François,

i could not know how to complied. just use conda install the dependencies and ploymer. and now follow the "ReadME" file and your suggestion, and download all auxdata from the website and "make". after that, run the code

error result:
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/example copy.py", line 4, in <module>
from polymer.main import run_atm_corr, Level1, Level2
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/polymer/main.py", line 8, in <module>
from polymer.luts import read_mlut_hdf, Idx
File "/Users/Binbin/Documents/pythondepen/polymer-v4.13/polymer/luts.py", line 20, in <module>
from scipy.interpolate import interp1d
ModuleNotFoundError: No module named 'scipy'


Thank you


Best


Binbin