Why there are many NaN in processing sentinel-2 ?

dingky
Posts: 22
Joined: Tue Jun 30, 2020 3:56 am
company / institution: China University of Geosciences
Location: China

Why there are many NaN in processing sentinel-2 ?

Post by dingky »

Dear

Hello, I am using polymer-v4.13 to process sentinel-2 data (cloudiness is less than 10%), and the result is that most of the pixels in non-water bodies have NaN values. What is the reason?
Image

Thinks.

Best regards,

ding
You do not have the required permissions to view the files attached to this post.
User avatar
fsteinmetz
Site Admin
Posts: 132
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Why there are many NaN in processing sentinel-2 ?

Post by fsteinmetz »

Dear ding,
Polymer is an atmospheric correction algorithm to retrieve water colour. It is not designed to work over land. With default processing parameters, the cloud mask triggers over most of the land pixels, and polymer returns bad results when it doesn't. An optional land mask (GSW) can be used to properly mask land pixels.
Cheers
dingky
Posts: 22
Joined: Tue Jun 30, 2020 3:56 am
company / institution: China University of Geosciences
Location: China

Re: Why there are many NaN in processing sentinel-2 ?

Post by dingky »

Dear fsteinmetz,


Could you further talk about how to implement GSW and where is the mask input?
Think you very much!


Best regards,

ding
User avatar
fsteinmetz
Site Admin
Posts: 132
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Why there are many NaN in processing sentinel-2 ?

Post by fsteinmetz »

A basic example of GSW usage is :

Code: Select all

from polymer.main import run_atm_corr
from polymer.level2_nc import Level2_NETCDF
from polymer.level1_msi import Level1_MSI
from polymer.gsw import GSW

run_atm_corr(
    Level1_MSI('<L1C product>', landmask=GSW('<gsw tiles storage directory>'))
    Level2_NETCDF(),
)
The tiles are downloaded automatically on the fly.

For more details about each class and its parameters, you should refer to the documentation provided in the source code (the docstring).

Cheers,
François
dingky
Posts: 22
Joined: Tue Jun 30, 2020 3:56 am
company / institution: China University of Geosciences
Location: China

Re: Why there are many NaN in processing sentinel-2 ?

Post by dingky »

Dear fsteinmetz,
Thank you very much。
When I execute the script, I get an error message.

Code: Select all

$ding@ding-virtual-machine:~/polymer-v4.13$ python3 /home/ding/polymer-v4.13/Sentinel2_process.py
The Sentinel2_process.py content here is:

Code: Select all

from polymer.main import run_atm_corr
from polymer.level2_hdf import Level2_hdf
from polymer.level1_msi import Level1_MSI
from polymer.gsw import GSW


run_atm_corr(
    Level1_MSI('/home/ding/S2A_MSIL1C_20170402T024541_N0204_R132_T50RNV_20170402T025302/-
    S2A_MSIL1C_20170402T024541_N0204_R132_T50RNV.SAFE/GRANULE/L1C_T50RNV_A009281_20170402T025302/', landmask=GSW('/home/ding/gsw'))
    
    Level2_hdf(filename='outfile')    #line 14
)

Code: Select all

Starting processing at 2020-09-03 18:44:58.698440
Initialize MSI projection EPSG:32650
/home/ding/.local/lib/python3.8/site-packages/pyproj/crs/crs.py:280: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable/gotchas.html#axis-order-changes-in-proj-6
  projstring = _prepare_from_string(projparams)
Downloading https://storage.googleapis.com/global-surface-water/downloads/occurrence/occurrence_110E_40N.tif
Killed
Is this how to use gsw.

Best,

ding
Last edited by dingky on Thu Sep 03, 2020 12:22 pm, edited 1 time in total.
dingky
Posts: 22
Joined: Tue Jun 30, 2020 3:56 am
company / institution: China University of Geosciences
Location: China

Re: Why there are many NaN in processing sentinel-2 ?

Post by dingky »

Dear fsteinmetz,

When processing sentinel2, I only get the reflectance of the water body after atmospheric correction. The land is all NaN. How do I get the reflectance of the land?

Thank you very much!

Best,

ding
User avatar
fsteinmetz
Site Admin
Posts: 132
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Why there are many NaN in processing sentinel-2 ?

Post by fsteinmetz »

Dear Ding,

Polymer is an atmospheric correction algorithm to retrieve water colour. It is not designed to work over land. With default processing parameters, the cloud mask triggers over most of the land pixels, and polymer returns bad results when it doesn't. An optional land mask (GSW) can be used to properly mask land pixels.

About GSW, yes this is how to use it. The code got killed by your OS for some reason, maybe insufficient RAM.

Cheers
dingky
Posts: 22
Joined: Tue Jun 30, 2020 3:56 am
company / institution: China University of Geosciences
Location: China

Re: Why there are many NaN in processing sentinel-2 ?

Post by dingky »

Dear fsteinmetz,

Thank you for your patience and help.

How to specify the downloaded global-surface-water parameters for run_atm_corr.Because I found that it may be because the global-surface-water data to be downloaded is large, I want to download it and then crop a piece as an input parameter. I don't know how to achieve

Best,

ding
User avatar
fsteinmetz
Site Admin
Posts: 132
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Why there are many NaN in processing sentinel-2 ?

Post by fsteinmetz »

The GSW data is downloaded on the fly, only the necessary tiles, so for a single S2 tile it should not be a large data amount.
The resolution of GSW data can be degraded by a factor agg=2^n, by passing the option agg to GSW (for example, GSW(agg=8). This makes the processing lighter.
Best
dingky
Posts: 22
Joined: Tue Jun 30, 2020 3:56 am
company / institution: China University of Geosciences
Location: China

Re: Why there are many NaN in processing sentinel-2 ?

Post by dingky »

Hello fsteinmetz ,


Thank you very much for your patience and help!
There is no difference between the results I got when using GSW and not using GSW, neither of them has the reflectivity of land.
Can polymer algorithm only perform atmospheric correction on water bodies? Can it perform atmospheric correction on land.


Best regards,

ding
Post Reply