Ancilliary data from Sentinel 3 products

Post Reply
patilla7
Posts: 5
Joined: Wed Apr 17, 2019 9:37 am
company / institution: University of Valencia
Location: Spain

Ancilliary data from Sentinel 3 products

Post by patilla7 »

Dear Francois,

First of all, thank you very much for developing POLYMER. This is my first time working with it, and I'm also new to Python. I'm wondering if there is a way for Polymer to use the ancillary data provided by the Sentinel 3 annotation data file instead of the data from external providers. Even though I set the .netrc file and could connect to the Earthdata server, I think I've been facing problems downloading the ancillary data.:

Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:22:57 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:02 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:08 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:14 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:20 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:26 ERROR 404: Not Found.
failure (2048)
Starting processing at 2023-04-14 12:23:26.645831
Removing file /home/purrego/S3_AC/S3_L2_Polymer/Spain/Alarcon/S3A_OL_2_EFR_20200702_0179_040_336_2340_LN1_O_NT_002.nc
Initializing output file "/home/purrego/S3_AC/S3_L2_Polymer/Spain/Alarcon/S3A_OL_2_EFR_20200702_0179_040_336_2340_LN1_O_NT_002.nc"
Processing block: size (100, 132), offset (0, 0)
Processing block: size (100, 132), offset (100, 0)
Processing block: size (18, 132), offset (200, 0)
Done in 0:00:08.730455

In the end, I was able to obtain an L2 product, but I'm not sure if POLYMER was able to download the ancillary data or where it is taking it from. I would really appreciate your comments.

Best regards,
Patricia
Kzy
Posts: 4
Joined: Mon Dec 11, 2023 10:38 am
company / institution: SYSU
Location: Guangzhou, China

Re: Ancilliary data from Sentinel 3 products

Post by Kzy »

I'm searching for help as well. I set a .netrc file as well, but I can't authenticate to EarthData. The way I set .netrc file is learned in this website https://urs.earthdata.nasa.gov/document ... l_and_wget. Can you tell me how to set .netrc file correctly? And which folder the files should be in? Thanks for any possible help.
patilla7
Posts: 5
Joined: Wed Apr 17, 2019 9:37 am
company / institution: University of Valencia
Location: Spain

Re: Ancilliary data from Sentinel 3 products

Post by patilla7 »

Hello,

You just need to configure the .netrc file by opening a terminal and following these steps:
> cd ~
> touch .netrc
> echo "machine urs.earthdata.nasa.gov login USERNAME password PASSWORD"
> chmod 0600 .netrc

Change USERNAME and PASSWORD according to your registration. Hope this helps.

Kind regards,
Patricia
sydney.baratta
Posts: 11
Joined: Tue Aug 08, 2023 6:40 pm
company / institution: University of Maine
Location: Orono

Re: Ancilliary data from Sentinel 3 products

Post by sydney.baratta »

I'm running into the same original problem as you, where I am getting 404 errors when trying to pull in the met data from Oceandata. I was told that "Those files are NRT (quick-look) MET data and are replaced ~45 days later with refined MET with file names like, 'GMAO_MERRA2.20220401T000000.MET.nc'" - did you end up finding a fix for this? Or does anyone know how to call in up-to-date files?

Thanks!
sydney.baratta
Posts: 11
Joined: Tue Aug 08, 2023 6:40 pm
company / institution: University of Maine
Location: Orono

Re: Ancilliary data from Sentinel 3 products

Post by sydney.baratta »

Update - I fixed this (I believe) by changing lines 29 and 37 in ancillary.py to read:

lambda date: [('GMAO_MERRA2.%Y%m%dT%H0000.MET.nc', d)

instead of

lambda date: [('GMAO_FP.%Y%m%dT%H0000.MET.NRT.nc', d)
mfestuary
Posts: 4
Joined: Thu Oct 28, 2021 7:35 pm
company / institution: EPA
Location: RTP, NC

Re: Ancilliary data from Sentinel 3 products

Post by mfestuary »

Thank you for posting this! I was experiencing the same problem with Sentinel2A processing, but this hack fixes it.
sydney.baratta
Posts: 11
Joined: Tue Aug 08, 2023 6:40 pm
company / institution: University of Maine
Location: Orono

Re: Ancilliary data from Sentinel 3 products

Post by sydney.baratta »

Happy to help!
User avatar
fsteinmetz
Site Admin
Posts: 306
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: Ancilliary data from Sentinel 3 products

Post by fsteinmetz »

Solved in the latest version of Polymer, now available on github at https://github.com/hygeos/polymer
Post Reply