Ancilliary data from Sentinel 3 products

[patilla7]

Dear Francois,

First of all, thank you very much for developing POLYMER. This is my first time working with it, and I'm also new to Python. I'm wondering if there is a way for Polymer to use the ancillary data provided by the Sentinel 3 annotation data file instead of the data from external providers. Even though I set the .netrc file and could connect to the Earthdata server, I think I've been facing problems downloading the ancillary data.:

Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:22:57 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:02 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:08 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:14 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:20 ERROR 404: Not Found.
failure (2048)
Trying to download https://oceandata.sci.gsfc.nasa.gov/cgi ... MET.NRT.nc ...
https://oceandata.sci.gsfc.nasa.gov/ob/ ... MET.NRT.nc:
2023-04-14 12:23:26 ERROR 404: Not Found.
failure (2048)
Starting processing at 2023-04-14 12:23:26.645831
Removing file /home/purrego/S3_AC/S3_L2_Polymer/Spain/Alarcon/S3A_OL_2_EFR_20200702_0179_040_336_2340_LN1_O_NT_002.nc
Initializing output file "/home/purrego/S3_AC/S3_L2_Polymer/Spain/Alarcon/S3A_OL_2_EFR_20200702_0179_040_336_2340_LN1_O_NT_002.nc"
Processing block: size (100, 132), offset (0, 0)
Processing block: size (100, 132), offset (100, 0)
Processing block: size (18, 132), offset (200, 0)
Done in 0:00:08.730455

In the end, I was able to obtain an L2 product, but I'm not sure if POLYMER was able to download the ancillary data or where it is taking it from. I would really appreciate your comments.

Best regards,
Patricia

[Kzy]

I'm searching for help as well. I set a .netrc file as well, but I can't authenticate to EarthData. The way I set .netrc file is learned in this website https://urs.earthdata.nasa.gov/document ... l_and_wget. Can you tell me how to set .netrc file correctly? And which folder the files should be in? Thanks for any possible help.

[patilla7]

Hello,

You just need to configure the .netrc file by opening a terminal and following these steps:
> cd ~
> touch .netrc
> echo "machine urs.earthdata.nasa.gov login USERNAME password PASSWORD"
> chmod 0600 .netrc

Change USERNAME and PASSWORD according to your registration. Hope this helps.

Kind regards,
Patricia

[sydney.baratta]

I'm running into the same original problem as you, where I am getting 404 errors when trying to pull in the met data from Oceandata. I was told that "Those files are NRT (quick-look) MET data and are replaced ~45 days later with refined MET with file names like, 'GMAO_MERRA2.20220401T000000.MET.nc'" - did you end up finding a fix for this? Or does anyone know how to call in up-to-date files?

Thanks!

[sydney.baratta]

Update - I fixed this (I believe) by changing lines 29 and 37 in ancillary.py to read:

lambda date: [('GMAO_MERRA2.%Y%m%dT%H0000.MET.nc', d)

instead of

lambda date: [('GMAO_FP.%Y%m%dT%H0000.MET.NRT.nc', d)

[mfestuary]

Thank you for posting this! I was experiencing the same problem with Sentinel2A processing, but this hack fixes it.

[sydney.baratta]

Happy to help!

[fsteinmetz]

Solved in the latest version of Polymer, now available on github at https://github.com/hygeos/polymer