Sanity check for ancillary data

[blake]

Hi,

Is this repeatable for anyone else (i am using 4.11) or have I done something else wrong?

When polymer goes to download the ancillary data required for S2MSI processing from nasa ocean colour I am only receiving the header file containing no data. (size 12.7kb) the normal is around 200kb for the ozone data. So polymer thinks it has the data then moves on until it attempts to read the file then errors out with a HDF read error as the file is not complete so cannot read it. Equally because it has the file (or thinks it does) it will not move onto any of the secondary options for download.

Thanks.

[fsteinmetz]

Hi Blake, and thanks for your feedback.

Indeed, you have done nothing wrong.
The problem is that downloading on oceancolor.gsfc.nasa.gov now (since January) requires an EarthData login. There will need to be a modification in polymer/ancillary.py to reflect this change.
In the meantime, I suggest you use polymer/ancillary_era5.py as an alternate source of ancillary data.
This can be done like so:

Code: Select all

from polymer.ancillary_era5 import Ancillary_ERA5
run_atm_corr(
    Level1_MSI(<S2_product>, ancillary=Ancillary_ERA5()),
    Level2...
    )

You will have to use a ECMWF login as described here : https://cds.climate.copernicus.eu/api-how-to

Cheers,
François

[blake]

Hi François,

Many thanks for your response but I am trying to keep everything the same so would prefer to use the nasa data.

by editing line 295 in anciallry.py to :

cmd = 'wget --user=insert_username --password=insert_password --auth-no-challenge=on -nv {} -O {} '.format(url, target+'.tmp')

Will make it run correctly however my understanding of using "--auth-no-challenge=on" is that it will send your username and password in plain text unencrypted for authorisation because that is they only way it will work with nasa ocean colour. So long as no one is re-using password then its maybe not so bad. Hopefully there is a better way for long term fix (and more elegant).

Cheers

[fsteinmetz]

Great, thanks for sending your solution !
It's not so bad as you say, but it will need a fix for the long term.
Cheers

[fsteinmetz]

Fixed in v4.13.