Page 1 of 1
Download ancillary data on-the-fly
Posted: Thu Apr 09, 2020 11:50 am
by scheunro
Hi,
How do I configure polymer to make the download of ancillary data work?
I tried executing the code suggested
here before calling polymer but it did not help.
Any help appreciated.
Best,
Rolf
polymer version: 4.13
OS: CentOS 7
wget: 1.14-18.el7_6.1.x86_64
Re: Download ancillary data on-the-fly
Posted: Fri Apr 10, 2020 2:11 pm
by fsteinmetz
Hello Rolf,
The different ancillary data modules (NASA, ERA-Interim, ERA5) should already download the necessary ancillary files by themselves.
Please have a try; the error messages should guide you if any further action is required on your side. If anything is unclear, let me know of the error message.
Cheers,
François
Re: Download ancillary data on-the-fly
Posted: Tue Apr 14, 2020 6:13 am
by scheunro
I am using the NASA module.
From error message it says me and from the code I see it requires a cookie file ".urs_cookie", how do I create this cookie file?
Re: Download ancillary data on-the-fly
Posted: Tue Apr 14, 2020 8:18 am
by scheunro
I found this here:
https://wiki.earthdata.nasa.gov/display ... L+And+Wget
I followed the instructions, but Step 4 does not work. What am I doing wrong?
What is the name of the application to be preauthorized in my
https://urs.earthdata.nasa.gov/ login?
Re: Download ancillary data on-the-fly
Posted: Tue Apr 14, 2020 8:53 pm
by fsteinmetz
What error message do you get ?
With the .netrc set, could you try with the following command ?
Code: Select all
wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge https://oceandata.sci.gsfc.nasa.gov/ob/getfile/N198101000_PROFILE_MERRA2_3h.nc
Re: Download ancillary data on-the-fly
Posted: Wed Apr 15, 2020 6:25 am
by scheunro
Thank you for your reply.
Sadly, there is no error message; it just hangs. After about a minute, I cancel it. To me it looks like the forwarding to
https://urs.earthdata.nasa.gov does not work with my wget.
Following is my output:
Code: Select all
$ wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge https://oceandata.sci.gsfc.nasa.gov/ob/getfile/N198101000_PROFILE_MERRA2_3h.nc
--2020-04-15 06:11:52-- https://oceandata.sci.gsfc.nasa.gov/ob/getfile/N198101000_PROFILE_MERRA2_3h.nc
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... 169.154.128.84, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|169.154.128.84|:443... connected.
^C
And with verbose and debug enabled:
Code: Select all
$ wget -d -v --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge https://oceandata.sci.gsfc.nasa.gov/ob/getfile/N198101000_PROFILE_MERRA2_3h.nc
Setting --verbose (verbose) to 1
Setting --load-cookies (loadcookies) to /home/eouser/.urs_cookies
Setting --save-cookies (savecookies) to /home/eouser/.urs_cookies
Setting --keep-session-cookies (keepsessioncookies) to 1
Setting --auth-no-challenge (authnochallenge) to 1
DEBUG output created by Wget 1.14 on linux-gnu.
URI encoding = ‘UTF-8’
Converted file name 'N198101000_PROFILE_MERRA2_3h.nc' (UTF-8) -> 'N198101000_PROFILE_MERRA2_3h.nc' (UTF-8)
Converted file name 'N198101000_PROFILE_MERRA2_3h.nc' (UTF-8) -> 'N198101000_PROFILE_MERRA2_3h.nc' (UTF-8)
--2020-04-15 06:12:23-- https://oceandata.sci.gsfc.nasa.gov/ob/getfile/N198101000_PROFILE_MERRA2_3h.nc
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... 169.154.128.84, 2001:4d0:2418:128::84
Caching oceandata.sci.gsfc.nasa.gov => 169.154.128.84 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|169.154.128.84|:443... connected.
Created socket 3.
Releasing 0x0000000000b47740 (new refcount 1).
Initiating SSL handshake.
^C
My .netrc looks like following:
Code: Select all
machine urs.earthdata.nasa.gov login scheunro password <super-secret-password>