ERROR:cdsapi:Reason: Request returned no data

Post Reply
gbourdin
Posts: 24
Joined: Wed Nov 04, 2020 8:49 pm
company / institution: University of Maine
Location: Maine, USA

ERROR:cdsapi:Reason: Request returned no data

Post by gbourdin »

Hi François,
It seems cdapi in polymer can't find any ancillary data on ERA5 for some NASA satellites in some rare cases:
MODISA: 8 out of 931 images
MODIST: 4 out of 985 images
VIIRSN: 8 of 1165 images
VIIRSJ1: 3 of 431 images

I am subsetting each image to only process a 20x20 pixels zone to save computing time.

For example running T2017330234000 would give me this error:

Code: Select all

ERROR:cdsapi:Message: no data is available within your requested subset
ERROR:cdsapi:Reason:  Request returned no data
ERROR:cdsapi:  Traceback (most recent call last):
ERROR:cdsapi:    File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 59, in handle_request
ERROR:cdsapi:      result = cached(context.method, proc, context, context.args, context.kwargs)
ERROR:cdsapi:    File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached
ERROR:cdsapi:      result = proc(context, *context.args, **context.kwargs)
ERROR:cdsapi:    File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 124, in __call__
ERROR:cdsapi:      return p(*args, **kwargs)
ERROR:cdsapi:    File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 60, in __call__
ERROR:cdsapi:      return self.proc(context, *args, **kwargs)
ERROR:cdsapi:    File "/home/cds/cdsservices/services/mars/mars.py", line 47, in internal
ERROR:cdsapi:      return mars(context, request, **kwargs)
ERROR:cdsapi:    File "/home/cds/cdsservices/services/mars/mars.py", line 19, in mars
ERROR:cdsapi:      execute_mars(context, requests)
ERROR:cdsapi:    File "/home/cds/cdsservices/services/mars/execute_mars.py", line 25, in execute_mars
ERROR:cdsapi:      raise NoDataException("Request returned no data", '')
ERROR:cdsapi:  cdsinf.exceptions.NoDataException: Request returned no data
Traceback (most recent call last):
  File "POLYbatchL2.py", line 424, in process_NASA_L1C_to_L2
    ancillary=Ancillary_ERA5(anc_dir)),
  File "/home/gbourdin/polymer-v4.13/polymer/level1_nasa.py", line 308, in __init__
    filename, sensor='MODIS', **kwargs)
  File "/home/gbourdin/polymer-v4.13/polymer/level1_nasa.py", line 124, in __init__
    self.init_ancillary()
  File "/home/gbourdin/polymer-v4.13/polymer/level1_nasa.py", line 130, in init_ancillary
    self.ozone = self.ancillary.get('ozone', self.date())
  File "/home/gbourdin/polymer-v4.13/polymer/ancillary_era5.py", line 57, in get
    file1 = self.ERA5.download_era5(t1)
  File "/home/gbourdin/polymer-v4.13/polymer/ancillary_era5.py", line 168, in download_era5
    target_tmp)
  File "/home/gbourdin/.conda/envs/Polymer/lib/python3.7/site-packages/cdsapi/api.py", line 317, in retrieve
    result = self._api('%s/resources/%s' % (self.url, name), request, 'POST')
  File "/home/gbourdin/.conda/envs/Polymer/lib/python3.7/site-packages/cdsapi/api.py", line 458, in _api
    raise Exception("%s. %s." % (reply['error'].get('message'), reply['error'].get('reason')))
Exception: no data is available within your requested subset. Request returned no data.
I noticed there was a new version of polymer but I had to run this analysis quickly without having to reinstall polymer, and according to the changelog, there was no update on the Ancillary_ERA5 module, am I right?

Best,
Guillaume
gbourdin
Posts: 24
Joined: Wed Nov 04, 2020 8:49 pm
company / institution: University of Maine
Location: Maine, USA

Re: ERROR:cdsapi:Reason: Request returned no data

Post by gbourdin »

I have set my program to create an ancillary folder for each image separately and found out that polymer is creating a folder dated to 1970/01/01

Code: Select all

ANCILLARY/ERA5/A2017315041000/1970/01/01/
for all the image the cdapi can't find ancillary data for:

Code: Select all

A2017315041000/
A2018035061500/
A2018043052500/
A2018101042500/
A2018180233000/
A2018182214000/
A2018185203500/
A2018205214500/
T2016281193000/
T2017090020500/
T2017330234000/
T2018181192000/
V2016185193000/
V2016242194800/
V2017010024800/
V2017038023000/
V2018006031800/
V2018051065400/
V2018072043000/
V2018194223000/
V2018261163000/
V2018261190000/
V2018270161200/
Best,
Guillaume
User avatar
fsteinmetz
Site Admin
Posts: 306
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: ERROR:cdsapi:Reason: Request returned no data

Post by fsteinmetz »

Hello Guillaume,

I have tried, and could process at least the first one A2017315041000 with Ancillary_ERA5 and polymer v4.14 (indeed, no change in Ancillary_ERA5 since v4.13) and seadas ocssw v2022.0 for L1C generation (a minor change was needed in make_L1C.py to account for changes with respect to scripts naming in recent ocssw - just in case, I am attaching it here since it was not included in latest release).
By subsetting, you mean providing the coordinates (sline, eline, srow, erow) to Ancillary_NASA, right ?
Cheers,
François
You do not have the required permissions to view the files attached to this post.
gbourdin
Posts: 24
Joined: Wed Nov 04, 2020 8:49 pm
company / institution: University of Maine
Location: Maine, USA

Re: ERROR:cdsapi:Reason: Request returned no data

Post by gbourdin »

Hi François,
I may have found the problem.
I am only processing the zone of interest of the L1A file into L1C using the same sline, eline, srow and erow on l2gen. The output file are usually ~600K except the ones that make the ancillary request crash on POLYMER. Those ones are ~200K so the zone I am processing must be close to the boundary of the scene. I just opened one L1C file (attached here
A2017315041000.L1C_LAC_OC.nc.zip
) and there is 2x24 pixels.
The zone is so small that I suspect this is what makes the ancillary request crash on POLYMER.
Cheers,
Guillaume
You do not have the required permissions to view the files attached to this post.
Post Reply