GSW httperror for 180E tiles

Post Reply
tsilva
Posts: 7
Joined: Tue Nov 02, 2021 5:32 pm
company / institution: CEFAS, UK government
Location: Lowestoft

GSW httperror for 180E tiles

Post by tsilva »

Hi,

When processing S3 OLCI images over the Southern Pacific I get an error when Polymer is downloading the corresponding GSW coastline tile. This has happened to occurrence_180E_10S.tif, occurrence_180E_20S.tif and occurrence_180E_30S.tif. I wonder if this tile_name should be 180W* due to the 180E/180W ambiguity.

As a second question, I have circumvented this problem by using run_atm_corr(Level1_OLCI( ,landmask=None)), but what is the effect of doing so on water pixels?

I am using 4.14 for consistency with previous results. I also have 4.16.1 installed but I didn't see any changes made in gsw.py so I didn't test it.


Thanks!

Code: Select all

polymerpath S3A_OL_1_EFR____20170106T213254.nc/S3A_OL_1_EFR____20170106T213254_20170106T213554_20180327T013414_0179_0
height=4091, width=4865
Downloading https://storage.googleapis.com/global-surface-water/downloads/occurrence/occurrence_180E_10S.tif
Traceback (most recent call last):
  File "/home/users/tsilva/git/river_plume_mapping/jasmin_chain/polymer_fu_regrid.py", line 206, in <module>
    main()
  File "/home/users/tsilva/git/river_plume_mapping/jasmin_chain/polymer_fu_regrid.py", line 158, in main
    _run_polymer(infile, polymerpath, mp=mp)
  File "/home/users/tsilva/git/river_plume_mapping/jasmin_chain/polymer_fu_regrid.py", line 25, in _run_polymer
    Level1_OLCI(str(infile),
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/level1_olci.py", line 68, in Level1_OLCI
    return Level1_SAFE(
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/level1_safe.py", line 101, in __init__
    self.init_landmask()
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/level1_safe.py", line 131, in init_landmask
    self.landmask_data = self.landmask.get(lat, lon)
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/gsw.py", line 183, in get
    data = get_sw(lat, lon, directory=self.directory, agg=self.agg)
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/gsw.py", line 147, in get_sw
    data = read_gsw_tile(tile_name, directory=directory, agg=agg)
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/gsw.py", line 118, in read_gsw_tile
    filename = get_gsw_tile(tile_name, directory=directory, agg=agg, verbose=verbose)
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/gsw.py", line 104, in get_gsw_tile
    aggregate(fetch_gsw_tile(tile_name, verbose=verbose), agg=agg),
  File "/home/users/tsilva/bin/polymer-v4.14/polymer/gsw.py", line 46, in fetch_gsw_tile
    with urlopen(url) as response:
  File "/home/users/tsilva/.conda/envs/polymer/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/users/tsilva/.conda/envs/polymer/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/home/users/tsilva/.conda/envs/polymer/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/home/users/tsilva/.conda/envs/polymer/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/home/users/tsilva/.conda/envs/polymer/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/home/users/tsilva/.conda/envs/polymer/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
User avatar
fsteinmetz
Site Admin
Posts: 306
Joined: Fri Sep 07, 2018 1:34 pm
company / institution: Hygeos
Location: Lille, France
Contact:

Re: GSW httperror for 180E tiles

Post by fsteinmetz »

Hello,

Many thanks for reporting this issue with GSW tiles, we will have a look.
The OLCI products have already a land/water mask in the Level1 products, and it is used by default (landmask='default'). So you shouldn't need to use GSW for OLCI. By passing landmask=None, you disactivate land masking : a lot of land will be flagged as clouds, but processing time wil still be longer and you will get bad pixels over land.
Cheers,
François
Post Reply