Skip to content

HTTPError when downloading AHN #377

@OnnoEbbens

Description

@OnnoEbbens

The eclipse server, that is used to get ahn tiles, seems to be unreliable. I often get a HTTPError but I don't always get it. My gut feeling is that I get it about 50% of the time which is too much for me. I know there has been some updates to the get_ahn function recently (#371, #370). Maybe @bdestombe and @rubencalje know an easy fix for this?

Full error:

HTTPError                                 Traceback (most recent call last)
Cell In[20], [line 2](vscode-notebook-cell:?execution_count=20&line=2)
      [1](vscode-notebook-cell:?execution_count=20&line=1) # get ahn
----> [2](vscode-notebook-cell:?execution_count=20&line=2) ds["ahn"] = nlmod.read.ahn.get_ahn(ds, cachedir=resultdir, cachename="ahn")["ahn"]

File ~\02_python\nlmod\nlmod\cache.py:236, in cache_netcdf.<locals>.decorator.<locals>.wrapper(cachedir, cachename, *args, **kwargs)
    [233](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:233)         return cached_ds
    [235](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:235) # create cache
--> [236](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:236) result = func(*args_adj, **kwargs_adj)
    [237](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:237) logger.info(f"caching data -> {cachename}")
    [239](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:239) if isinstance(result, xr.DataArray):
    [240](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:240)     # set the DataArray as a variable in a new Dataset

File ~\02_python\nlmod\nlmod\read\ahn.py:60, in get_ahn(ds, identifier, method, extent, **kwargs)
     [58](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:58) if extent is None and ds is not None:
     [59](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:59)     extent = get_extent(ds)
---> [60](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:60) ahn_ds_raw = _get_ahn_ellipsis(extent, identifier=identifier, **kwargs)
     [61](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:61) ahn_ds_raw = ahn_ds_raw.drop_vars("band")
     [63](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:63) if ds is None:

File ~\02_python\nlmod\nlmod\cache.py:123, in cache_netcdf.<locals>.decorator.<locals>.wrapper(cachedir, cachename, *args, **kwargs)
    [119](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:119) @functools.wraps(func)
    [120](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:120) def wrapper(*args, cachedir=None, cachename=None, **kwargs):
    [121](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:121)     # 1 check if cachedir and name are provided
    [122](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:122)     if cachedir is None or cachename is None:
--> [123](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:123)         return func(*args, **kwargs)
    [125](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:125)     if not cachename.endswith(".nc"):
    [126](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/cache.py:126)         cachename += ".nc"

File ~\02_python\nlmod\nlmod\read\ahn.py:576, in _get_ahn_ellipsis(extent, identifier, **kwargs)
    [559](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:559) @cache.cache_netcdf()
    [560](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:560) def _get_ahn_ellipsis(extent, identifier="AHN5_5M_M", **kwargs):
    [561](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:561)     """Download AHN5.
    [562](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:562) 
    [563](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:563)     Parameters
   (...)
    [574](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:574)         DataArray of the AHN
    [575](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:575)     """
--> [576](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:576)     tiles = _get_tiles_ellipsis(extent=extent, **kwargs)
    [577](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:577)     if identifier not in tiles.columns:
    [578](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:578)         raise (ValueError(f"Unknown ahn-identifier: {identifier}"))

File ~\02_python\nlmod\nlmod\read\ahn.py:314, in _get_tiles_ellipsis(extent, crs, timeout, base_url, path_id, timestamp_id)
    [312](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:312) r = requests.get(url, timeout=timeout)
    [313](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:313) if not r.ok:
--> [314](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:314)     raise (HTTPError(f"Request not successful: {r.url}"))
    [315](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:315) gdf = gpd.GeoDataFrame.from_features(r.json()["result"]["features"], crs=4326)
    [316](https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/oebbe/02_python/gxg_berging/~/02_python/nlmod/nlmod/read/ahn.py:316) gdf = gdf.to_crs(crs)

HTTPError: Request not successful: https://api.ellipsis-drive.com/v3/path/a9d410ad-a2f6-404c-948a-fdf6b43e77a6/vector/timestamp/87a21a71-c39f-4e92-a43b-207bc7dfe714/listFeatures

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions