fix(Terrain): back off after failed tile fetches to stop warning spam#14414
fix(Terrain): back off after failed tile fetches to stop warning spam#14414dakejahl wants to merge 2 commits into
Conversation
When the elevation server returns 500 for an uncached tile (e.g. uninitialized lat=0/lon=0 from a vehicle TERRAIN_REQUEST hitting Null Island), the prior code re-issued the request on every TerrainProtocolHandler timer tick (~12 Hz), producing continuous warning spam from QGeoTiledMapReplyQGC and TerrainTileManager::_terrainDone. - TerrainTileManager: remember failed tile hashes with a 5 s backoff. While a tile is in the failed set, getAltitudesForCoordinates returns NaN + error=true without dispatching a new network request. Demote repeat warnings to debug within the backoff window. - TerrainProtocolHandler: clear the mask bit on error too. Looping on a known-failed tile just hammers the same request every tick; the vehicle re-issues TERRAIN_REQUEST if it still needs the data, allowing a fresh attempt once the backoff expires.
Autopilots can emit TERRAIN_REQUEST before GPS lock with uninitialized (0, 0) coordinates. Bail out at the protocol entry point so we never even start a fetch for that tile, complementing the failed-tile backoff in TerrainTileManager.
|
Corresponding Ardupilot PR which fixes the root cause of the issue |
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (9.67%) is below the target coverage (30.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## master #14414 +/- ##
==========================================
+ Coverage 25.47% 26.44% +0.97%
==========================================
Files 769 767 -2
Lines 65912 66308 +396
Branches 30495 30682 +187
==========================================
+ Hits 16788 17536 +748
+ Misses 37285 36327 -958
- Partials 11839 12445 +606
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 104 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
Build ResultsPlatform Status
All builds passed. Pre-commit
Pre-commit hooks: 4 passed, 45 failed, 7 skipped. Test Resultslinux-coverage: 88 passed, 0 skipped Code CoverageCoverage: 60.4% No baseline available for comparison Artifact Sizes
Updated: 2026-05-20 22:37:22 UTC • Triggered by: Android |
1 participant
Summary
Eliminate the warning spam (and unnecessary network traffic) when the elevation server cannot return data for a tile, by caching recent failures in
TerrainTileManagerand not looping on a known-failed tile inTerrainProtocolHandler.Problem
When a vehicle sends
TERRAIN_REQUESTwith uninitialized lat=0/lon=0, the request maps to elevation tile (0, 0) — "Null Island". The Auterion terrain server returns HTTP 500 for that tile, butTerrainTileManagerhad no memory of recent failures, so everyTerrainProtocolHandlertimer tick (~12 Hz) dispatched a fresh HTTP request. Observed log shows continuous warnings from bothQtLocationPlugin.QGeoTiledMapReplyQGCandTerrain.TerrainTileManager::_terrainDoneat ~80–150 ms intervals:The handler also intentionally never cleared the mask bit on a non-success result, so even with backoff added, it would just transfer the spam to its own log line.
Solution
src/Terrain/TerrainTileManager.{h,cc}_failedTiles(tile hash → ms timestamp) with a 5 s backoff.getAltitudesForCoordinatesreturns NaN +error=truewithout dispatching a network request._terrainDonerecords the hash on failure, removes it on success, and rate-limits the warning (first failure at warning level; repeats inside the window at debug level).src/Vehicle/TerrainProtocolHandler.cc_sendTerrainDatanow clears the mask bit onerroras well as on success, so the handler stops looping on a known-failed tile. The vehicle re-issuesTERRAIN_REQUESTif it still needs the data, which then gets a fresh fetch once the backoff window expires.TerrainTileManager.