Skip to content

Fix race-condition when dealing with DNS lookups #130

@GUI

Description

@GUI

This was discovered and fixed yesterday. There was a race condition in how we were resolving the IP address of backend hostnames and then writing those IPs to new configuration files for nginx. This temporarily led to us routing a specific API backend to nowhere, which is quite bad. It would eventually restore itself on its own, but I believe this should now be fixed so this race condition shouldn't be possible.

The race condition was related to a DNS lookup occurring at the exact same time that we were writing a new config file with other DNS changes that had occurred. Given the timings of things, this was more likely to happen for domain name backends that have short TTLs, but then I think we only recently saw this because it was also more likely to occur as we add more and more backends (since there's more simultaneous DNS lookups or changes going on).

Fixed in: NREL/api-umbrella-gatekeeper@dc73bae

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions