Move away from raw.githubusercontent.com

[jonasmalacofilho]

In may GitHub lowered the rate limit for unauthenticated users to 60 requests/hour, which is pretty tight. And, in practice, that very low limit also seems to apply even to authenticated raw.githubusercontent.com requests.

Now raw.githubusercontent.com has been inaccessible from South America for most of the past 5 days, either due to a misconfigured rate limit = 0 or a straight up geoblock.

While a partial workaround is to locally change the import URLs to use jsDelivr, cargo vet still relies on the canonical location of the registry, as well as any URLs in it, for suggestions.

Therefore, could we move both the registry and its entries off of raw.githubusercontent.com?

Serving them through jsDelivr (or another similar service), if acceptable, would be the simplest approach, since it wouldn't require cooperation from the audit providers, or use of the GitHub API.

[bholley]

The advantage of using githubusercontent is that it avoids a dependency on a separate service that could be a vector for compromise.

I didn't know about the regional availability issues, will flag it with some contacts to see if there's anything that can be done.

[jonasmalacofilho]

@bholley,

Your concern is something I also considered. Although in my own risk analysis I concluded that it was acceptable, since the actual crates would not come from the same, potentially compromised, source. Still, your point holds.

An alternative would be for each audit set provider to publish their audits somewhere other than raw.githubusercontent.com themselves. However, that would make it harder to vet the audit set itself, since its git history would no longer be tied to what would be fetched.

---

I decided to wait a few weeks before replying to see how frequent the issues with raw.githubusercontent.com were. And in the past month and half South America was blocked a couple of times, once for maybe half day, the other time for 4 days. Besides the discussion I linked before, I've collected some pings from globalping.io to check that the issue was not local to my specific location.

So, still an issue, but also usable for now. That said, the official (or as close to official as there is) rate limit is pretty low and once GitHub actually starts enforcing it, it would effectively break cargo vet for everyone.