feat(LocalFeedRepository): fetch only enabled channel tabs #6941
Conversation
Updates the local feed extraction to only fetch channel tabs that are enabled. This cuts down the amount of requests that are send, but ultimately not used.
There was a problem hiding this comment.
Thank you!
Your implementation looks promising, I'll see what I can re-use for my implementation :)
My idea was to reduce the chances of being blocked by YouTube by refreshing the feed only once a day or so and cache it in the app's database, but we probably should add more measures to decrease chances of being blocked as much as possible.
I've also added some throttling, similar to how it is done in NewPipe, a couple of weeks ago, however I think the values need to be adjusted, since it, I still experience some rate-limiting. |
True, there's certainly a lot that could be done there, some throttling is probably a good idea too, feel free to open a PR for that 👍
Hmm, but we probably can't really automate guessing how often a channel uploads videos reliably, so users would need to set that manually 🤔 The best thing to avoid rate limits is probably to not fetch the whole feed at once but in small parts instead, but I'm not too sure how we could really implement that with a decent UX. Users probably want their feed to show all new videos when they refresh it, so fetching only like 10% of the channels subscribed to each day (so users always have a feed that's 10 days old for some channels that haven't been recently refreshed in this example) is also not a perfect solution. |
Throttles the local feed extration in-order to avoid rate-limiting. This is done, similary to NewPipe, by introducting an articificial delay every 50 fetched feeds of on average 1 second. These values may have to be changed in the future. Ref: libre-tube#6941 (comment)
Throttles the local feed extration in-order to avoid rate-limiting. This is done, similary to NewPipe, by introducting an articificial delay every 50 fetched feeds of on average 1 second. These values may have to be changed in the future. Ref: libre-tube#6941 (comment)
Throttles the local feed extration in-order to avoid rate-limiting. This is done, similary to NewPipe, by introducting an articificial delay every 50 fetched feeds of on average 1 second. These values may have to be changed in the future. Ref: libre-tube#6941 (comment)
Throttles the local feed extration in-order to avoid rate-limiting. This is done, similary to NewPipe, by introducting an articificial delay every 50 fetched feeds of on average 1 second. These values may have to be changed in the future. Ref: libre-tube#6941 (comment)
Throttles the local feed extration in-order to avoid rate-limiting. This is done, similary to NewPipe, by introducting an articificial delay every 50 fetched feeds of on average 1 second. These values may have to be changed in the future. Ref: libre-tube#6941 (comment)
Throttles the local feed extration in-order to avoid rate-limiting. This is done, similary to NewPipe, by introducting an articificial delay every 50 fetched feeds of on average 1 second. These values may have to be changed in the future. Ref: libre-tube#6941 (comment)
Throttles the local feed extration in-order to avoid rate-limiting. This is done, similary to NewPipe, by introducting an articificial delay every 50 fetched feeds of on average 1 second. These values may have to be changed in the future. Ref: libre-tube#6941 (comment)
2 participants
Updates the local feed extraction to only fetch channel tabs that are enabled. This cuts down the amount of requests that are sent, but ultimately not used.
As a side note, in my fork I've experimented with using the more general
FeedInfofirst, to check if there are even new videos available, which seems to reduce the chance to be rate-limited, so it might be worthwhile to take a look at ;)