🚀 Feature: Configurable rate limits

[ameerfayiz]

🔖 Feature description

instead of global Rate Limits of 60 requests in every 1 minutes per IP address. , there should be a provision to manually change that 60 request per minute limit.

🎤 Pitch

Because some functions should be allowed to execute 2 or 3 execution per minute like for OTP resending, etc.. now I have to manually control the limit by creating a proxy function and its hard to manage for functions to have different rate limits, even though possible to do it manually.

👀 Have you spent some time to check if this issue has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

[eldadfux]

This has been raised internally in the context of collections and document creation. We'd love to gather more community feedback before we push this feature to work.

[ameerfayiz]

im currently using tokenbucket algorithm inside all cloud functions which need individual rate limits to achieve the individual rate limit effect. which needs an extra table with 2 attributes for every such cloud functions.

[DankMemeGuy]

I agree on configurable rate limit. Servers are not all the same, and can handle different loads. Different websites have different levels of protection they really need, as well. Making this option configurable via UI would be a good solution.

If protection is really at front of mind, you could enable an 'alert' feature that triggers when a certain threshold is reached, whether site wide or per user/IP. Such as a user hitting 80% of their rate limit or something. That way an admin can just deal with the abuse when it happens, or see what abuse just occurred when they check the alerts next, or have the ability to easily impose a tighter limit if the website is undergoing an event.

[andreiverse]

This would a must-have for apps that have large amounts of functions

[huy-lv]

Hope this feature will be launch soon

[delqu]

I also waiting for this feature

[JoshiJoshiJoshi]

Is there an ETA when this is going to be worked on?

[webpatser]

I had a demo where 15 people tried a new app. All on the same wifi. So rate limit was triggered for account creation because they shared the IP address. This is rate limit should really be adjustable.

This endpoint is limited to 10 requests in every 60 minutes per IP address. We use rate limits to avoid service abuse by users and as a security practice. [Learn more about rate limiting](https://appwrite.io/docs/rate-limits).

[webpatser]

Setting _APP_OPTIONS_ABUSE=disabled in the .env seems to disable all forced upon rate limits. This is the way to scale and resolve abuse on a different level (like in nginx or an API gateway).

This should be in the docs?

[khare4]

This has been raised internally in the context of collections and document creation. We'd love to gather more community feedback before we push this feature to work.

this is must have feature so that people can set rate limit according to there need.

[jhooper04]

I think rate limiting would vary wildly on a per project basis of what is normal use of the system and what is abuse. _APP_OPTIONS_ABUSE=disabled with your own rate limiting proxy is a good approach but seems like that is going to be nearly every project that needs to do that beyond simple configuration type projects.

One of the great things I like about appwrite is that it is an all in one package to get an app up quickly and still be able to scale it later. Lots of features out of the box. Makes sense to me that if it is going to offer a rate limiting feature, it should be configurable per route; otherwise, most devs will have to disable it and implement that elsewhere. avoiding as much extra work as possible is what drew me to appwrite personally.

[sgaabdu4]

I'm not able to tweak the rate limiting for the cloud as we don't have access to .env or is there another way?

[joeyouss]

Hi @sgaabdu4 yes you cannot edit env on cloud yet but this is something we can probably discuss, however which rate limit you want to change? We might have some solution with Appwrite functions

[DH-555]

Also, It would be great to have the possibility to modify other abuse limits like pagination or max document fetched limits.
That could prevent anyone from fetching 5000 documents in an app where it's unnecessary fetching more than 30 documents with one query.

[jhooper04]

I think the functions would be the main need for custom rate limits since the function could do just about anything. Though you could implement that in the function itself. It would definitely be nice to be able to set a rate on any endpoint and simplify the functions.

[stnguyen90]

Related discussion:

• Custom abuse limits #7509

[xyvielyons]

i myself i would like a configurable rate limit especially in development when setting up authentication

[eldadfux]

We look to solve this problem in two main ways. First solution is development keys, these keys will be time limited to avoid accidental usage of them in a production environment. We aim to release this in Appwrite 1.7. As @stnguyen90 pointed out more information about this solution can be found here: #7509

This solution is designed to solve the problem with rate limits that are designed against brute force attack, especially on auth related endpoints.

2nd solution which is a bit more in early stages of discussion is the Appwrite Firewall. This is basically the idea of having something that is more similar to an L7 firewall aka WAF. This will allow greater level of flexibility in setting custom rules across all of your Appwrite endpoints.

[daveycodez]

Ahh I almost left Supabase for your built in rate limiting, but 60 per IP is too low sadly. It should be 60 per session and more like 600 per IP, definitely configurable. I'm building a turn based online game so I might have occurrences where up to 10 players are in a LAN. 60 requests per minute will not work there. For Supabase I just built a custom proxy endpoint solution so I can configure my own rate limits now... but it has added latency. Best choice I think for now.

Also the one minute window is too large, you want to rate limit users within 10 seconds, it needs multiple windows to be ideal

[Amitrochates]

much needed

[LorenzoBloedow]

I ran into this issue and created an open-source library to handle this, it supports 4 different algorithms, multiple strategies at the same time, allows you to self-host or use the rate limiting function inline for less latency, and a bunch of other options, fully typed with JSDoc.

I have plans in the future to have a paid managed option too, but I don't have one at the time of writing, so I wonder if it's ok to link it here so people can use the free version, I really don't want to spam or hijack this issue.

Thanks :)

[shlokstoira]

@LorenzoBloedow please share

[LorenzoBloedow]

@shlokstoira

Here you go:
https://github.com/borrowdev/borrow

Hopefully it helps. :)

If someone from Appwrite doesn't want me to share this please let me know and I'll delete all comments.
Again, don't want to hijack this issue.

[hortigado]

I need it for suppot offline, the upload of data is very slow when the user connect to the network by it limit

[ammarfaris]

Is there any update on this? just wondering;

oh i see it will be in the 1.7 release or sooner? thanks

[ammarfaris]

Why is it removed from 1.7 Release? @stnguyen90

[shlokstoira]

@LorenzoBloedow is there a way to self host this on my local machine, using docker and my own redis instead if Cloudflare workers and upstash redis. How can I go about doing that?

[xMrAfonso]

Indeed why was this removed? @stnguyen90

[Amitrochates]

@LorenzoBloedow Is borrow down currently?

[eldadfux]

UPDATE: The way we want to solve this is by introducing a global web application firewall, aka WAF. The idea is to allow each project, site or function to define its own custom rules.

Each rule will be defined by a set of condition which will result in action taken. Some of this actions will include:

• Bypass
• Deny
• Challenge
• RateLimit
• Redirect
• Log

Each action will allow you to set its own configuration like the target location for a redirect, or the timeframe, and count limits for the action RateLimit. Appwrite will be able to create conditions around your HTTP request (path, method, header etc) and will most likely also be able to introduce tighter integration with the platform with rules around (specific user, key, etc).

I have drafted a basic implementation of how the under the hood engine could look like. You can share feedback below.
https://github.com/utopia-php/waf