[Feature Request]: deploy using WSGI.

[shichunshan]

Is there an existing issue for the same feature request?

• I have checked the existing issues.

Is your feature request related to a problem?

./entrypoint.sh: line 25: 18337 Killed                  $PY api/ragflow_server.py
[WARNING] [2024-06-24 07:59:24,499] [synonym.__init__] [line:24]: Realtime synonym is disabled, since no redis connection.
SUM: [WARNING] Load term.freq FAIL!

    ____                 ______ __               
   / __ \ ____ _ ____ _ / ____// /____  _      __
  / /_/ // __ `// __ `// /_   / // __ \| | /| / /
 / _, _// /_/ // /_/ // __/  / // /_/ /| |/ |/ / 
/_/ |_| \__,_/ \__, //_/    /_/ \____/ |__/|__/  
              /____/                             

    
[INFO] [2024-06-24 07:59:36,021] [_internal._log] [line:96]: WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

Describe the feature you'd like

run_simple causes the service to be killed, interrupting the service. I hope to be able to deploy using WSGI.

Describe implementation you've considered

No response

Documentation, adoption, use case

No response

Additional information

No response

[r1ng13]

想问下这个问题解决了吗

[bwnjnOEI]

+1

[a1299646329]

+1

[tuoxiebushuijiao]

+1

[lichanghai]

+1

[pan003]

mark

[zanyatta]

+1

[zyoung1212]

+1

[which-W]

@shichunshan

Thanks so much for your suggestion — and apologies for the delayed response! ⏳

At the moment, our product does not directly bundle a WSGI server — you'll need to choose one that fits your needs based on the Flask framework (e.g., Gunicorn, uWSGI). ⚙️🐍

To better understand how this affects your workflow, could you share more details about your specific use case? We'd love to know whether the lack of a built-in WSGI server impacts any critical functionality on your side.

We’ll take this into consideration as we evaluate whether to include WSGI server support in future versions. Thanks again for the constructive feedback — and feel free to share more ideas anytime! 🙌

[ericwu0930]

I’m unsure if this is directly related, but the concurrency performance is poor.

When making a single call, the response time is around 1 second. However, when I initiated 100 concurrent requests, the performance degraded significantly (see attached results).

The code:

async def test_get_fund_manager():
    import asyncio
    
    async def single_query(i):
        start_time = time.time()
        result = await Rag_Api.retrieval_rag("请问如何撤单")
        end_time = time.time()
        duration = end_time - start_time
        
        logger.info(f"第{i+1}次查询耗时: {duration:.2f}秒")
        return result, duration

    tasks = [single_query(i) for i in range(100)]
    results = await asyncio.gather(*tasks)
    
    total_duration = sum(duration for _, duration in results)
    logger.info(f"并发查询总耗时: {total_duration:.2f}秒")

@which-W

[ZhenhangTung]

Related PR: #7845 And thanks @ericwu0930 for the details. Any further thoughts? @zyoung1212 @zanyatta @pan003 @lichanghai @tuoxiebushuijiao @a1299646329 @bwnjnOEI @r1ng13 @shichunshan :)

[ericwu0930]

@ZhenhangTung Coincidentally, I also tried using gunicorn instead of the original wsgi server and saw a 10x increase in concurrency performance. I hope to release this feature soon.

[ZhenhangTung]

@ZhenhangTung Coincidentally, I also tried using gunicorn instead of the original wsgi server and saw a 10x increase in concurrency performance. I hope to release this feature soon.

@ericwu0930 I saw @zyoung1212 has just finished work. You can leave your thoughts there. Could you also explain how you achieve 10x increase in concurrency performance? Any benchmark details?