[Bug] non-functional dataprep mivlus

[lianhao]

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source
• Other

Deploy method

• Docker
• Docker Compose
• Kubernetes Helm Charts
• Other

Running nodes

Single Node

What's the version?

docker id: opea/dataprep with digest 77aff00

Description

dataprep with milvus DB backend will report the following errors during data ingest:

[2025-02-17 01:30:39,914] [   ERROR] - opea_dataprep_microservice - Error during dataprep ingest invocation: OpeaMilvusDataprep.ingest_files() takes from 1 to 7 positional [10/202]
s but 8 were given
INFO:     192.168.103.211:40020 - "POST /v1/dataprep/ingest HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 174, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 172, in __call__
    await self.app(scope, receive, send_wrapper)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/comps/dataprep/src/opea_dataprep_microservice.py", line 68, in ingest_files
    response = await loader.ingest_files(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/comps/dataprep/src/opea_dataprep_loader.py", line 23, in ingest_files
    return await self.component.ingest_files(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: OpeaMilvusDataprep.ingest_files() takes from 1 to 7 positional arguments but 8 were given

Reproduce steps

Manually follow the steps in https://github.com/opea-project/GenAIComps/blob/main/tests/dataprep/test_dataprep_milvus.sh, to launch milvus, tei-embedding-serving, and dataprep-milvus, then run the following curl command:

echo "test file" > /tmp/file1.txt
curl http://localhost:6007/v1/dataprep/ingest -sS --fail-with-body \
        -X POST \
        -H "Content-Type: multipart/form-data" \
        -F "files=@/tmp/file1.txt"

Raw log

Attachments

No response

[xiguiw]

File "/home/user/comps/dataprep/src/opea_dataprep_microservice.py", line 68, in ingest_files
response = await loader.ingest_files(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/comps/dataprep/src/opea_dataprep_loader.py", line 23, in ingest_files
return await self.component.ingest_files(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: OpeaMilvusDataprep.ingest_files() takes from 1 to 7 positional arguments but 8 were given

It seems the number of arguments mismatched in the code.
@letonghan
Would you please help to check this issue?

[letonghan]

The argument issue of dataprep milvus will be fixed in this PR by Rita.

[xiguiw]

The argument issue of dataprep milvus will be fixed in this PR by Rita.

@lianhao
1292 is merged.
Could this be closed?

[yinghu5]

@lianhao thank you a lot for report, close the issue. please feel free to reopen it if any issues.