Skip to content

Conversation

@diegolovison
Copy link
Contributor

@diegolovison diegolovison commented Jul 10, 2025

Implement mcp-sse-client-server locally with ollama and mistral

See the discussion on #1605

@diegolovison
Copy link
Contributor Author

First error:

2025-07-10 10:35:57,431 INFO  [io.qua.lan.mcp.run.htt.McpHttpClientLogger] (vert.x-eventloop-thread-0) Request:
- method: POST
- url: http://localhost:8081/mcp/messages/YjMzMzBkMjMtNzExMi00M2U2LWJlMjItMjkwZmNiN2FhNDMx
- headers: [Accept: application/json], [Content-Type: application/json], [User-Agent: Quarkus REST Client], [content-length: 46]
- body: {"jsonrpc":"2.0","id":1,"method":"tools/list"}

2025-07-10 10:35:57,436 ERROR [io.qua.web.nex.run.Endpoints] (vert.x-worker-thread-1) Unable to consume text message in @OnTextMessage callback: WebSocket connection [endpointId=io.quarkiverse.langchain4j.sample.chatbot.ChatBotWebSocket, path=/chatbot, id=38d5c9ed-0222-4f4d-a6b9-1507655176f1]

Exception in ChatBotWebSocket.java:25
          23      @Blocking
          24      public String onMessage(String message) {
        → 25          return aiWeatherService.getWeather(message);
          26      }
          27  : dev.langchain4j.service.IllegalConfigurationException: Duplicated definition for tool: getAlerts
        at dev.langchain4j.service.tool.ToolProviderResult$Builder.add(ToolProviderResult.java:60)
        at dev.langchain4j.mcp.McpToolProvider.lambda$provideTools$2(McpToolProvider.java:114)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
        at dev.langchain4j.mcp.McpToolProvider.provideTools(McpToolProvider.java:113)
        at io.quarkiverse.langchain4j.mcp.runtime.QuarkusMcpToolProvider.provideTools(QuarkusMcpToolProvider.java:38)
        at dev.langchain4j.service.tool.ToolProvider_XijGBi814_1fDyr5SACAv287COI_Synthetic_ClientProxy.provideTools(Unknown Source)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.doImplement(AiServiceMethodImplementationSupport.java:203)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.implement(AiServiceMethodImplementationSupport.java:154)
        at io.quarkiverse.langchain4j.sample.chatbot.AiWeatherService$$QuarkusImpl.getWeather(Unknown Source)
        at io.quarkiverse.langchain4j.sample.chatbot.AiWeatherService$$QuarkusImpl_ClientProxy.getWeather(Unknown Source)
        at io.quarkiverse.langchain4j.sample.chatbot.ChatBotWebSocket.onMessage(ChatBotWebSocket.java:25)
        at io.quarkiverse.langchain4j.sample.chatbot.ChatBotWebSocket_WebSocketServerEndpoint.doOnTextMessage(Unknown Source)
        at io.quarkus.websockets.next.runtime.WebSocketEndpointBase$4.call(WebSocketEndpointBase.java:167)
        at io.quarkus.websockets.next.runtime.WebSocketEndpointBase$4.call(WebSocketEndpointBase.java:161)
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$4(ContextImpl.java:192)
        at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:270)
        at io.vertx.core.impl.ContextImpl$1.execute(ContextImpl.java:221)
        at io.vertx.core.impl.WorkerTask.run(WorkerTask.java:56)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
        at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
        at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
        at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:1583)

@diegolovison diegolovison changed the title Use ollama and Mistral Implement mcp-sse-client-server locally with ollama and mistral Jul 10, 2025
@geoand
Copy link
Collaborator

geoand commented Jul 10, 2025

Thanks!

@jmartisk do you want to take a look at the bug that this PR surfaces?

@geoand
Copy link
Collaborator

geoand commented Jul 10, 2025

I pushed an update to the PR, but it seems that the model you have chosen does not support tools:

Caused by: dev.langchain4j.exception.HttpException: {"error":"registry.ollama.ai/library/mistral:latest does not support tools"}                                                                                                                                                                                   
        at io.quarkiverse.langchain4j.jaxrsclient.JaxRsHttpClient.execute(JaxRsHttpClient.java:122)                                                                                                                                                                                                                
        at dev.langchain4j.http.client.log.LoggingHttpClient.execute(LoggingHttpClient.java:39)                                                                                                                                                                                                                    
        at dev.langchain4j.model.ollama.OllamaClient.chat(OllamaClient.java:93)                                                                                                                                                                                                                                    
        at dev.langchain4j.model.ollama.OllamaChatModel.lambda$doChat$0(OllamaChatModel.java:42)                                                                                                                                                                                                                   
        at dev.langchain4j.internal.ExceptionMapper.withExceptionMapper(ExceptionMapper.java:29)                                                                                                                                                                                                                   
        ... 32 more     

@diegolovison
Copy link
Contributor Author

I pushed an update to the PR, but it seems that the model you have chosen does not support tools:

Caused by: dev.langchain4j.exception.HttpException: {"error":"registry.ollama.ai/library/mistral:latest does not support tools"}                                                                                                                                                                                   
        at io.quarkiverse.langchain4j.jaxrsclient.JaxRsHttpClient.execute(JaxRsHttpClient.java:122)                                                                                                                                                                                                                
        at dev.langchain4j.http.client.log.LoggingHttpClient.execute(LoggingHttpClient.java:39)                                                                                                                                                                                                                    
        at dev.langchain4j.model.ollama.OllamaClient.chat(OllamaClient.java:93)                                                                                                                                                                                                                                    
        at dev.langchain4j.model.ollama.OllamaChatModel.lambda$doChat$0(OllamaChatModel.java:42)                                                                                                                                                                                                                   
        at dev.langchain4j.internal.ExceptionMapper.withExceptionMapper(ExceptionMapper.java:29)                                                                                                                                                                                                                   
        ... 32 more     

My example started from https://github.com/eugenp/tutorials/blob/master/quarkus-modules/quarkus-mcp-langchain/quarkus-mcp-client/src/main/resources/application.properties

I see:

quarkus.langchain4j.timeout=180s
quarkus.langchain4j.chat-model.provider=ollama
quarkus.langchain4j.ollama.chat-model.model-id=mistral
quarkus.langchain4j.ollama.base-url=http://localhost:11434

quarkus.langchain4j.mcp.default.transport-type=http
quarkus.langchain4j.mcp.default.url=http://localhost:9000/mcp/sse

@geoand
Copy link
Collaborator

geoand commented Jul 10, 2025

Right, I changed it because those values are wrong

@diegolovison
Copy link
Contributor Author

diegolovison commented Jul 10, 2025

Updated the configuration

quarkus.langchain4j.timeout=1000s
quarkus.log.category."io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport".level=DEBUG

It is slow on my machine, but I have also

2025-07-10 13:00:13,781 WARN  [io.ver.cor.imp.BlockedThreadChecker] (vertx-blocked-thread-checker) Thread Thread[vert.x-worker-thread-1,5,main] has been blocked for 60770 ms, time limit is 60000 ms

Exception in ChatBotWebSocket.java:25
          23      @Blocking
          24      public String onMessage(String message) {
        → 25          return aiWeatherService.getWeather(message);
          26      }
          27  

: io.vertx.core.VertxException: Thread blocked
        at java.base/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:221)
        at java.base/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1864)
        at java.base/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
        at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
        at java.base/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1898)
        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2072)
        at org.jboss.resteasy.reactive.client.impl.InvocationBuilderImpl.unwrap(InvocationBuilderImpl.java:202)
        at org.jboss.resteasy.reactive.client.impl.InvocationBuilderImpl.post(InvocationBuilderImpl.java:243)
        at io.quarkiverse.langchain4j.jaxrsclient.JaxRsHttpClient.execute(JaxRsHttpClient.java:117)
        at dev.langchain4j.http.client.log.LoggingHttpClient.execute(LoggingHttpClient.java:39)
        at dev.langchain4j.model.ollama.OllamaClient.chat(OllamaClient.java:93)
        at dev.langchain4j.model.ollama.OllamaChatModel.lambda$doChat$0(OllamaChatModel.java:42)
        at dev.langchain4j.internal.ExceptionMapper.withExceptionMapper(ExceptionMapper.java:29)
        at dev.langchain4j.internal.RetryUtils.lambda$withRetryMappingExceptions$2(RetryUtils.java:324)
        at dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry(RetryUtils.java:211)
        at dev.langchain4j.internal.RetryUtils.withRetry(RetryUtils.java:264)
        at dev.langchain4j.internal.RetryUtils.withRetryMappingExceptions(RetryUtils.java:324)
        at dev.langchain4j.internal.RetryUtils.withRetryMappingExceptions(RetryUtils.java:308)
        at dev.langchain4j.model.ollama.OllamaChatModel.doChat(OllamaChatModel.java:42)
        at dev.langchain4j.model.chat.ChatModel.chat(ChatModel.java:46)
        at dev.langchain4j.model.chat.ChatModel_CAOdzSz_JXOmbdPru8SBVdBro64_Synthetic_ClientProxy.chat(Unknown Source)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.executeRequest(AiServiceMethodImplementationSupport.java:513)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.executeRequest(AiServiceMethodImplementationSupport.java:522)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.executeRequest(AiServiceMethodImplementationSupport.java:528)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.doImplement(AiServiceMethodImplementationSupport.java:364)
        at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.implement(AiServiceMethodImplementationSupport.java:154)
        at io.quarkiverse.langchain4j.sample.chatbot.AiWeatherService$$QuarkusImpl.getWeather(Unknown Source)
        at io.quarkiverse.langchain4j.sample.chatbot.AiWeatherService$$QuarkusImpl_ClientProxy.getWeather(Unknown Source)
        at io.quarkiverse.langchain4j.sample.chatbot.ChatBotWebSocket.onMessage(ChatBotWebSocket.java:25)
        at io.quarkiverse.langchain4j.sample.chatbot.ChatBotWebSocket_WebSocketServerEndpoint.doOnTextMessage(Unknown Source)
        at io.quarkus.websockets.next.runtime.WebSocketEndpointBase$4.call(WebSocketEndpointBase.java:167)
        at io.quarkus.websockets.next.runtime.WebSocketEndpointBase$4.call(WebSocketEndpointBase.java:161)
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$4(ContextImpl.java:192)
        at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:270)
        at io.vertx.core.impl.ContextImpl$1.execute(ContextImpl.java:221)
        at io.vertx.core.impl.WorkerTask.run(WorkerTask.java:56)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
        at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
        at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
        at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:1583)



@diegolovison
Copy link
Contributor Author

Can quarkus.langchain4j.timeout=1000s be an ergonomic thing? If set, set also the vert.x timeout?

@geoand
Copy link
Collaborator

geoand commented Jul 11, 2025

I was able to get your sample to work and now I'm looking into the warning you are seeing.

Can quarkus.langchain4j.timeout=1000s be an ergonomic thing? If set, set also the vert.x timeout?

No, they are totally different things

@geoand
Copy link
Collaborator

geoand commented Jul 11, 2025

@diegolovison What you can do is the following:

quarkus.langchain4j.timeout=1000s
quarkus.vertx.max-worker-execute-time=${quarkus.langchain4j.timeout}

@diegolovison diegolovison marked this pull request as ready for review July 11, 2025 11:13
@diegolovison diegolovison requested a review from a team as a code owner July 11, 2025 11:13
@diegolovison
Copy link
Contributor Author

Ready for review

Copy link
Collaborator

@cescoffier cescoffier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use a profile to avoid overriding the default implementation, like %ollama.

@quarkus-bot

This comment has been minimized.

@diegolovison
Copy link
Contributor Author

Could you use a profile to avoid overriding the default implementation, like %ollama.

Do you mean something like ( First time contributing to quarkus / I read https://quarkus.io/guides/config-reference ):

Index: samples/mcp-sse-client-server/mcp-client/src/main/resources/application.properties
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>ISO-8859-1
===================================================================
diff --git a/samples/mcp-sse-client-server/mcp-client/src/main/resources/application.properties b/samples/mcp-sse-client-server/mcp-client/src/main/resources/application.properties
--- a/samples/mcp-sse-client-server/mcp-client/src/main/resources/application.properties	(revision fa1f7dcab6797cb15b71c46e71d8cd625036f935)
+++ b/samples/mcp-sse-client-server/mcp-client/src/main/resources/application.properties	(date 1752235940565)
@@ -1,12 +1,13 @@
 quarkus.langchain4j.log-requests=true
-quarkus.langchain4j.ollama.chat-model.model-id=mistral
+%ollama.quarkus.langchain4j.ollama.chat-model.model-id=mistral
 
 quarkus.langchain4j.mcp.weather.transport-type=http
 quarkus.langchain4j.mcp.weather.url=http://localhost:8081/mcp/sse/
 
 quarkus.http.port=8080
 
-quarkus.langchain4j.timeout=1000s
-quarkus.vertx.max-worker-execute-time=${quarkus.langchain4j.timeout}
+quarkus.langchain4j.timeout=30s
+%ollama.quarkus.langchain4j.timeout=1000s
+%ollama.quarkus.vertx.max-worker-execute-time=${%ollama.quarkus.langchain4j.timeout}
 
 #quarkus.log.category."io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport".level=DEBUG

But how about

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-openai</artifactId>
</dependency>
<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-ollama</artifactId>
</dependency>

@cescoffier
Copy link
Collaborator

Yes, you will need both extensions. We would need to see if each profile needs to be extended a bit to ensure it works in both cases.

We do something similar in https://github.com/cescoffier/langchain4j-deep-dive/blob/main/2-ai-services/5-langchain4j-ai-service-chat-memory/src/main/resources/application.properties

@diegolovison
Copy link
Contributor Author

diegolovison commented Jul 11, 2025

On Maven, it must be a different dependency. The default should be quarkus-langchain4j-openai. When we activate the ollama profile use quarkus-langchain4j-ollama. They cannot coexist.

@geoand
Copy link
Collaborator

geoand commented Jul 11, 2025

They cannot coexist.

They can if you set quarkus.langchain4j.chat-model.provider

@diegolovison
Copy link
Contributor Author

Done

mcp-client$ mvn quarkus:dev
mcp-client$ mvn quarkus:dev -Dquarkus.profile=ollama

@geoand geoand requested a review from cescoffier July 11, 2025 14:05
@quarkus-bot
Copy link

quarkus-bot bot commented Jul 14, 2025

Status for workflow Build (on pull request)

This is the status report for running Build (on pull request) on commit 189d2b5.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

@geoand geoand merged commit 3773395 into quarkiverse:main Jul 14, 2025
77 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants