bug: v0.6.6 Unable to load any models

[Azziemuth-Nocturne]

Version: v0.6.6

Describe the Bug

I used to be able to run a local server on Jan in 6.5 but ever since I updated to 6.6 It's been unable to load any models, even the ones I downloaded directly from the hub. Apologies if I'm getting any part of this format wrong, this is my first time on Github and my first time writing a report.

Steps to Reproduce

1. Start Jan
2. Attempt to load a model from llama.cpp
3. Press start
4. Error window pops up

Screenshots / Logs

app.log

Operating System

• MacOS
• Windows
• Linux

[LazyYuuki]

hey @Azziemuth-Nocturne, no worry we will take a look into this

cc @qnixsynapse it seems there are a bunch of user having problem starting their API server on Windows

This might be related also #6038

[qnixsynapse]

Hi. Can you copy the error in the popup and paste it here please?

[Azziemuth-Nocturne]

Hi. Can you copy the error in the popup and paste it here please?

Sure:

Failed to load llama-server: llamacpp error: Error: srv load_model: loading model 'C:\Users\Placeholder Username\Downloads\tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf'

Full stderr:
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce GTX 1050, compute capability 6.1, VMM: yes
build: 1 (f3471ce) with MSVC 19.38.33133.0 for Windows AMD64
system info: n_threads = 6, n_threads_batch = 6, total_threads = 12

system_info: n_threads = 6 (n_threads_batch = 6) / 12 | CUDA : ARCHS = 600,610,700,750,800,860,890 | F16 = 1 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |

Web UI is disabled
main: binding port with default address family
main: HTTP server is listening, hostname: 127.0.0.1, port: 3379, http threads: 11
main: loading model
srv load_model: loading model 'C:\Users\Placeholder Username\Downloads\tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf'

(I've also tried running it in CPU only mode to rule out if my GPU was the issue, I still had trouble getting it loaded.)

[qnixsynapse]

Probably a path issue because there are no such llamacpp errors. Will try to test and figure out.

Thank you!

[Azziemuth-Nocturne]

I'm not really sure what to do for now, the issue still isn't fixed and I have also tried reinstalling Jan, which leads to the same issue.