Skip to content

fix: Jan supports cortex with llama-server #4904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
May 12, 2025
Merged

fix: Jan supports cortex with llama-server #4904

merged 11 commits into from
May 12, 2025

Conversation

@vansangpfiev
Copy link
Contributor

@vansangpfiev vansangpfiev commented Apr 14, 2025

Describe Your Changes

This pull request includes significant updates to the engine management and inference cortex extensions. The changes primarily involve updating the engine version and renaming the engine from cortex.llamacpp to llama.cpp. Additionally, there are updates to the platform-specific download scripts and configurations.

Key changes include:

Engine Version and Naming Updates:

  • Updated CORTEX_ENGINE_VERSION to b5058 in rolldown.config.mjs files. [1] [2] [3]
  • Renamed engine from cortex.llamacpp to llama.cpp in index.ts and download scripts. [1] [2] [3] [4] [5]

Platform-Specific Updates:

  • Simplified and updated platform-specific identifiers in utils.ts:
    • Changed Windows identifier from windows-amd64 to win.
    • Updated macOS identifiers to macos-arm64 and macos-x64.
    • Changed Linux identifier from linux-amd64 to linux.
  • Updated CUDA version identifiers to cu12.0 and cu11.7 in utils.ts.

Download Script Updates:

  • Updated download URLs and subfolder names in download.bat and download.sh to reflect the new engine name and versioning scheme. [1] [2] [3] [4]

Additional Configuration Updates:

  • Added x64 to the engine variant strings in utils.ts.

Fixes Issues

  • Closes #
  • Closes #

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@louis-jan louis-jan closed this Apr 15, 2025
@github-project-automation github-project-automation bot moved this to Done in Jan Apr 15, 2025
@vansangpfiev vansangpfiev reopened this Apr 29, 2025
@vansangpfiev
Copy link
Contributor Author

vansangpfiev commented Apr 29, 2025

Re-open, but seems like we need to change the target to tauri branch

@vansangpfiev vansangpfiev changed the base branch from dev to feat/tauri-build-option April 29, 2025 10:45
@vansangpfiev vansangpfiev marked this pull request as ready for review May 12, 2025 02:22
@vansangpfiev vansangpfiev requested a review from louis-jan May 12, 2025 02:23
@louis-jan louis-jan requested a review from david-menloai May 12, 2025 02:33
louis-jan
louis-jan approved these changes May 12, 2025
View reviewed changes
Copy link
Contributor

@louis-jan louis-jan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@louis-jan louis-jan merged commit f3762a4 into feat/tauri-build-option May 12, 2025
@louis-jan louis-jan deleted the fix/llama-server branch May 12, 2025 05:52
@github-project-automation github-project-automation bot moved this from Done to QA in Jan May 12, 2025
@louis-jan louis-jan restored the fix/llama-server branch May 13, 2025 02:31
gau-nernst added a commit that referenced this pull request May 15, 2025
@gau-nernst
Squashed commit of the following:
aeb6a2c 
commit 04ddeff
Author: Thien Tran <[email protected]>
Date:   Thu May 15 16:15:35 2025 +0800

    remove dirs

commit ac68fff
Author: Thien Tran <[email protected]>
Date:   Thu May 15 16:13:26 2025 +0800

    refactor download logic. add download_hf_repo

commit 0c0e773
Author: Thien Tran <[email protected]>
Date:   Thu May 15 12:49:38 2025 +0800

    fixes

commit 60ebfcf
Author: Thien Tran <[email protected]>
Date:   Thu May 15 12:46:19 2025 +0800

    add download manager

commit edc48d1
Author: Thien Tran <[email protected]>
Date:   Wed Apr 23 19:19:16 2025 +0800

    use HF_TOKEN automatically

commit df26edc
Author: Thien Tran <[email protected]>
Date:   Wed Apr 23 15:36:57 2025 +0800

    support 3-part format

commit 9ebc1b4
Author: Thien Tran <[email protected]>
Date:   Wed Apr 23 14:18:14 2025 +0800

    emit correct event for cancelled download

commit 901f1ff
Author: Thien Tran <[email protected]>
Date:   Wed Apr 23 13:42:09 2025 +0800

    add cancel download

commit 42068ad
Author: Thien Tran <[email protected]>
Date:   Wed Apr 23 11:45:59 2025 +0800

    download with progress

commit 58f6774
Author: Thien Tran <[email protected]>
Date:   Tue Apr 22 08:49:41 2025 +0800

    reorganize files

commit 7fb525d
Author: Thien Tran <[email protected]>
Date:   Mon Apr 21 22:59:55 2025 +0800

    add list HF files

commit aee0ff5
Author: Thien Tran <[email protected]>
Date:   Mon Apr 21 15:41:47 2025 +0800

    add download fn

commit e1e375c
Author: vansangpfiev <[email protected]>
Date:   Wed May 14 20:46:01 2025 +0700

    chore: tauri codesign and CI/CD (#4961)

    * chore: build tauri

    * chore: codesign tauri app

    * chore: test update from electron to tauri

    * chore: test update from electron to tauri

    * chore: update csp config and cors

    * chore: nightly to 1317

    * fix: correct pre_install_path

    * chore: jan-nightly to 1320

    * chore: self sign tauri

    * chore: CI/CD for Windows, Linux

    commit 4897b2b
    Author: vansangpfiev <[email protected]>
    Date:   Fri Apr 25 15:32:37 2025 +0700

        chore: cleanup tauri config

    commit 66c5676
    Author: vansangpfiev <[email protected]>
    Date:   Fri Apr 25 14:37:04 2025 +0700

        fix: store path

    commit bc6560c
    Author: vansangpfiev <[email protected]>
    Date:   Thu Apr 24 09:39:50 2025 +0700

        chore: Linux CI

    commit b036275
    Author: vansangpfiev <[email protected]>
    Date:   Wed Apr 23 16:41:22 2025 +0700

        chore: updater windows

    commit e91b543
    Merge: dea80a8 4a54a37
    Author: vansangpfiev <[email protected]>
    Date:   Wed Apr 23 16:39:24 2025 +0700

        Merge branch 'chore/tauri-cicd' of https://github.com/menloresearch/jan into chore/tauri-cicd-windows

    commit dea80a8
    Author: Minh141120 <[email protected]>
    Date:   Wed Apr 23 11:47:04 2025 +0700

        chore: update azuresigntool install method

    commit 2ec2234
    Author: Minh141120 <[email protected]>
    Date:   Wed Apr 23 11:01:31 2025 +0700

        chore: add verbose option build tauri and targets app and dmg for macos build

    commit 42c7592
    Author: Minh141120 <[email protected]>
    Date:   Wed Apr 23 10:35:27 2025 +0700

        chore: update targets build

    commit 4c8ba44
    Author: Minh141120 <[email protected]>
    Date:   Wed Apr 23 09:53:21 2025 +0700

        refactor: remove debug step and upload electron build artifact

    commit 158c08b
    Author: Minh141120 <[email protected]>
    Date:   Wed Apr 23 09:21:08 2025 +0700

        chore: add script codesign on windows

    commit 4545b2b
    Author: vansangpfiev <[email protected]>
    Date:   Tue Apr 22 13:39:49 2025 +0700

        chore: update csp setting

    commit f64a1e1
    Author: Minh141120 <[email protected]>
    Date:   Tue Apr 22 10:15:14 2025 +0700

        chore: update azuresigntool installation

    commit 1f4b9d1
    Author: Minh141120 <[email protected]>
    Date:   Tue Apr 22 09:49:42 2025 +0700

        chore: update signcommand

    commit 911a3ab
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 19:19:23 2025 +0700

        chore: update codesign tauri windows

    commit fba15c4
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 19:04:29 2025 +0700

        chore: update path azuresigntool

    commit 8b8c950
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 18:38:56 2025 +0700

        chore: update azuresigntool path

    commit bd67a2b
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 17:47:33 2025 +0700

        chore: update azuresigntool url

    commit f70effc
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 17:33:32 2025 +0700

        chore: update azuretoolsign download

    commit 6679107
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 16:56:25 2025 +0700

        chore: update path azuresigntool

    commit f1610bf
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 16:52:36 2025 +0700

        chore: update azuresigntool path

    commit 0873d56
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 16:19:46 2025 +0700

        chore: add debug step

    commit 88e0b1a
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 15:58:39 2025 +0700

        ci(windows): download AzureSignTool to src-tauri for Tauri code signing

    commit 47f94e8
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 15:21:20 2025 +0700

        fix: AzureSignTool signcommand Path

    commit dc014a7
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 15:00:02 2025 +0700

        chore: add debug step azuresigntool tauri windows

    commit ee7b616
    Author: Minh141120 <[email protected]>
    Date:   Mon Apr 21 14:33:33 2025 +0700

        chore: tauri windows codesign

    commit 6607090
    Author: vansangpfiev <[email protected]>
    Date:   Thu Apr 17 10:29:50 2025 +0700

        chore: add windows download script

    commit 4b1a5cc
    Author: Nguyen Ngoc Minh <[email protected]>
    Date:   Mon Apr 21 13:44:34 2025 +0700

        chore: tauri codesign windows

    * chore: workflows for tauri

    * chore: test tauri manual build

    * chore: hide windows install detail

    * chore: upload artifacts

    * feat: run mcp with bundled bun and uv

    * chore: clean up

    * chore: update cicd

    * chore: remove deprecated workflows

    * chore: update allowed origins

    * chore: pull binaries windows, linux (#4963)

    * fix: get bun and uv from execution path

    * fix: macos

    * fix: typo

    * fix: remove old Jan binaries on windows

    * chore: build bun and uv universal

    * fix: appimage bundle bun issue

    * chore: libfuse2 for linux CI

    * feat: tauri cicd preview (#4975)

    * feat: tauri cicd preview

    * chore: add suffix preview for tauri build

    * chore: update condition for s3 upload nightly channel

    * chore: add debug step for tauri macos preview

    * chore: update aws s3 macos tauri build

    * refactor: remove debug code

    * chore: update artifact name tauri macos build

    * chore: add tauri build step for electron beta and stable

    * chore: update preview

    * chore: bump llama.cpp engine to b5351

    * chore: bump engine version

    * fix: cors windows

    ---------

    Co-authored-by: vansangpfiev <[email protected]>
    Co-authored-by: Service Account <[email protected]>
    Co-authored-by: Louis <[email protected]>
    Co-authored-by: hiento09 <[email protected]>
    Co-authored-by: Nguyen Ngoc Minh <[email protected]>

commit 833de44
Author: vansangpfiev <[email protected]>
Date:   Wed May 14 18:43:54 2025 +0700

    fix: add execution permission for bun and uv (#4990)

    * fix: add execution permission for bun

    * fix: add execution permission for uv

commit b7d385a
Author: Thien Tran <[email protected]>
Date:   Wed May 14 17:26:42 2025 +0800

    Use token.js for non-tools calls (#4973)

    * deprecate inference()

    * fix tool_choice. only startModel for Cortex

    * appease linter

    * remove sse

    * add stopInferencing support. temporarily with OpenAI

    * use abortSignal in token.js

    * bump token.js version

commit ebf79ff
Author: Louis <[email protected]>
Date:   Tue May 13 14:37:15 2025 +0700

    fix: download script should be mjs format

commit 02860ea
Author: Thien Tran <[email protected]>
Date:   Tue May 13 09:47:54 2025 +0700

    [feat] Download bun+uv on `make dev-tauri` (#4981)

    * only download if not exist

    * add download:bin to dev-tauri

commit e8fc2d6
Author: Louis <[email protected]>
Date:   Mon May 12 13:07:18 2025 +0700

    feat: run mcp with bundled bun and uv (#4949)

    * feat: run mcp with bundled bun and uv

    * chore: clean up

    * chore: pull binaries windows, linux (#4963)

    * fix: get bun and uv from execution path

    * fix: macos

    * fix: typo

    ---------

    Co-authored-by: vansangpfiev <[email protected]>

commit f3762a4
Author: vansangpfiev <[email protected]>
Date:   Sun May 11 22:52:51 2025 -0700

    fix: Jan supports cortex with llama-server (#4904)

    * fix: update logic to pull llama-cpp engine

    * chore: bump cortex version

    * fix: chat macos

    * chore: bump llama.cpp

    * chore: download windows

    * chore: download linux

    * chore: bump cortex to 1.0.13-rc6

    * fix: stop cortex before exit

    * chore: bum llama.cpp engine to b5350

    ---------

    Co-authored-by: vansangpfiev <[email protected]>

commit 3e922b8
Author: Louis <[email protected]>
Date:   Fri May 9 16:25:36 2025 +0700

    feat: simplify remote providers and tool use capability (#4970)

    * feat: built-in remote providers go to tokenjs

    * fix: error handling

    * fix: extend models

    * chore: error handling

    * chore: update advanced settings of built-in providers

    * chore: clean up message creation

    * chore: fix import

    * fix: engine name

    * fix: error handling
louis-jan pushed a commit that referenced this pull request May 15, 2025
@vansangpfiev @sangjanai @louis-jan
fix: Jan supports cortex with llama-server (#4904)
dea29ff 
* fix: update logic to pull llama-cpp engine

* chore: bump cortex version

* fix: chat macos

* chore: bump llama.cpp

* chore: download windows

* chore: download linux

* chore: bump cortex to 1.0.13-rc6

* fix: stop cortex before exit

* chore: bum llama.cpp engine to b5350

---------

Co-authored-by: vansangpfiev <[email protected]>
@louis-jan louis-jan mentioned this pull request May 15, 2025
Merged
3 tasks
@david-menloai david-menloai moved this from QA to Done in Jan Jun 23, 2025
@louis-jan louis-jan deleted the fix/llama-server branch July 20, 2025 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@louis-jan louis-jan louis-jan approved these changes

@david-menloai david-menloai Awaiting requested review from david-menloai

Assignees

@vansangpfiev vansangpfiev

@gau-nernst gau-nernst

@louis-jan louis-jan

Labels

None yet

Projects

Jan
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@vansangpfiev @louis-jan @gau-nernst @sangjanai