common: support negated args#17919
Conversation
|
Would it make sense to add positive variants to Or perhaps just leave the positive variant empty and handle that? Many of these can benefit from rewording. |
|
Yes I think we can add negation for most of the boolean variables to establish a pattern. The goal is to encourage future contributors to follow the same path. Wdyt @ggerganov ? |
|
Sounds good |
|
I added the support for most of the args with 2 exceptions:
|
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
|
A small nits is that some args have this in the help message: Probably we can have a |
|
already regen in e73c42b |
|
The server is not handling the negated arguments correctly: ./bin/llama-server --no-mmap
0.00.000.443 W main: setting n_parallel = 4 and kv_unified = true (add -kvu to disable this)
0.00.000.450 I build: 7376 (380b4c984) with AppleClang 17.0.0.17000319 for Darwin arm64 (debug)
0.00.000.466 I system info: n_threads = 16, n_threads_batch = 16, total_threads = 24
0.00.000.468 I
0.00.000.483 I system_info: n_threads = 16 (n_threads_batch = 16) / 24 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | REPACK = 1 |
0.00.000.485 I
0.00.000.516 I init: using 23 threads for HTTP server
WARNING: Using native backtrace. Set GGML_BACKTRACE_LLDB for more info.
WARNING: GGML_BACKTRACE_LLDB may cause native MacOS Terminal.app to crash.
See: https://github.com/ggml-org/llama.cpp/pull/17869
0 libggml-base.0.9.4.dylib 0x0000000103fad4c8 ggml_print_backtrace_symbols + 48
1 libggml-base.0.9.4.dylib 0x0000000103fad2f8 ggml_print_backtrace + 160
2 libggml-base.0.9.4.dylib 0x0000000103fc43c8 _ZL23ggml_uncaught_exceptionv + 12
3 libc++abi.dylib 0x00000001817f0c2c _ZSt11__terminatePFvvE + 16
4 libc++abi.dylib 0x00000001817f4394 __cxa_get_exception_ptr + 0
5 libc++abi.dylib 0x00000001817f433c _ZN10__cxxabiv1L12failed_throwEPNS_15__cxa_exceptionE + 0
6 llama-server 0x000000010244ede0 _Z19common_params_parseiPPc13llama_exampleRNSt3__13mapI10common_argNS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS2_4lessIS4_EENS8_INS2_4pairIKS4_SA_EEEEEE + 860
7 llama-server 0x00000001022ded9c _ZN14server_presetsC2EiPPcR13common_paramsRKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEE + 852
8 llama-server 0x00000001022df450 _ZN14server_presetsC1EiPPcR13common_paramsRKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEE + 60
9 llama-server 0x00000001022e06bc _ZN13server_modelsC2ERK13common_paramsiPPcS4_ + 180
10 llama-server 0x00000001022e1288 _ZN13server_modelsC1ERK13common_paramsiPPcS4_ + 60
11 llama-server 0x00000001022ad8d0 _ZN20server_models_routesC2ERK13common_paramsiPPcS4_ + 80
12 llama-server 0x00000001022ad870 _ZN20server_models_routesC1ERK13common_paramsiPPcS4_ + 60
13 llama-server 0x00000001022ad824 _ZNSt3__114__construct_atB8ne200100I20server_models_routesJR13common_paramsRiRPPcS7_EPS1_EEPT_SA_DpOT0_ + 72
14 llama-server 0x00000001022ad7bc _ZNSt3__123__optional_storage_baseI20server_models_routesLb0EE11__constructB8ne200100IJR13common_paramsRiRPPcS9_EEEvDpOT_ + 60
15 llama-server 0x0000000102291e54 _ZNSt3__18optionalI20server_models_routesE7emplaceB8ne200100IJR13common_paramsRiRPPcS9_EvEERS1_DpOT_ + 68
16 llama-server 0x000000010228f780 main + 1276
17 dyld 0x0000000181475d54 start + 7184
libc++abi: terminating due to uncaught exception of type std::invalid_argument: error: invalid argument: --no-mmap
Abort trap: 6 |
|
@ggerganov ah yeah, I forgot the change the function handling presets on server (it only affect router mode), the fix is here: #17991 |
* args: support negated args * update docs * fix typo * add more neg options * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * rm duplicated arg * fix LLAMA_ARG_NO_HOST * add test --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* args: support negated args * update docs * fix typo * add more neg options * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * rm duplicated arg * fix LLAMA_ARG_NO_HOST * add test --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Add better support for negated args like
-no-cnvor--no-jinja, example:For boolean options like
--mmapor--kv-offload, the environment variable is handled as shown in this example:LLAMA_ARG_MMAP=truemeans enabled, other accepted values are:1,on,enabledLLAMA_ARG_MMAP=falsemeans disabled, other accepted values are:0,off,disabledLLAMA_ARG_NO_MMAPis present (no matter the value), it means disabling mmapNote: Before this change,
LLAMA_ARG_JINJA=falsewill simply skip the--jinjaarg but does not set--no-jinja. Now, falsey values like0, false, no, disabledwill trigger the negated value.Same logic also apply to the recently-added preset feature: #17859