Skip to content

请问llm_type为llama时,模型权重文件必须为model/stable-vicuna-13B.ggml.q4_2.bin吗?权重文件是否可以为llama2?以及策略是否支持fp16? #534

@15229684931

Description

@15229684931

我在npu上进行测试时,迁移了wenda的代码,在适配llama时,config.yaml如下配置
llama: path: "/home/ch/stable-vicuna-13B-HF" strategy: "npu fp16"
image
但是在web端推理时,出现了回答不全的现象
231

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions