Skip to content

Let grammar escape backslashes#5865

Merged
oobabooga merged 46 commits intooobabooga:devfrom
altoiddealer:main
May 19, 2024
Merged

Let grammar escape backslashes#5865
oobabooga merged 46 commits intooobabooga:devfrom
altoiddealer:main

Conversation

@altoiddealer
Copy link
Copy Markdown
Contributor

@altoiddealer altoiddealer commented Apr 16, 2024

Checklist:

I was trying to develop a grammar string to respond in this format:

(path)"\"(name).safetensors [(hash)]

Here is what I tried...

  grammar_string: |-
    root ::= path? name ".safetensors" " [" hash "]"
    path ::= (folder "\\")*
    folder ::= [a-zA-Z0-9_]+
    name ::= [a-zA-Z0-9_]+
    hash ::= HEX HEX HEX HEX HEX HEX HEX HEX HEX HEX
    HEX ::= [0-9a-fA-F]

error: unknown escape

I then tried this:

path ::= (folder "\")*

error: out of range

Also this:

path ::= (folder "\\\\")*

error: unknown escape

It turns out, the parser was just incapable of escaping backslashes.

Simply adding this condition to the parser code resolves it:

        elif esc == "\\":
            return "\\", src[2:]

@oobabooga
Copy link
Copy Markdown
Owner

Thanks for the fix

@oobabooga oobabooga changed the base branch from main to dev May 19, 2024 23:25
@oobabooga oobabooga merged commit 818b4e0 into oobabooga:dev May 19, 2024
anon-contributor-0 pushed a commit to anon-contributor-0/text-generation-webui that referenced this pull request May 30, 2024
PoetOnTheRun pushed a commit to PoetOnTheRun/text-generation-webui that referenced this pull request Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants