llguidance build fixes for Windows

[mmoskal]

The actual changes is by @phil-scott-78 , see comment

I tested it on fresh Windows install, did minor naming fixes, and updated docs.

Also fixed link to VS download to be English not German.

cc @HanClinto

[phil-scott-78]

i was hoping to dig a bit more into the code this afternoon, but I've ran out of time before my kids come home from school. But it looks like llama_sampler_init_llg isn't exported either. Not sure if that's on purpose until the support isn't behind a flag. I'm using LlamaSharp and they build the samplers up by hand rather than calling common_sampler_init.

[mmoskal]

I suspect LlamaSharp is only linking against llama.dll ? The llguidance is currently in common.lib

[phil-scott-78]

Correct, it only takes a dependency on llama.dll that's shipped with the releases and calls that via .NET's DllImports to do the unmanaged calls to exported functions. So it would need to be exported.

I could see an argument that if the build was done with LLGuidance flag set then the existing llama_sampler_init_grammar and llama_sampler_init_grammar_lazy should be able to handle routing the grammar without the need for additional calls exported. much like the set up with common_sampler_init is handling it now, so maybe that's the design decision instead.

[mmoskal]

It was ggerganov decision to include the llguidance only in common.lib, which is not AFAIK included in llama.dll. So the code is not there, it's not only about exporting. I understand he's open to including it in llama.dll at a later date after it proves itself.

[phil-scott-78]

Ok, that makes sense. Then no worries, I think I can come up with some creative solutions until it's properly baked now that it's building on Windows

[ggerganov]

It was ggerganov decision to include the llguidance only in common.lib, which is not AFAIK included in llama.dll. So the code is not there, it's not only about exporting. I understand he's open to including it in llama.dll at a later date after it proves itself.

Yes, in the future we can include llguidance support to the core libllama. We have to see some use cases - atm, I am not at all familiar with the functionality so I cannot decide that it is worth it the extra maintenance and build logistics.

Sent you a collaborator invite - feel free to merge this PR if it is still needed.

[mmoskal]

@ggerganov thank you for the invite, merged.

As for use cases, let's see what people come up with!

I'm also fixing some stuff in llguidance to support tool calling better (#11607)

[phil-scott-78]

As for use cases, let's see what people come up with!

I've been working a c# -> grammar generator, bypassing json schema part of the typical grammar dance

Some of the Lark syntax looked exciting for possibilities in directing the output a bit cleaner than I was seeing with gbnf.

Unfortunately, I'm also using llamasharp so I do need it out of common (along with the Jinja parser) to really kick the tires. But I'll keep hacking away with some proof of concepts