finetune: SGD optimizer, more CLI args

[graehl]

add to ggml-opt learning rate (adamw alpha) cmdline arg, and an optimizer enum defaulting to adamw,
preparatory to work to support SGD

these are in common args a set of optimizer options active only for the new FINETUNE example (which includes all the previous finetune.cpp PERPLEXITY options as a precaution)

perhaps breaking with precedent, the ggml_opt_optimizer_params struct is included directly as args - if desired, we can instead just add learning rate and optimizer type to a struct independent of ggml-opt.h

as proposed in
#13835

[graehl]

perhaps no need to review until i have an actual SGD impl in a follow-on, @JohannesGaessler - but a few general questions about contributing:

1. is it ok to make small retouches to ggml/ sources in this (llama.cpp) project with the expectation of getting the changes into the actual ggml repo later? are there any plans to submodule a ggml-in-llama branch to keep things straight(er)?
2. is what i've got hee the expected way to add example-specific command line arguments? for finetune we definitely at least want to be able to vary the learning rate, which was formerly hard-coded.
3. were the PERPLEXITY args which i blindly added to the new FINETUNE example actually doing anything interesting? perhaps some should be dropped from finetune.
4. could you direct me to a .clang-format style file that might save me from accidentally re-indenting? i know i can set up clang-format to operate only on regions i've already changed ...

[WilliamTambellini]

you should better keep that change as it time to get more feedbacks/approval.

[JohannesGaessler]

is it ok to make small retouches to ggml/ sources in this (llama.cpp) project with the expectation of getting the changes into the actual ggml repo later? are there any plans to submodule a ggml-in-llama branch to keep things straight(er)?

Any changes made to the ggml source in this repository will eventually be synced to the ggml repository and vice versa; it is completely fine. I think the issue of a git submodule was previously brought up and rejected.

is what i've got hee the expected way to add example-specific command line arguments? for finetune we definitely at least want to be able to vary the learning rate, which was formerly hard-coded.

My opinion is that people serious about training should be writing a program rather than use a command line tool. Still, I think it's good to make things such as the learning rate configurable in the provided example program.

were the PERPLEXITY args which i blindly added to the new FINETUNE example actually doing anything interesting? perhaps some should be dropped from finetune.

I don't remember whether those args were put in by me when I copypasted code or by Georgi when he later refactored it but I myself definitely did not make an intentional choice to use these exact arguments.

could you direct me to a .clang-format style file that might save me from accidentally re-indenting? i know i can set up clang-format to operate only on regions i've already changed ...

I don't know, sorry.

[WilliamTambellini]

@ggerganov

[JohannesGaessler]

None of the previous perplexity-specific arguments are needed.

[JohannesGaessler]

For adding an SDG optimizer, add a new ggml op like OPT_STEP_SDG. Add a CPU implementation as a fallback for any backend without an implementation. Add a CUDA implementation since that is (I assume) the backend which you intend to use in production. Add a test to tests/test_backend_ops.cpp to assert that the CPU and CUDA backends produce consistent results. Extend ggml-opt.cpp to conditionally use the new SDG optimizer step, condition the allocation of the optimizer momenta on the optimizer type.

[graehl]

For adding an SDG optimizer, add a new ggml op like OPT_STEP_SDG. Add a CPU implementation as a fallback for any backend without an implementation. Add a CUDA implementation since that is (I assume) the backend which you intend to use in production. Add a test to tests/test_backend_ops.cpp to assert that the CPU and CUDA backends produce consistent results. Extend ggml-opt.cpp to conditionally use the new SDG optimizer step, condition the allocation of the optimizer momenta on the optimizer type.

yes, will do. should the actual SGD impl be a subsequent pull req (or several, e.g. starting first w/ just CPU impl) or do you want it all in one pull req?

[JohannesGaessler]

Either way would be fine with me as long as there are at no point broken or unfinished features on master.

[matiaslin]

Looking forward to the next PR(s).

[graehl]

you should see frivolous clang-format changes (using the project's .clang-format) only on lines changed in the PR (using git-clang-format). if there's something undesireable we could figure out what in the format config does it

[JohannesGaessler]

Don't autoformat code en masse unless it's done in a dedicated PR, it makes it unnecessarily difficult to track what was actually changed in a PR.

[JohannesGaessler]

Sorry, I didn't read the

only on lines changed in the PR

part.

[graehl]

Hi @WilliamTambellini @JohannesGaessler I think this is usable now, inviting code nitpicks etc :)
pretty new to the github interface honestly so let me know if this needs to be two separate PRs one for each commit or if it's reasonable to just review both commits here (obv. better to merge separately, first doesn't break any behavior, second impacts the finetune cmdline default learning rate but that should hurt no one)

[graehl]

Second (actual usable SGD) commit is 48a16bf (also shows above here)

[WilliamTambellini]

Mix up different projects: change of CLI/renaming and SGD. Need to split in 2 PRs.
@slaren ?

[graehl]

I'm not aware of anything I can do on my end to get this merged (is someone waiting on me that I'm unaware of?). I just marked a 'conflict' above resolved but I think ultimately the permission is with the llama.cpp maintainers

[lexasub]

@JohannesGaessler completed for final review/merge @graehl you may rebase to master to simply merge for JohannesGaessler

[graehl]

@JohannesGaessler completed for final review/merge @graehl you may rebase to master to simply merge for JohannesGaessler

Don't think there's anything I can currently do (please be specific if I'm mistaken, I'm new).

[0cc4m]

I'm not aware of anything I can do on my end to get this merged (is someone waiting on me that I'm unaware of?). I just marked a 'conflict' above resolved but I think ultimately the permission is with the llama.cpp maintainers

From my side, I'm basically waiting for you to look at graehl#1 and merge it if it's fine. I see it has a merge conflict again, I'll fix it.

[0cc4m]

You also didn't address 2 crucial comments from Johannes:

This problem causes issues with running tests, that's why I also fixed it in my branch. If you merge it this is resolved, too.

[lexasub]

Don't think there's anything I can currently do (please be specific if I'm mistaken, I'm new).

Rebase YOUR branch to master(then force push to your branch), see 0cc4m's changes, cherry pick 0cc4m(or rebase him changes to your's changes)

[graehl]

Don't think there's anything I can currently do (please be specific if I'm mistaken, I'm new).

Rebase YOUR branch to master(then force push to your branch), see 0cc4m's changes, cherry pick 0cc4m(or rebase him changes to your's changes)

Thanks for spelling this out, that was easy - didn't squash so we can keep occam's contrib. separate but it's all rebased and you should see it here.

[JohannesGaessler]

I'm not aware of anything I can do on my end to get this merged (is someone waiting on me that I'm unaware of?).

As I said, please use the human-readable parameters, and only the human-readable parameters, as the ones being passed to ggml_opt_step_sgd. If you are short on time I can take over and finish up this PR (or someone else can if they want to).

[graehl]

I'm not aware of anything I can do on my end to get this merged (is someone waiting on me that I'm unaware of?).

As I said, please use the human-readable parameters, and only the human-readable parameters, as the ones being passed to ggml_opt_step_sgd. If you are short on time I can take over and finish up this PR (or someone else can if they want to).

This is fine and done now, but I cannot be confident the vulkan end of things is correct after the change (I just haven't read up on how the vulkan API works, at all).

[0cc4m]

You can change what you want, once things are ready I'll do a proper review of the Vulkan parts and make sure they are okay.

[JohannesGaessler]

From my end I would consider this PR now essentially good to merge. So unless there is something else that is left to do I will make some cosmetic changes and rely on @0cc4m to fix Vulkan if necessary. After that I will approve and merge.

[graehl]

You can change what you want, once things are ready I'll do a proper review of the Vulkan parts and make sure they are okay.

I didn't change anything at all in vulkan - it's all greek to me :) Do take a look. Perhaps the tests weren't really running on vulkan (I had disabled them since I didn't have an impl). The change is that the op params tensor [1] is now sgd.wd instead of 1 - sgd.wd*sgd.alpha. ([0] is just sgd.alpha)

[0cc4m]

Yeah, no worries. Here's a diff that does that change on the Vulkan shader, and removes two unnecessary preprocessor steps.

diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/opt_step_sgd.comp b/ggml/src/ggml-vulkan/vulkan-shaders/opt_step_sgd.comp
index 3d5e1d98f..6426dedee 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/opt_step_sgd.comp
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/opt_step_sgd.comp
@@ -1,9 +1,6 @@
 #version 450
 
 #include "generic_head.comp"
-#include "types.comp"
-
-#extension GL_EXT_control_flow_attributes : enable
 
 layout(local_size_x = 512, local_size_y = 1, local_size_z = 1) in;
 
@@ -19,7 +16,7 @@ void main() {
     }
 
     const float alpha = data_params[0];
-    const float keep  = data_params[1];
+    const float keep = 1.f - alpha * data_params[1];
 
     data_x[i] = data_x[i] * keep - alpha * data_grad[i];
 }

If you apply that the CI should pass again.

[JohannesGaessler]

I made some changes and pushed them to johannesgaessler/finelayer. The changes are almost entirely cosmetic, the biggest functional change is that I'm using ggml_format_name in ggml-opt.cpp instead of manual string manipulation (as I should have done from the start). Training of LLaMA 3.2 1b is currently not working but that is not caused by this PR.

[graehl]

I believe I successfully applied both. If anything else can be done to get this merged let me know.

[0cc4m]

Please fix the build issue in the pipelines.

[JohannesGaessler]

The build failures are my fault. I don't know why, but for some reason std::powf is not available everywhere. I pushed a fix to my repository which just uses std::pow again, it's not performance relevant. As I eluded to in an added comment, the code should be refactored to use logf and expf anyways.

[0cc4m]

The Vulkan part is fine.

[JohannesGaessler]

Thanks for the work and the persistence, everyone. For bookkeeping I changed the title/commit message to also mention SGD.

[ggerganov]

I think this PR broke the SYCL build:

https://github.com/ggml-org/ci/blob/results/llama.cpp/22/67133881dd1425bd60d541c04a0e2bfd5a9090/ggml-6-x86-sycl/stdall#L688

Maybe just need to update the "supports" function

[WilliamTambellini]

Tks @JohannesGaessler