VowpalWabbitThreadedLearning is almost impossible to use

[michael-celani]

Describe the bug

I'd like to use threaded learning to speed up my model training, but it seems clunky and unintuitive. I'm only able to pass string examples (No ExampleBuilder examples) and no matter what I do, I can't seem to get the dang thing to save.

var settings = new VowpalWabbitSettings(args)
{
    EnableStringExampleGeneration = true,
    ParallelOptions = new ParallelOptions
    {
        MaxDegreeOfParallelism = 8

    },
    ExampleCountPerRun = 2000,
    ExampleDistribution = VowpalWabbitExampleDistribution.RoundRobin
};

using var model = new VowpalWabbitThreadedLearning(settings);

var examples = batch.Select(ToExample);
foreach (var example in examples)
{
    if (!string.IsNullOrWhiteSpace(example))
    {
        model.Learn(example);
    }
}

Logger.LogInformation("Saving model after pass.");

var saveModel = model.SaveModel(Options.Value.VowpalWabbitClassifierPath);
await model.Complete();

await model.SaveModel(Options.Value.VowpalWabbitClassifierPath);
await model.Complete();

I feel like this should work, but I have to buy an engagement ring for the thing. The example count per run has to match up exactly, I have no idea how to force it to synchronize, it's just completely unintuitive -- and this doesn't seem far off from the examples in the documentation. What am I doing wrong here?

How to reproduce

Train with the c# bindings using these options:
--oaa 30 --probabilities --loss_function logistic --progress 1000 --holdout_off

Version

9.11.2

OS

Linux

Language

C#

Additional context

No response

[JohnLangford]

I don't use C# much --- does 4913 help?

[michael-celani]

@JohnLangford If I were to describe my ideal C# threaded learning model, it would be more or less identical to a standard VowpalWabbit with the caveat that only learning methods are available. A dedicated Flush() method helps, though.

From the spec, it's unclear if Learn enqueues actions and returns immediately, so you have to wait on the Completion (typical TPL flow), or if it actually learns on the underlying model immediately so you can, for instance, learn in multiple threads explicitly. The corresponding VowpalWabbitThreadedPrediction is designed to work in multiple threads simultaneously, so if it's the TPL variant, it's less apparent.

[JohnLangford]

A redesign of the interface is more ambitious than I'm ready for, but it is a "patches welcome" kind of thing. We'd need to support the older interface for some time. Some comments documenting the interface have been added. Does that help?

[JohnLangford]

(reopen if this is still not working for some reason)