Add target based scaling support for Netherite

[bachuv]

This PR adds support for target based scaling. It builds upon Azure/azure-functions-durable-extension#2452. This PR is in draft mode since it hasn't been tested E2E yet.

Changes:

• Add NetheriteMetricsProvider and move CollectMetrics code from ScaleMonitor here
• Add NetheriteTargetScaler (implements ITargetScaler)
• Override TryGetTargetScaler in NetheriteProvider
• Add unit tests
• E2E testing

[sebastianburckhardt]

I have spent some time to modify the logic so it now computes a target instead of a scale recommendation. It still has the same overall structure but I had to use the data provided by the metrics to determine the current scale.

[bachuv]

Thanks Sebastian! I updated the PR and tested your changes locally.

[bachuv]

@sebastianburckhardt, can you point me to where I should add unit tests?

[sebastianburckhardt]

@sebastianburckhardt, can you point me to where I should add unit tests?

I think the best place to add the tests is the DurableTask.Netherite.Tests project.

The other tests in there are mostly end-to-end tests that use some special fixture. In your case, just plain normal unit tests are probably enough, so I don't think you need to use any of those.

[sebastianburckhardt]

Is this PR still needed?

[davidmrdavid]

I believe we've parked this for the time being. Right @bachuv? I suspect this may become relevant again once Flex Consumption enter it's GA-prep phase, but we're not there yet.

@sebastianburckhardt: is it a problem if this remains open, just perhaps in draft mode?

[bachuv]

I believe we've parked this for the time being. Right @bachuv? I suspect this may become relevant again once Flex Consumption enter it's GA-prep phase, but we're not there yet.

@davidmrdavid, yes that's right.

[davidmrdavid]

I've taken the liberty of marking this as draft to help signal that it doesn't need active review, and is simply an idle PR meant for future completion. Hope that's ok!

[ericleigh007]

Not to be too obvious, but I guess this is why we were often peppered with log entries warning that target-based scaling could not be used. The log entries are quite noisy.

[davidmrdavid]

Hey @ericleigh007 - do you know if your app somehow "opted in" to target based scaling? If you're getting these warnings without opting in, that could be a problem (and possibly a reason to re-prioritize this work)

[ericleigh007]

I was under the impression that target-based scaling was the default, and you had to opt-out. So we have done nothing specifically for this.
This does seem to look different than the result we get with AzureStorage.
As another peripheral point, I thought that target-based scaling pivoted back to metric-based because mainly lack of management roles, but that may be an incorrect perception.

[sebastianburckhardt]

PR looks good to me, though I think we need more unit tests. I will take a look at what to test.

[sebastianburckhardt]

found one more thing to fix, otherwise looks good.

[sebastianburckhardt]

This should be inside the conditional below, I think.

[nytian]

Make loadPublisher class property too like metricsProvider so that it can be shared among scalers.