Allow for faster scale-down times

[duglin]

In what area(s)?

/area autoscale
/kind proposal

Describe the feature

Currently there are two config options that control scaling times:

stable-window: 60s
scale-to-zero-grace-period: 30s

I believe the first one (stable-window) will control how long an instance is idle before the system determines that it is eligible for termination/scale-down. I believe there is a hard-coded minimum of 2 seconds.

The second one (scale-to-zero-grace-period) controls how long (more) to wait on the last instance, with a minimum value of 30 seconds.

These minimum values are much larger than what we'd like our system/users to experience. Like start-up times, we'd like scale-down times to be as small as possible - like 50ms time range. Yes this means more creation of instances but this is the semantics we're looking for.

Proposal: allow for values of "zero" to be specified for both to indicate the desire for things to be terminated immediately upon the instance being idle, or done processing all current requests. Then people can set it to a higher value as their needs dictate.

Note: as with I think most (if not all) of our config timing values, this would not be a guarantee but rather the desired behavior that they system would try to match. This is no different than the default values we have today of a combined value of 90 seconds actually showing up (for me) to be more like 100-120 seconds regularly. Not a guarantee, just something for the system (autoscaler) to shoot for.

[vagababov]

We already have #4453 which should try to lower down scale-to-zero-grace-period, but as @markusthoemmes we cannot guarantee request success with that number being 0.

[josephburnett]

Based on our discussion at today's scaling working group meeting, I think the desire is to have a way to express a use case. And to allow the implementation specific values to float underneath.

Perhaps enumerated autoscaling profiles would allow such an expression. E.g. Rapid Stable or Balanced.

I would currently configure Rapid to mean stable-window=6s and panic-window-percentage=10 and scale-to-zero-grace-period=0.

[duglin]

We discussed this a bit on today's scaling call - wanted to summarize what I think was said:

• this issue isn't about code changes to actually get to zero scale-down times - that'll happen in other issues/PRs
• this issue is more of a UX issue where I'm asking for a simple way for an operator to specify "scale down each instance as soon as possible" w/o them having to understand the specifics of each environment's auto-scaler to know what each one's minimum values for these config knobs might be. It's more of a UX-interop request.
• this will enable portability and consistency of tooling w/o having to understand each specific Kn install to get a common semantic behavior
• this is about scaling down in general - meaning all instances, not just the last one (scale to zero)
• Markus mentioned that (regardless of what knobs we offer) it might make sense to treat "zero" as special - something like "whatever minimum value the auto-scaler can achieve". This way as the number changes over time (due to the work being done in this WG) the operator doesn't need to know that or change their knobs each time. To them "zero" will always get them the lowest possible value.
• Joe mentioned that I might be looking for something like a "profile" where the operator can specify their "desired semantic outcome" and then behind the scenes the code maps that profile/value into the auto-scaler specific knobs to try to achieve those goals.
• I think a profile thing would be good from a UX perspective because it abstracts away the impl details from the user/operator. We just need to allow for advanced uses where people may want to also specify the auto-scaler specific knobs - so we may need to allow them to do "profile" or "low-level-knobs" but not both - otherwise they may be in conflict.
• while having a single knob to indicate this value would be great, at this point in time having to configure multiple knobs is ok as long as we can accurately document what knobs and values to use to get the "scale down as soon as possible" semantics.

[markusthoemmes]

To clarify this: The special case I mentioned for 0s windows would be to always return the latest reading. It's not "whatever minimum value the autoscaler can achieve".

[mattmoor]

We have #4453 for tracking what we plan in the near term. We should firm up what we want from this in the longer term.

[taragu]

/assign

[vagababov]

Note that #4453 is done. Which permits scaling down to 0 after stable window (which can be set as low as 6secs right now).

[knative-housekeeping-robot]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle stale.
Stale issues rot after an additional 30 days of inactivity and eventually close.
If this issue is safe to close now please do so by adding the comment /close.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/lifecycle stale