`--benchmark_min_time` command line option undocumented

[kpet]

The documentation mentions it once but I couldn't find comprehensive documentation for it. As far as I can tell, this option can be used for two things

1. Set the min duration of a repetition with --benchmark_min_time=<value> or --benchmark_min_time=<value>s.
2. Set the desired number of iterations per repetition with --benchmark_min_time=<value>x (for benchmarks not registered with a fixed number of iterations).

This seems like it relates to #746

Are both of these bits of behaviour officially supported? If yes, it would be helpful to document them.

[dmah42]

if they're in the code, they're supported. if we're missing some documentation, i'm happy to take PRs improving the docs.

[Proxihox]

I was messing around with the --benchmark_min_time flag and I observed:

1. If argument is <value>x then <value> is the exact number of iterations.
2. Anything other format, it assumes to be time in seconds. (But issues a warning if it doesn't end with s.
   Now what it does with this time value : Keeps running iterations, until time exceeds <value> seconds.
   i.e A benchmark which normally runs for any number of iterations, upon adding the --benchmark_min_time=0s will run exactly 1 iteration.
   This goes against the intuitive expectation that --benchmark_min_time sets a minimum limit on iterations (both wrt number and time).

I can see 2 ways to handle this:

1. Fix the functionality, to actually match what you would logically expect from this flag based on its name.
2. Replace --benchmark_min_time with --benchmark_time, and retain the current functionality. Add documentation explaining that we can use this flag to either specify the time in seconds (approximately) to execute the benchmarking, or specify the exact number of iterations to run in the benchmark. This also makes sense considering there isn't a '--benchmark_max_time` flag.

[LebedevRI]

The in-fix min is correct, it is impossible to guarantee that it will actually run for exactly that long,
it may run longer.

1. If argument is <value>x then <value> is the exact number of iterations.

Right.

2. Anything other format, it assumes to be time in seconds. (But issues a warning if it doesn't end with s.> Now what it does with this time value : Keeps running iterations, until time exceeds <value> seconds.
   i.e A benchmark which normally runs for any number of iterations, upon adding the --benchmark_min_time=0s will run exactly 1 iteration.
   This goes against the intuitive expectation that --benchmark_min_time sets a minimum limit on iterations (both wrt number and time).

Err, i don't agree, this is very much the expected behaviour.
This is not a bug.

[dmah42]

+1 it's doing exactly what you intuit. it's running for a minimum amount of time or a minimum amount of iterations. 1 iteration is > 0s so it runs 1 iteration.

what you seem to be looking for is --benchmark_max_time which doesn't exist.

[felmonon]

I am planning to take this as a docs-only PR. I will update docs/user_guide.md to document the supported --benchmark_min_time forms, clarify the preferred s suffix for time values, and keep behavior unchanged.