Skip to content

Conversation

@misgod-yy
Copy link

This PR change the defination of "kube-monkey/mtbf".
When we use kube-monkey to kill pod in k8s cluster, we find it too slow to generate a failure, because the label 'kube-monkey/mtbf' only support to generate one failure in days.
So I extend the time range of this label, to support failure in minutes, hours, and calculate mean time of the specified hour/minute.

available usage as follows:
kube-monkey/mtbf: "4h"
kube-monkey/mtbf: "30m"

@misgod-yy misgod-yy changed the title extend mtbf time range Extend mtbf time range Jan 9, 2020
@misgod-yy misgod-yy requested a review from asobti February 5, 2020 14:13
@minhnguyenvan95
Copy link

Look good

_, err := time.ParseDuration(mtbf)
if err != nil {
return -1, err
return "", fmt.Errorf("error parsing mtbf %s: %v", mtbf, err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it a breaking change. It would be good if we could still use values without a unit which would default to using days.

**`kube-monkey/enabled`**: Set to **`"enabled"`** to opt-in to kube-monkey
**`kube-monkey/mtbf`**: Mean time between failure (in days). For example, if set to **`"3"`**, the k8s app can expect to have a Pod
killed approximately every third weekday.
**`kube-monkey/mtbf`**: Mean time between failure (in hours/minutes etc. Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".). For example, if set to **`"3h"`**, the k8s app can expect to have a Pod
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using nano seconds for example would not be a valid use case. I think we should restrict to minutes/hours/days.

secOffset := r.Intn(60)
return time.Now().In(loc).Add(time.Duration(secOffset) * time.Second)
}
return calendar.RandomTimeInRange(config.StartHour(), config.EndHour(), loc)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is RandomTimeInRange still used?

return rangeStart.Add(offsetDuration)
}

func CustzRandomTimeInRange(mtbf string, startHour, endHour int, loc *time.Location) time.Time {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind using a different name? Not entirely sure what custz means.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused by the way this method is used.
If I set a mtbf of 1h I would expect my victim to be attacked roughly every hour, or 24 times during a single day.
However looking at the code, this will only schedule a single attack for that day.
Can you please clarify if that's the intention?

@yakovyohaev
Copy link

Hello, im experiencing the same issue/limitation.
Is there a solution to Extend mtbf time range?
configuring time frame as "minutes" will display the following in the kube-monkey pod log
0126 15:46:47.423502 1 eligible_deployments.go:29] Skipping eligible v1.Deployment rec-services-rigger because of error: strconv.Atoi: parsing "5m": invalid syntax W0126 15:46:47.427740 1 eligible_statefulsets.go:29] Skipping eligible v1.StatefulSet rec because of error: strconv.Atoi: parsing "5m": invalid syntax

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants