[GSK-1634] From MAD to 10% change per sample #1377
Conversation
GSK-1634 Decrease the threshold for the "makes prediction change" push feature
In this example for me there is nothing strange, that if we decrease age by 20, the prediction will change. It seems to me, that we need to decrease change size, which triggers this push-feature. |
python-client/giskard/ml_worker/testing/functions/transformation.py
Outdated
Show resolved
Hide resolved
| # Compute the MAD of the column | ||
| mad = compute_mad(ds.df[feature]) # Small issue: distribution might not be normal | ||
| # Compute 10% around the value to be perturbed | ||
| value_to_perturb = ds_slice.df[feature].iloc[0] |
There was a problem hiding this comment.
previously we were calculating MAD based on the whole series, now we're taking just the first element, isn't it less robust in case the first element is an outlier?
There was a problem hiding this comment.
by first element, it's the sample we get in the debugger. Yes, @jmsquare suggested based on @AbSsEnT 's feedback, that we reduce the perturbation to a fixed rate (in this case 10% or less of the sample-value) instead of using MAD because it's less clear where it comes from, plus the perturbation was really sizeable (see the linear card for examples).
…on.py Co-authored-by: Andrey Avtomonov <[email protected]>
|
Kudos, SonarCloud Quality Gate passed! |
Hartorn
deleted the
GSK-1634-from-mad-to-10-percent-perturbation-max
branch
Changing the perturbation of numerical features, from +/- 2*MAD, to 10%*value.