add test process_capabilities_fail

[kazmsk]

This implements the process_capabilities_fail validation in #361

[YJDoc2]

Hey, Thanks for the PR! There is some change needed - basically the current test validates the capabilities call itself fails, i.e. it will error out from line 9. However, we want to check that the container creation itself fails. For this we will need to create a spec with invalid value, and then assert that the container creation has failed.

For this, what we can do is this -

• create a valid spec, with a single valid capacity
• then use this spec with test_inside_container
  • here you'll need to update the code of test_inside_container in test_utils:204 to return error instead of calling unwrap.
  • In the function call, we get to provide a callback function which runs right before the container is created. This fn gets the rootfs path as a param. The spec (named config.json) it right outside the rootfs files for our tests, so you can edit the path to get the spec string, replace the cap name with an invalid name and re-write it there. Now we will have an invalid spec.
• With this, you can check that the test_inside_container fails because of the capabilities issue.

Please let me know if the above explanation is not clear/ you need any help.

[kazmsk]

@YJDoc2
I have modified it to test capability modification in the callback function.
Please review.

[YJDoc2]

Hey @kazmsk , thanks for the change, I'll try to take a look at this by the weekend.

[kazmsk]

Hi @YJDoc2 , how is the status of the review?

[YJDoc2]

Hey @kazmsk , sincere apologies from my side. I got busy with things, and I missed following up with this. May I ask you to sync with current main and resolve the conflicts?

Again, apologies for not keeping up with this properly 🙏

[kazmsk]

Hi @YJDoc2 , I apologize for not being able to respond earlier.
I have resolved the conflict, so could you please review it again?

[YJDoc2]

Hey, the test itself looks good, but one comment to fix. Also -

1. Can you add the comment on why we are loading the spec and then re-writing it ; basically what I mentioned in my comment before your recent changes.
2. Can you run cargo fmt and check just lint is passing? That is way the CI is failing.

After these are fixed, I think I'll go ahead and merge. Thanks :)

[YJDoc2]

I don't think we need this here, the default command should be ok. We don't actually intend this to reach the runtimetest stage, but instead fail in the creation itself. Right now, if we add this, and somehow the container is created successfully ; then we would still get error from the runtime test, which would result in test_inside_container returning Failed and thus our test incorrectly passing. You can simply set the args to sleep,1m or such (check what we set by default in utils .

[kazmsk]

I have set "sleep, 1m," but I'm not sure where the default is specifically referring to. It would be helpful if you could provide a link.

[kazmsk]

Hi @YJDoc2,
I have made the changes. Could you please check?

[kazmsk]

Hi @YJDoc2,
Could you kindly confirm if you've had a chance to review it?

[YJDoc2]

Hey, so I checked this and code wise it seems ok ; but I'm getting one issue here -
When I try to run this locally, the test passes ; but if I try to comment out the part where we modify the config json, it still fails, with error

container stderr was not empty, found : ERROR libcontainer::process::container_init_process: failed to drop capabilities err=SetCaps(CapsError("capset failure: Operation not permitted (os error 1)"))
ERROR libcontainer::process::container_intermediate_process: failed to initialize container process: failed syscall
ERROR libcontainer::process::container_main_process: failed to wait for init ready: exec process failed with error error in executing process : failed syscall
ERROR libcontainer::container::builder_impl: failed to run container process exec process failed with error error in executing process : failed syscall
ERROR youki: error in executing command: failed to create container: exec process failed with error error in executing process : failed syscall
error in executing command: failed to create container: exec process failed with error error in executing process : failed syscall
Error: failed to create container: exec process failed with error error in executing process : failed syscall

I suspect that this is related to my system not being configured correctly, and also some changes done in latest ubuntu bases systems , as seen in #3097 (comment)

Can you check if the test is passing and failing correctly on your system, and if so also add a check to the error we get to verify it is because the cap is wrong, and not because any other problem?

Apart from that, this is good, Thanks :)

[kazmsk]

Hi @YJDoc2,
I apologize for the delay in replying.

Is this the correct way to run tests in the local environment?
https://github.com/youki-dev/youki/blob/main/tests/contest/contest/README.md#usage

Do the commands seem correct?

[YJDoc2]

Hey,

1. to run contest, run just test-contest (after installing just) and just validate-contest-runc to validate it on runc.
2. There is some conflict in main.rs , please take a look and fix it.

[kazmsk]

Hi @YJDoc2,

Thank you, I was able to reproduce the issue in my environment as well.
I made some changes to the capability rewriting, but it seems like the container still starts even though an invalid capability is set.
Would it be possible to get some advice on this...?

[YJDoc2]

Hey @kazmsk , can you elaborate more on that? Why is it able to start when the spec has invalid cap? Do you have a way to re-create this locally so I can try it out? Thanks :)

[kazmsk]

Hi @YJDoc2,

When I ran just test-contest locally, the process_capabilities_fail_test result was not ok, and the message container creation succeeded unexpectedly. was printed.

# Start group process_capabilities_fail
1 / 1 : process_capabilities_fail_test : not ok
	container creation succeeded unexpectedly.
# End group process_capabilities_fail

This is because test_inside_container passes even though the capability is being overwritten with an invalid one, which led me to conclude that the container is actually starting up.

Please let me know if my understanding is incorrect.

[saku3]

Sorry to interrupt,

It seems that the spec is being overwritten after TEST_CAP is set.

Let's take a look at the test_inside_container function:

• At line 184 in test_inside_container, the closure(&|bundle| {...}) that sets TEST_CAP is executed:

  test_result!(setup_for_test(
      &bundle.as_ref().join("bundle").join("rootfs")
  ));

• Then at line 189, the following line overwrites the config.json using the spec passed as an argument:

  set_config(&bundle, spec).unwrap();

As a result, the modified config.json is overwritten with the original spec.

To confirm this, it would be helpful to print the contents of config.json right before the create_container.

[YJDoc2]

Hey @saku3 thanks for help!

Hey @kazmsk the change for config looks fine, but in test can we also validate in the test that the error we are getting when creation fails is specifically invalid cap error, and not some other error? Apart from that the PR looks fine.

[kazmsk]

Hi @YJDoc2,

I’ve made some additional changes, could you please take a look?

[YJDoc2]

Hey @kazmsk , yes this seems correct. However this fails on runc because the message is not matching - https://github.com/youki-dev/youki/actions/runs/14965059316/job/42053543183?pr=3010

Can we add corresponding error string check as well? (also comment regarding which string check if for which runtime) Thanks :)

[saku3]

to @kazmsk

[kazmsk]

Hi @YJDoc2,

Thank you for reviewing it multiple times!

I've updated the implementation to capture the error from the following message:

unexpected error: container stderr was not empty, found : 
time="2025-05-12T11:58:59Z" level=warning msg="ignoring unknown or unavailable capabilities: [TEST_CAP]"
time="2025-05-12T11:58:59Z" level=error msg="runc create failed: unable to start container process: unable to apply caps: operation not permitted"

Currently, if only a warning is detected (and no error), the test is treated as Failed.
However, if treating warning-only cases as Passed is acceptable, I'm happy to update the behavior—please let me know.

Additionally, I’ve added comments indicating which runtime each error message corresponds to, for clarity.

[YJDoc2]

Hey @kazmsk , so I suspect that the unable to apply error in runc might be because kernel support, so we should check for warning only, with the TEST_CAP word in it. Also in the invalid_variant error, can you update the comment to mention it is for youki? Apart from that LGTM.

[kazmsk]

Hi @YJDoc2,
I've updated the code to check for the warning message and also modified the comments. Please review again.

[YJDoc2]

LGTM!

Thanks for opening the PR and patiently making the changes requests :)