Skip to content

Conversation

@Leonidas-from-XIV
Copy link
Collaborator

@Leonidas-from-XIV Leonidas-from-XIV commented May 23, 2025

After discussing an alternate solution for #11827 with @rgrinberg we agreed that the best course of action is to just terminate the entire process group of the cram test after execution.

This PR reuses @Alizter's cram test repro case.

It introduces a slight race condition that we might be terminating an unrelated process group that accidentally got the same PID as the cram test. I am unaware of a way to avoid it, the chance is not substantial as OSes usually increment PIDs until they wrap around. In any case the engine is already terminating process groups if the process to be waited on has a timeout, so this code just extends it to the case where there is no timeout.

Fixes #11820

@Leonidas-from-XIV
Copy link
Collaborator Author

Surprising behavior on Windows when we kill_process_group which attempts to Unix.kill the PID which somehow causes corruption in the files that ocamldep writes. That's odd because at that time we have awaited the process so it should have exited already.

@Alizter
Copy link
Collaborator

Alizter commented May 24, 2025

@Leonidas-from-XIV that sounds to me like we have a race condition. Is it similar to the issue in #11010?

@rgrinberg
Copy link
Member

Seems like there's a failure in CI. Is it something relevant?

@Leonidas-from-XIV
Copy link
Collaborator Author

I'm trying to figure out. It works reliably on my system (only failures are from Melange), CI sometimes hangs forever which is unrelated to this PR but its hard to say whether the failure is related since I can't reproduce it.

@Alizter
Copy link
Collaborator

Alizter commented Jun 11, 2025

@Leonidas-from-XIV Could you push a temporary commit disabling all of exec-watch and perhaps watching. Those would be my goto culprits for tests that doesn't terminate.

Alizter and others added 3 commits June 29, 2025 18:54
Signed-off-by: Ali Caglayan <[email protected]>
Signed-off-by: Marek Kubica <[email protected]>
Signed-off-by: Rudi Grinberg <[email protected]>
@rgrinberg rgrinberg force-pushed the cram-terminate-pgroup branch from 968f8ec to 2088aee Compare June 29, 2025 17:55
@rgrinberg rgrinberg merged commit 7b57de6 into ocaml:main Jun 29, 2025
23 of 25 checks passed
maiste added a commit to maiste/opam-repository that referenced this pull request Jul 28, 2025
CHANGES:

### Fixed

- Stop re-running cram tests after promotion when it's not necessary (ocaml/dune#11994,
  @rgrinberg)

- fix: `$ dune subst` should not fail when adding the version field in opam
  files (ocaml/dune#11801, fixes ocaml/dune#11045, @btjorge)

- Kill all processes in the process group after the main process has
  terminated; in particular this avoids background processes in cram tests to
  stick around after the test finished (ocaml/dune#11841, fixes ocaml/dune#11820, @Alizter,
  @Leonidas-from-XIV)

### Added

- `(tests)` stanzas now generate aliases with the test name. To run
  `(test (name a))` you can do `dune build @runtest-a`. (ocaml/dune#11558, grants part of ocaml/dune#10239,
  @Alizter)

- Inline test libraries now produce aliases `runtest-name_of_lib`
  allowing users to run specific inline tests as `dune build
  @runtest-name_of_lib`. (ocaml/dune#11109, partially fixes ocaml/dune#10239, @Alizter)

- feature: `$ dune subst` use version from `dune-project` when no version
  control repository has been detected (ocaml/dune#11801, @btjorge)

- Allow `dune exec` to run concurrently with another instance of dune in watch
  mode (ocaml/dune#11840, @gridbugs)

- Introduce `%{os}`, `%{os_version}`, `%{os_distribution}`, and `%{os_family}`
  percent forms. These have the same values as their opam counterparts.
  (ocaml/dune#11863, @rgrinberg)

- Introduce option `(implicit_transitive_deps false-if-hidden-includes-supported)`
  that is equivalent to `(implicit_transitive_deps false)` when `-H` is
  supported by the compiler (OCaml >= 5.2) and equivalent to
  `(implicit_transitive_deps true)` otherwise. (ocaml/dune#11866, fixes ocaml/dune#11212, @nojb)

- Add `dune describe location` for printing the path to the executable that
  would be run (ocaml/dune#11905, @gridbugs)

- `dune runtest` can now understand absolute paths as well as run tests in
  specific build contexts (ocaml/dune#11936, @Alizter).

- Added 'empty' alias which contains no targets. (ocaml/dune#11556 ocaml/dune#11952 ocaml/dune#11955 ocaml/dune#11956,
  grants ocaml/dune#4161, @Alizter and @rgrinberg)

- Allow `dune promote` to properly run while a watch mode server is running
  (ocaml/dune#12010, @ElectreAAS)

- Add `--alias` and `--alias-rec` flags as an alternative to the `@` and `@@`
  syntax in the command line (ocaml/dune#12043, fixes ocaml/dune#5775, @rgrinberg)

### Changed

- Format long lists in s-expressions to fill the line instead of
  formatting them in a vertical way (ocaml/dune#10892, fixes ocaml/dune#10860, @nojb)

- Switch from MD5 to BLAKE3 for digesting targets and rules. BLAKE3 is both more
  performant and difficult to break than MD5 (ocaml/dune#11735, @rgrinberg, @Alizter)

- Print a warning when `dune build` runs over RPC (ocaml/dune#11836, @gridbugs)

- Stop emitting empty module group wrapper `.js` file in `melange.emit`
  (ocaml/dune#11987, fixes ocaml/dune#11986, @anmonteiro)
maiste added a commit to maiste/opam-repository that referenced this pull request Jul 30, 2025
CHANGES:

### Fixed

- Stop re-running cram tests after promotion when it's not necessary (ocaml/dune#11994,
  @rgrinberg)

- fix: `$ dune subst` should not fail when adding the version field in opam
  files (ocaml/dune#11801, fixes ocaml/dune#11045, @btjorge)

- Kill all processes in the process group after the main process has
  terminated; in particular this avoids background processes in cram tests to
  stick around after the test finished (ocaml/dune#11841, fixes ocaml/dune#11820, @Alizter,
  @Leonidas-from-XIV)

### Added

- `(tests)` stanzas now generate aliases with the test name. To run
  `(test (name a))` you can do `dune build @runtest-a`. (ocaml/dune#11558, grants part of ocaml/dune#10239,
  @Alizter)

- Inline test libraries now produce aliases `runtest-name_of_lib`
  allowing users to run specific inline tests as `dune build
  @runtest-name_of_lib`. (ocaml/dune#11109, partially fixes ocaml/dune#10239, @Alizter)

- feature: `$ dune subst` use version from `dune-project` when no version
  control repository has been detected (ocaml/dune#11801, @btjorge)

- Allow `dune exec` to run concurrently with another instance of dune in watch
  mode (ocaml/dune#11840, @gridbugs)

- Introduce `%{os}`, `%{os_version}`, `%{os_distribution}`, and `%{os_family}`
  percent forms. These have the same values as their opam counterparts.
  (ocaml/dune#11863, @rgrinberg)

- Introduce option `(implicit_transitive_deps false-if-hidden-includes-supported)`
  that is equivalent to `(implicit_transitive_deps false)` when `-H` is
  supported by the compiler (OCaml >= 5.2) and equivalent to
  `(implicit_transitive_deps true)` otherwise. (ocaml/dune#11866, fixes ocaml/dune#11212, @nojb)

- Add `dune describe location` for printing the path to the executable that
  would be run (ocaml/dune#11905, @gridbugs)

- `dune runtest` can now understand absolute paths as well as run tests in
  specific build contexts (ocaml/dune#11936, @Alizter).

- Added 'empty' alias which contains no targets. (ocaml/dune#11556 ocaml/dune#11952 ocaml/dune#11955 ocaml/dune#11956,
  grants ocaml/dune#4161, @Alizter and @rgrinberg)

- Allow `dune promote` to properly run while a watch mode server is running
  (ocaml/dune#12010, @ElectreAAS)

- Add `--alias` and `--alias-rec` flags as an alternative to the `@@` and `@`
  syntax in the command line (ocaml/dune#12043, fixes ocaml/dune#5775, @rgrinberg)

- Added a `(timeout <float>)` field to the `(cram)` stanza to specify per-test
  time limits. Tests exceeding the timeout are terminated with an error.
  (ocaml/dune#12041, @Alizter)

### Changed

- Format long lists in s-expressions to fill the line instead of
  formatting them in a vertical way (ocaml/dune#10892, fixes ocaml/dune#10860, @nojb)

- Switch from MD5 to BLAKE3 for digesting targets and rules. BLAKE3 is both more
  performant and difficult to break than MD5 (ocaml/dune#11735, @rgrinberg, @Alizter)

- Print a warning when `dune build` runs over RPC (ocaml/dune#11833, @gridbugs)

- Stop emitting empty module group wrapper `.js` file in `melange.emit`
  (ocaml/dune#11987, fixes ocaml/dune#11986, @anmonteiro)
maiste added a commit to maiste/opam-repository that referenced this pull request Aug 5, 2025
CHANGES:

### Fixed

- Stop re-running cram tests after promotion when it's not necessary (ocaml/dune#11994,
  @rgrinberg)

- fix: `$ dune subst` should not fail when adding the version field in opam
  files (ocaml/dune#11801, fixes ocaml/dune#11045, @btjorge)

- Kill all processes in the process group after the main process has
  terminated; in particular this avoids background processes in cram tests to
  stick around after the test finished (ocaml/dune#11841, fixes ocaml/dune#11820, @Alizter,
  @Leonidas-from-XIV)

### Added

- `(tests)` stanzas now generate aliases with the test name. To run
  `(test (name a))` you can do `dune build @runtest-a`. (ocaml/dune#11558, grants part of ocaml/dune#10239,
  @Alizter)

- Inline test libraries now produce aliases `runtest-name_of_lib`
  allowing users to run specific inline tests as `dune build
  @runtest-name_of_lib`. (ocaml/dune#11109, partially fixes ocaml/dune#10239, @Alizter)

- feature: `$ dune subst` use version from `dune-project` when no version
  control repository has been detected (ocaml/dune#11801, @btjorge)

- Allow `dune exec` to run concurrently with another instance of dune in watch
  mode (ocaml/dune#11840, @gridbugs)

- Introduce `%{os}`, `%{os_version}`, `%{os_distribution}`, and `%{os_family}`
  percent forms. These have the same values as their opam counterparts.
  (ocaml/dune#11863, @rgrinberg)

- Introduce option `(implicit_transitive_deps false-if-hidden-includes-supported)`
  that is equivalent to `(implicit_transitive_deps false)` when `-H` is
  supported by the compiler (OCaml >= 5.2) and equivalent to
  `(implicit_transitive_deps true)` otherwise. (ocaml/dune#11866, fixes ocaml/dune#11212, @nojb)

- Add `dune describe location` for printing the path to the executable that
  would be run (ocaml/dune#11905, @gridbugs)

- `dune runtest` can now understand absolute paths as well as run tests in
  specific build contexts (ocaml/dune#11936, @Alizter).

- Added 'empty' alias which contains no targets. (ocaml/dune#11556 ocaml/dune#11952 ocaml/dune#11955 ocaml/dune#11956,
  grants ocaml/dune#4161, @Alizter and @rgrinberg)

- Allow `dune promote` to properly run while a watch mode server is running
  (ocaml/dune#12010, @ElectreAAS)

- Add `--alias` and `--alias-rec` flags as an alternative to the `@@` and `@`
  syntax in the command line (ocaml/dune#12043, fixes ocaml/dune#5775, @rgrinberg)

- Added a `(timeout <float>)` field to the `(cram)` stanza to specify per-test
  time limits. Tests exceeding the timeout are terminated with an error.
  (ocaml/dune#12041, @Alizter)

### Changed

- Format long lists in s-expressions to fill the line instead of
  formatting them in a vertical way (ocaml/dune#10892, fixes ocaml/dune#10860, @nojb)

- Switch from MD5 to BLAKE3 for digesting targets and rules. BLAKE3 is both more
  performant and difficult to break than MD5 (ocaml/dune#11735, @rgrinberg, @Alizter)

- Print a warning when `dune build` runs over RPC (ocaml/dune#11833, @gridbugs)

- Stop emitting empty module group wrapper `.js` file in `melange.emit`
  (ocaml/dune#11987, fixes ocaml/dune#11986, @anmonteiro)
maiste added a commit to maiste/opam-repository that referenced this pull request Aug 6, 2025
CHANGES:

### Fixed

- Stop re-running cram tests after promotion when it's not necessary (ocaml/dune#11994,
  @rgrinberg)

- fix: `$ dune subst` should not fail when adding the version field in opam
  files (ocaml/dune#11801, fixes ocaml/dune#11045, @btjorge)

- Kill all processes in the process group after the main process has
  terminated; in particular this avoids background processes in cram tests to
  stick around after the test finished (ocaml/dune#11841, fixes ocaml/dune#11820, @Alizter,
  @Leonidas-from-XIV)

### Added

- `(tests)` stanzas now generate aliases with the test name. To run
  `(test (name a))` you can do `dune build @runtest-a`. (ocaml/dune#11558, grants part of ocaml/dune#10239,
  @Alizter)

- Inline test libraries now produce aliases `runtest-name_of_lib`
  allowing users to run specific inline tests as `dune build
  @runtest-name_of_lib`. (ocaml/dune#11109, partially fixes ocaml/dune#10239, @Alizter)

- feature: `$ dune subst` use version from `dune-project` when no version
  control repository has been detected (ocaml/dune#11801, @btjorge)

- Allow `dune exec` to run concurrently with another instance of dune in watch
  mode (ocaml/dune#11840, @gridbugs)

- Introduce `%{os}`, `%{os_version}`, `%{os_distribution}`, and `%{os_family}`
  percent forms. These have the same values as their opam counterparts.
  (ocaml/dune#11863, @rgrinberg)

- Introduce option `(implicit_transitive_deps false-if-hidden-includes-supported)`
  that is equivalent to `(implicit_transitive_deps false)` when `-H` is
  supported by the compiler (OCaml >= 5.2) and equivalent to
  `(implicit_transitive_deps true)` otherwise. (ocaml/dune#11866, fixes ocaml/dune#11212, @nojb)

- Add `dune describe location` for printing the path to the executable that
  would be run (ocaml/dune#11905, @gridbugs)

- `dune runtest` can now understand absolute paths as well as run tests in
  specific build contexts (ocaml/dune#11936, @Alizter).

- Added 'empty' alias which contains no targets. (ocaml/dune#11556 ocaml/dune#11952 ocaml/dune#11955 ocaml/dune#11956,
  grants ocaml/dune#4161, @Alizter and @rgrinberg)

- Allow `dune promote` to properly run while a watch mode server is running
  (ocaml/dune#12010, @ElectreAAS)

- Add `--alias` and `--alias-rec` flags as an alternative to the `@@` and `@`
  syntax in the command line (ocaml/dune#12043, fixes ocaml/dune#5775, @rgrinberg)

- Added a `(timeout <float>)` field to the `(cram)` stanza to specify per-test
  time limits. Tests exceeding the timeout are terminated with an error.
  (ocaml/dune#12041, @Alizter)

### Changed

- Format long lists in s-expressions to fill the line instead of
  formatting them in a vertical way (ocaml/dune#10892, fixes ocaml/dune#10860, @nojb)

- Switch from MD5 to BLAKE3 for digesting targets and rules. BLAKE3 is both more
  performant and difficult to break than MD5 (ocaml/dune#11735, @rgrinberg, @Alizter)

- Print a warning when `dune build` runs over RPC (ocaml/dune#11833, @gridbugs)

- Stop emitting empty module group wrapper `.js` file in `melange.emit`
  (ocaml/dune#11987, fixes ocaml/dune#11986, @anmonteiro)
maiste added a commit to maiste/opam-repository that referenced this pull request Aug 12, 2025
CHANGES:

### Fixed

- Stop re-running cram tests after promotion when it's not necessary (ocaml/dune#11994,
  @rgrinberg)

- fix: `$ dune subst` should not fail when adding the version field in opam
  files (ocaml/dune#11801, fixes ocaml/dune#11045, @btjorge)

- Kill all processes in the process group after the main process has
  terminated; in particular this avoids background processes in cram tests to
  stick around after the test finished (ocaml/dune#11841, fixes ocaml/dune#11820, @Alizter,
  @Leonidas-from-XIV)

### Added

- `(tests)` stanzas now generate aliases with the test name. To run
  `(test (name a))` you can do `dune build @runtest-a`. (ocaml/dune#11558, grants part of ocaml/dune#10239,
  @Alizter)

- Inline test libraries now produce aliases `runtest-name_of_lib`
  allowing users to run specific inline tests as `dune build
  @runtest-name_of_lib`. (ocaml/dune#11109, partially fixes ocaml/dune#10239, @Alizter)

- feature: `$ dune subst` use version from `dune-project` when no version
  control repository has been detected (ocaml/dune#11801, @btjorge)

- Allow `dune exec` to run concurrently with another instance of dune in watch
  mode (ocaml/dune#11840, @gridbugs)

- Introduce `%{os}`, `%{os_version}`, `%{os_distribution}`, and `%{os_family}`
  percent forms. These have the same values as their opam counterparts.
  (ocaml/dune#11863, @rgrinberg)

- Introduce option `(implicit_transitive_deps false-if-hidden-includes-supported)`
  that is equivalent to `(implicit_transitive_deps false)` when `-H` is
  supported by the compiler (OCaml >= 5.2) and equivalent to
  `(implicit_transitive_deps true)` otherwise. (ocaml/dune#11866, fixes ocaml/dune#11212, @nojb)

- Add `dune describe location` for printing the path to the executable that
  would be run (ocaml/dune#11905, @gridbugs)

- `dune runtest` can now understand absolute paths as well as run tests in
  specific build contexts (ocaml/dune#11936, @Alizter).

- Added 'empty' alias which contains no targets. (ocaml/dune#11556 ocaml/dune#11952 ocaml/dune#11955 ocaml/dune#11956,
  grants ocaml/dune#4161, @Alizter and @rgrinberg)

- Allow `dune promote` to properly run while a watch mode server is running
  (ocaml/dune#12010, @ElectreAAS)

- Add `--alias` and `--alias-rec` flags as an alternative to the `@@` and `@`
  syntax in the command line (ocaml/dune#12043, fixes ocaml/dune#5775, @rgrinberg)

- Added a `(timeout <float>)` field to the `(cram)` stanza to specify per-test
  time limits. Tests exceeding the timeout are terminated with an error.
  (ocaml/dune#12041, @Alizter)

### Changed

- Format long lists in s-expressions to fill the line instead of
  formatting them in a vertical way (ocaml/dune#10892, fixes ocaml/dune#10860, @nojb)

- Switch from MD5 to BLAKE3 for digesting targets and rules. BLAKE3 is both more
  performant and difficult to break than MD5 (ocaml/dune#11735, @rgrinberg, @Alizter)

- Print a warning when `dune build` runs over RPC (ocaml/dune#11833, @gridbugs)

- Stop emitting empty module group wrapper `.js` file in `melange.emit`
  (ocaml/dune#11987, fixes ocaml/dune#11986, @anmonteiro)
maiste added a commit to maiste/opam-repository that referenced this pull request Aug 19, 2025
CHANGES:

### Fixed

- Stop re-running cram tests after promotion when it's not necessary (ocaml/dune#11994,
  @rgrinberg)

- fix: `$ dune subst` should not fail when adding the version field in opam
  files (ocaml/dune#11801, fixes ocaml/dune#11045, @btjorge)

- Kill all processes in the process group after the main process has
  terminated; in particular this avoids background processes in cram tests to
  stick around after the test finished (ocaml/dune#11841, fixes ocaml/dune#11820, @Alizter,
  @Leonidas-from-XIV)

### Added

- `(tests)` stanzas now generate aliases with the test name. To run
  `(test (name a))` you can do `dune build @runtest-a`. (ocaml/dune#11558, grants part of ocaml/dune#10239,
  @Alizter)

- Inline test libraries now produce aliases `runtest-name_of_lib`
  allowing users to run specific inline tests as `dune build
  @runtest-name_of_lib`. (ocaml/dune#11109, partially fixes ocaml/dune#10239, @Alizter)

- feature: `$ dune subst` use version from `dune-project` when no version
  control repository has been detected (ocaml/dune#11801, @btjorge)

- Allow `dune exec` to run concurrently with another instance of dune in watch
  mode (ocaml/dune#11840, @gridbugs)

- Introduce `%{os}`, `%{os_version}`, `%{os_distribution}`, and `%{os_family}`
  percent forms. These have the same values as their opam counterparts.
  (ocaml/dune#11863, @rgrinberg)

- Introduce option `(implicit_transitive_deps false-if-hidden-includes-supported)`
  that is equivalent to `(implicit_transitive_deps false)` when `-H` is
  supported by the compiler (OCaml >= 5.2) and equivalent to
  `(implicit_transitive_deps true)` otherwise. (ocaml/dune#11866, fixes ocaml/dune#11212, @nojb)

- Add `dune describe location` for printing the path to the executable that
  would be run (ocaml/dune#11905, @gridbugs)

- `dune runtest` can now understand absolute paths as well as run tests in
  specific build contexts (ocaml/dune#11936, @Alizter).

- Added 'empty' alias which contains no targets. (ocaml/dune#11556 ocaml/dune#11952 ocaml/dune#11955 ocaml/dune#11956,
  grants ocaml/dune#4161, @Alizter and @rgrinberg)

- Allow `dune promote` to properly run while a watch mode server is running
  (ocaml/dune#12010, @ElectreAAS)

- Add `--alias` and `--alias-rec` flags as an alternative to the `@@` and `@`
  syntax in the command line (ocaml/dune#12043, fixes ocaml/dune#5775, @rgrinberg)

- Added a `(timeout <float>)` field to the `(cram)` stanza to specify per-test
  time limits. Tests exceeding the timeout are terminated with an error.
  (ocaml/dune#12041, @Alizter)

### Changed

- Format long lists in s-expressions to fill the line instead of
  formatting them in a vertical way (ocaml/dune#10892, fixes ocaml/dune#10860, @nojb)

- Switch from MD5 to BLAKE3 for digesting targets and rules. BLAKE3 is both more
  performant and difficult to break than MD5 (ocaml/dune#11735, @rgrinberg, @Alizter)

- Print a warning when `dune build` runs over RPC (ocaml/dune#11833, @gridbugs)

- Stop emitting empty module group wrapper `.js` file in `melange.emit`
  (ocaml/dune#11987, fixes ocaml/dune#11986, @anmonteiro)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Background tasks in Cram tests are not terminated

3 participants