diagnostic not correct for nested tests

[hildjj]

Given this test:

import test from "node:test";

test("root", t => {
  t.diagnostic("one");
  t.test("child", t => {
    t.diagnostic("two");
  });
  t.diagnostic("three");
});

I expect to get this result:

{
  "tests": [
    {
      "name": "root",
      "file": "dm.test.js",
      "tests": [
        {
          "name": "child",
          "file": "dm.test.js",
          "tests": [],
          "duration": 0.315334,
          "skip": false,
          "todo": false,
          "diagnostic": "two"
        }
      ],
      "duration": 0.731083,
      "skip": false,
      "todo": false,
      "diagnostic": "one\nthree\n"
    }
  ],
  "duration": 50.446458
}

But instead I get this:

{
  "tests": [
    {
      "name": "root",
      "file": "dm.test.js",
      "tests": [
        {
          "name": "child",
          "file": "dm.test.js",
          "tests": [],
          "duration": 0.315334,
          "skip": false,
          "todo": false
        }
      ],
      "duration": 0.731083,
      "skip": false,
      "todo": false,
      "diagnostic": "two\n"
    }
  ],
  "duration": 50.446458
}

Note that the diagnostic "two" is reported on the root test, the child test has no diagnostic, and the diagnostics "one" and "three" are lost.

[hildjj]

I've been working on a fix for this, but it's going to require a little more delicate work than expected. The "test:diagnostic" events are fired between "test:dequeue" and "test:complete", not between "test:start" and "test:pass" / "test:fail". Also, the test runner framework sends "test:diagnostic" itself at the end of a run, with no filePath.

[hildjj]

Also, "test:complete" may fire after the "test:dequeue" for the next test.

[hildjj]

Also, "test:complete" may fire before "test:start" for the same test.

[dosubot]

Hi @hildjj! I'm Dosu and I’m helping the node-test-parser team.

The parser currently doesn't handle the case where 'test:complete' fires before 'test:start' for the same test, which can lead to diagnostics and results being misattributed or lost. The event handling logic relies on a stack that expects 'test:start' before 'test:complete', so out-of-order events disrupt context tracking and diagnostic assignment. Robust handling of these scenarios would require buffering or correlating events by test ID, rather than strict stack order, to ensure diagnostics and results are correctly attributed even when events arrive out of sequence. This is a known challenge in the parser's design and isn't addressed in recent code or discussions. You can review the relevant parser logic here.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[hildjj]

@dosu correlating by testID (file, line, column) works. PR incoming once I've cleaned it up and added some more tests.

[hildjj]

@dosu please see #209.

[dosubot]

PR #209 is open and specifically targets the issue of diagnostics being misattributed in nested tests. The proposed fix assigns diagnostics to the correct test, addressing the problem where diagnostics like "one" and "three" were lost and "two" was misattributed to the parent test. The PR also includes dependency updates and is designed to handle cases where test events arrive out of order, such as "test:complete" firing before "test:start" for the same test. You can review the PR and its details here.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}