Skip to content

feat: enhance JSON output for multi-profile validation#74

Merged
kikkomep merged 8 commits intocrs4:developfrom
kikkomep:feat/improve-multiprofile-validation-output
Mar 7, 2025
Merged

feat: enhance JSON output for multi-profile validation#74
kikkomep merged 8 commits intocrs4:developfrom
kikkomep:feat/improve-multiprofile-validation-output

Conversation

@kikkomep
Copy link
Member

This PR refines the JSON output generated by validating a rocrate that requires multiple profiles, not belonging to the same inheritance hierarchy.

The main changes are as follows:

  • the output will be a single JSON object, unlike before when a separate JSON was produced for each validation profile used
  • the profile_identifier property is replaced by profile_identifiers, which will be populated with the list of validation profiles used
  • the issues property will contain the list of all issues encountered during the crate validation across the various involved validation profiles

@simleo
Copy link
Member

simleo commented Feb 25, 2025

I tried the following and got an error:

$ rocrate-validator -y validate -v -f json -o err.json -l REQUIRED tests/data/crates/valid/multi-profile-crate/
         This error may be due to a bug.
         Please report it to the issue tracker along with the following stack trace:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/simleo/git/rocrate-validator/rocrate_validator/cli/commands/validate.py:439 in validate    │
│                                                                                                  │
│   436 │   │   │   # Print the validation report to a file                                        │
│   437 │   │   │   if output_file:                                                                │
│   438 │   │   │   │   with open(output_file, "w") as f:                                          │
│ ❱ 439 │   │   │   │   │   f.write(json_output)                                                   │
│   440 │   │                                                                                      │
│   441 │   │   # using ctx.exit seems to raise an Exception that gets caught below,               │
│   442 │   │   # so we use sys.exit instead.                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: write() argument must be str, not dict


[ERROR] 

[FAILED] Unexpected error: write() argument must be str, not dict !!!

BTW, tests/data/crates/valid/multi-profile-crate does not seem to be valid with respect to https://w3id.org/ro/wftest/0.1, there is no test suite.

@simleo
Copy link
Member

simleo commented Feb 26, 2025

I ran:

rocrate-validator -y validate -v -f json -o err.json -l REQUIRED tests/data/crates/invalid/0_multi_profile_crate/

and got the attached err.json. profile_identifiers lists both workflow-testing-ro-crate-0.1 and provenance-run-crate-0.5, but no validation seems to have been performed with respect to workflow-testing-ro-crate-0.1: the fact that the root data entity does not refer to test suites via mentions (a MUST) is not reported.

Another weird thing is that if I set -p workflow-testing-ro-crate I only get issues if the output is in text format, while with JSON output I get this:

{
    "meta": {
        "version": "0.2"
    },
    "validation_settings": {
        "profile_identifier": "workflow-testing-ro-crate-0.1",
        "enable_profile_inheritance": true,
        "abort_on_first": false,
        "requirement_severity": "REQUIRED",
        "rocrate_validator_version": "0.6.1_bd0cc32+448-dirty"
    },
    "passed": false,
    "issues": []
}

@kikkomep kikkomep merged commit 079fed8 into crs4:develop Mar 7, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants