Harmonizing requirements between ARC (ARC scaffold) and ARC RO-Crate specifications

[kMutagene]

This is a necessary discussion as part of the ongoing efforts to implement the 'other side of the coin' representation of the ARC - the ARC RO-Crate.

Preface on mandatory information

The ARC ecosystem was initially created and grown with the user-facing ARC scaffold representation. To ease the process of manually creating an ever-evolving FAIR digital object, the specification for this ARC representation only specifies the mandatory structure of the information, without formulating requirements for the information itself. Any further requirements - e.g. ARCs in which information must satisfy domain-specific requirements - are formulated via the pull-model of ARC validation in regards to a target.

Now, the other side of the coin - the ARC RO-crate - is an increasing focus of development, as it will facilitate machine interaction with ARCs. We provide both sides of the coin by attaching the ARC RO-crate metadata as a json file to the ARC scaffold hosted on the DataHUB. However, there are differences in the required information that these formats have that need to be addressed.

Mandatory fields for being a valid RO-crate

In the JSON-LD world of RO-crates, there are some mandatory properties that MUST be present. Aside from technicalities, the relevant ones are:

• name
• description
• datePublished
• license

We must either find 1-to-1 mappings in the ARC scaffold for these or provide sensible default values. This is done for all other than license IIRC, and here is the respective discussion

Examples for maybe being too strict

• Our ISA profile has a mandatory field agent for each process. This is not the case for the ARC spec and may be unnecessary.
• Each process in our ISA profile must have input AND output. However, in the ARC scaffold (and ISA in general) you can create a process without output.

Examples for maybe being too lax

• A Sample input column can have an empty value according to ARC spec. This makes no sense, as it must be possible to identify a sample

I encourage others to have a look at both specs and give input on where to be more strict and where to be more lax:

• ARC specification (this is the spec of the ARC scaffold, ISA XLSX spec might be most relevant)
• ISA RO-crate profile (this is the most relevant of our RO crate profiles)

cc @HLWeil @floWetzels @muehlhaus @Brilator

[floWetzels]

@kMutagene Can you explain the Sample input example in more detail? Are you talking about the ARC scaffold spec? It would then still be an example for the RO-Crate being stricter than the ARC scaffold, right?

[HLWeil]

Yes the RO-Crate specification is stricter here in all case. The too strict and too lax could be translated here to relax ARC RO-Crate and stricten ARC Scaffold, though.

In more detail, staying with the example of Process output:

1. Outputs of Processes
   In ARC RO-Crate, the bio:LabProcess requires to have a value for the property schema:result. In Scaffold, this translates to an Annotation Table to require the output column, which is currently not the case. I think there are good arguments for both cases, but what we argue for here is that it would be possible to align the specs by relaxing ARC RO Crate by setting the object property to SHOULD.
2. Samples (e.g. as Process outputs)
   So now let's take a look at the actual ouput values of a process, in our example a Sample. In ARC RO-Crate, the bio:Sample requires to have a value for the property schema:name. In Scaffold, this translates to an entry in the Output [Sample] column to not be empty. This name is important for many different things, e.g. @id generation in JSON-LD or querying on the ARC Scaffold side. So we argue for align the specs by strictening the ARC Scaffold by requring to fill out all cells of Input and Output columns.

[HLWeil]

Another case, the PropertyValue:

In RO-Crate, both the name and the value property are defined as MUST. In ARC-Scaffold though, is was fine until now to leave cells for columns mapping to PropertyValue empty (e.g. Factor, Parameter, Characterstic). This is especially often the case when templates are used, but not immediately filled out.

Consider the following example table.

Input [Sample]	Chara [species]	Param [temp]	Output [Sample]
MyIn1	Arabidopsis		MyOut1
MyIn2		5° Cel	MyOut2

With the current RO-Crate profile, the Characteristic can't be properly depicted for the sample "MyIn2", as the value is missing in the tabular structure, but a value is required in RO-Crate.

So either, we set a random value (which I think is problematic here semantically) or we the leave the Characteristic out in RO-Crate. The second option would be problematic if we regard RO-Crate as a true exchange format, as the information gets lost on write and read.

My suggestion here would be to set the value property to SHOULD.

[kMutagene]

This is especially often the case when templates are used, but not immediately filled out

Note that we have no way of preserving empty tables (meaning a table with only the header row) with the RO-Crate profile, since the split information of Header cell + value cells is combined here. I do not think that this is a general issue worth "fixing", as the only solution i could come up with is creating a "dummy process" in the RO crate that has no values. With your proposal, this issue goes away once at least a single cell with a value exists, leading to all the other cells being property values without a value.

[kMutagene]

JFC here is the now closed epic for the isa ro crate profiles: nfdi4plants/ARC-specification#135

We are doing the same for the workflow/run part here: nfdi4plants/ARC-specification#150

[HLWeil]

Following a discussion between @muehlhaus @caroott and me:

Workflow and Run ARC Scaffold/RO-Crate-harmonization

In general, the discussion was about the topic of how and where to depict the information from the Workflow Run Crates in the ARC-Scaffold. Here we named two different types of information:

• Actionable metadata (prospective / cwl)
• Documentary metadata (retrospective / xlsx)

Conceptual explanations and ideas for representation in the ARC Scaffold for these two metadata follows:

Documentary metadata

The documentary metadata is meant to give additional information about the workflows and runs which is not required for actionability and can not be auto-generated from interpreting actionable files. This includes the process sequence, which should track the experimental workflow all the way from the source materials to the final computed result files.
It is not finally decided what specification we should use for this, but this is currently stored in the newly specified isa.workflow.xlsx and isa.run.xlsx files. Especially the isa.run.xlsx closely resembles an isa.assay.xlsx. Alternatives for this would be storage in YAML format, allowing for direct inclusion in the .cwl files.

Actionable metadata

The actionable metadata is meant to contain information necessary to reproducably run tools. As often discussed and as defined in the specification, this metadata is stored in cwl (Common Workflow Language) files.

But now we have the situation that the ARC RO-Crate will not only be produced and used by us, and other user-groups (like Galaxy) prefer other workflow languages than CWL (Galaxy Workflows, Nextflow etc.). The Workflow-Run-Crate (and by extension our profile) is unbiased towards the choice of workflows. Therefore, there is a strong incentive to handle also those cases where an ARC RO-Crate contains non-cwl workflows.

This includes mapping the ARC RO-Crate to ARC Scaffold, having the workflows runnable in methods native to ARC Scaffold and mapping it back to ARC RO-Crate in such a way that does not enforce usage of cwl in the RO-Crate representation.

To this effect here's the concept of wrapper CWL:

Wrapper CWL and other workflow languages

Wrapper CWLs are minimal CWL files which are used to wrap other workflow languages in the ARC Scaffold representation. They are assimilated into an Workflow description of the actual workflow file in the ARC RO-Crate representation.

These wrapper CWLs are NOT meant as generic wrappers for a group of wrapped workflows. Instead, Each Wrapper CWL is specific to the workflow it wraps, i.e. it contains the specific parameter names.

Example1

Let's say we have the following reduced workflow entity in the ARC RO-Crate (coming from RO documentation):

    {
        "@id": "Galaxy-Workflow-Hello_World.ga",
        "@type": ["File", "SoftwareSourceCode", "ComputationalWorkflow"],
        "programmingLanguage": {"@id": "https://w3id.org/workflowhub/workflow-ro-crate#galaxy"},
        "input": [
            {"@id": "#simple_input"}
        ],
        "output": [
            {"@id": "#reversed"}
        ]
    },

In the ARC Scaffold representation, we would map the input and output fields into a workflow.cwl file, and reference the "Galaxy-Workflow-Hello_World.ga", which will also be included in the workflow folder, like such:

flowchart TD

subgraph "Workflow Folder"
    ga1[Galaxy-Workflow-Hello_World.ga]
    w1[workflow.cwl]
end

w1 --wraps--> ga1

Loading

Example2: Orchestrated foreign workflows

Now, if the situation in the RO-Crate is more complex and the foreign workflows orchestrate each other, we might end up with the following ARC Scaffold:

flowchart TD

subgraph Workflow1
    ga1[step1.ga]
    w1[workflow.cwl]
end

subgraph Workflow2
    ga2[step2.ga]
    w2[workflow.cwl]
end

subgraph Run
    ga3[run.ga]
    y[run.yml]
    r[run.cwl]
end

ga3 --"orchestrates"--> ga2
w1 --wraps--> ga1
w2 --wraps--> ga2
r --wraps--> ga3

ga3 --"orchestrates"--> ga1

Loading

Notice three things:

• Each workflow/run contains a wrapping cwl file
• The run additionally contains a run.yml
• The "orchestration" is purely handled in the original .ga files

Handling process and workflow inputs

Another part of the actionable metadata is the concrete values given as the input parameters of the workflow in the context of a run/Create-Action. This is very important for reproducability, as only here these inputs are specified in a format parseable by the runner. In the ARC Scaffold, these values are stored in the run folder in the run.yml. In the ARC RO-Crate, these values will be stored in the object field of the CreateAction.
As stated before, we are not only interested in the actionable metadata but also want a complete process graph, connecting both the experimental and the computation part. To this effect, in the arc-wr-profile, we double typed the CreateAction with LabProcess, to store also the documentary metadata. The problem here is that the process inputs are also stored in the same object field.
A possible solution to differentiate between these two inputs might be looking for the exampleOfWork field inside the input entity. Those entities representing workflow inputs need to specify, which input parameter they fill.

If we take a look at the (reduced) example from the Workflow Run Crate specs again and already add the double typing with LabProcess, we might have the following:

{
    "@id": "#wfrun-5a5970ab-4375-444d-9a87-a764a66e3a47",
    "@type": ["CreateAction", "LabProcess"], 
    "name": "Galaxy workflow run 5a5970ab-4375-444d-9a87-a764a66e3a47",
    "instrument": {"@id": "Galaxy-Workflow-Hello_World.ga"},
    "object": [
        {"@id": "#verbose-pv"},
        {"@id": "MyInputFile.csv"}
    ]
},
{
   "@id": "MyInputFile.csv",
    "@type": "File"
},
{
    "@id": "#verbose-pv",
    "@type": "PropertyValue",
    "exampleOfWork": {"@id": "#verbose-param"},
    "name": "verbose",
    "value": "True"
}

Notice the exampleOfWork property of the input parameter value.

(Creating this example I noticed a possible issue with this apprach. In the original example, the CreateAction has two objects, one of the being a single file, which would be an object with the exact same @id in the LabProcess objects.

TLDR

• Disctinction between Actionable metadataand Documentary metadata
  • Actionable metadata stored in workflow.cwl, run.cwl and run.yml
  • Documentary metadata (currently) stored in isa.workflow.xlsx and isa.run.xlsx
• We want to allow usage of non-cwl workflows in ARC RO-Crate and ARC Scaffold (eg. Galaxy Workflows)
  • In ARC RO-Crate then no CWL
  • In ARC Scaffold wrapper CWL containing minimal information
• We need a method to distinguish between actionable and documentary Run-Crate inputs (both in the object property)
  • Checking ExampleOfWork property

@kMutagene @floWetzels

[caroott]

We could also move the .ga orchestrating the other .ga to the workflows folder:

flowchart TD
subgraph Workflow1
    ga1[step1.ga]
    w1[workflow.cwl]
end

subgraph Workflow2
    ga2[step2.ga]
    w2[workflow.cwl]
end

subgraph WorkflowCombined
    ga3[run.ga]
    w3[workflow.cwl]
end

subgraph Run
    y[run.yml]
    r[run.cwl]
end

ga3 --"orchestrates"--> ga2
w1 --wraps--> ga1
w2 --wraps--> ga2
w3 --wraps--> ga3
r --hasStep--> w3

ga3 --"orchestrates"--> ga1

Loading

This way we won't lose any information. We would require one additional wrapper, but we would move the compatibility problem to the workflows folder only, reducing the complexity.

[caroott]

(Creating this example I noticed a possible issue with this apprach. In the original example, the CreateAction has two objects, one of the being a single file, which would be an object with the exact same @id in the LabProcess objects.

In a discussion with @HLWeil we found another issue. Currently, a LabProcess contains only one object and result, directly linking those together. Multiple objects and results are part of a ProcessSequence. This can be done analogous for the multitype CreateAction/LabProcess. The problem is, that this would result in many duplicates of the "same" CreateAction, which would result in unnecessary bloat, since it generally has more properties than LabProcess.

One possible solution would be to separate LabProcess and CreateAction again and add the ProcessSequence to the ARC Run. This way we could also avoid the @id duplications.

Another solution could be to expand the possible types of result to allow the isBasedOn property. That would allow us to point to the source of a result, making it unambiguous and also allowing objects to be inputs of a workflow and source of a result. This change could also be adapted for the ISA RO Crate.

[kMutagene]

After further discussion with @caroott, i would suggest we add a separate profile for the "wrapper cwl", and expect an object following that profile when attempting to import/convert arc-wr crates that describe non-cwl orchestrations. That way we have a clean separation and no need to attempt any extraction of logic for auto-generating such files.

[caroott]

One point we also have to take into consideration is the executability of workflows described in a Workflow RO-Crate. The crate contains information about the executed workflow file, be it a .cwl or .ga file or another type of workflow and the expected inputs. In a run RO-Crate we also have the information about the actual values for the input.

What we are missing is the execution environment for the workflow. CWL for example requires an installed cwl runner. Galaxy would require a galaxy instance with corresponding running environment. Workflows can refer to the workflow type via the property programmingLanguage:

{
  "@id": "https://w3id.org/workflowhub/workflow-ro-crate#cwl",
  "@type": "ComputerLanguage",
  "name": "Common Workflow Language",
  "alternateName": "CWL",
  "identifier": {
    "@id": "https://w3id.org/cwl/v1.2/"
  },
  "url": {
    "@id": "https://www.commonwl.org/"
  }
}

or

{
  "@id": "https://w3id.org/workflowhub/workflow-ro-crate#galaxy",
  "@type": "ComputerLanguage",
  "name": "Galaxy",
  "identifier": {
    "@id": "https://galaxyproject.org/"
  },
  "url": {
    "@id": "https://galaxyproject.org/"
  }
}

By that we know for which type of workflow our environment should be set up. This would then still require a runner, that can parse this information and have preset containers or can install the required tools to be actually executable.

[kMutagene]

A possible solution to differentiate between these two inputs might be looking for the exampleOfWork field inside the input entity. Those entities representing workflow inputs need to specify, which input parameter they fill.

On another note, in addition to the distinction of process parameters and workflow inputs on the Workflow Invocation via exampleOfWork, we should discuss to potentially duplicate process parameters to the parameterValue property, as this is the position where parameters of ISA Assay/Study graphs are located. It would look weird from a query perspective to have this unified model via LabProcess, but use different fields for parameters. I get that we need to have these parameters in the input of the create action to be compatibloe with the existing workflow run profile, so IMO the only way to play nice with our ISA profile is to duplicate these properties.