Fix handling of cos_object_prefix pipeline property

[kiersten-stokes]

Prior to this PR, pipelines with a cos_object_prefix defined will fail to run on main and the 3.12 release.

What changes were proposed in this pull request?

This PR does a handful of related things. First, the pipeline_parameters property on the pipeline_definition.py Pipeline object is removed - it is no longer necessary now that cos_object_prefix is stored in the pipeline_defaults stanza of the pipeline JSON. (Prior to the changes in #2780, cos_object_prefix existed as a sibling to the pipeline_defaults stanza). The PIPELINE_META_PROPERTIES constant is also no longer needed and is removed. A handful of helper methods are added to component_parameter.py and, as a result, the convert_elyra_owned_properties methods are simplified.

The key-value pairs in the pipeline_defaults stanza are passed to the pipeline.py Pipeline object and accessed by the runtime processors in the same way as before, but now this property is named pipeline_properties rather than pipeline_parameters (in order to prepare for an implementation of #2936).

Of note here is that no pipeline migration is necessary right now. In order to avoid a migration so soon in 3.13 (3.12 just required migration), a workaround is added to the backend that supports cos_object_prefix both as a child of the pipeline_defaults and as a sibling. See pipeline_definition:191:197.

How was this pull request tested?

• Airflow - a case was added to an existing test to cover cos_object_prefix functionality; existing case that uses the same sample pipeline was also updated accordingly
• KFP -added a new test (test_create_yaml_complex_pipeline) to cover cos_object_prefix and other functionality; mimics the create_test_file test in test_airflow_processor.py
• Manual test with cos_object_prefix defined in the pipeline_defaults stanza of the pipeline json
• Manual test with cos_object_prefix defined in the app_data stanza of the pipeline json

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

[elyra-bot]

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link:

[ptitzler]

LGTM!

[akchinSTC]

FYI, I think using | to represent the typehint Unions only work for python >=3.10.

anywho not a biggie, LGTM

[kevin-bates]

FYI, I think using | to represent the typehint Unions only work for python >=3.10.

Older python versions can use that syntax with from __future__ import annotations. That said, I've found that the overrides package (which we're exploring) has heartburn equating, for example, signatures of type | None to Optional[type] (or Union[type, None]) and when either signature is not in the local package, you're forced to choose the approach of the foreign signature (speaking from experience). So I guess my point is that we should be consistent. 😄

anywho not a biggie

Agreed.