Mark declarative config as stable

[jack-berg]

Resolves #4374.

We've cut 3 release candidates: https://github.com/open-telemetry/opentelemetry-configuration/releases

There are 4 languages up to date with the latest release, 1.0-rc.3:

• c++
• go
• java
• js

Additionally, php is up to date with 1.0-rc.2

Besides the schema as defined in opentelemetry-configuration, its also important to understand the status of implementations with respect to the spec language itself. This is tracked here: https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md#declarative-configuration

Summary of the items being stabilized:

• The schema for the data model, as defined in opentelemetry-configuration
• The YAML representation of the data model in files
• The in-memory representation of the data model
• The generic representation of a YAML mapping node, ConfigProperties
• The mechanism for referencing custom plugin components in the data model, PluginComponentProvider
• The SDK operations for parsing a YAML file and instantiating SDK components, Parse and Create
• The standard env var to indicate that declarative config should be used and to point to the path of a config file OTEL_CONFIG_FILE

cc @open-telemetry/configuration-maintainers

[jack-berg]

This is ready for review, but SHOULD not be merged. Assuming we get the required approvals, I'll coordinate merging this with cutting the corresponding release of opentelemetry-configuration.

[tsloughter]

A concern I have is that this sort of implies that a spec for the behavior of AutoConfiguredOpenTelemetrySdk.initialize().getOpenTelemetrySdk(); is stable, which I think requires that #4591 be done.

Can the spec of config be stable before SDK initialization is defined?

[trask]

🙌

[MikeGoldsmith]

Looks good.

FYI I've started on implementing in Python too.

[pellared]

FYI the Go implementation is not compliant with the current version of the specification.
For instance:

• otelconf: Register ComponentProvider opentelemetry-go-contrib#7564 (this one is important)
• [Bug]: Invalid Variable Substitution in Configuration Files opentelemetry-go-contrib#8405

I do not say that it is a blocker for stabilization.

Probably it would be also good to link to: https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md#declarative-configuration

[jack-berg]

I do not say that it is a blocker for stabilization. However, personally I would not list Go as being up to date.

Yes there are different dimensions for an implementation. Go is up to date with the latest schema, but SDK implementation isn't complete. From my perspective, the signal I want is whether the implementers / maintainers of Go see problems with the spec that would prohibit implementing.

open-telemetry/opentelemetry-go-contrib#7564 (this one is important)

Yup this one is important 👍

open-telemetry/opentelemetry-go-contrib#8405

The go implementation is less restrictive than the spec. I suspect this is for compatibility with collector substitution? If yes, the difference in substitution between collector and declarative config spec is known and intentional. The go implementation is in an interesting position because declarative config is used within the collector and outside the collector. Maybe this suggests a special flag which enables less restrictive substitution to match the collector semantics, which are disabled by default outside of the collector. Can discuss more on the issue.

Probably it would be also good to link to: https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md#declarative-configuration

👍 updated PR description

[pellared]

Regarding

From my perspective, the signal I want is whether the implementers / maintainers of Go see problems with the spec that would prohibit implementing.

My biggest concern is that I am not sure if any implementation allows "nesting" of plugins e.g. making exporter wrappers. Related PR: #4407. Is this post v1 or should it already be supported? Or is it out of scope?

[jack-berg]

My biggest concern is that I am not sure if any implementation allows "nesting" of plugins e.g. making exporter wrappers. Related PR: #4407. Is this post v1 or should it already be supported? Or is it out of scope?

The ComponentPluginProvider mechanism allows you to define custom exporters which wrap others. For example, I could write:

file_format: 1.0-rc.3
logger_provider:
  processors:
    batch:
      exporter:
        filtering_exporter:
          predicate: // some custom DSL syntax for filtering log records
          delegate:
            otlp_http:
              endpoint: http://endpoint:4318

With the interpretation:

• Define and register (via PluginComponentProvider mechanism) a custom log record exporter called filtering_exporter
• filtering exporter delegates to another exporter (in this case, the built-in OTLP http exporter) after filtering records according to some predicate

It would be easier to do if there were standard built-in exporters for standard wrapper / nesting / delegation tasks, but it is possible. This task is common for samplers, which use composition models in a variety of the built-in implementations.

[jpkrohling]

I believe that this is the biggest quality of life improvement that OTel users can have at the moment. I'm very excited about this one!

[trask]

Congrats @open-telemetry/configuration-approvers and everyone else who contributed to this effort!!