[service] Move batchprocessor metrics to normal level and update level guidelines (#12525)

#### Description

This PR:
- requires "level: normal" before outputting batch processor metrics (in
addition to one specific metric which was already restricted to "level:
detailed")
- clarifies wording in the telemetry level guidelines and documentation,
and adds said guidelines to the requirements for stable components.

Some rationale for these changes can be found in the tracking issue and
[this
comment](#7890 (comment)).

#### Link to tracking issue
Resolves #7890

#### To be discussed

Should we add a feature gate for this, in case a user relies on "level:
basic" outputting batch processor metrics? This feels like a niche use
case, so considering the "alpha" stability level of these metrics, I
don't think it's really necessary.

Considering batch processor metrics had already been switched to
"normal" once (#9767), but were turned back to basic at some later point
(not sure when), we might also want to add tests to avoid further
regressions (especially as the handling of telemetry levels is bound to
change further with #11754).

---------

Co-authored-by: Dmitrii Anoshin <[email protected]>

Author: jade-guiton-dd

.chloggen/telemetry-level-update.yaml

@@ -0,0 +1,33 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: breaking
+
+# The name of the component, or a single word describing the area of concern, (e.g. otlpreceiver)
+component: service
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Batch processor telemetry is no longer emitted at "basic" verbosity level
+
+# One or more tracking issues or pull requests related to the change
+issues: [7890]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext: |
+  According to the guidelines, basic-level telemetry should be reserved for core Collector APIs.
+  Components such as the batch processor should emit telemetry starting from the "normal" level
+  (which is also the default level).
+
+  Migration: If your Collector telemetry was set to `level: basic` and you want to keep seeing
+  batch processor-related metrics, consider switching to `level: normal` instead.
+
+  
+
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: [user]

config/configtelemetry/configtelemetry.go

@@ -10,13 +10,13 @@ import (
 )
 
 const (
-	// LevelNone indicates that no telemetry data should be collected.
+	// LevelNone indicates that no telemetry should be collected.
 	LevelNone Level = iota - 1
-	// LevelBasic is the recommended and covers the basics of the service telemetry.
+	// LevelBasic indicates that only core Collector telemetry should be collected.
 	LevelBasic
-	// LevelNormal adds some other indicators on top of basic.
+	// LevelNormal indicates that all low-overhead telemetry should be collected.
 	LevelNormal
-	// LevelDetailed adds dimensions and views to the previous levels.
+	// LevelDetailed indicates that all available telemetry should be collected.
 	LevelDetailed
 
 	levelNoneStr     = "None"

config/configtelemetry/doc.go

@@ -15,33 +15,28 @@
 //
 // 2. configtelemetry.Basic
 //
-// Telemetry associated with this level provides essential coverage of the collector telemetry.
-// It should only be used for internal collector telemetry generated by the collector core API. Components outside of
-// the core API MUST NOT record additional telemetry at this level.
+// Telemetry associated with this level provides essential coverage of the Collector telemetry.
+// It should only be used for telemetry generated by the core Collector API.
+// Components outside of the core API MUST NOT record telemetry at this level.
 //
 // 3. configtelemetry.Normal
 //
-// Telemetry associated with this level provides complete coverage of the collector telemetry.
+// Telemetry associated with this level provides intermediate coverage of the Collector telemetry.
 // It should be the default for component authors.
 //
-// Component authors using this telemetry level can use this guidance:
-//
-//   - The signals associated with this level must control cardinality.
-//     It is acceptable at this level for cardinality to scale linearly with the monitored resources.
-//
-//   - The signals associated with this level must represent a controlled data volume. Examples follow:
+// Normal-level telemetry should have limited cardinality and data volume, though it is acceptable
+// for them to scale linearly with the monitored resources.
+// For example, there may be a limit of 5 attribute sets or 5 spans generated per request.
 //
-//     a. A max cardinality (total possible combinations of dimension values) for a given metric of at most 100.
-//
-//     b. At most 5 spans actively recording simultaneously per active request.
+// Normal-level telemetry should also have a low computational cost: it should not contain values
+// requiring significant additional computation compared to the normal flow of processing.
 //
 // This is the default level recommended when running the Collector.
 //
 // 4. configtelemetry.Detailed
 //
 // Telemetry associated with this level provides complete coverage of the collector telemetry.
 //
-// The signals associated with this level may exhibit high cardinality and/or high dimensionality.
-//
-// There is no limit on data volume.
+// The signals associated with this level may exhibit high cardinality, high data volume, or high
+// computational cost.
 package configtelemetry // import "go.opentelemetry.io/collector/config/configtelemetry"

docs/component-stability.md

@@ -214,6 +214,9 @@ If data can be dropped/created/held at multiple distinct points in a component's
 scraping, validation, processing, etc.), it is recommended to define additional attributes to help
 diagnose the specific source of the discrepancy, or to define different signals for each.
 
+The breakdown of emitted telemetry per telemetry level (basic / normal / detailed) should follow
+the guidelines in [the Go package documentation for `configtelemetry`](/config/configtelemetry/doc.go).
+
 ### Deprecated
 
 The component is planned to be removed in a future version and no further support will be provided. Note that new issues will likely not be worked on. When a component enters "deprecated" mode, it is expected to exist for at least two minor releases. See the component's readme file for more details on when a component will cease to exist.

service/service.go

@@ -509,8 +509,12 @@ func configureViews(level configtelemetry.Level) []config.View {
 	}
 
 	// Batch processor metrics
-	if level < configtelemetry.LevelDetailed {
-		scope := ptr("go.opentelemetry.io/collector/processor/batchprocessor")
+	scope := ptr("go.opentelemetry.io/collector/processor/batchprocessor")
+	if level < configtelemetry.LevelNormal {
+		views = append(views, dropViewOption(&config.ViewSelector{
+			MeterName: scope,
+		}))
+	} else if level < configtelemetry.LevelDetailed {
 		views = append(views, dropViewOption(&config.ViewSelector{
 			MeterName:      scope,
 			InstrumentName: ptr("otelcol_processor_batch_batch_send_size_bytes"),