Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 18 additions & 15 deletions content/en/docs/collector/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ service:
pipelines: # section that can contain multiple subsections, one per pipeline
traces: # type of the pipeline
receivers: [otlp, zipkin]
processors: [memory_limiter, batch]
processors: [memory_limiter]
exporters: [otlp, zipkin]
```

Expand All @@ -102,7 +102,7 @@ service:
pipelines:
traces: # a pipeline of “traces” type
receivers: [otlp]
processors: [memory_limiter, batch]
processors: [memory_limiter]
exporters: [otlp]
traces/2: # another pipeline of “traces” type
receivers: [otlp]
Expand Down Expand Up @@ -210,26 +210,29 @@ The same name of the processor can be referenced in the `processors` key of
multiple pipelines. In this case, the same configuration is used for each of
these processors, but each pipeline always gets its own instance of the
processor. Each of these processors has its own state, and the processors are
never shared between pipelines. For example, if `batch` processor is used in
several pipelines, each pipeline has its own batch processor, but each batch
processor is configured exactly the same way if they reference the same key in
the configuration. See the following configuration:
never shared between pipelines. For example, if the `transform` processor is
used in several pipelines, each pipeline has its own transform processor, but
each transform processor is configured exactly the same way if they reference
the same key in the configuration. See the following configuration:

```yaml
processors:
batch:
send_batch_size: 10000
timeout: 10s
transform:
error_mode: ignore
trace_statements:
- set(resource.attributes["namespace"],
resource.attributes["k8s.namespace.name"])
- delete_key(resource.attributes, "k8s.namespace.name")

service:
pipelines:
traces: # a pipeline of “traces” type
receivers: [zipkin]
processors: [batch]
processors: [transform]
exporters: [otlp]
traces/2: # another pipeline of “traces” type
receivers: [otlp]
processors: [batch]
processors: [transform]
exporters: [otlp]
```

Expand All @@ -240,7 +243,7 @@ When the Collector loads this config, the result looks like this diagram:
title: Pipeline "traces"
---
flowchart LR
R1("`zipkin Receiver`") --> P1["`#quot;batch#quot; Processor`"]
R1("`zipkin Receiver`") --> P1["`#quot;transform#quot; Processor`"]
P1 --> E1[["`#quot;otlp#quot; Exporter`"]]
```

Expand All @@ -249,12 +252,12 @@ flowchart LR
title: Pipeline "traces/2"
---
flowchart LR
R1("`otlp Receiver`") --> P1["`#quot;batch#quot; Processor`"]
R1("`otlp Receiver`") --> P1["`#quot;transform#quot; Processor`"]
P1 --> E1[["`#quot;otlp#quot; Exporter`"]]
```

Note that each `batch` processor is an independent instance, although they are
configured the same way with a `send_batch_size` of `10000`.
Note that each `transform` processor is an independent instance, although they
are configured the same way with a `send_batch_size` of `10000`.

> The same name of the processor must not be referenced multiple times in the
> `processors` key of a single pipeline.
Expand Down
22 changes: 7 additions & 15 deletions content/en/docs/collector/building/receiver.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,9 @@ Collector's components and pipelines.
touch config.yaml
```

For now, you just need a basic traces pipeline with the `otlp` receiver, the
`otlp` and `debug`[^1] exporters, and optionally the `batch` processor. Here is
what your `config.yaml` file should look like:
For now, you just need a basic traces pipeline with the `otlp` receiver and the
`otlp` and `debug`[^1] exporters. Here is what your `config.yaml` file should
look like:

> config.yaml

Expand All @@ -110,9 +110,6 @@ receivers:
grpc:
endpoint: 0.0.0.0:4317

processors:
batch:

exporters:
# NOTE: Prior to v0.86.0 use `logging` instead of `debug`.
debug:
Expand All @@ -121,12 +118,13 @@ exporters:
endpoint: localhost:14317
tls:
insecure: true
sending_queue:
batch:

service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp/jaeger, debug]
telemetry:
logs:
Expand Down Expand Up @@ -975,7 +973,6 @@ import (
"go.opentelemetry.io/collector/receiver"
debugexporter "go.opentelemetry.io/collector/exporter/debugexporter"
otlpexporter "go.opentelemetry.io/collector/exporter/otlpexporter"
batchprocessor "go.opentelemetry.io/collector/processor/batchprocessor"
otlpreceiver "go.opentelemetry.io/collector/receiver/otlpreceiver"
tailtracer "github.com/open-telemetry/opentelemetry-tutorials/trace-receiver/tailtracer" // newly added line
)
Expand Down Expand Up @@ -1007,7 +1004,6 @@ func components() (otelcol.Factories, error) {
}

factories.Processors, err = otelcol.MakeFactoryMap[processor.Factory](
batchprocessor.NewFactory(),
)
if err != nil {
return otelcol.Factories{}, err
Expand Down Expand Up @@ -1046,9 +1042,6 @@ receivers:
interval: 1m
number_of_traces: 1

processors:
batch:

exporters:
# NOTE: Prior to v0.86.0 use `logging` instead of `debug`.
debug:
Expand All @@ -1057,12 +1050,13 @@ exporters:
endpoint: localhost:14317
tls:
insecure: true
sending_queue:
batch:

service:
pipelines:
traces:
receivers: [otlp, tailtracer]
processors: [batch]
exporters: [otlp/jaeger, debug]
telemetry:
logs:
Expand All @@ -1084,7 +1078,6 @@ The output should look like this:
2023-11-08T21:38:36.621+0800 info [email protected]/telemetry.go:201 Serving Prometheus metrics {"address": ":8888", "level": "Basic"}
2023-11-08T21:38:36.621+0800 info [email protected]/exporter.go:275 Development component. May change in the future. {"kind": "exporter", "data_type": "traces", "name": "debug"}
2023-11-08T21:38:36.621+0800 debug [email protected]/exporter.go:273 Stable component. {"kind": "exporter", "data_type": "traces", "name": "otlp/jaeger"}
2023-11-08T21:38:36.621+0800 debug [email protected]/processor.go:287 Stable component. {"kind": "processor", "name": "batch", "pipeline": "traces"}
2023-11-08T21:38:36.621+0800 debug [email protected]/receiver.go:294 Stable component. {"kind": "receiver", "name": "otlp", "data_type": "traces"}
2023-11-08T21:38:36.621+0800 debug [email protected]/receiver.go:294 Alpha component. May change in the future. {"kind": "receiver", "name": "tailtracer", "data_type": "traces"}
2023-11-08T21:38:36.622+0800 info [email protected]/service.go:143 Starting otelcol-dev... {"Version": "1.0.0", "NumCPU": 10}
Expand Down Expand Up @@ -2283,7 +2276,6 @@ And you should see the output like this after a few minutes:
2023-11-09T11:38:19.890+0800 info [email protected]/telemetry.go:201 Serving Prometheus metrics {"address": ":8888", "level": "Basic"}
2023-11-09T11:38:19.890+0800 debug [email protected]/exporter.go:273 Stable component. {"kind": "exporter", "data_type": "traces", "name": "otlp/jaeger"}
2023-11-09T11:38:19.890+0800 info [email protected]/exporter.go:275 Development component. May change in the future. {"kind": "exporter", "data_type": "traces", "name": "debug"}
2023-11-09T11:38:19.890+0800 debug [email protected]/processor.go:287 Stable component. {"kind": "processor", "name": "batch", "pipeline": "traces"}
2023-11-09T11:38:19.891+0800 debug [email protected]/receiver.go:294 Stable component. {"kind": "receiver", "name": "otlp", "data_type": "traces"}
2023-11-09T11:38:19.891+0800 debug [email protected]/receiver.go:294 Alpha component. May change in the future. {"kind": "receiver", "name": "tailtracer", "data_type": "traces"}
2023-11-09T11:38:19.891+0800 info [email protected]/service.go:143 Starting otelcol-dev... {"Version": "1.0.0", "NumCPU": 10}
Expand Down
26 changes: 7 additions & 19 deletions content/en/docs/collector/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,12 @@ receivers:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
batch:

exporters:
otlp:
endpoint: otelcol:4317
sending_queue:
batch:

extensions:
health_check:
Expand All @@ -131,15 +131,12 @@ service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
logs:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
```

Expand All @@ -161,15 +158,15 @@ receivers:
grpc:
endpoint: 0.0.0.0:55690

processors:
batch:
batch/test:

exporters:
otlp:
endpoint: otelcol:4317
sending_queue:
batch:
otlp/2:
endpoint: otelcol2:4317
sending_queue:
batch:

extensions:
health_check:
Expand All @@ -184,19 +181,15 @@ service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
traces/2:
receivers: [otlp/2]
processors: [batch/test]
exporters: [otlp/2]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
logs:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
```

Expand Down Expand Up @@ -367,9 +360,6 @@ processors:
- key: email
action: hash

# Data sources: traces, metrics, logs
batch:

# Data sources: metrics, metrics, logs
filter:
error_mode: ignore
Expand Down Expand Up @@ -640,11 +630,10 @@ service:
pipelines:
metrics:
receivers: [opencensus, prometheus]
processors: [batch]
exporters: [opencensus, prometheus]
traces:
receivers: [opencensus, jaeger]
processors: [batch, memory_limiter]
processors: [memory_limiter]
exporters: [opencensus, zipkin]
```

Expand All @@ -659,7 +648,6 @@ service:
# ...
traces/2:
receivers: [opencensus]
processors: [batch]
exporters: [zipkin]
```

Expand Down
7 changes: 1 addition & 6 deletions content/en/docs/collector/custom-collector.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Building a custom collector
weight: 29
# prettier-ignore
cSpell:ignore: batchprocessor chipset darwin debugexporter gomod loggingexporter otlpexporter otlpreceiver wyrtw
cSpell:ignore: chipset darwin debugexporter gomod loggingexporter otlpexporter otlpreceiver wyrtw
---

If you are planning to build and debug custom collector receivers, processors,
Expand Down Expand Up @@ -314,8 +314,6 @@ receivers:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
batch:

exporters:
debug:
Expand All @@ -325,15 +323,12 @@ service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [debug]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [debug]
logs:
receivers: [otlp]
processors: [batch]
exporters: [debug]
```

Expand Down
16 changes: 4 additions & 12 deletions content/en/docs/collector/deployment/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,18 +45,16 @@ receivers:
http:
endpoint: 0.0.0.0:4318

processors:
batch:

exporters:
otlp/jaeger: # Jaeger supports OTLP directly
endpoint: https://jaeger.example.com:4317
sending_queue:
batch:

service:
pipelines:
traces/dev:
receivers: [otlp]
processors: [batch]
exporters: [otlp/jaeger]
```

Expand All @@ -69,18 +67,16 @@ receivers:
http:
endpoint: 0.0.0.0:4318

processors:
batch:

exporters:
prometheusremotewrite: # the PRW exporter, to ingest metrics to backend
endpoint: https://prw.example.com/v1/api/remote_write
sending_queue:
batch:

service:
pipelines:
metrics/prod:
receivers: [otlp]
processors: [batch]
exporters: [prometheusremotewrite]
```

Expand All @@ -93,9 +89,6 @@ receivers:
http:
endpoint: 0.0.0.0:4318

processors:
batch:

exporters:
file: # the File Exporter, to ingest logs to local file
path: ./app42_example.log
Expand All @@ -105,7 +98,6 @@ service:
pipelines:
logs/dev:
receivers: [otlp]
processors: [batch]
exporters: [file]
```

Expand Down
4 changes: 1 addition & 3 deletions content/en/docs/collector/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,10 +286,8 @@ The Collector might drop data for a variety of reasons, but the most common are:
- The exporter destination is unavailable or accepting the data too slowly.

To mitigate drops, configure the
[`batch` processor](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md).
In addition, it might be necessary to configure the
[queued retry options](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/exporterhelper#configuration)
on enabled exporters.
on enabled exporters, including the `sending_queue::batch` section.

#### Collector is not receiving data

Expand Down
Loading