Skip to content

Integrate codecs::Encoder with sinks #9459

@binarylogic

Description

@binarylogic

Like #9404 but for encoding/sinks

Issues that will be resolved with the integration:

Native encoding issues:

Other issues:

Priority of sinks (check means "has open PR" or "marked as not applicable"):

  • socket
  • http
  • kafka
  • datadog_archives
  • datadog_events
  • datadog_logs
  • datadog_traces
  • aws_s3
  • azure_blob
  • gcp_cloudstorage
  • gcp_pubsub
  • gcp_stackdriver_logs
  • console
  • file
  • vector
  • aws_sqs
  • elasticsearch
  • splunk_hec_logs
  • loki
  • aws_cloudwatch_logs
  • aws_kinesis_firehose
  • aws_kinesis_streams
  • azure_monitor_logs
  • blackhole
  • clickhouse
  • honeycomb
  • humio_logs
  • influxdb
  • logdna
  • nats
  • new_relic
  • new_relic_logs
  • papertrail
  • pulsar
  • redis
  • sematext
  • websocket
Sink Issue PR Status Comment Legacy encoding
aws_cloudwatch_logs - #12175 Merged - EncodingConfig<StandardEncodings>
aws_kinesis_firehose - #12176 Merged - EncodingConfig<StandardEncodings>
aws_kinesis_streams - #12177 Merged - EncodingConfig<StandardEncodings>
aws_s3 - #12136 Merged - EncodingConfig<StandardEncodings>
aws_sqs - #12550 Merged - EncodingConfig<(Text,Json)>
azure_blob - #12179 Merged - EncodingConfig<StandardEncodings>
azure_monitor_logs - Blocked Needs improvements to Encoder to emit JSON value when serializer supports JSON. (Part of #12495.) EncodingConfigWithDefault<(Default)>
blackhole - - Not applicable - -
clickhouse - - Not applicable Has opinionated fixed encoding EncodingConfigWithDefault<(Default)>
console - #12181 In review EncodingConfig<StandardEncodings>
datadog_archives - - Not applicable Has opinionated fixed encoding -
datadog_events - - Not applicable Has opinionated fixed encoding -
datadog_logs - - Not applicable Has opinionated fixed encoding -
datadog_traces - - Not applicable Has opinionated fixed encoding -
elasticsearch - Needs clarification The Elasticsearch Bulk API expects entries in JSON format. EncodingConfigFixed<{doc_type,suppress_type_name}>
file - #12548 Merged - EncodingConfig<(Text,Ndjson)>
gcp_cloud_storage - #12488 Merged - EncodingConfig<StandardEncodings>
gcp_pubsub - #12718 In review EncodingConfigWithDefault<(Default)>
gcp_stackdriver_logs - #12719 Blocked Needs improvements to Encoder to emit JSON value when serializer supports JSON. (Part of #12495.) EncodingConfigWithDefault<(Default)>
honeycomb - #12764 Blocked Needs improvements to Encoder to emit JSON value when serializer supports JSON. (Part of #12495.) -
http #11579 #11647 Merged - EncodingConfig<(Text,Ndjson,Json)>
humio_logs - #12495 In review EncodingConfig<(Json,Text)>
influxdb - #12583 Merged - EncodingConfigWithDefault<(Default)>
kafka - #12133 Merged - EncodingConfig<StandardEncodings>
logdna - EncodingConfigWithDefault<(Default)>
loki - #12558 Merged - EncodingConfig<(Json,Text,Logfmt)>
nats - #12586 Merged - EncodingConfig<(Text,Json)>
new_relic - EncodingConfigFixed<(Default)>
new_relic_logs - EncodingConfigWithDefault<(Json)>
papertrail - #12589 Merged - EncodingConfig<(Text,Json)>
pulsar - #13139 In review EncodingConfig<(Text,Json,Avro)>
redis - #12596 Merged - EncodingConfig<(Text,Json)>
sematext - EncodingConfigFixed<{doc_type,suppress_type_name}>
socket - #10684 Merged - EncodingConfig<(Text,Json)>
splunk_hec_logs - #12495 In review EncodingConfig<(Json,Text)>
vector - - Not applicable - -
websocket - #13054 In review - EncodingConfig<StandardEncodings>

Metadata

Metadata

Assignees

Labels

domain: codecsAnything related to Vector's codecs (encoding/decoding)domain: sinksAnything related to the Vector's sinkstype: enhancementA value-adding code change that enhances its existing functionality.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions