enhancement(gcp_pubsub sink, codecs)!: Integrate encoding::Encoder with gcp_pubsub sink

[pablosichert]

Part of #9459.

This is a breaking change since the encoding is no longer optional.

[netlify]

👷 Deploy Preview for vector-project processing.

Name	Link
🔨 Latest commit	2927ced
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/627e26c2c6c0b9000889c28b

[netlify]

👷 Deploy Preview for vector-project processing.

Name	Link
🔨 Latest commit	2927ced
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/627e26c2c6c0b9000889c28b

[netlify]

✅ Deploy Preview for vector-project ready!

Name	Link
🔨 Latest commit	b2f3299
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/62a261040cbfe4000829c4b4
😎 Deploy Preview	https://deploy-preview-12718--vector-project.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[jszwedko]

Can we add a note to the upgrade guide since this is a breaking change?

[jszwedko]

Also, why is it no longer optional? Could we maintain the existing default?

[pablosichert]

Also, why is it no longer optional? Could we maintain the existing default?

Largely to be consistent with all the other sinks. Previously there only existed a default here because the user had no choice at all. So I'm not sure it's worth to break consistency to remain backwards compatible here, especially since the migration path is very straight-forward.

[jszwedko]

Thanks for adding the upgrade note. Do we need to update the docs too? Currently that option is undocumented for this sink.

[pablosichert]

@jszwedko added the json and text codecs for now until we document all new ones for the integration with #9459

[jszwedko]

Sweet, thanks!

[github-actions]

Soak Test Results

Baseline: b471333
Comparison: b2f3299
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
syslog_humio_logs	880.9KiB	5.05	100.00%	17.05MiB	215.54KiB	4.42KiB	0	0.012345	17.91MiB	305.89KiB	6.26KiB	0	0.0166776	False	False
syslog_splunk_hec_logs	647.72KiB	3.75	100.00%	16.88MiB	610.74KiB	12.47KiB	0	0.0353334	17.51MiB	777.35KiB	15.88KiB	0	0.0433477	False	False
http_to_http_acks	602.23KiB	3.37	99.01%	17.43MiB	7.91MiB	165.28KiB	0	0.453511	18.02MiB	7.88MiB	164.71KiB	0	0.43727	True	True
splunk_transforms_splunk3	394.48KiB	2.95	100.00%	13.06MiB	2.22MiB	46.35KiB	0	0.169965	13.44MiB	2.23MiB	46.76KiB	0	0.166101	False	False
datadog_agent_remap_datadog_logs	843.35KiB	1.13	100.00%	72.95MiB	2.81MiB	58.73KiB	0	0.0385529	73.77MiB	3.84MiB	80.23KiB	0	0.0520379	False	False
datadog_agent_remap_blackhole_acks	686.01KiB	0.94	100.00%	71.52MiB	3.61MiB	75.22KiB	0	0.0504075	72.19MiB	2.39MiB	49.98KiB	0	0.0331443	False	False
datadog_agent_remap_datadog_logs_acks	671.55KiB	0.89	100.00%	73.71MiB	446.99KiB	9.13KiB	0	0.00592048	74.37MiB	3.11MiB	65.0KiB	0	0.0418112	False	False
http_to_http_noack	93.92KiB	0.39	100.00%	23.75MiB	978.94KiB	19.93KiB	0	0.0402367	23.85MiB	267.48KiB	5.46KiB	0	0.0109517	False	False
syslog_regex_logs2metric_ddmetrics	37.48KiB	0.29	94.22%	12.58MiB	658.54KiB	13.41KiB	0	0.0510959	12.62MiB	712.98KiB	14.51KiB	0	0.0551594	False	False
syslog_log2metric_splunk_hec_metrics	38.85KiB	0.22	83.69%	17.58MiB	1005.26KiB	20.47KiB	0	0.0558224	17.62MiB	926.63KiB	18.88KiB	0	0.0513456	False	False
http_pipelines_blackhole_acks	9.36KiB	0.21	43.15%	4.26MiB	564.18KiB	11.54KiB	0	0.12923	4.27MiB	571.59KiB	11.67KiB	0	0.130647	False	False
datadog_agent_remap_blackhole	130.44KiB	0.18	99.77%	71.31MiB	1.85MiB	38.77KiB	0	0.025947	71.43MiB	888.83KiB	18.17KiB	0	0.0121487	False	False
splunk_hec_route_s3	10.84KiB	0.05	13.99%	20.01MiB	2.06MiB	42.97KiB	0	0.10271	20.02MiB	2.1MiB	44.0KiB	0	0.105085	False	False
fluent_elasticsearch	28.51KiB	0.04	97.56%	79.45MiB	626.12KiB	12.62KiB	0	0.00769492	79.47MiB	51.96KiB	1.05KiB	0	0.00063837	False	False
http_to_http_json	587.7B	0	4.85%	23.85MiB	326.48KiB	6.67KiB	0	0.0133679	23.85MiB	326.83KiB	6.67KiB	0	0.0133817	False	False
splunk_hec_to_splunk_hec_logs_noack	-1.23KiB	-0.01	10.05%	23.84MiB	333.8KiB	6.83KiB	0	0.0136721	23.84MiB	341.07KiB	6.98KiB	0	0.0139703	False	False
socket_to_socket_blackhole	-7.22KiB	-0.05	37.42%	14.03MiB	566.76KiB	11.54KiB	0	0.0394371	14.02MiB	455.05KiB	9.28KiB	0	0.03168	False	False
splunk_hec_indexer_ack_blackhole	-23.72KiB	-0.1	74.74%	23.79MiB	665.99KiB	13.56KiB	0	0.0273363	23.76MiB	770.78KiB	15.68KiB	0	0.0316687	False	False
splunk_hec_to_splunk_hec_logs_acks	-28.83KiB	-0.12	81.66%	23.78MiB	690.09KiB	14.05KiB	0	0.0283311	23.75MiB	810.92KiB	16.49KiB	0	0.0333315	False	False
file_to_blackhole	-165.51KiB	-0.17	94.31%	95.33MiB	2.91MiB	60.48KiB	0	0.0305167	95.17MiB	3.02MiB	62.4KiB	0	0.0317394	False	False
syslog_loki	-85.44KiB	-0.55	100.00%	15.18MiB	584.61KiB	11.91KiB	0	0.0375987	15.1MiB	377.7KiB	7.71KiB	0	0.0244253	False	False
http_pipelines_blackhole	-44.1KiB	-0.94	98.85%	4.59MiB	587.26KiB	12.03KiB	0	0.124817	4.55MiB	619.1KiB	12.62KiB	0	0.13283	False	False
http_pipelines_no_grok_blackhole	-197.97KiB	-1.11	99.99%	17.4MiB	1.69MiB	35.39KiB	0	0.0971406	17.21MiB	1.7MiB	35.52KiB	0	0.0989315	False	False
syslog_log2metric_humio_metrics	-207.13KiB	-1.38	100.00%	14.66MiB	386.17KiB	7.87KiB	0	0.0257107	14.46MiB	160.99KiB	3.3KiB	0	0.0108687	False	False