Upgrade arrow to improve zstd payload compression performance in OTAP Exporter

[albertlockett]

Closes #1129

arrow-rs 57.0 has an improvement for the performance of zstd IPC compression apache/arrow-rs#8405

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.83%. Comparing base (4cf9802) to head (6816bf3).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1433      +/-   ##
==========================================
- Coverage   83.88%   83.83%   -0.06%     
==========================================
  Files         398      398              
  Lines      108990   108990              
==========================================
- Hits        91428    91371      -57     
- Misses      17028    17085      +57     
  Partials      534      534              

Components	Coverage Δ
otap-dataflow	85.66% <ø> (-0.10%)	⬇️
query_abstraction	80.61% <ø> (ø)
query_engine	90.48% <ø> (ø)
syslog_cef_receivers	∅ <ø> (∅)
otel-arrow-go	53.50% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[albertlockett]

Tested perf manually:

	main (arrow 56.1)	albert/1129 (arrow 57.0)
cpu average	54.3%	6.3%
cpu max	86.8%	14.7%

load gen (3 cores)

settings:
  default_pipeline_ctrl_msg_channel_size: 100
  default_node_ctrl_msg_channel_size: 100
  default_pdata_channel_size: 100

nodes:
  receiver:
    kind: receiver
    plugin_urn: "urn:otel:otap:fake_data_generator:receiver"
    out_ports:
      out_port:
        destinations:
          - exporter
        dispatch_strategy: round_robin
    config:
      traffic_config:
        # max_signal_count: 1000
        max_batch_size: 1000
        signals_per_second: 50000
        log_weight: 100
      registry_path: https://github.com/open-telemetry/semantic-conventions.git[model]
  exporter:
    kind: exporter
    plugin_urn: "urn:otel:otap:exporter"
    config:
      grpc_endpoint: "http://127.0.0.1:4317"
      compression_method: zstd
      arrow:
        payload_compression: none

system under test (1 core)

settings:
  default_pipeline_ctrl_msg_channel_size: 100
  default_node_ctrl_msg_channel_size: 100
  default_pdata_channel_size: 100

nodes:
  receiver:
    kind: receiver
    plugin_urn: "urn:otel:otap:receiver"
    out_ports:
      out_port:
        destinations:
          - exporter
        dispatch_strategy: round_robin
    config:
      listening_addr: "127.0.0.1:4317"
      response_stream_channel_size: 256
      compression_method: zstd
  exporter:
    kind: exporter
    plugin_urn: "urn:otel:otap:exporter"
    config:
      grpc_endpoint: "http://127.0.0.1:4318"
      compression_method: zstd
      arrow:
        payload_compression: zstd 

[lquerel]

LGTM

[lquerel]

Thanks for the improvement you did in the Arrow crate!