Support in Elastic Beanstalk worker tier to propagate tracing context #362
Description
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
- Please do not leave "+1" or "me too" comments. They generate extra noise for issue followers and do not help prioritize the request.
Tell us about your request
What do you want us to build?
To enhance Elastic Beanstalk's worker tier to propagate the tracing context from the SQS message.
Is this request specific to an Elastic Beanstalk platform?
If so, which one(s)?
The pattern applies across the various platforms to the worker tier.
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
What outcome are you trying to achieve, ultimately, and why is it hard/impossible to do right now? What is the impact of not having this problem solved? The more details you can provide, the better we'll be able to understand and solve the problem.
We extensively use Elastic Beanstalk’s worker tier for asynchronous workloads. Unfortunately aws-sqsd doesn’t propagate the trace context in the SQS message. This means the http request sent by aws-sqsd to Nginx and the underlying app is disconnected from the SQS message's tracing context making system behavior harder to reason about. End-to-end distributed tracing of both synchronous and asynchronous communication patterns is essential for achieving comprehensive observability in distributed systems.
Are you currently working around this issue?
How are you currently solving this problem?
We're living with the lower observability.
Additional context
Anything else we should know?
We understand AWS is transitioning to OpenTelemetry as its primary instrumentation standard for tracing. We hope Beanstalk's work as part of this issue aligns with that direction. The expectation it would make Beanstalk's behavior compatible with observability tooling such as Datadog, Dynatrace, as well as AWS X-Ray.
Here is relevant content:
- https://repost.aws/questions/QUi9eJBPPMSWmYbaazKTZNyw/does-aws-elastic-beanstalk-worker-tier-support-propagating-traces
- https://docs.dynatrace.com/docs/ingest-from/amazon-web-services/integrate-into-aws/aws-lambda-integration/aws-lambda-classic/opentelemetry-interoperability/lambda-otel-bridge-java#java-sqs-sns
- https://docs.dynatrace.com/docs/observe/application-observability/distributed-tracing/tracking-transactions#propagation-mechanisms
- Support distributed trace propagation across SQS DataDog/dd-trace-java#1823 (comment)
Relevant AWS docs:
- https://aws.amazon.com/what-is/distributed-tracing/
- https://docs.aws.amazon.com/xray/latest/devguide/xray-tracelinking.html
- https://docs.aws.amazon.com/sns/latest/dg/sns-active-tracing.html
- https://docs.aws.amazon.com/xray/latest/devguide/xray-services-sqs.html
- https://docs.aws.amazon.com/xray/latest/devguide/xray-services-beanstalk.html
- https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-migration.html
Attachments
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)