contrib/labstack/echo.v4: option to ignore based on the response

[rafaeljusto]

The existing WithIgnoreRequest won't contain the response status code since it's checked before the underlying controller is executed.

Discussion #3341

What does this PR do?

Adds a new WithIgnoreResponse option to the Echo v4 middleware.

Motivation

The existing WithIgnoreRequest doesn't have the response information when executed. This doesn't cover the scenario when you need to skip the trace based on a request path with a specific returning status code.

Reviewer's Checklist

• Changed code has unit tests for its functionality at or near 100% coverage.
• System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
• There is a benchmark for any new code, or changes to existing code.
• If this interacts with the agent in a new way, a system test has been added.
• Add an appropriate team label so this PR gets put in the right place for the release notes.
• Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
• For internal contributors, a matching PR should be created to the v2-dev branch and reviewed by @DataDog/apm-go.

[rafaeljusto]

Hey @darccio 👋 Could you please have a look at this one? 🙏

[mtoffl01]

Hey @rafaeljusto , thanks for your contribution! I left a small suggestion for improvement. Once addressed, we can look at merging this.

[mtoffl01]

Hey @rafaeljusto ,

Thanks for making those changes!

After taking a closer look, I realized I may have misunderstood your intention initially. I thought you were trying to modify the status code based on the response, but it looks like your goal is to conditionally drop the span depending on the response. Is that right?

As it stands, your changes will start a span on line 71 that is never finished. That would leave the span open indefinitely, or until the process exits, rather than cleanly dropping it.

I'm trying to better understand your motivation. Are you aiming to:

1. "Drop" spans/traces conditionally, like we do with sampling, where spans that are created can later be discarded based on various criteria (including request/response data)? (Example: https://docs.datadoghq.com/tracing/guide/ignoring_apm_resources/?tab=datadogyaml#sampling)
2. Avoid creating spans entirely for certain responses?

Can you share more about the use case behind this change? Understanding the problem you're solving will help me determine if this is the right approach. 🙇

[rafaeljusto]

That would leave the span open indefinitely, or until the process exits, rather than cleanly dropping it.

Oh, that's not good. I don't want to cause any memory leak here. 😄 Thanks for the heads up! What would be the proper way to drop the span?

2. Avoid creating spans entirely for certain responses?

This one! I'm looking at dropping the entire trace when the response status code is 503 (service unavailable). I don't want to add DD_APM_FILTER_TAGS_REJECT at the agent level as it would apply it to all services. Is there a better way?

[mtoffl01]

Hey @rafaeljusto ,

Yes, you can configure sampling rules based on various criteria, including span tags — which would cover your status code criteria.

Here's an example setup:

	rules := []tracer.SamplingRule{
		{
			Tags: map[string]*regexp.Regexp{
				ext.HTTPCode: regexp.MustCompile("503"),
			},
			Rate: 0.0,
		},
	}
	tracer.Start(tracer.WithSamplingRules(rules))
	defer tracer.Stop()

Or via environment variable: DD_TRACE_SAMPLING_RULES='[{"sample_rate": 0.0,"tags": {"http.status_code":"503"}}]'

The above configurations will drop all traces that contain a span with a 503 status code. It's a "head based sampling" mechanism; you can read more about head based sampling in tracer libraries here, if you like.

Let me know how this sounds to you!

[rafaeljusto]

Oh, the tracer.SamplingRule is what I was looking for! Thank you!

Closing this one.