Remote Sampling is not working as expected

[Narenderbhcu]

Description

We have configured our trace generator using GO SDK remote sampling configuration similar to way mentioned in https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/samplers/jaegerremote.

jaegerRemoteSampler := jaegerremote.New(
"your-service-name",
jaegerremote.WithSamplingServerURL("http://{sampling_service_host_name}:5778"),
jaegerremote.WithSamplingRefreshInterval(10time.Second),
jaegerremote.WithInitialSampler(trace.TraceIDRatioBased(0.5)),
)
tp := trace.NewTracerProvider(
trace.WithSampler(jaegerRemoteSampler),
...
)
otel.SetTracerProvider(tp)

After configuring the SDK using the above configuration,samplingstrategies.json file configuration is successfully fetched from Jaeger-collector
But when we tried generating the traces, the sampling_strategies.json file of collector is still not applied and did not show any effect on the sampling of traces.
we would like to know the reason for not applying the fetched sampling configuration, are we missing anything ? could you please help us?
We have also raised ticket on Jaeger, please check "jaegertracing/jaeger#3925" for more information, they concluded like, sampling information(sampler.params and sampler.type) should be visible in Jaeger UI to work remote sampling, currently with OTEL SDK we can not see sampling information(sampler.params and sampler.type) in Jaeger UI.

Environment

• OS: Linux Based OS
• Go Version: 1.18
• opentelemetry-go version:

Expected behavior

Remote Sampling should work as expected.

[MrAlias]

Please update this with deterministic steps to reproduce and a clear statement of expected behavior. It is not clear what you need from this project with the current state of this issue.

[Narenderbhcu]

Hi @MrAlias

Thanks for supporting!
we are trying to apply remote sampling on spans generated using OTEL SDK, along with Jaeger 1.38.1.

Using the below code as mentioned in the link, tried to generate the spans. https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/samplers/jaegerremote.

Configurations changes on OTEL SDK:

jaegerRemoteSampler := jaegerremote.New(
"your-service-name",
jaegerremote.WithSamplingServerURL("http://{sampling_service_host_name}:5778"),
jaegerremote.WithSamplingRefreshInterval(10time.Second),
jaegerremote.WithInitialSampler(trace.TraceIDRatioBased(0.5)),
)
tp := trace.NewTracerProvider(
trace.WithSampler(jaegerRemoteSampler),
...
)
otel.SetTracerProvider(tp)

tracer:= otel.Tracer("component-foo")
ctx, span := tracer.Start(ctx, "foo")

Configurations on Jaeger-collector side:

we have configured the samplingstrategies.json file as mentioned in below link while deploying jaeger-collector and mentioned the sampling rate as "param": 0.1", which means the expected behavior is if we generate 10 spans only 1 should be reported to Jaeger UI, but it is not working like this, we could see some random numbers(5,6,4 spans) of spans reported out of 10 spans to Jaeger UI the behavior is inconsistent.

https://www.jaegertracing.io/docs/1.38/sampling/#file-sampling

We would like to understand, is our configuration changes on OTEL SDK is fine? or do we need to do any more changes to make the remote sampling works?

Thanks,
Narender

[Narenderbhcu]

Hi @MrAlias ,

Could you please check the information you have asked and help us in fixing this issue.

Thanks,
BR,
Narender

[Narenderbhcu]

Hi @MrAlias

How to configure below client sampling configuration using OTEL SDK

https://www.jaegertracing.io/docs/1.39/sampling/#client-sampling-configuration.

sampler.type as "const" or "probabilistic" or "ratelimiting" or "remote"

Thanks,
BR,
Narender

[Narenderbhcu]

Hi, can someone comment on this issue?