Skip to content

Make SPM integration test faster by flushing metrics more frequently#7861

Merged
yurishkuro merged 1 commit into
jaegertracing:mainfrom
Don-Assamongkol1:5608-SPM-IT-see-if-metrics-are-getting-scraped
Feb 1, 2026
Merged

Make SPM integration test faster by flushing metrics more frequently#7861
yurishkuro merged 1 commit into
jaegertracing:mainfrom
Don-Assamongkol1:5608-SPM-IT-see-if-metrics-are-getting-scraped

Conversation

@Don-Assamongkol1
Copy link
Copy Markdown
Contributor

@Don-Assamongkol1 Don-Assamongkol1 commented Jan 10, 2026

Which problem is this PR solving?

Description of the changes

  • Speed up integration test by changing the interval at which jaeger flushes metrics.
    • The set up is microsim sends spans to jaeger where Jaeger is running an OTLP collector. Previously, the collector was configured to flush metrics out using the default of 1 min, which is why the integration test had to wait.
  • log with timestamp
  • Configured prometheus to scrape poll more frequently.
  • Query metrics API with a step param = 10 seconds so prometheus computes metrics more frequently
  • remove old test code that expected zero value metric data points. We don't expect this anymore because metrics are getting populated more quickly

How was this change tested?

  • Ran locally. Entire test now takes 1 minute instead of 3 (I think?) previously. (can verify here in CI to double check)

Checklist

@Don-Assamongkol1 Don-Assamongkol1 changed the title make integration test faster by flushing metrics frequently make integration test faster by flushing metrics more frequently Jan 10, 2026
@Don-Assamongkol1 Don-Assamongkol1 marked this pull request as ready for review January 10, 2026 19:22
@Don-Assamongkol1 Don-Assamongkol1 requested a review from a team as a code owner January 10, 2026 19:22
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 10, 2026

Metrics Comparison Summary

Total changes across all snapshots: 0

Detailed changes per snapshot

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

Total Changes: 0

  • 🆕 Added: 0 metrics
  • ❌ Removed: 0 metrics
  • 🔄 Modified: 0 metrics
  • 🚫 Excluded: 106 metrics

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

Total Changes: 0

  • 🆕 Added: 0 metrics
  • ❌ Removed: 0 metrics
  • 🔄 Modified: 0 metrics
  • 🚫 Excluded: 106 metrics

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

Total Changes: 0

  • 🆕 Added: 0 metrics
  • ❌ Removed: 0 metrics
  • 🔄 Modified: 0 metrics
  • 🚫 Excluded: 106 metrics

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

Total Changes: 0

  • 🆕 Added: 0 metrics
  • ❌ Removed: 0 metrics
  • 🔄 Modified: 0 metrics
  • 🚫 Excluded: 53 metrics

➡️ View full metrics file

@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.57%. Comparing base (0c6cd40) to head (23ba328).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7861      +/-   ##
==========================================
+ Coverage   95.55%   95.57%   +0.02%     
==========================================
  Files         316      316              
  Lines       16726    16726              
==========================================
+ Hits        15982    15986       +4     
+ Misses        580      577       -3     
+ Partials      164      163       -1     
Flag Coverage Δ
badger_v1 9.11% <ø> (ø)
badger_v2 1.90% <ø> (ø)
cassandra-4.x-v1-manual 13.41% <ø> (ø)
cassandra-4.x-v2-auto 1.89% <ø> (ø)
cassandra-4.x-v2-manual 1.89% <ø> (ø)
cassandra-5.x-v1-manual 13.41% <ø> (ø)
cassandra-5.x-v2-auto 1.89% <ø> (ø)
cassandra-5.x-v2-manual 1.89% <ø> (ø)
clickhouse 1.98% <ø> (ø)
elasticsearch-6.x-v1 17.27% <ø> (ø)
elasticsearch-7.x-v1 17.30% <ø> (ø)
elasticsearch-8.x-v1 17.45% <ø> (ø)
elasticsearch-8.x-v2 1.90% <ø> (ø)
elasticsearch-9.x-v2 1.90% <ø> (ø)
grpc_v1 8.44% <ø> (ø)
grpc_v2 1.90% <ø> (ø)
kafka-3.x-v2 1.90% <ø> (ø)
memory_v2 1.90% <ø> (ø)
opensearch-1.x-v1 17.34% <ø> (ø)
opensearch-2.x-v1 17.34% <ø> (ø)
opensearch-2.x-v2 1.90% <ø> (ø)
opensearch-3.x-v2 1.90% <ø> (ø)
query 1.90% <ø> (ø)
tailsampling-processor 0.54% <ø> (ø)
unittests 94.26% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread cmd/jaeger/config-spm.yaml Outdated
Comment thread docker-compose/monitor/prometheus.yml Outdated
Comment thread scripts/e2e/spm.sh Outdated
Comment thread scripts/e2e/spm.sh Outdated
Comment thread scripts/e2e/spm.sh Outdated
Comment thread scripts/e2e/spm.sh
Comment thread .cursor/commands/pr.md Outdated
Comment thread docker-compose/monitor/docker-compose-opensearch.yml Outdated
Comment thread docker-compose/monitor/docker-compose-elasticsearch.yml Outdated
@Don-Assamongkol1
Copy link
Copy Markdown
Contributor Author

Hey @yurishkuro sorry, I'm still working on this. Would you mind reviewing after I've gotten all changes to pass?

Not sure what the flow of this repo is for having an already open PR, then having to fix it-- should I push to another branch or mark it as review for later?

@Don-Assamongkol1 Don-Assamongkol1 force-pushed the 5608-SPM-IT-see-if-metrics-are-getting-scraped branch 5 times, most recently from a58ab44 to ecc976d Compare January 12, 2026 15:28
Comment thread scripts/e2e/spm.sh Outdated
@Don-Assamongkol1 Don-Assamongkol1 force-pushed the 5608-SPM-IT-see-if-metrics-are-getting-scraped branch from ecc976d to a3dc783 Compare January 12, 2026 21:20
@Don-Assamongkol1
Copy link
Copy Markdown
Contributor Author

The label check is failing-- Could a maintainer please add a label like changelog:ci ?

@yurishkuro yurishkuro added the changelog:ci Change related to continuous integration / testing label Jan 13, 2026
@Don-Assamongkol1
Copy link
Copy Markdown
Contributor Author

@yurishkuro Pr ready for review again

@Don-Assamongkol1 Don-Assamongkol1 force-pushed the 5608-SPM-IT-see-if-metrics-are-getting-scraped branch from a3dc783 to bce6c91 Compare January 15, 2026 22:08
@Don-Assamongkol1 Don-Assamongkol1 force-pushed the 5608-SPM-IT-see-if-metrics-are-getting-scraped branch 2 times, most recently from 364f319 to 0df3bdc Compare January 27, 2026 02:06
@Don-Assamongkol1
Copy link
Copy Markdown
Contributor Author

@yurishkuro this PR should be ready to review again. Please lmk if I'm missing anything!

@Don-Assamongkol1 Don-Assamongkol1 force-pushed the 5608-SPM-IT-see-if-metrics-are-getting-scraped branch from 0df3bdc to dbc450d Compare January 31, 2026 18:06
@Don-Assamongkol1 Don-Assamongkol1 marked this pull request as draft January 31, 2026 18:06
@Don-Assamongkol1 Don-Assamongkol1 marked this pull request as ready for review January 31, 2026 18:06
@dosubot dosubot Bot added the performance label Jan 31, 2026
Comment thread docker-compose/monitor/prometheus.yml Outdated
Comment thread scripts/e2e/spm.sh
Comment thread scripts/e2e/spm.sh Outdated
@yurishkuro yurishkuro added the waiting-for-author PR is waiting for author to respond to maintainer's comments label Jan 31, 2026
@Don-Assamongkol1 Don-Assamongkol1 force-pushed the 5608-SPM-IT-see-if-metrics-are-getting-scraped branch from dbc450d to 3c18749 Compare January 31, 2026 21:33
@github-actions github-actions Bot removed the waiting-for-author PR is waiting for author to respond to maintainer's comments label Jan 31, 2026
Signed-off-by: Don Assamongkol <don.assamongkol@gmail.com>
@Don-Assamongkol1 Don-Assamongkol1 force-pushed the 5608-SPM-IT-see-if-metrics-are-getting-scraped branch from b048fc5 to 23ba328 Compare January 31, 2026 21:57
@Don-Assamongkol1
Copy link
Copy Markdown
Contributor Author

Hmm, not sure why check-label bot is failing this-- The changelog label is still applied I believe. But anyways tested it locally so let's see if the build passes here too

@yurishkuro yurishkuro changed the title make integration test faster by flushing metrics more frequently Make SPM integration test faster by flushing metrics more frequently Feb 1, 2026
@yurishkuro yurishkuro merged commit 39b645c into jaegertracing:main Feb 1, 2026
63 of 64 checks passed
SoumyaRaikwar pushed a commit to SoumyaRaikwar/jaeger that referenced this pull request Feb 13, 2026
…aegertracing#7861)

## Which problem is this PR solving?
- follow-up to jaegertracing#5608

## Description of the changes
- Speed up integration test by changing the interval at which jaeger
flushes metrics.
- The set up is `microsim` sends spans to jaeger where Jaeger is running
an OTLP collector. Previously, the collector was configured to flush
metrics out using the default of 1 min, which is why the integration
test had to wait.
- log with timestamp
- Configured prometheus to scrape poll more frequently. 
- Query metrics API with a step param = 10 seconds so prometheus
computes metrics more frequently
- remove old test code that expected zero value metric data points. We
don't expect this anymore because metrics are getting populated more
quickly

## How was this change tested?
- Ran locally. Entire test now takes 1 minute instead of 3 (I think?)
previously. (can verify here in CI to double check)

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `npm run lint` and `npm run test`

Signed-off-by: Don Assamongkol <don.assamongkol@gmail.com>
singhvibhanshu pushed a commit to singhvibhanshu/jaeger that referenced this pull request Mar 18, 2026
…aegertracing#7861)

## Which problem is this PR solving?
- follow-up to jaegertracing#5608

## Description of the changes
- Speed up integration test by changing the interval at which jaeger
flushes metrics.
- The set up is `microsim` sends spans to jaeger where Jaeger is running
an OTLP collector. Previously, the collector was configured to flush
metrics out using the default of 1 min, which is why the integration
test had to wait.
- log with timestamp
- Configured prometheus to scrape poll more frequently. 
- Query metrics API with a step param = 10 seconds so prometheus
computes metrics more frequently
- remove old test code that expected zero value metric data points. We
don't expect this anymore because metrics are getting populated more
quickly

## How was this change tested?
- Ran locally. Entire test now takes 1 minute instead of 3 (I think?)
previously. (can verify here in CI to double check)

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `npm run lint` and `npm run test`

Signed-off-by: Don Assamongkol <don.assamongkol@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog:ci Change related to continuous integration / testing performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants