Skip to content

Benchmark tests prototype#2591

Merged
liquidsec merged 62 commits intodevfrom
benchmark-tests
Sep 4, 2025
Merged

Benchmark tests prototype#2591
liquidsec merged 62 commits intodevfrom
benchmark-tests

Conversation

@liquidsec
Copy link
Collaborator

No description provided.

- Add pytest-benchmark dependency and configuration
- Create bloom filter performance benchmarks covering add/check/mixed operations
- Add branch-based benchmark comparison script for PR automation
- Benchmarks measure critical DNS brute-forcing performance paths
@liquidsec liquidsec marked this pull request as draft August 14, 2025 22:28
@codecov
Copy link

codecov bot commented Aug 21, 2025

Codecov Report

❌ Patch coverage is 16.22103% with 470 lines in your changes missing coverage. Please review.
✅ Project coverage is 92%. Comparing base (848266c) to head (e76e1b8).
⚠️ Report is 63 commits behind head on dev.

Files with missing lines Patch % Lines
...est/benchmarks/test_event_validation_benchmarks.py 9% 197 Missing ⚠️
bbot/test/benchmarks/test_excavate_benchmarks.py 15% 108 Missing ⚠️
bbot/test/benchmarks/test_ipaddress_benchmarks.py 18% 71 Missing ⚠️
...ot/test/benchmarks/test_bloom_filter_benchmarks.py 21% 42 Missing ⚠️
...t/test/benchmarks/test_closest_match_benchmarks.py 29% 27 Missing ⚠️
...est/benchmarks/test_weighted_shuffle_benchmarks.py 35% 25 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff           @@
##             dev   #2591    +/-   ##
======================================
- Coverage     93%     92%    -1%     
======================================
  Files        404     410     +6     
  Lines      33387   33947   +560     
======================================
+ Hits       30922   31019    +97     
- Misses      2465    2928   +463     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 22, 2025

🚀 Performance Benchmark Report

ℹ️ No baseline benchmark data available

Showing current results for benchmark-tests only.

📊 Current Results (benchmark-tests) - Click to expand

Results

Test Name Mean Time Ops/sec Min Max
Bloom Filter Dns Mutation Tracking Performance 8.02ms 124.7 ops/sec 7.72ms 8.54ms
Bloom Filter Large Scale Dns Brute Force 37.20ms 26.9 ops/sec 36.22ms 39.08ms
Large Closest Match Lookup 347.62ms 2.9 ops/sec 345.03ms 353.00ms
Realistic Closest Match Workload 190.01ms 5.3 ops/sec 188.33ms 191.62ms
Event Validation Full Scan Startup Small Batch 442.96ms 2.3 ops/sec 434.96ms 447.98ms
Event Validation Full Scan Startup Large Batch 800.78ms 1.2 ops/sec 754.53ms 838.61ms
Make Event Autodetection Small 31.25ms 32.0 ops/sec 30.84ms 34.29ms
Make Event Autodetection Large 318.98ms 3.1 ops/sec 317.56ms 320.52ms
Make Event Explicit Types 13.94ms 71.7 ops/sec 13.87ms 14.01ms
Excavate Single Thread Small 4.135s 0.2 ops/sec 4.023s 4.179s
Excavate Single Thread Large 9.856s 0.1 ops/sec 9.792s 9.908s
Excavate Parallel Tasks Small 4.322s 0.2 ops/sec 4.221s 4.375s
Excavate Parallel Tasks Large 7.434s 0.1 ops/sec 7.335s 7.500s
Is Ip Performance 3.22ms 310.1 ops/sec 3.10ms 4.40ms
Make Ip Type Performance 11.34ms 88.2 ops/sec 11.25ms 11.78ms
Mixed Ip Operations 4.46ms 224.1 ops/sec 4.42ms 4.87ms
Typical Queue Shuffle 62.40µs 16.0K ops/sec 60.25µs 102.97µs
Priority Queue Shuffle 712.97µs 1.4K ops/sec 694.26µs 902.76µs

🐍 Python Version 3.11.13

@blacklanternsecurity blacklanternsecurity deleted a comment from github-actions bot Aug 22, 2025
@liquidsec liquidsec merged commit e03356b into dev Sep 4, 2025
17 checks passed
@liquidsec liquidsec deleted the benchmark-tests branch September 4, 2025 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants