Skip to content

Add all-route greedy timeout fallback and skip section search after timeout recovery#111

Open
Abse2001 wants to merge 13 commits into
tscircuit:mainfrom
Abse2001:main
Open

Add all-route greedy timeout fallback and skip section search after timeout recovery#111
Abse2001 wants to merge 13 commits into
tscircuit:mainfrom
Abse2001:main

Conversation

@Abse2001
Copy link
Copy Markdown
Contributor

@Abse2001 Abse2001 commented May 29, 2026

Motivation

SRJ18 samples were failing even when cloned ports should have made the routing problem solvable. Investigation showed the
failures were not all true static reachability failures:

  • DuplicateCongestedPortSolver dropped solvedRoutes, causing cloned graphs to lose endpoint hints and remap some routes to
    fallback centermost ports.
  • The section pipeline used a fixed outer iteration cap that could stop before the inner core solver reached its own timeout
    acceptance logic.
  • Some large cloned graphs are only recoverable through the core solver’s emergency greedy fallback, but the fallback only
    retried remaining routes from the current partial state.
  • When solveGraph succeeds via emergency timeout fallback, automatic section optimization can be prohibitively expensive and
    not useful.

Fix

  • Preserve solvedRoutes when cloning congested ports so route endpoint hints survive topology repair.
  • Add a general core timeout fallback that retries all routes greedily from scratch after the existing remaining-routes
    fallback fails.
  • Derive the section pipeline iteration cap from the inner solveGraph and section solver caps, plus overhead, instead of
    using a fixed 200_000.
  • Skip automatic section search when solveGraph completed via greedy timeout fallback, using an empty section mask to
    preserve the valid solveGraph output.

@vercel
Copy link
Copy Markdown

vercel Bot commented May 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
tiny-hypergraph Ready Ready Preview, Comment May 30, 2026 3:26pm

Request Review

@tscircuitbot
Copy link
Copy Markdown

Benchmark This PR

Run benchmarks by commenting on this PR:

/benchmark [benchmark.sh args...]
/benchmark --dataset <hg07|srj18> --limit <n> --solver <core|poly> --families <default|default+deep|all> --concurrency <n|auto>

Everything after /benchmark is forwarded directly to ./benchmark.sh.
The result comment shows the matching dataset from the latest stored main-branch benchmark artifact beside the PR run; main is not rerun for each PR benchmark request.

Examples:

  • /benchmark -> hg07, all samples, core solver
  • /benchmark --dataset srj18 --limit 20 -> first 20 srj18 samples
  • /benchmark --dataset srj18 --concurrency auto -> srj18 samples with runner CPU-count concurrency
  • /benchmark --sample sample002 -> one hg07 sample
  • /benchmark --limit 20 --solver poly -> first 20 hg07 samples with the poly solver
  • /benchmark --limit 40 --families default+deep -> first 40 hg07 samples with deeper candidate families

Any PR whose title contains [BENCHMARK TEST] will automatically run the benchmark workflow on PR updates.

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark --dataset srj18

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 29, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main srj18 core default 31.3% 2.049s 6.221s
PR srj18 core default 75.0% 2.347s 38.901s
Main Branch Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 31.3%
Improved rate 60.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 0.565
Avg final max region cost 0.499
Avg max region delta 0.066
Avg candidate count 5.200
Avg duration 2.160s
P50 duration 2.049s
P95 duration 6.221s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.103s
sample002 success 0.391 0.358 0.033 6 1.232s
sample003 success 0.429 0.429 0.000 5 0.671s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 0.864s
sample006 failed n/a n/a n/a 0 0.199s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 failed n/a n/a n/a 0 6.221s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample008 failed n/a n/a n/a 0 2.049s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample009 failed n/a n/a n/a 0 0.193s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 failed n/a n/a n/a 0 3.195s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample011 failed n/a n/a n/a 0 3.817s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample012 failed n/a n/a n/a 0 5.315s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample013 failed n/a n/a n/a 0 0.215s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 failed n/a n/a n/a 0 2.643s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample015 failed n/a n/a n/a 0 3.404s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample016 success 0.912 0.707 0.205 4 2.413s
PR Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 75.0%
Improved rate 33.3%
Zero final max region cost rate 0.0%
Avg baseline max region cost 12.113
Avg final max region cost 11.986
Avg max region delta 0.127
Avg candidate count 4.833
Avg duration 7.702s
P50 duration 2.347s
P95 duration 38.901s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.414s
sample002 success 0.391 0.358 0.033 6 1.316s
sample003 success 0.429 0.429 0.000 5 0.606s
sample004 failed n/a n/a n/a 0 0.025s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 1.187s
sample006 failed n/a n/a n/a 0 0.252s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 success 1.612 1.612 0.000 5 13.456s
sample008 success 2.859 2.859 0.000 7 6.156s
sample009 failed n/a n/a n/a 0 0.253s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 success 3.411 3.411 0.000 9 8.050s
sample011 success 1.792 0.599 1.193 6 18.207s
sample012 success 43.704 43.704 0.000 0 38.901s
sample013 failed n/a n/a n/a 0 0.405s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 success 14.175 14.175 0.000 5 7.994s
sample015 success 74.975 74.975 0.000 0 21.666s
sample016 success 0.912 0.707 0.205 4 2.347s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26646756234

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 29, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main hg07 core default 99.0% 0.114s 0.603s
PR hg07 core default 99.0% 0.107s 0.562s
Main Branch Details
Metric Value
Dataset hg07
Solver core
Families default
Samples 105
Success rate 99.0%
Improved rate 58.7%
Zero final max region cost rate 1.0%
Avg baseline max region cost 0.657
Avg final max region cost 0.264
Avg max region delta 0.393
Avg candidate count 7.990
Avg duration 0.185s
P50 duration 0.114s
P95 duration 0.603s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.017 0.017 0.000 5 0.067s
sample002 success 0.391 0.365 0.025 8 0.359s
sample003 success 0.242 0.220 0.022 8 0.167s
sample004 success 0.175 0.149 0.026 8 0.151s
sample005 success 0.042 0.042 0.000 5 0.022s
sample006 success 0.301 0.301 0.000 9 0.051s
sample007 success 0.143 0.143 0.000 10 0.047s
sample008 success 0.121 0.121 0.000 9 0.068s
sample009 success 0.301 0.301 0.000 10 0.130s
sample010 success 0.183 0.043 0.140 10 0.085s
sample011 success 0.301 0.301 0.000 10 0.055s
sample012 success 0.156 0.054 0.102 9 0.142s
sample013 success 0.233 0.233 0.000 7 0.102s
sample014 failed n/a n/a n/a 0 0.003s Error: Connection "source_trace_69" could not be mapped to route endpoints
sample015 success 0.039 0.039 0.000 5 0.016s
sample016 success 0.059 0.059 0.000 5 0.018s
sample017 success 0.105 0.105 0.000 8 0.092s
sample018 success 0.369 0.289 0.080 7 0.514s
sample019 success 0.156 0.156 0.000 10 0.067s
sample020 success 0.387 0.223 0.164 9 0.404s
sample021 success 0.878 0.426 0.452 8 0.382s
sample022 success 0.236 0.209 0.027 10 0.230s
sample023 success 0.525 0.337 0.188 9 0.395s
sample024 success 0.375 0.296 0.079 8 0.300s
sample025 success 0.280 0.217 0.063 10 0.123s
sample026 success 0.240 0.240 0.000 5 0.138s
sample027 success 0.250 0.250 0.000 6 0.318s
sample028 success 0.253 0.237 0.016 10 0.116s
sample029 success 0.545 0.491 0.055 7 0.349s
sample030 success 0.302 0.302 0.000 9 0.080s
sample031 success 0.636 0.405 0.231 9 0.133s
sample032 success 0.508 0.421 0.087 6 0.603s
sample033 success 0.047 0.047 0.000 5 0.021s
sample034 success 0.217 0.217 0.000 10 0.052s
sample035 success 0.304 0.301 0.003 7 0.227s
sample036 success 0.053 0.053 0.000 10 0.014s
sample037 success 0.722 0.368 0.354 9 0.848s
sample038 success 0.156 0.156 0.000 10 0.121s
sample039 success 0.411 0.240 0.171 9 0.271s
sample040 success 0.156 0.156 0.000 9 0.087s
sample041 success 0.378 0.234 0.144 7 0.308s
sample042 success 0.122 0.087 0.035 8 0.050s
sample043 success 0.175 0.175 0.000 10 0.026s
sample044 success 0.272 0.240 0.032 10 0.154s
sample045 success 0.100 0.100 0.000 10 0.025s
sample046 success 0.000 0.000 0.000 0 0.003s
sample047 success 0.217 0.188 0.029 9 0.038s
sample048 success 0.223 0.223 0.000 10 0.052s
sample049 success 0.450 0.450 0.000 9 0.071s
sample050 success 0.433 0.300 0.133 9 0.561s
sample051 success 0.280 0.265 0.016 6 0.105s
sample052 success 0.133 0.090 0.043 7 0.055s
sample053 success 0.244 0.244 0.000 9 0.286s
sample054 success 0.268 0.217 0.051 8 0.278s
sample055 success 0.628 0.391 0.237 9 0.477s
sample056 success 0.401 0.247 0.154 10 0.605s
sample057 success 0.298 0.192 0.106 9 0.321s
sample058 success 0.242 0.178 0.064 9 0.278s
sample059 success 0.998 0.713 0.285 6 0.343s
sample060 success 0.398 0.351 0.047 5 0.206s
sample061 success 0.368 0.365 0.003 8 0.246s
sample062 success 0.390 0.335 0.055 6 0.402s
sample063 success 0.270 0.153 0.117 9 0.025s
sample064 success 0.217 0.217 0.000 10 0.096s
sample065 success 0.334 0.301 0.033 9 0.185s
sample066 success 0.062 0.062 0.000 9 0.034s
sample067 success 0.714 0.581 0.133 5 0.630s
sample068 success 0.489 0.361 0.128 5 0.632s
sample069 success 0.041 0.041 0.000 9 0.012s
sample070 success 0.253 0.253 0.000 9 0.186s
sample071 success 0.793 0.672 0.121 6 0.345s
sample072 success 0.110 0.108 0.002 9 0.091s
sample073 success 0.741 0.654 0.087 9 0.199s
sample074 success 0.117 0.103 0.015 10 0.017s
sample075 success 0.217 0.217 0.000 9 0.009s
sample076 success 0.386 0.386 0.000 7 0.076s
sample077 success 25.120 0.623 24.497 7 0.264s
sample078 success 0.101 0.049 0.052 8 0.028s
sample079 success 0.107 0.107 0.000 10 0.015s
sample080 success 0.182 0.077 0.105 9 0.043s
sample081 success 0.148 0.148 0.000 10 0.027s
sample082 success 0.217 0.217 0.000 7 0.036s
sample083 success 0.493 0.449 0.043 6 0.433s
sample084 success 0.250 0.110 0.141 10 0.027s
sample085 success 0.172 0.147 0.025 9 0.085s
sample086 success 0.182 0.095 0.087 8 0.140s
sample087 success 0.156 0.156 0.000 10 0.034s
sample088 success 0.758 0.758 0.000 2 0.231s
sample089 success 0.324 0.217 0.107 8 0.222s
sample090 success 0.096 0.065 0.031 8 0.066s
sample091 success 0.386 0.325 0.061 5 0.455s
sample092 success 0.181 0.181 0.000 8 0.025s
sample093 success 11.141 1.090 10.051 6 0.276s
sample094 success 1.556 1.063 0.493 5 0.226s
sample095 success 0.197 0.118 0.079 10 0.028s
sample096 success 0.032 0.032 0.000 10 0.016s
sample097 success 0.791 0.740 0.051 6 0.879s
sample098 success 1.053 0.534 0.519 7 0.515s
sample099 success 0.247 0.141 0.106 7 0.114s
sample100 success 0.211 0.150 0.061 8 0.134s
sample101 success 0.989 0.989 0.000 9 0.111s
sample102 success 0.301 0.301 0.000 6 0.035s
sample103 success 0.058 0.043 0.014 9 0.041s
sample104 success 0.074 0.074 0.000 10 0.029s
sample105 success 0.187 0.187 0.000 6 0.072s
PR Details
Metric Value
Dataset hg07
Solver core
Families default
Samples 105
Success rate 99.0%
Improved rate 58.7%
Zero final max region cost rate 1.0%
Avg baseline max region cost 0.657
Avg final max region cost 0.264
Avg max region delta 0.393
Avg candidate count 7.990
Avg duration 0.176s
P50 duration 0.107s
P95 duration 0.562s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.017 0.017 0.000 5 0.057s
sample002 success 0.391 0.365 0.025 8 0.297s
sample003 success 0.242 0.220 0.022 8 0.132s
sample004 success 0.175 0.149 0.026 8 0.131s
sample005 success 0.042 0.042 0.000 5 0.016s
sample006 success 0.301 0.301 0.000 9 0.034s
sample007 success 0.143 0.143 0.000 10 0.032s
sample008 success 0.121 0.121 0.000 9 0.053s
sample009 success 0.301 0.301 0.000 10 0.126s
sample010 success 0.183 0.043 0.140 10 0.107s
sample011 success 0.301 0.301 0.000 10 0.027s
sample012 success 0.156 0.054 0.102 9 0.060s
sample013 success 0.233 0.233 0.000 7 0.090s
sample014 failed n/a n/a n/a 0 0.002s Error: Connection "source_trace_69" could not be mapped to route endpoints
sample015 success 0.039 0.039 0.000 5 0.010s
sample016 success 0.059 0.059 0.000 5 0.011s
sample017 success 0.105 0.105 0.000 8 0.054s
sample018 success 0.369 0.289 0.080 7 0.364s
sample019 success 0.156 0.156 0.000 10 0.047s
sample020 success 0.387 0.223 0.164 9 0.369s
sample021 success 0.878 0.426 0.452 8 0.407s
sample022 success 0.236 0.209 0.027 10 0.220s
sample023 success 0.525 0.337 0.188 9 0.397s
sample024 success 0.375 0.296 0.079 8 0.294s
sample025 success 0.280 0.217 0.063 10 0.115s
sample026 success 0.240 0.240 0.000 5 0.108s
sample027 success 0.250 0.250 0.000 6 0.311s
sample028 success 0.253 0.237 0.016 10 0.100s
sample029 success 0.545 0.491 0.055 7 0.342s
sample030 success 0.302 0.302 0.000 9 0.061s
sample031 success 0.636 0.405 0.231 9 0.110s
sample032 success 0.508 0.421 0.087 6 0.562s
sample033 success 0.047 0.047 0.000 5 0.010s
sample034 success 0.217 0.217 0.000 10 0.051s
sample035 success 0.304 0.301 0.003 7 0.218s
sample036 success 0.053 0.053 0.000 10 0.010s
sample037 success 0.722 0.368 0.354 9 0.886s
sample038 success 0.156 0.156 0.000 10 0.115s
sample039 success 0.411 0.240 0.171 9 0.275s
sample040 success 0.156 0.156 0.000 9 0.084s
sample041 success 0.378 0.234 0.144 7 0.300s
sample042 success 0.122 0.087 0.035 8 0.047s
sample043 success 0.175 0.175 0.000 10 0.024s
sample044 success 0.272 0.240 0.032 10 0.140s
sample045 success 0.100 0.100 0.000 10 0.026s
sample046 success 0.000 0.000 0.000 0 0.003s
sample047 success 0.217 0.188 0.029 9 0.035s
sample048 success 0.223 0.223 0.000 10 0.058s
sample049 success 0.450 0.450 0.000 9 0.065s
sample050 success 0.433 0.300 0.133 9 0.537s
sample051 success 0.280 0.265 0.016 6 0.097s
sample052 success 0.133 0.090 0.043 7 0.053s
sample053 success 0.244 0.244 0.000 9 0.276s
sample054 success 0.268 0.217 0.051 8 0.276s
sample055 success 0.628 0.391 0.237 9 0.467s
sample056 success 0.401 0.247 0.154 10 0.591s
sample057 success 0.298 0.192 0.106 9 0.312s
sample058 success 0.242 0.178 0.064 9 0.279s
sample059 success 0.998 0.713 0.285 6 0.333s
sample060 success 0.398 0.351 0.047 5 0.205s
sample061 success 0.368 0.365 0.003 8 0.238s
sample062 success 0.390 0.335 0.055 6 0.382s
sample063 success 0.270 0.153 0.117 9 0.028s
sample064 success 0.217 0.217 0.000 10 0.095s
sample065 success 0.334 0.301 0.033 9 0.183s
sample066 success 0.062 0.062 0.000 9 0.034s
sample067 success 0.714 0.581 0.133 5 0.623s
sample068 success 0.489 0.361 0.128 5 0.608s
sample069 success 0.041 0.041 0.000 9 0.012s
sample070 success 0.253 0.253 0.000 9 0.186s
sample071 success 0.793 0.672 0.121 6 0.350s
sample072 success 0.110 0.108 0.002 9 0.093s
sample073 success 0.741 0.654 0.087 9 0.197s
sample074 success 0.117 0.103 0.015 10 0.016s
sample075 success 0.217 0.217 0.000 9 0.008s
sample076 success 0.386 0.386 0.000 7 0.072s
sample077 success 25.120 0.623 24.497 7 0.264s
sample078 success 0.101 0.049 0.052 8 0.027s
sample079 success 0.107 0.107 0.000 10 0.015s
sample080 success 0.182 0.077 0.105 9 0.042s
sample081 success 0.148 0.148 0.000 10 0.026s
sample082 success 0.217 0.217 0.000 7 0.024s
sample083 success 0.493 0.449 0.043 6 0.394s
sample084 success 0.250 0.110 0.141 10 0.027s
sample085 success 0.172 0.147 0.025 9 0.085s
sample086 success 0.182 0.095 0.087 8 0.151s
sample087 success 0.156 0.156 0.000 10 0.032s
sample088 success 0.758 0.758 0.000 2 0.232s
sample089 success 0.324 0.217 0.107 8 0.232s
sample090 success 0.096 0.065 0.031 8 0.066s
sample091 success 0.386 0.325 0.061 5 0.463s
sample092 success 0.181 0.181 0.000 8 0.025s
sample093 success 11.141 1.090 10.051 6 0.276s
sample094 success 1.556 1.063 0.493 5 0.219s
sample095 success 0.197 0.118 0.079 10 0.027s
sample096 success 0.032 0.032 0.000 10 0.016s
sample097 success 0.791 0.740 0.051 6 0.851s
sample098 success 1.053 0.534 0.519 7 0.505s
sample099 success 0.247 0.141 0.106 7 0.108s
sample100 success 0.211 0.150 0.061 8 0.132s
sample101 success 0.989 0.989 0.000 9 0.128s
sample102 success 0.301 0.301 0.000 6 0.049s
sample103 success 0.058 0.043 0.014 9 0.039s
sample104 success 0.074 0.074 0.000 10 0.027s
sample105 success 0.187 0.187 0.000 6 0.076s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26646762387

@Abse2001 Abse2001 changed the title Preserve solved-route hints during port duplication and add clone-port benchmarking support Add all-route greedy timeout fallback and skip section search after timeout recovery May 29, 2026
Copy link
Copy Markdown
Contributor

@seveibar seveibar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Abse2001 just summarizing some stuff from our meeting:

  • Instead of doing a "greedy panic", can we try to have a greedy initialization OR greedy initialization, my concern is that we're calling greedySolver.solve() which is difficult to debug and breaks the regular iteration system
  • In our meeting i thought the solved routes from the duplicate port congestion solver would be usable as a fallback, i think this is actually NOT possible because the solved routes likely share ports illegally!

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark --dataset srj18

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 29, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main srj18 core default 31.3% 2.049s 6.221s
PR srj18 core default 75.0% 2.207s 42.746s
Main Branch Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 31.3%
Improved rate 60.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 0.565
Avg final max region cost 0.499
Avg max region delta 0.066
Avg candidate count 5.200
Avg duration 2.160s
P50 duration 2.049s
P95 duration 6.221s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.103s
sample002 success 0.391 0.358 0.033 6 1.232s
sample003 success 0.429 0.429 0.000 5 0.671s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 0.864s
sample006 failed n/a n/a n/a 0 0.199s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 failed n/a n/a n/a 0 6.221s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample008 failed n/a n/a n/a 0 2.049s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample009 failed n/a n/a n/a 0 0.193s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 failed n/a n/a n/a 0 3.195s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample011 failed n/a n/a n/a 0 3.817s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample012 failed n/a n/a n/a 0 5.315s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample013 failed n/a n/a n/a 0 0.215s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 failed n/a n/a n/a 0 2.643s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample015 failed n/a n/a n/a 0 3.404s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample016 success 0.912 0.707 0.205 4 2.413s
PR Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 75.0%
Improved rate 33.3%
Zero final max region cost rate 0.0%
Avg baseline max region cost 12.113
Avg final max region cost 11.986
Avg max region delta 0.127
Avg candidate count 4.833
Avg duration 7.261s
P50 duration 2.207s
P95 duration 42.746s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.485s
sample002 success 0.391 0.358 0.033 6 1.324s
sample003 success 0.429 0.429 0.000 5 0.593s
sample004 failed n/a n/a n/a 0 0.025s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 1.223s
sample006 failed n/a n/a n/a 0 0.256s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 success 1.612 1.612 0.000 5 13.248s
sample008 success 2.859 2.859 0.000 7 5.471s
sample009 failed n/a n/a n/a 0 0.255s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 success 3.411 3.411 0.000 9 6.857s
sample011 success 1.792 0.599 1.193 6 16.078s
sample012 success 43.704 43.704 0.000 0 42.746s
sample013 failed n/a n/a n/a 0 0.384s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 success 14.175 14.175 0.000 5 6.638s
sample015 success 74.975 74.975 0.000 0 16.381s
sample016 success 0.912 0.707 0.205 4 2.207s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26655698427

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark --dataset srj18

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 29, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main srj18 core default 31.3% 2.049s 6.221s
PR srj18 core default 75.0% 2.945s 29.830s
Main Branch Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 31.3%
Improved rate 60.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 0.565
Avg final max region cost 0.499
Avg max region delta 0.066
Avg candidate count 5.200
Avg duration 2.160s
P50 duration 2.049s
P95 duration 6.221s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.103s
sample002 success 0.391 0.358 0.033 6 1.232s
sample003 success 0.429 0.429 0.000 5 0.671s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 0.864s
sample006 failed n/a n/a n/a 0 0.199s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 failed n/a n/a n/a 0 6.221s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample008 failed n/a n/a n/a 0 2.049s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample009 failed n/a n/a n/a 0 0.193s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 failed n/a n/a n/a 0 3.195s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample011 failed n/a n/a n/a 0 3.817s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample012 failed n/a n/a n/a 0 5.315s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample013 failed n/a n/a n/a 0 0.215s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 failed n/a n/a n/a 0 2.643s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample015 failed n/a n/a n/a 0 3.404s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample016 success 0.912 0.707 0.205 4 2.413s
PR Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 75.0%
Improved rate 25.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 16.548
Avg final max region cost 16.518
Avg max region delta 0.031
Avg candidate count 4.917
Avg duration 6.411s
P50 duration 2.945s
P95 duration 29.830s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.654 0.508 0.146 5 5.808s
sample002 success 0.252 0.231 0.022 8 1.554s
sample003 success 0.972 0.972 0.000 8 0.824s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.582 0.582 0.000 9 0.903s
sample006 failed n/a n/a n/a 0 0.196s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 success 1.612 1.410 0.201 6 29.830s
sample008 success 2.859 2.859 0.000 0 9.641s
sample009 failed n/a n/a n/a 0 0.193s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 success 3.411 3.411 0.000 9 6.039s
sample011 success 0.569 0.569 0.000 5 7.358s
sample012 success 43.704 43.704 0.000 0 19.008s
sample013 failed n/a n/a n/a 0 0.225s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 success 14.175 14.175 0.000 5 5.701s
sample015 success 127.904 127.904 0.000 0 12.325s
sample016 success 1.888 1.888 0.000 4 2.945s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26655973718

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 29, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main hg07 core default 99.0% 0.114s 0.603s
PR hg07 core default 99.0% 0.098s 0.588s
Main Branch Details
Metric Value
Dataset hg07
Solver core
Families default
Samples 105
Success rate 99.0%
Improved rate 58.7%
Zero final max region cost rate 1.0%
Avg baseline max region cost 0.657
Avg final max region cost 0.264
Avg max region delta 0.393
Avg candidate count 7.990
Avg duration 0.185s
P50 duration 0.114s
P95 duration 0.603s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.017 0.017 0.000 5 0.067s
sample002 success 0.391 0.365 0.025 8 0.359s
sample003 success 0.242 0.220 0.022 8 0.167s
sample004 success 0.175 0.149 0.026 8 0.151s
sample005 success 0.042 0.042 0.000 5 0.022s
sample006 success 0.301 0.301 0.000 9 0.051s
sample007 success 0.143 0.143 0.000 10 0.047s
sample008 success 0.121 0.121 0.000 9 0.068s
sample009 success 0.301 0.301 0.000 10 0.130s
sample010 success 0.183 0.043 0.140 10 0.085s
sample011 success 0.301 0.301 0.000 10 0.055s
sample012 success 0.156 0.054 0.102 9 0.142s
sample013 success 0.233 0.233 0.000 7 0.102s
sample014 failed n/a n/a n/a 0 0.003s Error: Connection "source_trace_69" could not be mapped to route endpoints
sample015 success 0.039 0.039 0.000 5 0.016s
sample016 success 0.059 0.059 0.000 5 0.018s
sample017 success 0.105 0.105 0.000 8 0.092s
sample018 success 0.369 0.289 0.080 7 0.514s
sample019 success 0.156 0.156 0.000 10 0.067s
sample020 success 0.387 0.223 0.164 9 0.404s
sample021 success 0.878 0.426 0.452 8 0.382s
sample022 success 0.236 0.209 0.027 10 0.230s
sample023 success 0.525 0.337 0.188 9 0.395s
sample024 success 0.375 0.296 0.079 8 0.300s
sample025 success 0.280 0.217 0.063 10 0.123s
sample026 success 0.240 0.240 0.000 5 0.138s
sample027 success 0.250 0.250 0.000 6 0.318s
sample028 success 0.253 0.237 0.016 10 0.116s
sample029 success 0.545 0.491 0.055 7 0.349s
sample030 success 0.302 0.302 0.000 9 0.080s
sample031 success 0.636 0.405 0.231 9 0.133s
sample032 success 0.508 0.421 0.087 6 0.603s
sample033 success 0.047 0.047 0.000 5 0.021s
sample034 success 0.217 0.217 0.000 10 0.052s
sample035 success 0.304 0.301 0.003 7 0.227s
sample036 success 0.053 0.053 0.000 10 0.014s
sample037 success 0.722 0.368 0.354 9 0.848s
sample038 success 0.156 0.156 0.000 10 0.121s
sample039 success 0.411 0.240 0.171 9 0.271s
sample040 success 0.156 0.156 0.000 9 0.087s
sample041 success 0.378 0.234 0.144 7 0.308s
sample042 success 0.122 0.087 0.035 8 0.050s
sample043 success 0.175 0.175 0.000 10 0.026s
sample044 success 0.272 0.240 0.032 10 0.154s
sample045 success 0.100 0.100 0.000 10 0.025s
sample046 success 0.000 0.000 0.000 0 0.003s
sample047 success 0.217 0.188 0.029 9 0.038s
sample048 success 0.223 0.223 0.000 10 0.052s
sample049 success 0.450 0.450 0.000 9 0.071s
sample050 success 0.433 0.300 0.133 9 0.561s
sample051 success 0.280 0.265 0.016 6 0.105s
sample052 success 0.133 0.090 0.043 7 0.055s
sample053 success 0.244 0.244 0.000 9 0.286s
sample054 success 0.268 0.217 0.051 8 0.278s
sample055 success 0.628 0.391 0.237 9 0.477s
sample056 success 0.401 0.247 0.154 10 0.605s
sample057 success 0.298 0.192 0.106 9 0.321s
sample058 success 0.242 0.178 0.064 9 0.278s
sample059 success 0.998 0.713 0.285 6 0.343s
sample060 success 0.398 0.351 0.047 5 0.206s
sample061 success 0.368 0.365 0.003 8 0.246s
sample062 success 0.390 0.335 0.055 6 0.402s
sample063 success 0.270 0.153 0.117 9 0.025s
sample064 success 0.217 0.217 0.000 10 0.096s
sample065 success 0.334 0.301 0.033 9 0.185s
sample066 success 0.062 0.062 0.000 9 0.034s
sample067 success 0.714 0.581 0.133 5 0.630s
sample068 success 0.489 0.361 0.128 5 0.632s
sample069 success 0.041 0.041 0.000 9 0.012s
sample070 success 0.253 0.253 0.000 9 0.186s
sample071 success 0.793 0.672 0.121 6 0.345s
sample072 success 0.110 0.108 0.002 9 0.091s
sample073 success 0.741 0.654 0.087 9 0.199s
sample074 success 0.117 0.103 0.015 10 0.017s
sample075 success 0.217 0.217 0.000 9 0.009s
sample076 success 0.386 0.386 0.000 7 0.076s
sample077 success 25.120 0.623 24.497 7 0.264s
sample078 success 0.101 0.049 0.052 8 0.028s
sample079 success 0.107 0.107 0.000 10 0.015s
sample080 success 0.182 0.077 0.105 9 0.043s
sample081 success 0.148 0.148 0.000 10 0.027s
sample082 success 0.217 0.217 0.000 7 0.036s
sample083 success 0.493 0.449 0.043 6 0.433s
sample084 success 0.250 0.110 0.141 10 0.027s
sample085 success 0.172 0.147 0.025 9 0.085s
sample086 success 0.182 0.095 0.087 8 0.140s
sample087 success 0.156 0.156 0.000 10 0.034s
sample088 success 0.758 0.758 0.000 2 0.231s
sample089 success 0.324 0.217 0.107 8 0.222s
sample090 success 0.096 0.065 0.031 8 0.066s
sample091 success 0.386 0.325 0.061 5 0.455s
sample092 success 0.181 0.181 0.000 8 0.025s
sample093 success 11.141 1.090 10.051 6 0.276s
sample094 success 1.556 1.063 0.493 5 0.226s
sample095 success 0.197 0.118 0.079 10 0.028s
sample096 success 0.032 0.032 0.000 10 0.016s
sample097 success 0.791 0.740 0.051 6 0.879s
sample098 success 1.053 0.534 0.519 7 0.515s
sample099 success 0.247 0.141 0.106 7 0.114s
sample100 success 0.211 0.150 0.061 8 0.134s
sample101 success 0.989 0.989 0.000 9 0.111s
sample102 success 0.301 0.301 0.000 6 0.035s
sample103 success 0.058 0.043 0.014 9 0.041s
sample104 success 0.074 0.074 0.000 10 0.029s
sample105 success 0.187 0.187 0.000 6 0.072s
PR Details
Metric Value
Dataset hg07
Solver core
Families default
Samples 105
Success rate 99.0%
Improved rate 58.7%
Zero final max region cost rate 1.0%
Avg baseline max region cost 0.415
Avg final max region cost 0.258
Avg max region delta 0.157
Avg candidate count 8.038
Avg duration 0.173s
P50 duration 0.098s
P95 duration 0.588s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.022 0.017 0.004 10 0.098s
sample002 success 0.489 0.404 0.086 5 0.600s
sample003 success 0.242 0.220 0.022 8 0.121s
sample004 success 0.175 0.149 0.026 8 0.140s
sample005 success 0.042 0.042 0.000 5 0.015s
sample006 success 0.301 0.301 0.000 9 0.040s
sample007 success 0.143 0.143 0.000 10 0.039s
sample008 success 0.121 0.121 0.000 9 0.067s
sample009 success 0.301 0.301 0.000 10 0.106s
sample010 success 0.183 0.043 0.140 10 0.074s
sample011 success 0.045 0.045 0.000 4 0.021s
sample012 success 0.156 0.054 0.102 9 0.066s
sample013 success 0.233 0.233 0.000 7 0.096s
sample014 failed n/a n/a n/a 0 0.002s Error: Connection "source_trace_69" could not be mapped to route endpoints
sample015 success 0.039 0.039 0.000 5 0.008s
sample016 success 0.059 0.059 0.000 5 0.010s
sample017 success 0.105 0.105 0.000 8 0.042s
sample018 success 0.420 0.225 0.195 8 0.389s
sample019 success 0.156 0.156 0.000 10 0.041s
sample020 success 0.296 0.296 0.000 8 0.157s
sample021 success 10.495 0.804 9.691 8 0.266s
sample022 success 0.236 0.209 0.027 10 0.208s
sample023 success 0.328 0.249 0.079 5 0.245s
sample024 success 0.408 0.296 0.112 7 0.216s
sample025 success 0.280 0.217 0.063 10 0.079s
sample026 success 0.240 0.240 0.000 4 0.106s
sample027 success 0.250 0.250 0.000 8 0.351s
sample028 success 0.237 0.237 0.000 9 0.066s
sample029 success 0.545 0.491 0.055 7 0.279s
sample030 success 0.302 0.302 0.000 10 0.064s
sample031 success 0.636 0.337 0.298 9 0.120s
sample032 success 0.599 0.575 0.023 8 0.517s
sample033 success 0.047 0.047 0.000 5 0.012s
sample034 success 0.217 0.217 0.000 10 0.069s
sample035 success 0.411 0.222 0.189 5 0.200s
sample036 success 0.053 0.053 0.000 10 0.012s
sample037 success 0.702 0.460 0.242 8 0.735s
sample038 success 0.156 0.156 0.000 10 0.092s
sample039 success 0.411 0.240 0.171 9 0.250s
sample040 success 0.156 0.156 0.000 8 0.079s
sample041 success 0.302 0.262 0.041 7 0.197s
sample042 success 0.147 0.087 0.060 9 0.065s
sample043 success 0.175 0.175 0.000 10 0.023s
sample044 success 0.272 0.229 0.043 10 0.152s
sample045 success 0.100 0.100 0.000 10 0.017s
sample046 success 0.000 0.000 0.000 0 0.002s
sample047 success 0.217 0.188 0.029 9 0.042s
sample048 success 0.223 0.223 0.000 10 0.091s
sample049 success 0.450 0.450 0.000 9 0.070s
sample050 success 0.499 0.300 0.200 9 0.588s
sample051 success 0.245 0.217 0.028 9 0.093s
sample052 success 0.176 0.069 0.107 10 0.079s
sample053 success 0.244 0.244 0.000 9 0.289s
sample054 success 0.268 0.217 0.051 8 0.294s
sample055 success 0.635 0.397 0.238 7 0.514s
sample056 success 0.363 0.340 0.022 8 0.344s
sample057 success 0.200 0.180 0.020 9 0.341s
sample058 success 0.326 0.257 0.069 5 0.188s
sample059 success 1.018 0.754 0.265 6 0.368s
sample060 success 0.458 0.351 0.108 7 0.219s
sample061 success 0.570 0.326 0.243 7 0.305s
sample062 success 0.391 0.301 0.090 6 0.583s
sample063 success 0.270 0.153 0.117 9 0.039s
sample064 success 0.217 0.217 0.000 10 0.109s
sample065 success 0.301 0.301 0.000 8 0.105s
sample066 success 0.062 0.062 0.000 9 0.030s
sample067 success 0.962 0.673 0.289 4 0.643s
sample068 success 0.578 0.447 0.131 4 0.287s
sample069 success 0.045 0.041 0.005 10 0.014s
sample070 success 0.425 0.425 0.000 10 0.220s
sample071 success 1.058 0.707 0.351 9 0.417s
sample072 success 0.110 0.108 0.002 9 0.084s
sample073 success 0.741 0.654 0.087 9 0.177s
sample074 success 0.117 0.103 0.015 10 0.022s
sample075 success 0.217 0.217 0.000 9 0.010s
sample076 success 0.617 0.346 0.271 8 0.099s
sample077 success 0.565 0.565 0.000 6 0.265s
sample078 success 0.101 0.049 0.052 8 0.027s
sample079 success 0.107 0.107 0.000 10 0.019s
sample080 success 0.182 0.077 0.105 9 0.049s
sample081 success 0.148 0.148 0.000 10 0.025s
sample082 success 0.217 0.217 0.000 8 0.025s
sample083 success 0.355 0.345 0.010 5 0.339s
sample084 success 0.250 0.110 0.141 10 0.026s
sample085 success 0.172 0.147 0.025 9 0.082s
sample086 success 0.182 0.095 0.087 8 0.169s
sample087 success 0.156 0.156 0.000 10 0.041s
sample088 success 0.486 0.447 0.039 6 0.389s
sample089 success 0.289 0.175 0.115 7 0.165s
sample090 success 0.072 0.065 0.007 8 0.070s
sample091 success 0.319 0.286 0.033 6 0.449s
sample092 success 0.181 0.181 0.000 8 0.030s
sample093 success 0.982 0.565 0.416 8 0.382s
sample094 success 1.582 1.196 0.386 7 0.306s
sample095 success 0.197 0.118 0.079 10 0.030s
sample096 success 0.032 0.032 0.000 10 0.017s
sample097 success 0.747 0.602 0.145 6 0.899s
sample098 success 0.478 0.415 0.063 8 0.596s
sample099 success 0.247 0.141 0.106 7 0.110s
sample100 success 0.211 0.150 0.061 8 0.133s
sample101 success 0.989 0.989 0.000 9 0.123s
sample102 success 0.301 0.301 0.000 9 0.059s
sample103 success 0.069 0.043 0.026 10 0.051s
sample104 success 0.074 0.074 0.000 10 0.026s
sample105 success 0.187 0.187 0.000 6 0.065s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26656046446

@Abse2001 Abse2001 requested a review from seveibar May 29, 2026 19:19
Copy link
Copy Markdown
Contributor

@seveibar seveibar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dies this work? Did it increase scores


const DEFAULT_SECTION_SOLVER_MAX_ITERATIONS = 50_000
const DEFAULT_SECTION_PIPELINE_MAX_ITERATIONS = 200_000
const SECTION_PIPELINE_ITERATION_OVERHEAD = 10_000
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?? What is this concept 🤔

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it works and I removed the SECTION_PIPELINE_ITERATION_OVERHEAD must of been codex added it and I didn't catch it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and yes I ran another benchmark and it still works with 75% on srj18

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark --dataset srj18

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 29, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main srj18 core default 31.3% 2.049s 6.221s
PR srj18 core default 75.0% 2.936s 26.107s
Main Branch Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 31.3%
Improved rate 60.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 0.565
Avg final max region cost 0.499
Avg max region delta 0.066
Avg candidate count 5.200
Avg duration 2.160s
P50 duration 2.049s
P95 duration 6.221s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.103s
sample002 success 0.391 0.358 0.033 6 1.232s
sample003 success 0.429 0.429 0.000 5 0.671s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 0.864s
sample006 failed n/a n/a n/a 0 0.199s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 failed n/a n/a n/a 0 6.221s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample008 failed n/a n/a n/a 0 2.049s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample009 failed n/a n/a n/a 0 0.193s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 failed n/a n/a n/a 0 3.195s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample011 failed n/a n/a n/a 0 3.817s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample012 failed n/a n/a n/a 0 5.315s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample013 failed n/a n/a n/a 0 0.215s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 failed n/a n/a n/a 0 2.643s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample015 failed n/a n/a n/a 0 3.404s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample016 success 0.912 0.707 0.205 4 2.413s
PR Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 75.0%
Improved rate 25.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 16.548
Avg final max region cost 16.518
Avg max region delta 0.031
Avg candidate count 4.917
Avg duration 6.639s
P50 duration 2.936s
P95 duration 26.107s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.654 0.508 0.146 5 5.858s
sample002 success 0.252 0.231 0.022 8 1.562s
sample003 success 0.972 0.972 0.000 8 0.806s
sample004 failed n/a n/a n/a 0 0.028s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.582 0.582 0.000 9 0.881s
sample006 failed n/a n/a n/a 0 0.201s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 success 1.612 1.410 0.201 6 26.107s
sample008 success 2.859 2.859 0.000 0 8.967s
sample009 failed n/a n/a n/a 0 0.188s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 success 3.411 3.411 0.000 9 6.196s
sample011 success 0.569 0.569 0.000 5 7.175s
sample012 success 43.704 43.704 0.000 0 21.527s
sample013 failed n/a n/a n/a 0 0.252s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 success 14.175 14.175 0.000 5 7.088s
sample015 success 127.904 127.904 0.000 0 16.461s
sample016 success 1.888 1.888 0.000 4 2.936s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26657795991

@Abse2001 Abse2001 requested a review from seveibar May 29, 2026 19:35
@seveibar
Copy link
Copy Markdown
Contributor

Do we know the impact on tscircuit autorouter?

@Abse2001
Copy link
Copy Markdown
Contributor Author

will update on autorouter rn

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark --dataset srj18

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 29, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main srj18 core default 31.3% 2.049s 6.221s
PR srj18 core default 68.8% 2.629s 94.941s
Main Branch Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 31.3%
Improved rate 60.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 0.565
Avg final max region cost 0.499
Avg max region delta 0.066
Avg candidate count 5.200
Avg duration 2.160s
P50 duration 2.049s
P95 duration 6.221s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.103s
sample002 success 0.391 0.358 0.033 6 1.232s
sample003 success 0.429 0.429 0.000 5 0.671s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 0.864s
sample006 failed n/a n/a n/a 0 0.199s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 failed n/a n/a n/a 0 6.221s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample008 failed n/a n/a n/a 0 2.049s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample009 failed n/a n/a n/a 0 0.193s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 failed n/a n/a n/a 0 3.195s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample011 failed n/a n/a n/a 0 3.817s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample012 failed n/a n/a n/a 0 5.315s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample013 failed n/a n/a n/a 0 0.215s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 failed n/a n/a n/a 0 2.643s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample015 failed n/a n/a n/a 0 3.404s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample016 success 0.912 0.707 0.205 4 2.413s
PR Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 68.8%
Improved rate 36.4%
Zero final max region cost rate 0.0%
Avg baseline max region cost 9.731
Avg final max region cost 9.678
Avg max region delta 0.053
Avg candidate count 6.727
Avg duration 14.749s
P50 duration 2.629s
P95 duration 94.941s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.654 0.508 0.146 5 5.869s
sample002 success 0.252 0.231 0.022 8 1.633s
sample003 success 0.972 0.972 0.000 8 0.779s
sample004 failed n/a n/a n/a 0 0.023s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.582 0.582 0.000 9 1.069s
sample006 failed n/a n/a n/a 0 0.243s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 success 1.612 1.410 0.201 6 31.068s
sample008 success 10.453 10.239 0.214 9 13.253s
sample009 failed n/a n/a n/a 0 0.246s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 success 3.411 3.411 0.000 9 7.775s
sample011 success 0.569 0.569 0.000 5 8.474s
sample012 failed n/a n/a n/a 0 94.941s RangeError: Out of memory
sample013 failed n/a n/a n/a 0 0.335s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 success 14.175 14.175 0.000 5 6.324s
sample015 success 72.478 72.478 0.000 6 61.319s
sample016 success 1.888 1.888 0.000 4 2.629s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26663209805

@Abse2001
Copy link
Copy Markdown
Contributor Author

/benchmark --dataset srj18

@tscircuitbot
Copy link
Copy Markdown

tscircuitbot commented May 30, 2026

Tiny Hypergraph Benchmark Results

Run Dataset Solver Families Success P50 P95
Main srj18 core default 31.3% 2.049s 6.221s
PR srj18 core default 75.0% 3.570s 66.830s
Main Branch Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 31.3%
Improved rate 60.0%
Zero final max region cost rate 0.0%
Avg baseline max region cost 0.565
Avg final max region cost 0.499
Avg max region delta 0.066
Avg candidate count 5.200
Avg duration 2.160s
P50 duration 2.049s
P95 duration 6.221s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.670 0.579 0.090 4 2.103s
sample002 success 0.391 0.358 0.033 6 1.232s
sample003 success 0.429 0.429 0.000 5 0.671s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.423 0.423 0.000 7 0.864s
sample006 failed n/a n/a n/a 0 0.199s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 failed n/a n/a n/a 0 6.221s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample008 failed n/a n/a n/a 0 2.049s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample009 failed n/a n/a n/a 0 0.193s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 failed n/a n/a n/a 0 3.195s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample011 failed n/a n/a n/a 0 3.817s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample012 failed n/a n/a n/a 0 5.315s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample013 failed n/a n/a n/a 0 0.215s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 failed n/a n/a n/a 0 2.643s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample015 failed n/a n/a n/a 0 3.404s Error: TinyHyperGraphSectionPipelineSolver ran out of iterations
sample016 success 0.912 0.707 0.205 4 2.413s
PR Details
Metric Value
Dataset srj18
Solver core
Families default
Samples 16
Success rate 75.0%
Improved rate 33.3%
Zero final max region cost rate 0.0%
Avg baseline max region cost 12.562
Avg final max region cost 12.514
Avg max region delta 0.049
Avg candidate count 6.167
Avg duration 10.929s
P50 duration 3.570s
P95 duration 66.830s
Sample Status Baseline Final Delta Attempts Duration Error
sample001 success 0.654 0.508 0.146 5 6.404s
sample002 success 0.252 0.231 0.022 8 1.732s
sample003 success 0.972 0.972 0.000 8 0.827s
sample004 failed n/a n/a n/a 0 0.029s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample005 success 0.582 0.582 0.000 9 1.137s
sample006 failed n/a n/a n/a 0 0.253s Error: Static reachability precheck failed: 9 route(s) have no legal path under the current reservation and start-reg...
sample007 success 1.612 1.410 0.201 6 32.372s
sample008 success 10.453 10.239 0.214 9 17.851s
sample009 failed n/a n/a n/a 0 0.253s Error: Static reachability precheck failed: 2 route(s) have no legal path under the current reservation and start-reg...
sample010 success 3.411 3.411 0.000 9 7.826s
sample011 success 0.569 0.569 0.000 5 7.502s
sample012 success 43.704 43.704 0.000 0 21.383s
sample013 failed n/a n/a n/a 0 0.338s Error: Static reachability precheck failed: 11 route(s) have no legal path under the current reservation and start-re...
sample014 success 14.175 14.175 0.000 5 6.556s
sample015 success 72.478 72.478 0.000 6 66.830s
sample016 success 1.888 1.888 0.000 4 3.570s

Workflow: View run
Artifact: https://github.com/tscircuit/tiny-hypergraph/actions/runs/26681335852

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants