idea-8.txt

Great—this is the spine of the whole system. Below is a concrete, implementable heuristic scoring spec that is:

fast (no deep parsing)

explainable

composable

tunable

Rust-friendly

Think of this as a probabilistic rules engine, not if/else spaghetti.

Heuristic Scoring Rules

Semantic Inference Engine v0.1

0. Core model (mental + code)

Each file accumulates evidence for multiple roles.

At the end, you normalize → select → compute confidence.

struct RoleScore {
    weights: HashMap<Role, f32>,
    signals: HashMap<Role, Vec<Signal>>,
}


Rules add weight, never overwrite.

1. Scoring philosophy (important)

Start with no assumptions

Prefer path > filename > extension > content

Multiple weak signals beat one strong one

Confidence comes from agreement, not magnitude

2. Base roles & priors

No global priors by default (avoid bias).

Optional minimal priors (very light):

prod        +0.05
docs        +0.05 (for *.md, *.mdx only)
generated   +0.02


Everything else starts at 0.

3. Path-based heuristics (strongest signal)
Rule class: PathIntent

Applies if path contains directory segment match (case-insensitive).

Path fragment	Role	Weight
/test/, /tests/, /__tests__/	test	+0.60
/spec/, /specs/	test	+0.55
/infra/, /terraform/, /pulumi/, /helm/	infra	+0.65
/.github/workflows/	infra	+0.70
/docs/, /doc/, /site/	docs	+0.65
/config/, /configs/	config	+0.55
/scripts/, /tools/, /bin/	scripts	+0.55
/examples/, /samples/, /demo/	examples	+0.55
/vendor/, /third_party/, /node_modules/	vendor	+0.90

Notes

Vendor is near-certain

CI paths outrank generic infra

Path match applies once per file

4. Filename morphology (strong, precise)
Rule class: FilenamePattern
Pattern	Role	Weight
*_test.*	test	+0.75
*.spec.*	test	+0.70
*.e2e.*	test:e2e	+0.80
*.integration.*	test:integration	+0.80
*_fixture.*	test:fixture	+0.60
Dockerfile*	infra	+0.85
docker-compose.*	infra	+0.80
Makefile	infra	+0.65
*.tf, *.tfvars	infra	+0.90
helmfile.*	infra	+0.85
*.env*	config	+0.85
config.*, settings.*	config	+0.60

Filename beats path if conflict exists.

5. Extension bias (weak signal)
Rule class: ExtensionBias

Used only to nudge, never decide.

Extension	Role	Weight
.md, .mdx	docs	+0.20
.yaml, .yml	config	+0.15
.json	config	+0.10
.sql (in /migrations/)	infra:data	+0.50
.proto	interface	+0.40
.lock	generated	+0.90

Extension bias is ignored if path/filename already decisive.

6. Neighborhood inference (medium signal, powerful)
Rule class: DirectoryConsensus

If ≥70% of sibling files resolve to the same role with confidence ≥0.7:

Apply that role with +0.40

Record signal Neighborhood

This is run after first-pass inference.

This is how unlabelled helpers get classified.

7. Shallow content probe (optional, capped)

Strict limits

Read max first 10 lines

Regex only

No AST

Rule class: HeaderProbe
Pattern	Role	Weight
Code generated by	generated	+0.95
DO NOT EDIT	generated	+0.90
terraform {	infra	+0.80
describe\( / test\(	test	+0.60
package main in /cmd/	prod	+0.60
@generated	generated	+0.90

Header probe can override weak conflicts, but not vendor.

8. Conflict resolution

After scoring:

Select role with highest weight

If second-highest within 0.15, mark as ambiguous

Reduce confidence by 20% if ambiguous

test: 0.72
prod: 0.61   ← ambiguous


Result:

role = test
confidence = 0.72 × 0.8 = 0.58

9. Confidence calculation (simple, honest)
confidence =
    clamp(
        max_weight × agreement_factor,
        0.0,
        1.0
    )


Where:

agreement_factor = min(1.0, signals.len() × 0.25)

1 signal → 0.25

2 signals → 0.50

3+ signals → 0.75–1.0

This rewards multiple independent confirmations.

10. Overrides (human intent always wins)

Optional semantic-overrides.yaml:

prod:
  - internal/core/**
infra:
  - ops/**


Overrides:

Apply with weight = 1.0

Signal = Override

Confidence = 1.0

Overrides are explicit, auditable, rare.

11. Final role selection order (tie-break)

If equal weight:

Override

Vendor

Generated

Test

Infra

Prod

Docs

Config

Scripts / Examples

This avoids misclassifying third-party or generated code.

12. Explainability output (critical)

Every file should be explainable:

{
  "role": "test",
  "confidence": 0.82,
  "signals": ["path", "filename"]
}


This is non-negotiable.

13. Why this works in practice

Humans encode intent structurally

Files cluster by purpose

Tests are named loudly

Infra lives in obvious places

Generated code self-identifies

Ambiguity is rare and surfaced

You don’t need perfect classification—
you need stable, honest aggregates.

14. MVP defaults (recommended)

Enable: path, filename, extension, neighborhood

Disable by default: header probe (opt-in)

Emit confidence always

Allow overrides but don’t require them

15. What to add later (v0.2+)

Language-specific probes

Import-based boundary detection

Churn-weighted confidence

Repo-specific heuristics packs