Repository navigation
Give QueryEvaluator the Content fallback, and pin both query providers to one corpus (#3511) - #3534
Conversation
…s to one corpus (#3511) The query language has ONE grammar and TWO executors, in two repositories: QueryEvaluator here (in-memory, FileSystem and static-node hosts -- and, on EVERY backend, live-query relevance via PostgreSqlPartitionedMeshQuery.IsRelevant and merged-result ordering via MeshQuery.ClipMergedInitial) and PostgreSqlSqlGenerator.MapSelector in MeshWeaver.Plugins. Nothing compared them. MapSelector has always resolved a selector as: known column, else content.X walk, else n.content->>'X'. QueryEvaluator had the first two and not the third -- an unknown selector reflected against MeshNode and returned null, so the predicate matched NOTHING. Measured on a live Postgres mesh 2026-09-07: compilationStatus:Error returned 5, compilationStatus:Ok returned 195, disjoint; the same queries matched nothing in the evaluator. The negated form was worse than blind: CompareEqual(null, "Ok") is false, so -compilationStatus:Ok matched EVERY node -- a filter written to exclude returned what it excluded. The fix is the RULE, not a special case; compilationStatus is named nowhere. Ordering, and why: the object's own property wins and content is only the fallback. That is MapSelector's order (PropertyMap before the n.content->> default) and the only order that changes nothing already working -- content-first would re-point every live query whose selector names both, and name/description/ category/icon/order/state/version are common content field names. This way round the only selectors whose resolution moves are the ones that resolved to null, i.e. matched nothing. The fallback keys on the property being ABSENT, never on its value being null (GetDirectPropertyValue split into a presence-reporting TryGetDirectPropertyValue), because SQL answers the n.description column for description: and never falls through. First hop only, for the same parity. SelectorResolutionCorpus (test/MeshWeaver.Fixture, referenced by both repos) is the shared statement of the rule: per selector, the side each provider reads it from and the exact column expression SQL must emit. Core asserts QueryEvaluator resolves every row on its recorded side; the Plugins twin asserts MapSelector emits every row's SqlExpression. Divergences are named, not merely absent. Falsification, both arms: with the fallback removed 7 of 36 fail; restored, 36/36 pass. Full project 440/440, documentation suite 304/304. Two defects found on the way: - #3515's assertions used `GetPropertyValue(...)?.ToString().Should().Be(...)`, which short-circuits the whole chain on null -- .Should() is never reached and the test passes having asserted nothing. Both stayed GREEN against the un-fixed evaluator. Now stringified through interpolation (null renders ""), which is what took the red arm from 5 failures to 7. - MapSelector's PropertyMap omits createdBy/createdDate/lastModifiedBy/desiredId/ syncBehavior/excludeFromContext although all six are real mesh_nodes columns, so Postgres reads the content field of the same name. Same shape as Plugins#1310. Recorded in KnownDivergences; the fix is a Plugins change. AGENTS.md's pre-deploy sweep becomes content.compilationStatus:Error -- not because the bare form stays broken, but because the sweep runs against whatever image a portal is ALREADY running. The line is NOT in a shared-rule block (only rest-not-graphql and conserve-work-products are) and no satellite carries it, so this changes one file. Doc: Doc/Architecture/QueryProviderParity, the selector-resolution rule in Doc/DataMesh/QuerySyntax, and BuildIdentityAdmission corrected off "pinned rather than changed". Closes #3511 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
The changes correctly align selector resolution semantics across providers and are backed by stronger, control-based tests; only minor naming nits remain.
Pull request overview
This PR fixes a long-standing parity gap between the in-memory query executor (QueryEvaluator in core) and the Postgres SQL translator (MapSelector in MeshWeaver.Plugins) by adding a first-hop Content fallback in QueryEvaluator and introducing a shared selector-resolution corpus to keep both providers pinned to the same behavior. It also updates operational guidance and documentation so the recommended sweep/query spellings remain correct across deployed images and backends.
Changes:
- Add first-hop selector resolution fallback in
QueryEvaluator(node property first, thenContentof the same name) to match SQL behavior and fix negation behaving “worse than blind” for unknown selectors. - Introduce
SelectorResolutionCorpus(shared fixture data) and expand/strengthen the existing sweep selector test to validate the new resolution rule and its controls. - Update AGENTS guidance and documentation to describe the three-step selector resolution rule and the operationally safe sweep query form.
File summaries
| File | Description |
|---|---|
| test/MeshWeaver.Hosting.Test/SweepSelectorReachesTheCompileStatusTest.cs | Expands tests to cover the new fallback behavior, ordering decision, negation control, and validates resolver behavior against the shared corpus. |
| test/MeshWeaver.Fixture/SelectorResolutionCorpus.cs | Adds a shared selector-resolution corpus (data-only) to pin both providers’ behavior to one canonical list, including named divergences. |
| src/MeshWeaver.Hosting/Persistence/Query/QueryEvaluator.cs | Implements first-hop Content fallback with presence-aware property resolution (distinguishing absent vs null). |
| src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-07-searching-on-a-content-field-means-one-thing-everywhere.md | Adds a user-facing “Fix” entry describing the behavioral impact (live results, sorting, negation filters). |
| src/MeshWeaver.Documentation/Data/DataMesh/QuerySyntax.md | Documents the three-step selector resolution rule and recommends the explicit content.X form for operational stability. |
| src/MeshWeaver.Documentation/Data/Architecture/QueryProviderParity.md | New architecture doc explaining the two-provider model, the drift, the fix, the shared corpus, and remaining divergences. |
| src/MeshWeaver.Documentation/Data/Architecture/BuildIdentityAdmission.md | Updates the build identity admission guidance to reflect the fixed evaluator and the still-recommended dotted sweep form. |
| AGENTS.md | Updates the prescribed pre-prod sweep query to use content.compilationStatus and explains why the prefix remains required operationally. |
Review details
Suppressed comments (1)
test/MeshWeaver.Hosting.Test/SweepSelectorReachesTheCompileStatusTest.cs:129
- Test method name contains an all-caps run ("HOLDING") which reduces readability and isn’t consistent with typical method naming in this test project. Consider renaming to standard PascalCase.
public void AKnownFieldHOLDINGNullDoesNotFallIntoContent()
- Files reviewed: 8/8 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| [Fact] | ||
| public void TheBareSelectorDoesNotReachIt_WhichIsWhyAGreenSweepHereIsNotEvidence() | ||
| => new QueryEvaluator().GetPropertyValue(BrokenType(), "compilationStatus") | ||
| public void ANodeFieldBEATSASameNamedContentField() |
…chable (#3511) Found while checking the blast radius of the Content fallback, and reproduced deterministically before fixing. `sort:` resolves through the same GetPropertyValue as a filter, and MeshQuery.ClipMergedInitial runs OrderResults to order merged results on EVERY backend, Postgres included. Before the fallback a bare content selector resolved to null for every row, so every sort key was null and nothing could be mis-compared. The fallback widens that to the content's real values -- and JSON is not type-uniform: the same key is a string on one node and a number on another. Comparer<object>.Default answers that pair with InvalidOperationException: Failed to compare two elements in the array ---> ArgumentException: Object must be of type String so a widened selector would turn a merely differently-ordered query into a FAILED one, on the merge path every backend runs. The same latent hole already swallowed int against long; nothing reached it because nothing sorted on a mixed key. OrderResults now orders through a total comparer. Uniform keys keep their existing order EXACTLY -- same type + IComparable is dispatched first and untouched, so strings still sort as strings and ticks still sort chronologically. Only the pairs the default comparer REFUSED are newly decided: mixed numerics numerically, anything else by its invariant string form (ordinal, never resolved from the ambient culture). Postgres has no equivalent hazard because n.content->>'X' is always TEXT. SortingOnAContentFieldOfMixedTypes pins it; it was written first and observed to fail with the exception above. Falsification re-measured against the final suite: fallback removed 7 of 37 fail, restored 37/37. MeshWeaver.Hosting.Test 441/441, MeshWeaver.Graph.Test 1383/1383, MeshWeaver.Documentation.Test 304/304. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes #3511.
The defect, as the issue finally scoped it
The query language has ONE grammar and TWO executors, in two repositories:
QueryEvaluator(core)PostgreSqlPartitionedMeshQuery.IsRelevant) and merged-result ordering (MeshQuery.ClipMergedInitial)PostgreSqlSqlGenerator.MapSelector(MeshWeaver.Plugins)contentJSONB on every portalNothing compared them.
MapSelectorhas always resolved known column →content.Xwalk →n.content->>'X'.QueryEvaluatorhad the first two and not the third: an unknown selector reflected againstMeshNodeand returnednull, so the predicate matched nothing.Measured on a live Postgres mesh, 2026-09-07 — the numbers that re-scoped the issue:
nodeType:NodeType compilationStatus:ErrornodeType:NodeType compilationStatus:OknodeType:NodeType content.compilationStatus:Error🚨 The negated form was worse than blind:
CompareEqual(null, "Ok")isfalse, soNotEqualinverted totrue—-compilationStatus:Okmatched every node, and-status:Decommissionedmatched every deployment. A filter written to exclude returned what it was written to exclude.The fix
GetPropertyValueresolves the first hop as the object's own property, else the same name out ofContent— the rule SQL always had.compilationStatusis named nowhere; it is reachable because the rule reaches it.Three decisions, each stated in the code:
MapSelector's order and the only order that changes nothing already working. Content-first would silently re-point every live query whose selector names both — andname,description,category,icon,order,state,versionare common content field names. This way round, the only selectors whose resolution moves are the ones that resolved tonull— i.e. matched nothing.n.descriptioncolumn fordescription:and never falls through.GetDirectPropertyValueis split into a presence-reportingTryGetDirectPropertyValue.n.content->>'X'.The test that compares the two providers
test/MeshWeaver.Fixture/SelectorResolutionCorpus.cs— pure data, in a project both repos reference. Per selector: the side each provider reads it from, and the exact column expression SQL must emit. Core assertsQueryEvaluatorresolves every row on its recorded side; the Plugins twin assertsMapSelectoremits every row'sSqlExpression. Same shape as the existingQueryRouteClassifier. Divergences are named, not merely absent.The
#3515control (a healthy type answersOk, notError) is kept, and a second control added on the negation arm.Falsification — both arms, with numbers
MeshWeaver.Hosting.TestMeshWeaver.Graph.TestMeshWeaver.Documentation.Test(all ratchet/static/async guards)dotnet build -c Release -warnaserrorclean, one project per invocation, onMeshWeaver.Hosting,MeshWeaver.Fixture,MeshWeaver.Hosting.Test,MeshWeaver.Graph.Test,MeshWeaver.Documentation.Test.🚨 A third defect, found by checking the blast radius (second commit)
sort:resolves through the sameGetPropertyValue, andMeshQuery.ClipMergedInitialrunsOrderResultsto order merged results on every backend, Postgres included. Before the fallback a bare content selector resolved tonullfor every row, so every sort key was null and nothing could be mis-compared. The fallback widens that to the content's real values — and JSON is not type-uniform. Reproduced deterministically before fixing:So a widened selector would have turned a merely differently-ordered query into a failed one, on the merge path. (The same latent hole already swallowed
intagainstlong; nothing reached it because nothing sorted on a mixed key.)OrderResultsnow orders through a total comparer. Uniform keys keep their existing order exactly — same type +IComparableis dispatched first and untouched — so only the pairs the default comparer refused are newly decided: mixed numerics numerically, anything else by its invariant string form (ordinal, never the ambient culture). Postgres has no equivalent hazard:n.content->>'X'is alwaysTEXT.🚨 Two more defects found on the way
1. #3515's assertions could not fail.
GetPropertyValue(...)?.ToString().Should().Be("Error")short-circuits the whole chain on null —.Should()is never reached and the test passes having asserted nothing. Both stayed GREEN against the un-fixed evaluator. Now stringified through interpolation (null renders""). That repair is what took the red arm from 5 failures to 7.2.
PropertyMapomits six real columns.createdBy,createdDate,lastModifiedBy,desiredId,syncBehavior,excludeFromContextare all realmesh_nodescolumns (AuthorColumns,SyncBehaviorColumn,ExcludeFromContextColumn) thatMapSelectordoes not list, so Postgres reads the content field of the same name instead. Same shape as Plugins#1310.createdByis in live use (the chat history selector). This is a Plugins fix; recorded inKnownDivergencesand pinned so it cannot grow silently.The AGENTS.md sweep string — and a corrected premise
Changed to
search 'nodeType:NodeType content.compilationStatus:Error'— not because the bare form stays broken, but because the sweep runs against whatever image a portal is ALREADY running, where the fix has not shipped.🚨 The issue's premise that this line sits in a shared-rule block is FALSE. Core's
AGENTS.mdcarries exactly two blocks —rest-not-graphql(L142–143) andconserve-work-products(L145–146). The sweep line is L55, outside both, and no satellite carries the string at all (measured over all six via REST: Education/Reinsurance/SocialMedia/Manufacturing each have one unrelatedcompilationStatusmention; Plugins and Memex have none). So this changes ONE file andcheck-shared-rules.pyis not engaged.Work products committed
Doc/Architecture/QueryProviderParity(new) — the two executors, what the drift cost, the ordering decision, the corpus, the residual divergence table, and the third resolver (MeshSearchView.razor.cs, which falls back on null VALUE rather than absence and should delegate).Doc/DataMesh/QuerySyntax— the three-step selector-resolution rule as an author sees it.Doc/Architecture/BuildIdentityAdmission— corrected off "the evaluator is pinned rather than changed".Category: Fix) — live results not refreshing, content-field sorts being no-ops, and-status:Xreturning what it excluded.Pairs-with: none — no public type or member is removed (
GetPropertyValue's signature is unchanged; everything added). The Plugins follow-up (widenPropertyMap, add the corpus-drivenMapSelectortest, makeMeshSearchView's third resolver delegate) needs this merged plus a pin bump first, and is filed as Systemorph/MeshWeaver.Plugins#1439.🤖 Generated with Claude Code