[WIP] feat(agent): training-ready data recipes, learnable-value mappers, cross-model similarity

[yxdyc]

• Add training-data YAMLs (R0 bridge, R1–R3, CPU-only R3), recipes README, TRAIN_DATA_FIELD_GUIDE.

• New mappers: cross-model cohorts (exact | normalized_query | simhash_lsh + match_basis), syslog/harness noise, error taxonomy (string evidence leaves), learnable value scorer, safety gate, distill trajectory, rewrite hints, training card (JSON string; forces turbo map).

• Wire MetaKeys and mapper init exports; extend bad_case / insight / dialog normalize for the training-data stack.

• Add a training-free before/after report (migration matrix, stage retention, hardness/quality ranking checks).

• Stabilize HF Arrow meta typing: tool_success_ratio uses -1.0 when undefined; usage total_tokens always int; bad_case respects ratio >= 0.

• Docs: demos/agent README links; analysis recipe points to R0 + field guide.

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive suite of new Mapper operators and recipes designed to transform agent interaction logs into high-quality training datasets. Key additions include mappers for cross-model pairing, error taxonomy classification, learnable value scoring, and trajectory distillation using teacher models. The PR also updates existing mappers to support these new features and provides detailed documentation and recipes for end-to-end training data pipelines. A critical issue was identified in the AgentCrossModelPairMapper regarding its scalability, as it currently loads the entire dataset into memory, which may lead to out-of-memory errors for large-scale data processing.

[gemini-code-assist]

This implementation loads the entire dataset into memory using dataset.to_list(), which will not scale to large datasets and can lead to out-of-memory errors. This is a critical issue for a data processing operator, as it bypasses the streaming capabilities of the underlying datasets library.

For group_key_mode set to exact or normalized_query, this can be implemented more scalably. Consider the following approach:

1. Use dataset.map to compute the grouping key for each sample and add it as a new column.
2. Sort the dataset by this new grouping key column using dataset.sort().
3. Iterate through the sorted dataset to process samples group by group. This avoids loading the entire dataset into memory at once.

For simhash_lsh mode, a global view of the data is harder to avoid. However, the current implementation's memory limitation should be clearly documented in the class docstring, noting that it's only suitable for small to medium-sized datasets. For future scalability, you might consider exploring approximate nearest neighbor (ANN) libraries that support distributed or out-of-core computation.

[ShenQianli]

here has_contrast only captures data with >=2 models, regardless of version. for example, {model=qwen-plus, version=3.5}, {model=qwen-plus, version=3.6} -> len(peer_models)=1 -> has_contrast=False. feel free to ignore if it is desired:)