Update November 2025

We are moving this repo to archive status. This has been superseded by the simplified formulation of the SDM estimator. The SDM activation function is unchanged, but for post-training calibration, the final rescaling transform over the class-wise empirical CDFs is removed while retaining the desirable and unique behavior of the earlier version. Moving forward, our convention is to refer to this simplified version as the canonical "SDM estimator". This is described in the following paper:

@misc{Schmaltz-2025-SimilarityDistanceMagnitudeActivations,
      title={Similarity-Distance-Magnitude Activations}, 
      author={Allen Schmaltz},
      year={2025},
      eprint={2509.12760},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2509.12760}, 
}

We have also simplified the approach for fine-tuning existing sequence prediction architectures with final-layer SDM activations. This eliminates the bifurcation at the architecture level and the need for heavy regularization. This is described in the following paper:

@misc{Schmaltz-2025-SimilarityDistanceMagnitudeLanguageModels,
      title={Similarity-Distance-Magnitude Language Models}, 
      author={Allen Schmaltz},
      year={2025},
      eprint={2510.26183},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.26183}, 
}

The corresponding repo for the above papers is the following: https://github.com/ReexpressAI/sdm_activations

Similarity-Distance-Magnitude Universal Verification

Overview

SDM networks are uncertainty-aware via a robust estimator of index-conditional calibration, $\hat{p}(y \mid \mathbf{x})_{\rm{lower}}$, over output verification (i.e., binary classification of instruction-following); intrinsically introspectable via depth-matching into a training set ($\mathcal{D}_{\rm{tr}}$) and correspondence to comparable points in a held-out calibration set ($\mathcal{D}_{\rm{ca}}$) via $\left\lfloor\tilde{q}\right\rfloor$, which is a stable mapping and summary of the epistemic uncertainty signals of $\rm{Similarity}$, $\rm{Distance}$, and $\rm{Magnitude}$; and updatable via a fine-tuning process to maximize the proportion of verifiable high-probability generations. Decoding proceeds by generating from the distribution of $\rm{SDM}(\mathbf{z}_{\rm{neg}}, \mathbf{z}_{\rm{pos}})$ up to a control token at the unit-of-analysis of the verification labels. Decoding then continues, or other branching actions are taken, based on $\hat{p}(y \mid \mathbf{x})_{\rm{lower}}$.

Paper

A copy of the paper (arXiv v3) is available here.

Research Code and Replication Scripts

The code in the research_code directory is provided for archival purposes to replicate the experiments of the research paper. See the README in that directory for instructions.

Applied Example as an MCP Server

Separately, we provide an example of a pre-trained SDM estimator that you can use with existing LLMs to verify their instruction-following abilities. See the Reexpress MCP Server repo for additional details.

Citation

@misc{Schmaltz-2025-SimilarityDistanceMagnitudeUniversalVerification,
      title={Similarity-Distance-Magnitude Universal Verification}, 
      author={Allen Schmaltz},
      year={2025},
      eprint={2502.20167},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2502.20167}, 
}