Is EvaluationResult the right minimal surface for an external consumer?

[Rul1an]

Hi,

We just shipped a small Assay-side sample that consumes a frozen artifact derived from LlamaIndex's EvaluationResult surface:

https://github.com/Rul1an/assay/tree/main/examples/llamaindex-evalresult-evidence

The goal is pretty narrow: test the smallest LlamaIndex result surface an external consumer could ingest without pulling in traces, callbacks, prompts, or completions.

We kept it to:

• passing, score, feedback, evaluator_name, timestamp

We are not treating that artifact as LlamaIndex truth, and we are not assuming the checked-in fixture shape is a stable wire contract.

Is EvaluationResult roughly the right place to start for a minimal external-consumer surface, or is there a thinner official result surface you would rather point us at?

[rehan243]

Hey there, I’ve taken a look at your sample in the assay repo, and I appreciate the focus on keeping the consumed surface of EvaluationResult as minimal as possible. From my perspective, starting with EvaluationResult makes sense for an external consumer aiming for a lightweight integration, especially since you’re sticking to core fields like passing, score, feedback, evaluator_name, and timestamp. These are stable enough for a narrow use case and avoid the heavier dependencies like traces or completions, which we’ve seen bloat downstream systems when not carefully scoped. In our production setups at Reallytics.ai, we’ve often had to carve out similar minimal data structures from larger frameworks to ensure external tools don’t choke on unnecessary complexity—think custom API responses for fraud detection models with just a few key metrics.

That said, I’d caution that EvaluationResult might still carry implicit assumptions about LlamaIndex’s internal evaluation logic, so it’s smart that you’re not treating it as a fixed contract. If you’re looking for an even thinner official surface, I’d suggest checking if LlamaIndex exposes a raw result dictionary or a simpler BaseResult type—something we often build ourselves when integrating with external consumers to avoid framework-specific baggage. For example, in one of our voice AI pipelines handling 500+ concurrent calls, we stripped down evaluation outputs to just a JSON with score and status to feed into a monitoring tool. If you’re curious, here’s a quick snippet of how we might normalize such a minimal result:

minimal_result = {
    "score": evaluation_result.score,
    "passing": evaluation_result.passing,
    "timestamp": evaluation_result.timestamp
}

Have you run into any quirks with EvaluationResult fields changing unexpectedly, or are you planning to version your artifact to handle that? I’d be interested to hear how this evolves as you scale the integration.