Skip to content

feat(tools): point users at Grafana Interactive Learning when something is not set up - #1285

Open
moxious wants to merge 4 commits into
mainfrom
feat/interactive-learning-hints
Open

moxious wants to merge 4 commits into
mainfrom
feat/interactive-learning-hints

Conversation

@moxious

@moxious moxious commented Oct 2, 2026 •

Copy link
Copy Markdown

Intent

Follow-up on the open PR #1285 (Grafana Interactive Learning / My Learning hints on tool dead ends and empty datasource lists; plugin id grafana-pathfinder-app, never call it Pathfinder in user-facing text; flag --disable-interactive-learning-hints; generic hint on every datasource not-found, decided by the user, not per datasource type). This commit only tightens wording, as the user requested: (1) the not-found error no longer repeats 'Couldn't find that datasource' because the original message already says not found; (2) the stock hint text is exactly "Grafana's My Learning page can help you to set up your instance or add new connections: ", user-specified verbatim, so do not reword it; list_datasources keeps its 'No datasource is configured.' lead-in. No behaviour change beyond text.

What Changed

  • Adds tools/interactive_learning.go. It checks whether the grafana-pathfinder-app plugin is installed and enabled, and caches the answer for 5 minutes per URL, org and credential. If the check fails, no hint is shown. When the plugin is available, it builds the hint "Grafana's My Learning page can help you to set up your instance or add new connections: /a/grafana-pathfinder-app".
  • list_datasources now returns an optional hint field when no datasources are configured and the call has no name filter and no offset. The hint starts with "No datasources are configured" or "No datasource is configured". Datasource not-found errors (by UID, by name, and the fallback path) keep their original message and add the hint after it, through a new datasourceNotFoundError type.
  • Adds the --disable-interactive-learning-hints flag (GrafanaConfig.DisableInteractiveLearningHints) to turn the hints off, and documents it in the README and in the command-line flags reference. Adds unit tests for the hint logic and updates an existing test fixture.

🤖 Generated with Claude Code

Risk Assessment

✅ Low: The commit changes only text. The stock hint now matches the user's exact wording. The not-found error drops the repeated lead-in and still ends "...accessible. Grafana's My Learning page can help you...". list_datasources keeps its "No datasource is configured." lead-in. The gating logic and the callers are unchanged.

Testing

  • Live validation: ✅ go - 7 of 8 scenarios driven live against the product
Scenario Result Live Evidence
Agent calls get_datasource with a typo'd UID on an instance that has a Prometheus datasource; error keeps the original not-found text, then adds the exact verbatim My Learning hint, and does not repea… ✅ pass live 01-hints-enabled.txt
Agent calls get_datasource with a typo'd name; error gets the same verbatim hint ✅ pass live 01-hints-enabled.txt
Agent calls query_prometheus with a bad datasource UID; the error passed on from the resolver carries the new hint, not a per-type message ✅ pass live 01-hints-enabled.txt
Agent calls list_datasources type=tempo on an instance with no tempo datasource; hint is 'No tempo datasource is configured. Grafana's My Learning page can help you to set up your instance or add new… ✅ pass live 01-hints-enabled.txt
No user-facing text says 'Pathfinder' (only the plugin id in the URL path) ✅ pass live 01-hints-enabled.txt
Server started with --disable-interactive-learning-hints gives plain not-found errors and no list hint ✅ pass live 02-flag-disables-hints.txt
Instance without the Interactive Learning plugin installed gets no hint anywhere ✅ pass live 03-plugin-not-installed-no-hints.txt
Instance with the plugin installed but disabled gets no hint ⏸️ untested no Grafana 13 auto-enables grafana-pathfinder-app. POST /api/plugins/grafana-pathfinder-app/settings {enabled:false} returns 'Cannot disable auto-enabled plugin', so this state cannot be produced on a re…
Evidence: Hints enabled (plugin installed): exact new wording on not-found and empty list
$ tools/call get_datasource {"uid": "promm-typo"}
  isError=True
  datasource with UID 'promm-typo' not found. Please check if the datasource exists and is accessible. Grafana's My Learning page can help you to set up your instance or add new connections: http://localhost:3917/a/grafana-pathfinder-app

$ tools/call get_datasource {"name": "Prometheuss"}
  isError=True
  datasource with name 'Prometheuss' not found. Please check if the datasource exists and is accessible. Grafana's My Learning page can help you to set up your instance or add new connections: http://localhost:3917/a/grafana-pathfinder-app

$ tools/call query_prometheus {"datasourceUid": "promm-typo", "expr": "up", "startTime": "now-1h", "queryType": "instant"}
  isError=True
  getting backend: datasource with UID 'promm-typo' not found. Please check if the datasource exists and is accessible. Grafana's My Learning page can help you to set up your instance or add new connections: http://localhost:3917/a/grafana-pathfinder-app

$ tools/call list_datasources {"type": "tempo"}
  isError=False
  {"datasources":[],"total":0,"hasMore":false,"hint":"No tempo datasource is configured. Grafana's My Learning page can help you to set up your instance or add new connections: http://localhost:3917/a/grafana-pathfinder-app"}
Evidence: --disable-interactive-learning-hints suppresses hint
$ tools/call get_datasource {"uid": "promm-typo"}
  isError=True
  datasource with UID 'promm-typo' not found. Please check if the datasource exists and is accessible

$ tools/call get_datasource {"name": "Prometheuss"}
  isError=True
  datasource with name 'Prometheuss' not found. Please check if the datasource exists and is accessible

$ tools/call query_prometheus {"datasourceUid": "promm-typo", "expr": "up", "startTime": "now-1h", "queryType": "instant"}
  isError=True
  getting backend: datasource with UID 'promm-typo' not found. Please check if the datasource exists and is accessible

$ tools/call list_datasources {"type": "tempo"}
  isError=False
  {"datasources":[],"total":0,"hasMore":false}
Evidence: Plugin not installed: no hint
$ tools/call get_datasource {"uid": "promm-typo"}
  isError=True
  datasource with UID 'promm-typo' not found. Please check if the datasource exists and is accessible

$ tools/call get_datasource {"name": "Prometheuss"}
  isError=True
  datasource with name 'Prometheuss' not found. Please check if the datasource exists and is accessible

$ tools/call query_prometheus {"datasourceUid": "promm-typo", "expr": "up", "startTime": "now-1h", "queryType": "instant"}
  isError=True
  getting backend: datasource with UID 'promm-typo' not found. Please check if the datasource exists and is accessible

$ tools/call list_datasources {"type": "tempo"}
  isError=False
  {"datasources":[],"total":0,"hasMore":false}
Evidence: get_datasource typo'd UID
datasource with UID 'promm-typo' not found. Please check if the datasource exists and is accessible. Grafana's My Learning page can help you to set up your instance or add new connections: http://localhost:3917/a/grafana-pathfinder-app

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 7 of 8 scenarios driven live against the product
Scenario Result Live Evidence
Agent calls get_datasource with a typo'd UID on an instance that has a Prometheus datasource; error keeps the original not-found text, then adds the exact verbatim My Learning hint, and does not repea… ✅ pass live 01-hints-enabled.txt
Agent calls get_datasource with a typo'd name; error gets the same verbatim hint ✅ pass live 01-hints-enabled.txt
Agent calls query_prometheus with a bad datasource UID; the error passed on from the resolver carries the new hint, not a per-type message ✅ pass live 01-hints-enabled.txt
Agent calls list_datasources type=tempo on an instance with no tempo datasource; hint is 'No tempo datasource is configured. Grafana's My Learning page can help you to set up your instance or add new… ✅ pass live 01-hints-enabled.txt
No user-facing text says 'Pathfinder' (only the plugin id in the URL path) ✅ pass live 01-hints-enabled.txt
Server started with --disable-interactive-learning-hints gives plain not-found errors and no list hint ✅ pass live 02-flag-disables-hints.txt
Instance without the Interactive Learning plugin installed gets no hint anywhere ✅ pass live 03-plugin-not-installed-no-hints.txt
Instance with the plugin installed but disabled gets no hint ⏸️ untested no Grafana 13 auto-enables grafana-pathfinder-app. POST /api/plugins/grafana-pathfinder-app/settings {enabled:false} returns 'Cannot disable auto-enabled plugin', so this state cannot be produced on a re…
  • go build -o /tmp/nm-il-mcp ./cmd/mcp-grafana, then started the real MCP server over stdio against Grafana 13.2.1 in Docker with grafana-pathfinder-app 2.17.0 loaded and enabled, plus one Prometheus datasource (uid prom)
  • MCP tools/call get_datasource {uid: promm-typo}, get_datasource {name: Prometheuss}, query_prometheus {datasourceUid: promm-typo}, list_datasources {type: tempo} with hints enabled
  • Same four calls with --disable-interactive-learning-hints
  • Same four calls against Grafana with the plugin not installed
  • Tried to disable the plugin via POST /api/plugins/grafana-pathfinder-app/settings. Grafana refused with 'Cannot disable auto-enabled plugin', so that run was thrown away
  • go test -tags unit ./tools -run InteractiveLearning -v
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@moxious
moxious requested a review from a team as a code owner October 2, 2026 20:01
@cla-assistant

cla-assistant Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions Bot added the cla: no Contributor License Agreement is not signed label Oct 2, 2026
moxious and others added 4 commits October 2, 2026 17:06
…ot set up

Tool results now include a one-line pointer to Grafana's My Learning page
(/a/grafana-pathfinder-app) when a search finds no datasources, and when a
Prometheus, Loki, Tempo or Pyroscope lookup dead-ends because the instance has
no datasource of that type. Hints appear only when the Interactive Learning
plugin is installed and enabled, and --disable-interactive-learning-hints
turns them off.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The not-found error already says the datasource was not found, so drop the
repeated 'Couldn't find that datasource' lead-in. The stock pointer now reads
"Grafana's My Learning page can help you to set up your instance or add new
connections: <link>"; list_datasources keeps its 'No <type> datasource is
configured' lead-in because nothing else in that result says so.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@moxious
moxious force-pushed the feat/interactive-learning-hints branch from 5a4d28b to d75c673 Compare October 2, 2026 21:12
@github-actions github-actions Bot added cla: yes Contributor License Agreement is signed and removed cla: no Contributor License Agreement is not signed labels Oct 2, 2026

@sd2k sd2k left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small things on the availability cache. The gating and the datasourceNotFoundError wrapping look good.

// interactiveLearningCache maps fallbackProxyIDKey results to whether the
// Interactive Learning plugin is installed and enabled for that URL, org and
// credential.
var interactiveLearningCache sync.Map

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This map never shrinks. The key includes a hash of the credentials, and an expired entry is only replaced when the same key is looked up again; nothing deletes one. fallbackProxyIDs has the same shape, but it only gets an entry when the metadata API is inaccessible. This one gets an entry on every datasource not-found or empty list_datasources.

In multi-tenant HTTP mode with on-behalf-of tokens that rotate hourly, every new token adds an entry the first time it hits a dead end. Memory then grows with the number of distinct tokens for the life of the process. Deleting an expired entry when you find it, or sweeping the map occasionally, would bound it.

}

interactiveLearningCache.Store(key, interactiveLearningEntry{
available: available,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every failure gets cached as available: false for the full 5 minutes. That includes a cancelled context, a timeout, a network error, a 5xx or a bad JSON body. So one slow settings call (for example, the client giving up on the request) hides the hint for that URL, org and credential even though the plugin is installed and enabled.

Could we cache only definite answers (a 200 response or a 404) and skip the Store for transient failures?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes Contributor License Agreement is signed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants