An error occurred within the plugin

[earlbread]

When querying a Quickwit index with Grafana Datasource, the query fails with a downstream error.

Index doc mapping

{
  "doc_mapping_uid": "01JYK2J58VAC6HJX2H90K9F7R6",
  "mode": "dynamic",
  "dynamic_mapping": {
    "indexed": true,
    "tokenizer": "raw",
    "record": "basic",
    "stored": true,
    "expand_dots": true,
    "fast": {
      "normalizer": "raw"
    }
  },
  "field_mappings": [
    {
      "name": "actor",
      "type": "object",
      "field_mappings": [
        {
          "description": "Actor Type (Employee, User, System)",
          "fast": {
            "normalizer": "raw"
          },
          "fieldnorms": false,
          "indexed": true,
          "name": "type",
          "record": "basic",
          "stored": true,
          "tokenizer": "raw",
          "type": "text"
        },
        {
          "description": "Actor Metadata",
          "expand_dots": true,
          "fast": false,
          "indexed": true,
          "name": "metadata",
          "record": "basic",
          "stored": true,
          "tokenizer": "default",
          "type": "json"
        }
      ]
    },
    {
      "name": "event",
      "type": "object",
      "field_mappings": [
        {
          "description": "Event Type (Normal, Authorization, Privacy, Location)",
          "fast": {
            "normalizer": "raw"
          },
          "fieldnorms": false,
          "indexed": true,
          "name": "type",
          "record": "basic",
          "stored": true,
          "tokenizer": "raw",
          "type": "text"
        },
        {
          "description": "Event Operation",
          "fast": {
            "normalizer": "raw"
          },
          "fieldnorms": false,
          "indexed": true,
          "name": "operation",
          "record": "basic",
          "stored": true,
          "tokenizer": "default",
          "type": "text"
        },
        {
          "description": "Event Reason",
          "fast": false,
          "fieldnorms": false,
          "indexed": true,
          "name": "reason",
          "record": "basic",
          "stored": true,
          "tokenizer": "default",
          "type": "text"
        },
        {
          "field_mappings": [
            {
              "description": "Event Resource Type",
              "fast": {
                "normalizer": "raw"
              },
              "fieldnorms": false,
              "indexed": true,
              "name": "type",
              "record": "basic",
              "stored": true,
              "tokenizer": "raw",
              "type": "text"
            },
            {
              "description": "Event Resource Value",
              "fast": {
                "normalizer": "raw"
              },
              "fieldnorms": false,
              "indexed": true,
              "name": "value",
              "record": "basic",
              "stored": true,
              "tokenizer": "default",
              "type": "text"
            }
          ],
          "name": "resource",
          "type": "object"
        },
        {
          "description": "Event Metadata",
          "expand_dots": true,
          "fast": false,
          "indexed": true,
          "name": "metadata",
          "record": "basic",
          "stored": true,
          "tokenizer": "default",
          "type": "json"
        }
      ]
    },
    {
      "name": "source",
      "type": "object",
      "field_mappings": [
        {
          "description": "Source Type (Admin, Service)",
          "fast": {
            "normalizer": "raw"
          },
          "fieldnorms": false,
          "indexed": true,
          "name": "type",
          "record": "basic",
          "stored": true,
          "tokenizer": "raw",
          "type": "text"
        },
        {
          "field_mappings": [
            {
              "description": "Source Name",
              "fast": {
                "normalizer": "raw"
              },
              "fieldnorms": false,
              "indexed": true,
              "name": "name",
              "record": "basic",
              "stored": true,
              "tokenizer": "default",
              "type": "text"
            },
            {
              "description": "Source Country Code",
              "fast": {
                "normalizer": "raw"
              },
              "fieldnorms": false,
              "indexed": true,
              "name": "country_code",
              "record": "basic",
              "stored": true,
              "tokenizer": "raw",
              "type": "text"
            },
            {
              "description": "Source URL",
              "fast": false,
              "fieldnorms": false,
              "indexed": true,
              "name": "url",
              "record": "basic",
              "stored": true,
              "tokenizer": "default",
              "type": "text"
            },
            {
              "description": "Source Request ID",
              "fast": {
                "normalizer": "raw"
              },
              "fieldnorms": false,
              "indexed": true,
              "name": "request_id",
              "record": "basic",
              "stored": true,
              "tokenizer": "raw",
              "type": "text"
            }
          ],
          "name": "metadata",
          "type": "object"
        }
      ]
    },
    {
      "name": "timestamp",
      "type": "datetime",
      "description": "Log occurrence timestamp",
      "fast": true,
      "fast_precision": "seconds",
      "indexed": true,
      "input_formats": [
        "iso8601",
        "unix_timestamp"
      ],
      "output_format": "unix_timestamp_nanos",
      "stored": true
    },
    {
      "name": "env",
      "type": "text",
      "description": "Environment (alpha, prod)",
      "fast": {
        "normalizer": "raw"
      },
      "fieldnorms": false,
      "indexed": true,
      "record": "basic",
      "stored": true,
      "tokenizer": "raw"
    },
    {
      "name": "region",
      "type": "text",
      "description": "Region (kr, ca, jp, gb)",
      "fast": {
        "normalizer": "raw"
      },
      "fieldnorms": false,
      "indexed": true,
      "record": "basic",
      "stored": true,
      "tokenizer": "raw"
    }
  ],
  "timestamp_field": "timestamp",
  "tag_fields": [],
  "max_num_partitions": 200,
  "index_field_presence": false,
  "store_document_size": false,
  "store_source": false,
  "tokenizers": []
}

Quickwit Query Result

{
  "num_hits": 3,
  "hits": [
    {
      "actor": {
        "metadata": {
          "admin_role": "unknown",
          "admin_service_name": "unknown",
          "email": "daey@daangn.com",
          "ip": "0.0.0.0"
        },
        "type": "Employee"
      },
      "env": "alpha",
      "event": {
        "metadata": {
          "after": "{\"id\":\"locale_push_inbox_t\",\"title\":\"locale_push_inbox_t\",\"description\":\"locale test\",\"creator_email\":\"daey@daangn.com\",\"assignee_service_code\":\"\",\"category\":\"CATEGORY_UNSPECIFIED\",\"assignee_department_code\":\"\",\"is_dry_run\":false,\"running_notification_experiment\":null,\"delivery_statistics\":null,\"delete_time\":null,\"create_time\":\"2024-12-13T08:41:22.147200682Z\",\"update_time\":\"2024-12-13T08:41:22.147200682Z\",\"archive_time\":null,\"archive_scheduled_time\":null,\"unmet_monitoring_criteria\":null,\"contact_email\":\"daey@daangn.com\"}",
          "before": "",
          "field": "notification_scenario"
        },
        "operation": "create",
        "resource": {
          "type": "NotificationScenario",
          "value": "locale_push_inbox_t"
        },
        "type": "Normal"
      },
      "region": "kr",
      "source": {
        "metadata": {
          "country_code": "alpha-kr",
          "name": "NotificationCenter",
          "request_id": "237c3974-0315-46fc-8ea4-5ea2e4356398",
          "url": "https://example.com/"
        },
        "type": "Admin"
      },
      "timestamp": 1735262123293169000
    },
    {
      "actor": {
        "metadata": {
          "admin_role": "unknown",
          "admin_service_name": "unknown",
          "email": "daey@daangn.com",
          "ip": "0.0.0.0"
        },
        "type": "Employee"
      },
      "env": "alpha",
      "event": {
        "metadata": {
          "after": "{\"id\":\"locale_push_inbox_t\",\"title\":\"locale_push_inbox_t\",\"description\":\"locale test\",\"creator_email\":\"daey@daangn.com\",\"assignee_service_code\":\"\",\"category\":\"CATEGORY_UNSPECIFIED\",\"assignee_department_code\":\"\",\"is_dry_run\":false,\"running_notification_experiment\":null,\"delivery_statistics\":null,\"delete_time\":null,\"create_time\":\"2024-12-13T08:41:22.147200682Z\",\"update_time\":\"2024-12-13T08:41:22.147200682Z\",\"archive_time\":null,\"archive_scheduled_time\":null,\"unmet_monitoring_criteria\":null,\"contact_email\":\"daey@daangn.com\"}",
          "before": "",
          "field": "notification_scenario"
        },
        "operation": "create",
        "resource": {
          "type": "NotificationScenario",
          "value": "locale_push_inbox_t"
        },
        "type": "Normal"
      },
      "region": "kr",
      "source": {
        "metadata": {
          "country_code": "alpha-kr",
          "name": "NotificationCenter",
          "request_id": "237c3974-0315-46fc-8ea4-5ea2e4356398",
          "url": "https://example.com/"
        },
        "type": "Admin"
      },
      "timestamp": 1735262123293169000
    },
    {
      "actor": {
        "metadata": {
          "admin_role": "unknown",
          "admin_service_name": "unknown",
          "email": "daey@daangn.com",
          "ip": "0.0.0.0"
        },
        "type": "Employee"
      },
      "env": "alpha",
      "event": {
        "metadata": {
          "after": "{\"id\":\"locale_push_inbox_t\",\"title\":\"locale_push_inbox_t\",\"description\":\"locale test\",\"creator_email\":\"daey@daangn.com\",\"assignee_service_code\":\"\",\"category\":\"CATEGORY_UNSPECIFIED\",\"assignee_department_code\":\"\",\"is_dry_run\":false,\"running_notification_experiment\":null,\"delivery_statistics\":null,\"delete_time\":null,\"create_time\":\"2024-12-13T08:41:22.147200682Z\",\"update_time\":\"2024-12-13T08:41:22.147200682Z\",\"archive_time\":null,\"archive_scheduled_time\":null,\"unmet_monitoring_criteria\":null,\"contact_email\":\"daey@daangn.com\"}",
          "before": "",
          "field": "notification_scenario"
        },
        "operation": "create",
        "resource": {
          "type": "NotificationScenario",
          "value": "locale_push_inbox_t"
        },
        "type": "Normal"
      },
      "region": "kr",
      "source": {
        "metadata": {
          "country_code": "alpha-kr",
          "name": "NotificationCenter",
          "request_id": "237c3974-0315-46fc-8ea4-5ea2e4356398",
          "url": "https://example.com/"
        },
        "type": "Admin"
      },
      "timestamp": 1735262123293169000
    }
  ],
  "elapsed_time_micros": 27088,
  "errors": []
}

Query Inspection

traceId:undefined
request:Object
url:"api/ds/query?ds_type=quickwit-quickwit-datasource&requestId=explore_1dw"
method:"POST"
data:Object
queries:Array[1]
0:Object
metrics:Array[1]
0:Object
id:"1"
type:"logs"
query:""
refId:"A"
datasource:Object
type:"quickwit-quickwit-datasource"
uid:"ceq08xjzvsdtse"
key:"Q-34f88434-080d-47a4-83a4-ff7d739fbd35-0"
alias:""
bucketAggs:Array[1]
0:Object
type:"date_histogram"
id:"2"
settings:Object
interval:"auto"
field:""
timeField:""
datasourceId:166
intervalMs:60000
maxDataPoints:1720
queryCachingTTL:undefined
from:"1735225200000"
to:"1735311599000"
hideFromInspector:false
response:Object
message:"An error occurred within the plugin"
messageId:"plugin.downstreamError"
statusCode:500

[earlbread]

I've identified the cause. The first field (actor) in the index doc_mapping is of object type, and it seems like it's only looking for the timestamp field within the inner fields (type, metadata) of this object. As a result, it determines that there's no timestamp.

This issue does not occur if the timestamp field is moved to the top level of the doc_mapping.

	refTimestampFieldName, refTimestampOutputFormat := FindTimestampFieldInfos(indexMetadataList[0])
	if refTimestampFieldName == "" || refTimestampOutputFormat == "" {
		return "", "", fmt.Errorf("Fuck Invalid timestamp field infos for %s: %s, %s", indexMetadataList[0].IndexConfig.IndexID, refTimestampFieldName, refTimestampOutputFormat)
	}

It seems that the cause is that indexMetadataList[0] is being passed as an argument to FindTimestampFieldInfos.

[fmassot]

@earlbread @rachel-mj-park will be fixed by #154

[earlbread]

@fmassot Thank you!

[fmassot]

@earlbread: if you want to test, pre release is here https://github.com/quickwit-oss/quickwit-datasource/releases/tag/v0.5-beta.2