-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Description
Is your feature request related to a problem? Please describe.
Metrics sent to the Splunk HEC in some cases have a timestamp of 0 - Splunk accepts these timestamps without modification and consequently it is impossible to analyse these metrics meaningfully.
Describe the solution you'd like
In the case of metrics that have a 'zero' timestamp (i.e. beginning of Unix Epoch time), omit the timestamp field so that Splunk automatically sets it at ingestion time.
Describe alternatives you've considered
Splunk has options available to override the timestamps (https://docs.splunk.com/Documentation/Splunk/8.0.6/Data/Configuretimestamprecognition) in events, but despite repeated attempts these options have not worked. I think it's something special related to the way the HEC itself works.
Additional context
An example SignalFx Smart Agent monitor that publishes metrics without timestamps is https://docs.signalfx.com/en/latest/integrations/agent/monitors/telegraf-win_services.html, which is how I uncovered this behaviour.