open telemetry agent problems with Hostname, Timestamp add on attributes list

[plrrag]

Hi All,

I'm trying to extract and past "hostname" and "timestamp" into attributeslist, but both settings don't work, they return empty
This is the complete config.yaml.
The agent version is Otelcontrib 0.72 and it is running on windows system

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
        
  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
          system.cpu.time:
            enabled: false
      disk:
      #load:
      filesystem:
      memory:
      network:
      #paging:
      #process:




processors:
  batch:
    timeout: 1s

    
  resource:
    attributes:
      - key: hostname
        value: "LAB-DB"
        action: insert
      - key: ipaddress
        value: "127.0.0.0"
        #from_context: host.name  it doesn't work
        action: insert
      - key: servertype
        value: "Database Lab"
        action: insert
      - key: isactive
        value: "1"
        action: insert
      #- key: timeStamp
        #value: ${event.time} it doesn't work
        #action: upsert
      - key: timestamp
        value: "${timestamp}" it doesn't work
        action: upsert
        

exporters:
  otlp:
    endpoint: 0.0.0.0:4320
    tls:
      insecure: true
  
  logging:
    verbosity: detailed
    sampling_initial: 5
    sampling_thereafter: 5




service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch,resource]
      exporters: [logging,otlp]
      
    metrics:
      receivers: [otlp,hostmetrics]
      processors: [batch,resource]
      exporters: [logging,otlp]

Does anyone know how to fix the problem?

Many thanks

[atoulme]

You should not use the attributes processor for this. The transformprocessor is probably what you're after.

[plrrag]

could you send me the remodified yaml config?

receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318

hostmetrics:
collection_interval: 60s
scrapers:
cpu:
metrics:
system.cpu.utilization:
enabled: true
system.cpu.time:
enabled: false
disk:
#load:
filesystem:
memory:
network:
#paging:
#process:

processors:
resourcedetection:
detectors: [env,system]

attributes:
actions:
- key: "timestamp_unix"
value: "${timestamp_unix}"
action: upsert
- key: hostNAME
value: "nodeIp"
action: upsert
- key: test
#from_attribute: host.name
value: "HOST.NAME"
action: upsert

#transform:

transforms:

- action: insert

key: timestamp_unix

value: ${timestamp_unix}

format: int64

#attributes:

actions:

- key: "timestamp_unix"

#value: "${time}"

#value: ${event.time}

#value: "${timestamp_unix}"

value: '{{ time.Now().Unix() }}'

action: upsert

batch:
timeout: 1s

#resource:

attributes:

- key: hostname

value: "DEBUG-HOSTNAME"

action: upsert

#cumulativetodelta:

#resource:

attributes:

- key: hostname

value: "LAB-DB"

action: upsert

- key: timestamp

value: "${timestamp}"

action: upsert

resource:
attributes:
- key: hostname
value: "LAB-DB"
action: insert

- key: ipaddress

value: "x.y.z.k"

#from_context: host.name

action: insert

- key: servertype

value: "Database Lab"

action: insert

- key: isactive

value: "1"

action: insert

#- key: timeStamp

#value: ${event.time}

#action: upsert

- key: timestamp

value: "${timestamp}"

action: upsert

exporters:
otlp:
endpoint: x.y.z.k:4320
tls:
insecure: true

logging:
verbosity: detailed
sampling_initial: 5
sampling_thereafter: 5

service:
pipelines:
traces:
receivers: [otlp]
processors: [batch,resource]
#processors: [batch]
exporters: [logging,otlp]

metrics:
  receivers: [otlp,hostmetrics]
  #receivers: [otlp,sqlserver]
  #processors: [batch,resource]
  processors: [batch,resourcedetection,attributes,resource]
  exporters: [logging,otlp]

how can I capture the timestamp value I see from the agent terminal and insert it into the attribute list?

NumberDataPoints open-telemetry/opentelemetry-collector#6
Data point attributes:
-> device: Str(L:)
-> mode: Str(rw)
-> mountpoint: Str(L:)
-> type: Str(NTFS)
-> state: Str(used)
-> timestamp_unix: Str()
-> hostNAME: Str(nodeIp)
-> test: Str(HOST.NAME)
StartTimestamp: 2023-11-26 19:16:47 +0000 UTC
Timestamp: 2023-11-30 10:22:30.9672223 +0000 UTC
Value: 47042797568

many thanks

[github-actions]

Pinging code owners for processor/attributes: @boostchicken. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

Pinging code owners for processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[plrrag]

every attempt made so far has failed, I just want to expose the timestamp of the metric in the attribute list.

this is the metric exported to prometheus ,

Gateway_system_cpu_time{NRcpuNEW="",cpu="cpu8",host_name="serverPerformance",ipaddress="x.y.z.k",os_type="windows",servertype="Server Performance",state="interrupt",timestamp=""} 68.328125

the attributes NRcpuNEW and timestamp are always empty!!!!

a notation like this ${timestamp_unix} is correct or not?

processors:
batch:
timeout: 1s

resourcedetection:
detectors: [env, system]
#custom_timestamp:

attributes:
actions:
- key: "timestamp"
value: "${timestamp_unix}"
action: upsert

resource:
attributes:
- key: ipaddress
action: insert
value: "x.y.z.k"
- key: servertype
action: insert
value: "Server Performance"
- key: NRcpuNEW
value: "${host.cpu.count}"
action: upsert

[evan-bradley]

@plrrag I don't think you should need to set timestamps as attributes if the data points already have timestamps on them. If you're using the Prometheus exporter but aren't seeing them in Prometheus, perhaps there's an option on the exporter to set these. I don't have much experience with the Prometheus exporter, but does the send_timestamps option shown here work for you?

For setting other attributes, I'd suggest using the transform processor instead of the attributes processor to accomplish these sorts of modifications. With OTTL's datapoint context you should be able to access data in fields on the resource, metric, or data point to set on the data point's attributes.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.