Skip to content

Metrics aren't being sent when using redis as store #101

@BRMatt

Description

@BRMatt

I upgraded our l2met instance this evening and thought it'd be a good opportunity to setup metchan metrics. As it seems the store related metrics are only available when using redis, I switched our instance from an in-memory store to using a redistogo nano instance. All metchan metrics were then being reported, but l2met stopped sending our normal app metrics:

2013ear-08-14_selection

The point at which the store.throughput crashes is where I disabled redis (I'd stepped away for a while and didn't notice. Luckily we're not using these metrics for alerts! 😄), and the flat line in the outlet.throughput and outlet.latency are where it's not reporting anything, but the averages of averages config for the instrument is filling in the blanks.

During the outage we saw a lot of the following in the logs:

2013-08-13T22:25:20.597416+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.601086+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.615588+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.617556+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.605537+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.612221+00:00 app[web.1]: error=redis-time-not-available
2013-08-13T22:25:20.619599+00:00 app[web.1]: error=redis-time-not-available

Switching back to in memory "fixed" the issue, but it seems odd that this happened.

I'll try running an earlier version of l2met tomorrow, pre adf9daa, but hopefully this information is somewhat useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions