fix bug in meter traversal logic

[vyzo]

See libp2p/go-libp2p-core#65

[Stebalien]

There's actually a trick to this:

newMeters := sw.meters[:0]
for _, m := range sw.meters {
    if m != nil {
        newMeters = append(newMeters, m)
    }
}
sw.meters = newMeters

[vyzo]

neat, but it probably generates the same code underneath :)

[vyzo]

do you want me to apply it? I am neutral either way and I usually prefer explicitness.

[vyzo]

actually, the trick might generate slower code because it has to do bounds checking.
Maybe the compiler is smart enough to elide it though, I wouldn't bet against it.

[Stebalien]

The trick will do less bounds checking, IIRC. But I'm fine with this as it is.

[Stebalien]

@vyzo could you write a test for this? We can probably do this by:

1. Starting 3 meters in sequence.
2. Making the first and last go idle.
3. Re-activate the last meter.

The total for that last meter should see a massive spike.

[vyzo]

well, not with the fix! But we can haz a test.

[vyzo]

This is harder to trigger than it looks; we need to be actively putting stuff while the sweeper is running so that we avoid the short-circuit from the optimistic load to generate a race condition.

[vyzo]

I managed to trigger it -- I made a small test that fails consistently before this patch, and passes after the patch.

[albrow]

@vyzo @Stebalien this does appear to fix the issue on our end. At the very least the issue is a lot less frequent. We're going to deploy this to all of our nodes and then keep an eye on things for a few days to know for sure.