polygon/sync: message listener to preserve peer events ordering (#10032)

Observed the following issue in a long running Astrid sync on
bor-mainnet:
```
[DBUG] [04-17|14:25:43.504] [p2p.peerEventObserver] received new peer event id=Disconnect peerId=51935aa1eeabdb73b70d36c7d5953a3bfdf5c84e88241c44a7d16d508b281d397bdd8504c934bfb45af146b86eb5899ccea85e590774f9823d056a424080b763
[DBUG] [04-17|14:25:43.504] [p2p.peerEventObserver] received new peer event id=Connect peerId=51935aa1eeabdb73b70d36c7d5953a3bfdf5c84e88241c44a7d16d508b281d397bdd8504c934bfb45af146b86eb5899ccea85e590774f9823d056a424080b763
```

Note the timestamps are the same on the millisecond level, however the
disconnect was processed before the connect which is wrong (connect
should always be first).

This then got the `PeerTracker` in a bad state - it kept on returning
peer
`51935aa1eeabdb73b70d36c7d5953a3bfdf5c84e88241c44a7d16d508b281d397bdd8504c934bfb45af146b86eb5899ccea85e590774f9823d056a424080b763`
as a valid peer to download from, which caused repeated `peer not found`
errors when sending messages to it.

Fix is to have the message listener wait for all observers to finish
processing peer event 1 before proceeding to notifying them about peer
event 2.

Author: taratorio

polygon/p2p/message_listener.go

@@ -170,7 +170,18 @@ func (ml *messageListener) notifyPeerEventObservers(peerEvent *sentry.PeerEvent)
 	ml.observersMu.Lock()
 	defer ml.observersMu.Unlock()
 
-	notifyObservers(ml.peerEventObservers, peerEvent)
+	// wait on all observers to finish processing the peer event before notifying them
+	// with subsequent events in order to preserve the ordering of the sentry messages
+	var wg sync.WaitGroup
+	for _, observer := range ml.peerEventObservers {
+		wg.Add(1)
+		go func(observer MessageObserver[*sentry.PeerEvent]) {
+			defer wg.Done()
+			observer(peerEvent)
+		}(observer)
+	}
+
+	wg.Wait()
 	return nil
 }