Avoid race condition for subscribers

If several monitors (each is a subscriber) are attached to an everest
server, the server could crash with a RuntimeError when tearing down;

Traceback (most recent call last):
  File "/data/projects/ert/src/ert/dark_storage/endpoints/experiment_server.py", line 341, in run
    for sub in shared_data.subscribers.values():
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: dictionary changed size during iteration

This is because there is an await in the loop execution body, which
allows the async loop to add a new subscriber while we are iterating.

By expanding the values() iterator to a fixed list before looping, we
avoid the possibility of a race condition (but we are leaving behind the
possible late added subscriber)

Repeated testing with multiple monitors with this patch has not
revealed any further issues.

Author: berland

src/ert/dark_storage/endpoints/experiment_server.py

@@ -338,7 +338,7 @@ async def run(self) -> None:
 
                 if isinstance(item, EndEvent):
                     # Wait for subscribers to receive final events
-                    for sub in shared_data.subscribers.values():
+                    for sub in list(shared_data.subscribers.values()):
                         await sub.is_done()
                     break
             await simulation_future