The latest dev of agent_loop involved inspecting cluster resources, and placing workers in a custom, PACK-like strategy. However, there is a potential problem when inspecting resources:
def _init_agent_loop_workers(self):
self.agent_loop_workers = []
...
node_ids = [node["NodeID"] for node in ray.nodes() if node["Alive"] and node["Resources"]["CPU"] == 0]
In Ray clusters, CPU as a resource key is optional. It is a very common practice to set Ray head node to num_cpus=0 and Ray will not include "CPU" key in its resource dict.