Skip to content

[Python SDK] graceful shutdown does not enforce timeout-based task cancellation #357

@santoshkumarradha

Description

@santoshkumarradha

Summary

Even when an in-flight task survives past the configured timeout_seconds, agentfield.agent_server.AgentServer._graceful_shutdown does not force-cancel it. The process exits while the task is still running, leaving an orphaned coroutine in the event loop. This is the second half of the graceful-shutdown gap (the first being task tracking — see #356).

Where

  • File: sdk/python/agentfield/agent_server.py
  • Function: AgentServer._graceful_shutdown
  • Discovered by: sdk/python/tests/test_agent_graceful_shutdown.py::test_graceful_shutdown_force_cancels_tasks_after_timeout (skipped with pytest.skip("source bug: graceful shutdown does not enforce timeout-based task cancellation"))

Reproduction

agent = make_shutdown_agent()
server = AgentServer(agent)

task = asyncio.create_task(asyncio.sleep(60))   # task that will outlive the deadline
await server._graceful_shutdown(timeout_seconds=0)

# Expected: task.cancelled() is True (the deadline forced a cancel)
# Actual:   task is still running — no cancel issued

Expected behavior

_graceful_shutdown(timeout_seconds=N) should:

  1. Wait up to N seconds for in-flight tasks to drain (depends on [Python SDK] graceful shutdown does not track or cancel in-flight tasks #356)
  2. After the deadline expires, call task.cancel() on every still-running tracked task
  3. await asyncio.gather(*tasks, return_exceptions=True) to let cancellation propagate
  4. Only then proceed to exit

A timeout_seconds=0 value should mean "cancel immediately", not "wait forever".

Acceptance criteria

  • After timeout_seconds elapses, all still-running tracked tasks receive cancel()
  • Cancellation is awaited so the cancelled-state is observable on each task
  • timeout_seconds=0 cancels immediately
  • The skipped test in test_agent_graceful_shutdown.py::test_graceful_shutdown_force_cancels_tasks_after_timeout is unskipped and passes

Related

Discovered via

PR #352 (test coverage improvements). One of 5 source bugs surfaced while writing failure-mode tests for the Python SDK.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-friendlyWell-documented task suitable for AI-assisted developmentbugSomething isn't workinghelp wantedExtra attention is neededsdk:pythonPython SDK relatedtestsUnit test improvements and coverage

    Type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions