APScheduler (v3.10.3) + BlockingScheduler + Daemon Thread: Jobs Terminate Unexpectedly on Overlap

[harshalizode]

Hi all,

I'm running APScheduler v3.10.3 using a BlockingScheduler, and I've run into a strange issue when executing the scheduler inside a daemon thread managed by a subprocess. Here's a breakdown of my setup and the problem:

🔧 Architecture Overview:

• A Python script launches a multiprocessing subprocess, which runs a function (run_subprocess).
• Inside that function, I start the BlockingScheduler in a daemon thread.
• This subprocess (with the daemon thread) is terminated and restarted every hour, to ensure jobs don’t run indefinitely.
• The job functions are synchronous and shared across jobs.

🧩 Problem:

When multiple jobs (scheduled to run at the same second or with overlap) are triggered:

• Some jobs terminate silently before completing their execution.
• If Job A is still running when Job B starts (calling the same sync function), Job A is terminated, and Job B begins, without any error or traceback.
• It appears that job execution is not isolated, even though max_instances is set to 20.

📜 Code Summary:

def run_subprocess():
    # Create a daemon thread that runs the scheduler
    daemon_thread = threading.Thread(target=start_schedulers)
    daemon_thread.daemon = True
    daemon_thread.start()
    daemon_thread.join()

if __name__ == "__main__":
    while True:
        try:
            subprocess_process = multiprocessing.Process(
                target=run_subprocess
            )
            subprocess_process.start()
            
            with open("scheduler.pid", "w", encoding="utf-8") as f:
                f.write(str(subprocess_process.pid))
                
            subprocess_process.join()
        except Exception as e:
            print(f"Exception in main process: {e}")

And the scheduler configuration:

scheduler = BlockingScheduler({
    "apscheduler.job_defaults.coalesce": "true",
    "apscheduler.job_defaults.max_instances": "20",
    "apscheduler.timezone": "UTC",
})

All jobs use the same synchronous function and are scheduled using cron triggers.

❓ Questions:

1. Is running BlockingScheduler inside a daemon thread within a subprocess a supported or safe pattern?
2. Why do jobs get terminated or preempted when overlapping, despite max_instances=20?
3. Would switching to BackgroundScheduler or explicitly configuring ThreadPoolExecutor help preserve job isolation?
4. Is this termination behavior related to Python threads or APScheduler internals?
5. Could the function being reused across jobs cause any side effects in thread-based execution?

Any guidance on how to prevent premature job termination and ensure stable concurrent job execution would be much appreciated.

Thanks in advance!

[agronholm]

Daemon threads in general are not safe. What does start_schedulers() do? This implies you're starting multiple schedulers. Why?

[harshalizode]

Yeah, we are providing the signal to terminate the scheduler. We have the flag into the Redis database to know when we have to terminate the scheduler gracefully. For this we have the decorator that will check all this stuff.

[harshalizode]

Is multiprocessing necessary to trigger the job isolation problem?

It is not compulsory, we can use the threading as well as follows:

threading.Thread(
        target=start_schedulers

[agronholm]

Do you need the thread there for some reason?

[agronholm]

Where are you setting up the signal handler that gracefully stops the scheduler?

[harshalizode]

Where are you setting up the signal handler that gracefully stops the scheduler?

Before starting the task function execution, we are checking the flag to restart the scheduler gracefully. To maintain the flag we are using the Redis DB.

The flag was set with certain conditions like the timeout of thread etc.

[harshalizode]

Any guidance on how to prevent premature job termination and ensure stable concurrent job execution?
Thanks in advance !

[agronholm]

Jobs can only be forcibly terminated if the process itself is killed. This is where the signal handlers would come into play.

[harshalizode]

Okay, I will check it once in signal handler.