Overflow error in NautilusKernel initialization due to negative timestamp calculation

[HaakonFlaaronning]

Bug Report

Confirmation

Before opening a bug report, please confirm:

• I’ve re-read the relevant sections of the documentation.
• I’ve searched existing issues and discussions to avoid duplicates.
• I’ve reviewed or skimmed the source code (or examples) to confirm the behavior is not by design.
• I’ve tested this issue using a recent development wheel (dev develop or a nightly) and can still reproduce it.

Expected Behavior

The NautilusKernel should initialize successfully in multiprocessing environments without raising overflow errors. The build time calculation should either be removed (as it is purely informational) or gracefully handle edge cases where system clock adjustments occur during initialization.

Note: I have only encountered the error when running backtests in a multiprocessing environment. I am not sure if one can encounter the error running in a single-process environment. In any case, seeing that the bug is caused by a calculation that is purely informational I believe it should be fixed. Parallel operations is a common use case for performance optimization, and although not natively supported in the project, such implementations should not be blocked by unnecessary timing code.

Actual Behavior

When running backtests in a multiprocessing environment (using for example multiprocessing.Pool), the NautilusKernel initialization fails with an OverflowError due to a negative timestamp calculation in the build time logging code.

The error occurs at line 542 in kernel.py:

When time.time_ns() returns a timestamp earlier than the original ts_build captured at the beginning of the initialization, the subtraction results in a negative value that cannot be converted to an unsigned integer by nanos_to_millis().

Steps to Reproduce the Problem

1. Set up a multiprocessing environment with multiple worker processes
2. Create multiple BacktestEngine instances simultaneously across different processes
3. Run the backtest operations in parallel
4. The error occurs intermittently likely when system clock synchronization or minor clock adjustments happen during the lengthy kernel initialization process

Code Snippets or Logs

Error traceback:

Traceback (most recent call last):
  File "/usr/lib/python3.12/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/haafla/privdev/nautilus-algo-trading/src/utils/backtest_utils.py", line 284, in run_single_low_level_backtest
    raise e
  File "/home/haafla/privdev/nautilus-algo-trading/src/utils/backtest_utils.py", line 191, in run_single_low_level_backtest
    engine = BacktestEngine(config=engine_config)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "nautilus_trader/backtest/engine.pyx", line 250, in nautilus_trader.backtest.engine.BacktestEngine.__init__
  File "/home/haafla/privdev/nautilus-algo-trading/libs/nautilus_trader/nautilus_trader/system/kernel.py", line 539, in __init__
    build_time_ms = nanos_to_millis(time.time_ns() - ts_build)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: can't convert negative int to unsigned

Proposed Solution

Solution 1: Remove the lines of code causing the error. The build time logging is purely informational and serves no functional purpose in the system. The simplest and solution is to remove these lines entirely:

build_time_ms = nanos_to_millis(time.time_ns() - ts_build)
self._log.info(f"Initialized in {build_time_ms}ms")

Solution 2: Gracefully handle timing edge cases. Something like:

build_time_diff = time.time_ns() - ts_build

# Handle negative time difference (clock adjustment during init)
if build_time_diff < 0:
    self._log.warning(
        f"Detected system clock adjustment during initialization "
        f"(time diff: {build_time_diff}ns). Using fallback timing."
    )
    build_time_ms = 0
else:
    build_time_ms = nanos_to_millis(build_time_diff)

self._log.info(f"Initialized in {build_time_ms}ms")

Specifications

• OS platform: Linux
• Python version: 3.12
• nautilus_trader version: develop branch commit 490a629 (latest)

[cjdsellers]

Hi @HaakonFlaaronning

Thanks for the detailed report.

The overflow on build time duration has been fixed in de32be2 by adding a guard similar to your option 2, so fallback is graceful instead of a crash.

Please note that multiprocessing is not officially supported. While this fix makes the code more robust to clock adjustments, you may still encounter other unforeseen issues when using multiprocessing.