[Interactive Brokers] Incorrectly skips producing position reports for flat positions

[idobz]

Bug Report

Confirmation

Before opening a bug report, please confirm:

• [v ] I’ve re-read the relevant sections of the documentation.
• [v ] I’ve searched existing issues and discussions to avoid duplicates.
• [v ] I’ve reviewed or skimmed the source code (or examples) to confirm the behavior is not by design.
• [v ] I’ve tested this issue using a recent development wheel (dev develop or a nightly) and can still reproduce it.

Expected Behavior

When positions have been closed in the client but remain open in the cache (due to DB persistence), the expected behavior is for NT to identify this and remove the phantom positions

Actual Behavior

In practice, this case is not recognized by NT, and the phantom positions remain open in the cache/DB, causing downstream mayhem

Steps to Reproduce the Problem

The set up is a bit complex, and I have only thought it out for NETTING accounts:

1. the setup is a cache that is persisted to DB, and a pattern of running NT that includes node shutdowns and then restarts (which makes sense when running strategies for days and weeks)
2. run NT and have your strategy open a position
3. make sure that the DB is reflecting this open position
4. 'offline' (w/o NT running) close the position through direct interaction with the client

This is where the expected behavior comes in:

run NT again - NT is expected to recognize that the DB and the client are not aligned, and update the DB by closing the positions

Code Snippets or Logs

Specifications

• OS platform: Ubuntu
• Python version: 3.12.7
• nautilus_trader version: 1.221.0a20251002

[qtnx]

I encountered the same issue on Bybit exchange, and adding filter_unclaimed_external_orders=True seemed to fix it. However, keep in mind that this option may also filter out external orders.

My configuration:

exec_engine=LiveExecEngineConfig(
            reconciliation=True,
            reconciliation_lookback_mins=120,
            generate_missing_orders=True,
            filter_unclaimed_external_orders=True,
            snapshot_orders=False,
            snapshot_positions=False,
            open_check_interval_secs=5.0,
        ),

[aixohub]

I encountered the same issue on InteractiveBrokers exchange. After successfully buying one share of stock , The number of stocks obtained through self.cache.positions(instrument_id=instrument_id) has not changed

[idobz]

I encountered the same issue on Bybit exchange, and adding filter_unclaimed_external_orders=True seemed to fix it. However, keep in mind that this option may also filter out external orders.

My configuration:

exec_engine=LiveExecEngineConfig(
            reconciliation=True,
            reconciliation_lookback_mins=120,
            generate_missing_orders=True,
            filter_unclaimed_external_orders=True,
            snapshot_orders=False,
            snapshot_positions=False,
            open_check_interval_secs=5.0,
        ),

Thanks @qtnx , I actually use external_order_claims, to incorporate external orders into the reconciliation process. Without it, positions opened in a previous run and which still exist are not dealt with. In any case though, I checked to see if this setting has an effect in this case - but it doesn't seem to have any.

[idobz]

I encountered the same issue on InteractiveBrokers exchange. After successfully buying one share of stock , The number of stocks obtained through self.cache.positions(instrument_id=instrument_id) has not changed

Thanks @aixohub ,
This sounds like a grave issue, but I don't think that it is related. I'd suggest that you try to find a way to reproduce this so that it can be fixed as a separate bug.
One thing that I did see in the context of the IB client, is some confusion with respect to the venue being used. The NT engine and its various components seem to be using the IB_VENUE constant, but positions have the actual venue (in my case NYSE) recorded. So searching for positions with the venue filter (something like self.cache.positions(venue=IB_VENUE)) produces the wrong results.

[cjdsellers]

Hi @idobz

Thanks for the report.

This turns out to be a fairly simple spec oversight with the Interactive Brokers execution client where it doesn't produce a position report for flat positions. The live execution engine needs this confirmation downstream for correct reconciliation processing.

Also make sure you don't have any configs filtering position reports (usually needed if you have multiple nodes trading the same instrument).

Let me know if you continue to experience issues and we can reopen this.

[idobz]

Wow, you're fast man!
In any case, I actually tried to implement something similar and create a flat position report (only it took me several hours to figure out).
For me though the result was that multiple positions were opened per instrument as a result of this - despite the netting account.
In short I'll put it to the test once its built

[idobz]

Hi @cjdsellers ,

I tried the new code, in a situation where:

1. client has no open positions
2. DB reports of a short position for 1 instrument

Results:

1. there is some issue with venue concept in the context of IB. In line

   nautilus_trader/nautilus_trader/live/execution_engine.py

   Line 1261 in 705b327

   for position in self._cache.positions_open(venue):

   - venue is set to IB_VENUE, but open positions in the cache have venue set to Venue("NYSE"). So the whole reconciliation is skipped

2. manually setting venue to None and skipping this hurdle, reconciliation proceeds as such:

2025-10-05T07:17:42.710786520Z [INFO] IB_LIVE-001.ExecEngine: Reconciling ExecutionMassStatus for INTERACTIVE_BROKERS
2025-10-05T07:18:56.064630534Z [INFO] IB_LIVE-001.ExecEngine: Position(SHORT 5 CCL.NYSE, id=CCL.NYSE-CCL_spread-017) pending reconciliation
2025-10-05T07:19:00.798949666Z [INFO] IB_LIVE-001.ExecEngine: Awaiting 1 position reports for INTERACTIVE_BROKERS
2025-10-05T07:19:56.870073550Z [INFO] IB_LIVE-001.ExecEngine: Reconciling NET position for CCL.NYSE
2025-10-05T07:20:17.182578438Z [INFO] IB_LIVE-001.ExecEngine: report.signed_decimal_qty=Decimal('0')
2025-10-05T07:20:18.385346758Z [INFO] IB_LIVE-001.ExecEngine: position_signed_decimal_qty=Decimal('-5')
2025-10-05T07:20:28.296260035Z [INFO] IB_LIVE-001.ExecEngine: diff_quantity=Quantity(5)
2025-10-05T07:22:15.576638314Z [WARN] IB_LIVE-001.ExecEngine: Cannot calculate exact reconciliation price for CCL.NYSE: position report lacks average price information, using last quote fallback
2025-10-05T07:23:52.205983392Z [INFO] IB_LIVE-001.CCL_spread: <--[EVT] OrderAccepted(instrument_id=CCL.NYSE, client_order_id=O-c7aec962-ab4a-437c-9572-3307fd61452f, venue_order_id=1b4b1530-5699-4190-ad39-84a39afd6b06, account_id=INTERACTIVE_BROKERS-DUM955954, ts_event=1759648884507877755)
2025-10-05T07:24:33.436952462Z [INFO] IB_LIVE-001.ExecEngine: Generated inferred OrderFilled(instrument_id=CCL.NYSE, client_order_id=O-c7aec962-ab4a-437c-9572-3307fd61452f, venue_order_id=1b4b1530-5699-4190-ad39-84a39afd6b06, account_id=INTERACTIVE_BROKERS-DUM955954, trade_id=e9a0a831-1408-4ff3-bf19-0a36631cd57c, position_id=CCL.NYSE-EXTERNAL, order_side=BUY, order_type=LIMIT, last_qty=5, last_px=31.67 USD, commission=0.00 USD, liquidity_side=NO_LIQUIDITY_SIDE, ts_event=1759648884507877755)
2025-10-05T07:24:58.662906164Z [INFO] IB_LIVE-001.CCL_spread: <--[EVT] OrderFilled(instrument_id=CCL.NYSE, client_order_id=O-c7aec962-ab4a-437c-9572-3307fd61452f, venue_order_id=1b4b1530-5699-4190-ad39-84a39afd6b06, account_id=INTERACTIVE_BROKERS-DUM955954, trade_id=e9a0a831-1408-4ff3-bf19-0a36631cd57c, position_id=CCL.NYSE-CCL_spread-017, order_side=BUY, order_type=LIMIT, last_qty=5, last_px=31.67 USD, commission=0.00 USD, liquidity_side=NO_LIQUIDITY_SIDE, ts_event=1759648884507877755)
2025-10-05T07:24:58.663186575Z [INFO] IB_LIVE-001.CCL_spread: <--[EVT] PositionClosed(instrument_id=CCL.NYSE, position_id=CCL.NYSE-CCL_spread-017, account_id=INTERACTIVE_BROKERS-DUM955954, opening_order_id=CCL.NYSE, closing_order_id=O-c7aec962-ab4a-437c-9572-3307fd61452f, entry=SELL, side=FLAT, signed_qty=0.0, quantity=0, peak_qty=5, currency=USD, avg_px_open=31.67, avg_px_close=31.67, realized_return=0.00000, realized_pnl=0.00 USD, unrealized_pnl=0.00 USD, ts_opened=1759227773432756284, ts_last=1759648884507877755, ts_closed=1759648884507877755, duration_ns=421111075121471)
2025-10-05T07:24:58.663485024Z [INFO] IB_LIVE-001.ExecEngine: Reconciliation for CCL.NYSE succeeded
2025-10-05T07:24:58.663719245Z [INFO] IB_LIVE-001.ExecEngine: Reconciliation for INTERACTIVE_BROKERS succeeded

When the run is stopped (SIGINT - causing the required cleanup AFAIK) I get these lines:

2025-10-05T07:26:23.944322284Z [WARN] IB_LIVE-001.Cache: Residual MarketOrder(BUY 5 CCL.NYSE MARKET GTC, status=ACCEPTED, client_order_id=O-20250930-134612-001-017-2, venue_order_id=103, position_id=None, tags=None)

I think that this is good (?)

3. but, the DB still reports positions for the instrument: on top of the original sell there is now also a buy filled order. Technically I guess the position is flat - so I am not sure whether these filled orders should be in the DB under positions 'folder', or not