Skip to content

Fix sonic-utilities submodule update failure due to ijson library#4256

Merged
saiarcot895 merged 10 commits intosonic-net:masterfrom
venkit-nexthop:master
Feb 10, 2026
Merged

Fix sonic-utilities submodule update failure due to ijson library#4256
saiarcot895 merged 10 commits intosonic-net:masterfrom
venkit-nexthop:master

Conversation

@venkit-nexthop
Copy link
Contributor

What I did

Forced the use of python backend for the ijson library

How I did it

- import ijson
+ import ijson.backends.python as ijson

How to verify it

Run the CI process of running sonic-mgmt test in a KVM environment.

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

This diff modifies the route_check.py to not
invoke "show" and rather invoke the vtysh cmd directly.
It then attempt to interpret one route at
a time in a paginated manner. This prevents a sudden transient memory
buildup. The zebra process already does the right thing and backs off
when the output socket buffers are full. There is probably scope to
improve that further
(Refer to
https://sonicfoundation.dev/2025-sonic-hackathon-most-impactful-award-spotlight-optimizing-output-buffer-memory-for-show-commands/)

Signed-off-by: Venkit Kasiviswanathan <[email protected]>
Signed-off-by: Venkit Kasiviswanathan <[email protected]>
Signed-off-by: Venkit Kasiviswanathan <[email protected]>
There was a failure when sonic-mgmt tests were run in a KVM. The failure appears to be due to the environment where it is running. It seems like on this environment ijson is not able to find the C-libraries required to set a default backend. Force a python backend to iterm.

Signed-off-by: Venkit Kasiviswanathan <[email protected]>
@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: Venkit Kasiviswanathan <[email protected]>
@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@venkit-nexthop
Copy link
Contributor Author

@saiarcot895 @deepak-singhal0408 @prsunny

I made a fix recommended by @saiarcot895. But code coverage is failing due to lack of coverage in the following lines:

try:
import ijson.backends.yajl2 as ijson
except ImportError: <--------------------------------------
import ijson.backends.python as ijson <------------------

I am not sure if it is worth adding coverage for this. I am not sure how to add coverage for imports.
Is there a way to override the coverage check?

Alternatively, I can simply add the python backend.

The alternative C backend has an issue that is best described by a
comment from saiarcot895 in
sonic-net#4205

Signed-off-by: Venkit Kasiviswanathan <[email protected]>
@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@venkit-nexthop
Copy link
Contributor Author

@saiarcot895 @deepak-singhal0408
I decided to simply use the python backend due to the test coverage issue I described earlier. The primary issue the earlier PR was solving was to save on memory and this change satisfies that objective equally. Although it was slightly slower than the C backend, the performance was not too bad.
Please approve this PR.

@saiarcot895 saiarcot895 merged commit 8103627 into sonic-net:master Feb 10, 2026
9 checks passed
@saiarcot895
Copy link
Contributor

@venkit-nexthop Looks like this method doesn't actually work:

2026 Feb 19 09:02:20.452788 vlab-03 WARNING monit[872]: 'routeCheck' status failed (1) -- Traceback (most recent call last):#012  File "/usr/local/bin/route_check.py", line 40, in <module>#012    import ijson.backends.python as ijson#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 36, in <module>#012    backend = _default_backend()#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 32, in _default_backend#012    return get_backend(backend)#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 24, in get_backend#012    return importlib.import_module('ijson.backends.' + backend)#012           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^#012  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module#012    return _bootstrap._gcd_import(name[level:], package, level)#012           ~~~

The _default_backend function in the __init__.py for the ijson module is still being run, I guess because it needs to be imported before ijson.backend.python can be imported.

I'm thinking setting the IJSON_BACKEND environment variable for monit is the next best method?

@venkit-nexthop
Copy link
Contributor Author

@venkit-nexthop Looks like this method doesn't actually work:

2026 Feb 19 09:02:20.452788 vlab-03 WARNING monit[872]: 'routeCheck' status failed (1) -- Traceback (most recent call last):#012  File "/usr/local/bin/route_check.py", line 40, in <module>#012    import ijson.backends.python as ijson#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 36, in <module>#012    backend = _default_backend()#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 32, in _default_backend#012    return get_backend(backend)#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 24, in get_backend#012    return importlib.import_module('ijson.backends.' + backend)#012           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^#012  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module#012    return _bootstrap._gcd_import(name[level:], package, level)#012           ~~~

The _default_backend function in the __init__.py for the ijson module is still being run, I guess because it needs to be imported before ijson.backend.python can be imported.

I'm thinking setting the IJSON_BACKEND environment variable for monit is the next best method?

I had an earlier version of this PR where I was not using ijson and instead had my own parsing logic.
@deepak-singhal0408 recommended using ijson instead. I am now wondering if I should go back to that code.
@deepak-singhal0408 @saiarcot895 please let me know what you think here.

@deepak-singhal0408
Copy link
Contributor

@venkit-nexthop Looks like this method doesn't actually work:

2026 Feb 19 09:02:20.452788 vlab-03 WARNING monit[872]: 'routeCheck' status failed (1) -- Traceback (most recent call last):#012  File "/usr/local/bin/route_check.py", line 40, in <module>#012    import ijson.backends.python as ijson#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 36, in <module>#012    backend = _default_backend()#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 32, in _default_backend#012    return get_backend(backend)#012  File "/usr/local/lib/python3.13/dist-packages/ijson/__init__.py", line 24, in get_backend#012    return importlib.import_module('ijson.backends.' + backend)#012           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^#012  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module#012    return _bootstrap._gcd_import(name[level:], package, level)#012           ~~~

The _default_backend function in the __init__.py for the ijson module is still being run, I guess because it needs to be imported before ijson.backend.python can be imported.
I'm thinking setting the IJSON_BACKEND environment variable for monit is the next best method?

I had an earlier version of this PR where I was not using ijson and instead had my own parsing logic. @deepak-singhal0408 recommended using ijson instead. I am now wondering if I should go back to that code. @deepak-singhal0408 @saiarcot895 please let me know what you think here.

Hi @venkit-nexthop, @saiarcot895
i suggest setting this env variable before importing ijson in the route_check script itself..
Lets not go back to custom parser code..

os.environ.setdefault('IJSON_BACKEND', 'python')
import ijson

this is the official recommendation from ijson documenation as well..

https://github.com/ICRAR/ijson#backends
relevant content:

Importing the top level library as import ijson uses the first available backend in the same order of the list above, and its name is recorded under ijson.backend. If the IJSON_BACKEND environment variable is set its value takes precedence and is used to select the default backend.

@venkit-nexthop
Copy link
Contributor Author

Sounds good @deepak-singhal0408 I will try setting env variable before importing in the route_check script.

DannyIsa pushed a commit to DannyIsa/sonic-utilities that referenced this pull request Mar 4, 2026
…nic-net#4256)

* Fix route_check.py to not hog a lot of memory

This diff modifies the route_check.py to not
invoke "show" and rather invoke the vtysh cmd directly.
It then attempt to interpret one route at
a time in a paginated manner. This prevents a sudden transient memory
buildup. The zebra process already does the right thing and backs off
when the output socket buffers are full. There is probably scope to
improve that further
(Refer to
https://sonicfoundation.dev/2025-sonic-hackathon-most-impactful-award-spotlight-optimizing-output-buffer-memory-for-show-commands/)

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Fix merge conflicts related test failure from upstream

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Fix precommit check failure

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Revert back to using the TIMEOUT from the earlier code.

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Fixed review comments from upstream

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Removed CHUNK_SIZE as it is not used any more

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Fix failure with ijson library

There was a failure when sonic-mgmt tests were run in a KVM. The failure appears to be due to the environment where it is running. It seems like on this environment ijson is not able to find the C-libraries required to set a default backend. Force a python backend to iterm.

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Incorporate feedback from Sai

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

* Pick the python backend for ijson

The alternative C backend has an issue that is best described by a
comment from saiarcot895 in
sonic-net#4205

Signed-off-by: Venkit Kasiviswanathan <[email protected]>

---------

Signed-off-by: Venkit Kasiviswanathan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants