Skip to content

Execute with subprocess list#15165

Merged
ko3n1g merged 2 commits intomainfrom
fix_shell_execution_ais
Dec 10, 2025
Merged

Execute with subprocess list#15165
ko3n1g merged 2 commits intomainfrom
fix_shell_execution_ais

Conversation

@nithinraok
Copy link
Member

Important

The Update branch button must only be pressed in very rare occassions.
An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.

What does this PR do ?

Fixes security related bug

Collection: ASR

Changelog

  • Update to disable shell execution

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this 

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
) as proc:
stream = proc.stdout
if stream.peek(1):
done = True

Check notice

Code scanning / CodeQL

Unused local variable Note

Variable done is not used.

Copilot Autofix

AI 3 months ago

To fix the problem, remove the unused variable done from the function open_datastore_object_with_binary. Its only assignment is immediately followed by a return from the function, and the subsequent branch based on if not done: can simply be left as is after the loop, since done will always be False when execution reaches it. Specifically, eliminate the line done = False, and the inner assignment to done = True, along with simplifying the code to avoid using this variable at all.

Changes needed:

  • Remove initialization of done = False before the loop.
  • Remove assignment done = True inside the loop.
  • Remove if not done:—since that branch is always taken if the loop does not return, replace it with a code block that follows the loop naturally.

No imports, methods, or definitions are required.

Suggested changeset 1
nemo/utils/data_utils.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/nemo/utils/data_utils.py b/nemo/utils/data_utils.py
--- a/nemo/utils/data_utils.py
+++ b/nemo/utils/data_utils.py
@@ -211,7 +211,6 @@
 
         cmd = [binary, 'get', path, '-']
 
-        done = False
 
         for _ in range(num_retries):
             with subprocess.Popen(
@@ -219,18 +218,17 @@
             ) as proc:
                 stream = proc.stdout
                 if stream.peek(1):
-                    done = True
                     return stream
 
-        if not done:
-            with subprocess.Popen(
-                cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=False
-            ) as proc:
-                error = proc.stderr.read().decode("utf-8", errors="ignore").strip()
-            raise ValueError(
-                f"{path} couldn't be opened with AIS binary "
-                f"after {num_retries} attempts because of the following exception: {error}"
-            )
+        # If all attempts fail, raise error
+        with subprocess.Popen(
+            cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=False
+        ) as proc:
+            error = proc.stderr.read().decode("utf-8", errors="ignore").strip()
+        raise ValueError(
+            f"{path} couldn't be opened with AIS binary "
+            f"after {num_retries} attempts because of the following exception: {error}"
+        )
     return None
 
 
EOF
@@ -211,7 +211,6 @@

cmd = [binary, 'get', path, '-']

done = False

for _ in range(num_retries):
with subprocess.Popen(
@@ -219,18 +218,17 @@
) as proc:
stream = proc.stdout
if stream.peek(1):
done = True
return stream

if not done:
with subprocess.Popen(
cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=False
) as proc:
error = proc.stderr.read().decode("utf-8", errors="ignore").strip()
raise ValueError(
f"{path} couldn't be opened with AIS binary "
f"after {num_retries} attempts because of the following exception: {error}"
)
# If all attempts fail, raise error
with subprocess.Popen(
cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=False
) as proc:
error = proc.stderr.read().decode("utf-8", errors="ignore").strip()
raise ValueError(
f"{path} couldn't be opened with AIS binary "
f"after {num_retries} attempts because of the following exception: {error}"
)
return None


Copilot is powered by AI and may make mistakes. Always verify output.
@github-actions
Copy link
Contributor

[🤖]: Hi @nithinraok 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully.

So it might be time to merge this PR or get some approvals.

//cc @chtruong814 @ko3n1g @pablo-garay @thomasdhc

@nithinraok nithinraok requested a review from artbataev December 10, 2025 02:15
Copy link
Contributor

@ko3n1g ko3n1g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nithinraok offline asked for a fast-merge

@ko3n1g ko3n1g merged commit 5e806c4 into main Dec 10, 2025
286 of 293 checks passed
@ko3n1g ko3n1g deleted the fix_shell_execution_ais branch December 10, 2025 03:03
chtruong814 pushed a commit that referenced this pull request Jan 4, 2026
* Execute with subprocess list

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* fix pylint issues

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

---------

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
chtruong814 pushed a commit that referenced this pull request Jan 6, 2026
* Execute with subprocess list

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* fix pylint issues

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

---------

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
chtruong814 added a commit that referenced this pull request Jan 6, 2026
* Execute with subprocess list



* fix pylint issues



---------

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Rudra-Tiwari-codes added a commit to Rudra-Tiwari-codes/NeMo that referenced this pull request Jan 23, 2026
Security Fix: Convert amr-nb, ogg, and g711 codecs in TranscodePerturbation from shell=True to secure subprocess patterns using explicit argument lists.

- Use subprocess.Popen with pipes for amr-nb and ogg codecs

- Use subprocess.run with list arguments for g711 codec

- Remove unused 'Any' import to fix linting errors

Related to NVIDIA-NeMo#15165

Signed-off-by: Rudra Tiwari <tiwarirudra2006@gmail.com>
Rudra-Tiwari-codes added a commit to Rudra-Tiwari-codes/NeMo that referenced this pull request Jan 23, 2026
Security: Convert amr-nb, ogg, and g711 codecs in TranscodePerturbation
from shell=True to secure subprocess patterns using list arguments.

Changes:
- Replace subprocess.check_output with shell=True to subprocess.Popen
  and subprocess.run with explicit argument lists for amr-nb and ogg codecs
- Replace subprocess.check_output with shell=True to subprocess.run
  with list arguments for g711 codec
- Use subprocess.DEVNULL for stderr to suppress unnecessary output
- Properly close stdout pipes and wait for encoder process completion
  to prevent potential resource leaks
- Remove unused 'Any' import from typing module

This follows the precedent set in PR NVIDIA-NeMo#15165 which fixed similar
security vulnerabilities in data_utils.py.

Signed-off-by: Rudra-Tiwari-codes <rudratiwari2006@gmail.com>
Signed-off-by: Rudra Tiwari <tiwarirudra2006@gmail.com>
AkCodes23 pushed a commit to AkCodes23/NeMo that referenced this pull request Jan 28, 2026
* Execute with subprocess list

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* fix pylint issues

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

---------

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: Akhil Varanasi <akhilvaranasi23@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants