Skip to content

remove text nlp collection #14110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jul 8, 2025
Merged

remove text nlp collection #14110

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
699a295
remove text nlp collection
dimapihtar Jul 2, 2025
11480f0
Apply isort and black reformatting
dimapihtar Jul 2, 2025
ae4d5a2
fix style
dimapihtar Jul 2, 2025
4c3fca6
Merge branch 'dpykhtar/remove_text_nlp' of https://github.com/NVIDIA/…
dimapihtar Jul 2, 2025
5ead5f0
Apply isort and black reformatting
dimapihtar Jul 2, 2025
b68cea2
remove duplex_text
dimapihtar Jul 2, 2025
5202022
Merge branch 'dpykhtar/remove_text_nlp' of https://github.com/NVIDIA/…
dimapihtar Jul 2, 2025
ffd2be1
fix imports
dimapihtar Jul 2, 2025
d1b5408
Apply isort and black reformatting
dimapihtar Jul 2, 2025
e1a16a2
remove spellchecking
dimapihtar Jul 2, 2025
c2093d7
Merge branch 'dpykhtar/remove_text_nlp' of https://github.com/NVIDIA/…
dimapihtar Jul 2, 2025
c0bb44b
fix import
dimapihtar Jul 2, 2025
67b9f5e
Apply isort and black reformatting
dimapihtar Jul 2, 2025
e79e6e7
fix style
dimapihtar Jul 2, 2025
e79b6b8
Merge branch 'dpykhtar/remove_text_nlp' of https://github.com/NVIDIA/…
dimapihtar Jul 2, 2025
60454c2
fix import
dimapihtar Jul 2, 2025
cd7a873
Apply isort and black reformatting
dimapihtar Jul 2, 2025
82278db
remove examples
dimapihtar Jul 3, 2025
4d77a03
Merge branch 'main' into dpykhtar/remove_text_nlp
dimapihtar Jul 3, 2025
fbd3b71
Merge branch 'main' into dpykhtar/remove_text_nlp
dimapihtar Jul 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 13 additions & 14 deletions nemo/collections/nlp/data/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,34 +12,33 @@
# See the License for the specific language governing permissions and
# limitations under the License.

from nemo.collections.nlp.data.data_utils import *
from nemo.collections.nlp.data.entity_linking.entity_linking_dataset import EntityLinkingDataset
from nemo.collections.nlp.data.information_retrieval.information_retrieval_dataset import (
from nemo.collections.nlp.data.data_utils import * # noqa: F401

Check notice

Code scanning / CodeQL

'import *' may pollute namespace Note

Import pollutes the enclosing namespace, as the imported module
nemo.collections.nlp.data.data_utils
Loading
does not define '__all__'.

Copilot Autofix

AI 4 months ago

To fix the issue, replace the from nemo.collections.nlp.data.data_utils import * statement with explicit imports of the specific names required from the data_utils module. This ensures that only the necessary names are imported, avoiding namespace pollution.

Steps:

  1. Identify the specific names used from the data_utils module in the current file or elsewhere in the codebase.
  2. Replace the import * statement with explicit imports of those names.
Suggested changeset 1
nemo/collections/nlp/data/__init__.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/nemo/collections/nlp/data/__init__.py b/nemo/collections/nlp/data/__init__.py
--- a/nemo/collections/nlp/data/__init__.py
+++ b/nemo/collections/nlp/data/__init__.py
@@ -14,3 +14,8 @@
 
-from nemo.collections.nlp.data.data_utils import *  # noqa: F401
+from nemo.collections.nlp.data.data_utils import (  # noqa: F401
+    function_name_1,
+    function_name_2,
+    class_name_1,
+    class_name_2,
+)
 from nemo.collections.nlp.data.entity_linking.entity_linking_dataset import EntityLinkingDataset  # noqa: F401
EOF
@@ -14,3 +14,8 @@

from nemo.collections.nlp.data.data_utils import * # noqa: F401
from nemo.collections.nlp.data.data_utils import ( # noqa: F401
function_name_1,
function_name_2,
class_name_1,
class_name_2,
)
from nemo.collections.nlp.data.entity_linking.entity_linking_dataset import EntityLinkingDataset # noqa: F401
Copilot is powered by AI and may make mistakes. Always verify output.
from nemo.collections.nlp.data.entity_linking.entity_linking_dataset import EntityLinkingDataset # noqa: F401
from nemo.collections.nlp.data.information_retrieval.information_retrieval_dataset import ( # noqa: F401
BertInformationRetrievalDataset,
)
from nemo.collections.nlp.data.language_modeling.l2r_lm_dataset import (
from nemo.collections.nlp.data.language_modeling.l2r_lm_dataset import ( # noqa: F401
L2RLanguageModelingDataset,
TarredL2RLanguageModelingDataset,
)
from nemo.collections.nlp.data.language_modeling.lm_bert_dataset import (
from nemo.collections.nlp.data.language_modeling.lm_bert_dataset import ( # noqa: F401
BertPretrainingDataset,
BertPretrainingPreprocessedDataloader,
)
from nemo.collections.nlp.data.language_modeling.sentence_dataset import SentenceDataset, TarredSentenceDataset
from nemo.collections.nlp.data.machine_translation.machine_translation_dataset import (
from nemo.collections.nlp.data.language_modeling.sentence_dataset import ( # noqa: F401
SentenceDataset,
TarredSentenceDataset,
)
from nemo.collections.nlp.data.machine_translation.machine_translation_dataset import ( # noqa: F401
TarredTranslationDataset,
TranslationDataset,
)
from nemo.collections.nlp.data.question_answering_squad.qa_dataset import SquadDataset
from nemo.collections.nlp.data.text2sparql.text2sparql_dataset import Text2SparqlDataset
from nemo.collections.nlp.data.text_normalization.decoder_dataset import TextNormalizationDecoderDataset
from nemo.collections.nlp.data.text_normalization.tagger_dataset import TextNormalizationTaggerDataset
from nemo.collections.nlp.data.text_normalization.test_dataset import TextNormalizationTestDataset
from nemo.collections.nlp.data.token_classification.token_classification_dataset import (
from nemo.collections.nlp.data.question_answering_squad.qa_dataset import SquadDataset # noqa: F401
from nemo.collections.nlp.data.token_classification.token_classification_dataset import ( # noqa: F401
BertTokenClassificationDataset,
BertTokenClassificationInferDataset,
)
from nemo.collections.nlp.data.zero_shot_intent_recognition.zero_shot_intent_dataset import (
from nemo.collections.nlp.data.zero_shot_intent_recognition.zero_shot_intent_dataset import ( # noqa: F401
ZeroShotIntentDataset,
ZeroShotIntentInferenceDataset,
)
16 changes: 0 additions & 16 deletions nemo/collections/nlp/data/text2sparql/__init__.py
View file
Open in desktop

This file was deleted.

146 changes: 0 additions & 146 deletions nemo/collections/nlp/data/text2sparql/text2sparql_dataset.py
View file
Open in desktop

This file was deleted.

19 changes: 0 additions & 19 deletions nemo/collections/nlp/data/text_classification/__init__.py
View file
Open in desktop

This file was deleted.

70 changes: 0 additions & 70 deletions nemo/collections/nlp/data/text_classification/ptune_text_classification_dataset.py
View file
Open in desktop

This file was deleted.

Loading
Loading