Skip to content

Commit 57884eb

Browse files
authored
[MINOR:TYPO] Fix docstrings (#1653)
* [MINOR:TYPO] Update pre_tokenizers.rs * [MINOR:TYPO] Update __init__.pyi
1 parent 5e223ce commit 57884eb

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

bindings/python/py_src/tokenizers/pre_tokenizers/__init__.pyi

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -421,7 +421,7 @@ class Split(PreTokenizer):
421421
422422
Args:
423423
pattern (:obj:`str` or :class:`~tokenizers.Regex`):
424-
A pattern used to split the string. Usually a string or a a regex built with `tokenizers.Regex`.
424+
A pattern used to split the string. Usually a string or a regex built with `tokenizers.Regex`.
425425
If you want to use a regex pattern, it has to be wrapped around a `tokenizer.Regex`,
426426
otherwise we consider is as a string pattern. For example `pattern="|"`
427427
means you want to split on `|` (imagine a csv file for example), while

bindings/python/src/pre_tokenizers.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -334,7 +334,7 @@ impl PyWhitespaceSplit {
334334
///
335335
/// Args:
336336
/// pattern (:obj:`str` or :class:`~tokenizers.Regex`):
337-
/// A pattern used to split the string. Usually a string or a a regex built with `tokenizers.Regex`.
337+
/// A pattern used to split the string. Usually a string or a regex built with `tokenizers.Regex`.
338338
/// If you want to use a regex pattern, it has to be wrapped around a `tokenizer.Regex`,
339339
/// otherwise we consider is as a string pattern. For example `pattern="|"`
340340
/// means you want to split on `|` (imagine a csv file for example), while

0 commit comments

Comments
 (0)