Skip to content

Commit 039e628

Browse files
committed
test: add test adjust rules
1 parent 7d7fd40 commit 039e628

3 files changed

Lines changed: 7 additions & 1 deletion

File tree

tests/data/trie_dialect_pack/dictionary/rules/adjust_rules.tsv renamed to tests/data/trie_dialect_pack/adjustments/rules/adjust_rules.tsv

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,4 @@
1313
["ལ་ལ་"] ["ལ་ལ་"] 2 = [pos="PART"]
1414
["ལ་ལ་"] ["ལ་ལ་"] 1-2 :: [pos="PART"] [pos="PART"]
1515
["ལ་"] ["ལ་"] ["ལ་ལ་"] 3-2 :: [pos="PART"] [pos="PART"]
16-
["ལ་"] ["ལ་"] ["ལ་"] ["ལ་"] 2 + [pos="PART"]
16+
["ལ་"] ["ལ་"] ["ལ་"] ["ལ་"] 2 + [pos="PART"]

tests/data/trie_dialect_pack/adjustments/rules/test.tsv

Whitespace-only changes.

tests/modifytokens/test_matchers.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,12 @@ def test_match_replace(tokens):
142142
def test_adjust_tokens(wt):
143143
string = "ལ་ལ་ལ་ལ་ལ་བ་ཡོད།"
144144
token_list = wt.tokenize(string, split_affixes=False)
145+
146+
# add test adjust rule to adjustments rules
147+
wt.config.adjustments["rules"].append(
148+
Path("./tests/data/trie_dialect_pack/adjustments/rules/adjust_rules.tsv")
149+
)
150+
145151
at = AdjustTokens(
146152
main=wt.config.dictionary["rules"], custom=wt.config.adjustments["rules"]
147153
)

0 commit comments

Comments
 (0)