-
Notifications
You must be signed in to change notification settings - Fork 235
Support non-ASCII characters in function arguments #2584
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 35 commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
75778bf
Support non-ASCII characters
seisman e840c99
Support all ISOlatin1 characters
seisman 1b6d940
Support more ISOLatin1+ characters
seisman 1b88cd6
fix
seisman 311b976
Update pygmt/helpers/utils.py
seisman 672413b
Refactor to make it more readable
seisman 2dcc288
Need to remove single quote
seisman c9b8254
[ci skip] Use a better reference for ASCII table
seisman e2947fa
Support Symbols charset
seisman 1a84634
Support ZapfDingbats charset
seisman 636ace0
Refactor and add more doctests
seisman ffabaee
Fix a symbol which is incorrectly copied from PDF
seisman e1b43b2
Replace octal codes with non-ASCII character in two examples
seisman 7ea78da
Fix a typo in doctest
seisman 487f2d8
Merge branch 'main' into non-ascii-support
seisman 288486c
Fix some characters
seisman 97a223d
Fix symbol characters
seisman 4ff2e56
Add one more reference
seisman 388109f
Add two more references
seisman 2270d9d
Update ZapfDingbats charset
seisman 117b6e5
Make it clear that Symbol/ZapfDingbats are from Adobe
seisman 1798ccc
Update for ISOLatin1+ charset
seisman e4bedb9
Fix registered sign, copyright sign and trade mark sign
seisman 5ef84f9
Add more notes
seisman 490535c
Add a test for non-ascii support
seisman 418adc5
Update the dvc file
seisman 4b66b8a
Fix styling issue
seisman 37b0c6a
Add docstrings
seisman 3362b31
Update examples/gallery/embellishments/colorbar.py
seisman 695c59b
Merge branch 'main' into non-ascii-support
seisman bbc223b
Remove an unused pylint directive
seisman 35306bf
Fix a typo in doctest
seisman e56359e
Merge branch 'main' into non-ascii-support
seisman 633b2f9
Merge branch 'main' into non-ascii-support
seisman cdb4ab1
Merge branch 'main' into non-ascii-support
seisman 646465c
Update pygmt/helpers/utils.py
seisman 9ce0217
Merge branch 'main' into non-ascii-support
seisman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,6 +4,7 @@ | |
| import os | ||
| import pathlib | ||
| import shutil | ||
| import string | ||
| import subprocess | ||
| import sys | ||
| import time | ||
|
|
@@ -196,6 +197,119 @@ def data_kind(data=None, x=None, y=None, z=None, required_z=False, required_data | |
| return kind | ||
|
|
||
|
|
||
| def non_ascii_to_octal(argstr): | ||
| r""" | ||
| Translate non-ASCII characters to their corresponding octal codes. | ||
|
|
||
| Currently, only characters in the ISOLatin1+ charset and | ||
| Symbol/ZapfDingbats fonts are supported. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| argstr : str | ||
| The string to be translated. | ||
|
|
||
| Returns | ||
| ------- | ||
| str | ||
| The translated string. | ||
|
|
||
| Examples | ||
| -------- | ||
| >>> non_ascii_to_octal("•‰“”±°ÿ") | ||
| '\\31\\214\\216\\217\\261\\260\\377' | ||
| >>> non_ascii_to_octal("αζΔΩ∑π∇") | ||
| '@~\\141@~@~\\172@~@~\\104@~@~\\127@~@~\\345@~@~\\160@~@~\\321@~' | ||
| >>> non_ascii_to_octal("✁❞❡➾") | ||
| '@%34%\\41@%%@%34%\\176@%%@%34%\\241@%%@%34%\\376@%%' | ||
| >>> non_ascii_to_octal("ABC ±120° DEF α ♥") | ||
| 'ABC \\261120\\260 DEF @~\\141@~ @%34%\\252@%%' | ||
| """ | ||
| # Dictionary mapping non-ASCII characters to octal codes | ||
| mapping = {} | ||
|
|
||
| # Adobe Symbol charset | ||
| # References: | ||
| # 1. https://en.wikipedia.org/wiki/Symbol_(typeface) | ||
| # 2. https://unicode.org/Public/MAPPINGS/VENDORS/ADOBE/symbol.txt | ||
| # Notes: | ||
| # 1. \322 and \342 are "REGISTERED SIGN SERIF" and | ||
| # "REGISTERED SIGN SANS SERIF" respectively, but only "REGISTERED SIGN" | ||
| # is available in the unicode table. So both are mapped to | ||
| # "REGISTERED SIGN". \323, \343, \324 and \344 also have the same | ||
| # problem. | ||
| # 2. Characters for \140, \275, \276 are incorrect. | ||
| mapping.update( | ||
| { | ||
| c: "@~\\" + format(i, "o") + "@~" | ||
| for c, i in zip( | ||
| " !∀#∃%&∋()∗+,−./" # \04x-05x | ||
| + "0123456789:;<=>?" # \06x-07x | ||
| + "≅ΑΒΧΔΕΦΓΗΙϑΚΛΜΝΟ" # \10x-11x | ||
| + "ΠΘΡΣΤΥςΩΞΨΖ[∴]⊥_" # \12x-13x | ||
| + "αβχδεφγηιϕκλμνο" # \14x-15x | ||
| + "πθρστυϖωξψζ{|}∼" # \16x-17x. \177 is undefined | ||
| + "€ϒ′≤⁄∞ƒ♣♦♥♠↔←↑→↓" # \24x-\25x | ||
| + "°±″≥×∝∂•÷≠≡≈…↵" # \26x-27x | ||
| + "ℵℑℜ℘⊗⊕∅∩∪⊃⊇⊄⊂⊆∈∉" # \30x-31x | ||
| + "∠∇®©™∏√⋅¬∧∨⇔⇐⇑⇒⇓" # \32x-33x | ||
| + "◊〈®©™∑" # \34x-35x | ||
| + "〉∫⌠⌡", # \36x-37x. \360 and \377 are undefined | ||
|
Comment on lines
+250
to
+257
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| [*range(32, 127), *range(160, 240), *range(241, 255)], | ||
| ) | ||
| } | ||
| ) | ||
|
|
||
| # Adobe ZapfDingbats charset | ||
| # References: | ||
| # 1. https://en.wikipedia.org/wiki/Zapf_Dingbats | ||
| # 2. https://unicode.org/Public/MAPPINGS/VENDORS/ADOBE/zdingbat.txt | ||
| mapping.update( | ||
| { | ||
| c: "@%34%\\" + format(i, "o") + "@%%" | ||
| for c, i in zip( | ||
| " ✁✂✃✄☎✆✇✈✉☛☞✌✍✎✏" # \04x-\05x | ||
| + "✐✑✒✓✔✕✖✗✘✙✚✛✜✝✞✟" # \06x-\07x | ||
| + "✠✡✢✣✤✥✦✧★✩✪✫✬✭✮✯" # \10x-\11x | ||
| + "✰✱✲✳✴✵✶✷✸✹✺✻✼✽✾✿" # \12x-\13x | ||
| + "❀❁❂❃❄❅❆❇❈❉❊❋●❍■❏" # \14x-\15x | ||
| + "❐❑❒▲▼◆❖◗❘❙❚❛❜❝❞" # \16x-\17x. \177 is undefined | ||
| + "❡❢❣❤❥❦❧♣♦♥♠①②③④" # \24x-\25x. \240 is undefined | ||
| + "⑤⑥⑦⑧⑨⑩❶❷❸❹❺❻❼❽❾❿" # \26x-\27x | ||
| + "➀➁➂➃➄➅➆➇➈➉➊➋➌➍➎➏" # \30x-\31x | ||
| + "➐➑➒➓➔→↔↕➘➙➚➛➜➝➞➟" # \32x-\33x | ||
| + "➠➡➢➣➤➥➦➧➨➩➪➫➬➭➮➯" # \34x-\35x | ||
| + "➱➲➳➴➵➶➷➸➹➺➻➼➽➾", # \36x-\37x. \360 and \377 are undefined | ||
| [*range(32, 127), *range(161, 240), *range(241, 255)], | ||
| ) | ||
| } | ||
| ) | ||
|
|
||
| # Adobe ISOLatin1+ charset (i.e., ISO-8859-1 with extensions) | ||
| # References: | ||
| # 1. https://en.wikipedia.org/wiki/ISO/IEC_8859-1 | ||
| # 2. https://docs.generic-mapping-tools.org/dev/cookbook/octal-codes.html | ||
| # 3. https://www.adobe.com/jp/print/postscript/pdfs/PLRM.pdf | ||
| mapping.update( | ||
| { | ||
| c: "\\" + format(i, "o") | ||
| for c, i in zip( | ||
| "•…™—–fiž" # \03x. \030 is undefined | ||
| + "š" # \177 | ||
| + "Œ†‡Ł⁄‹Š›œŸŽł‰„“”" # \20x-\21x | ||
| + "ı`´ˆ˜¯˘˙¨‚˚¸'˝˛ˇ", # \22x-\23x | ||
| [*range(25, 32), *range(127, 160)], | ||
| ) | ||
| } | ||
| ) | ||
| # \240-\377 | ||
| mapping.update({chr(i): "\\" + format(i, "o") for i in range(160, 256)}) | ||
|
|
||
| # Remove any printable characters | ||
| mapping = {k: v for k, v in mapping.items() if k not in string.printable} | ||
| return argstr.translate(str.maketrans(mapping)) | ||
|
|
||
|
|
||
| def build_arg_string(kwdict, confdict=None, infile=None, outfile=None): | ||
| r""" | ||
| Convert keyword dictionaries and input/output files into a GMT argument | ||
|
|
@@ -318,7 +432,7 @@ def build_arg_string(kwdict, confdict=None, infile=None, outfile=None): | |
| gmt_args = [str(infile)] + gmt_args | ||
| if outfile: | ||
| gmt_args.append("->" + str(outfile)) | ||
| return " ".join(gmt_args) | ||
| return non_ascii_to_octal(" ".join(gmt_args)) | ||
|
|
||
|
|
||
| def is_nonstr_iter(value): | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| outs: | ||
| - md5: d93bed7495d77cd2ef7cc1b64edb9b3a | ||
| size: 19563 | ||
| path: test_non_ascii_to_octal.png |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.

Uh oh!
There was an error while loading. Please reload this page.