fix(tools): fix spacing errors in CJK and Latin mixed paths#1925
Closed
Sakuranda wants to merge 1 commit intoQwenLM:mainfrom
Closed
fix(tools): fix spacing errors in CJK and Latin mixed paths#1925Sakuranda wants to merge 1 commit intoQwenLM:mainfrom
Sakuranda wants to merge 1 commit intoQwenLM:mainfrom
Conversation
Collaborator
|
@Sakuranda 感谢你的贡献!我们排查发现这是模型 API 返回结果的问题,影响范围包括所有中英文混杂的情况,不局限于文件目录。我们正在联系相关方排查解决中,具体细节内容请参考 issue #1922 |
Collaborator
|
这个 issue 不考虑 hotfix,等待模型侧修复。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
本PR修复了LLM在生成路径时,会错误在CJK字符集(中日韩)与拉丁字符(a-zA-Z0-9)混用的路径段中插入空格的问题。
(例如,
image 图片与真实文件夹image图片)。本PR添加了一个回退机制:
Dive Deeper
Root Cause
LLM有时会把路径字符串当成自然语言格式化掉,而不会遵循严格的字符串规矩。(英文与中文相邻时,LLM会倾向于插入空格来提高回复的可读性。)
所以LLM会偶发地在中英混合的路径段上错误地加上空格。
Fix
添加了
resolvePathWithMixedScriptSpacingFix(...)函数,并设有严格的边界:中文平假名片假名韩文字母和A-Za-z0-9之间的间距。应用于:
packages/core/src/tools/read-file.tspackages/core/src/tools/ls.tspackages/core/src/tools/edit.tsTests
在以下位置添加了测试,可供参考:
packages/core/src/tools/read-file.test.tspackages/core/src/tools/ls.test.tspackages/core/src/tools/edit.test.tspackages/core/src/utils/paths.test.tsReviewer Test Plan
Testing Matrix
Linked issues / bugs
Fixes #1897
Fixes #1923