Skip to content

UPSTREAM PR #17738: common: use native MultiByteToWideChar#420

Open
loci-dev wants to merge 1 commit intomainfrom
upstream-PR17738-branch_angt-common-use-native-multibytetowidechar
Open

UPSTREAM PR #17738: common: use native MultiByteToWideChar#420
loci-dev wants to merge 1 commit intomainfrom
upstream-PR17738-branch_angt-common-use-native-multibytetowidechar

Conversation

@loci-dev
Copy link

@loci-dev loci-dev commented Dec 3, 2025

Mirrored from ggml-org/llama.cpp#17738

std::codecvt_utf8<wchar_t> is deprecated and produces warnings:

common/common.cpp:792:31: warning: 'codecvt_utf8<wchar_t>' is deprecated [-Wdeprecated-declarations]
  792 |     std::wstring_convert<std::codecvt_utf8<wchar_t>> converter;
      |

Please, review carefully as I’ve used Gemini to generate utf8_to_wstring() and I'm absolutely not a Windows dev.

`std::codecvt_utf8<wchar_t>` is deprecated and produces warnings:

    common/common.cpp:792:31: warning: 'codecvt_utf8<wchar_t>' is deprecated [-Wdeprecated-declarations]
      792 |     std::wstring_convert<std::codecvt_utf8<wchar_t>> converter;
          |

Signed-off-by: Adrien Gallouët <[email protected]>
@loci-review
Copy link

loci-review bot commented Dec 3, 2025

Explore the complete analysis inside the Version Insights

Performance Analysis Summary: PR #420

Analysis Overview

This PR replaces deprecated std::codecvt_utf8 with native Windows API MultiByteToWideChar in fs_create_directory_with_parents function within common/common.cpp. The change affects filesystem initialization code, not inference or tokenization paths.

Performance Impact Assessment

Condition 1 applies: No meaningful performance changes detected.

The modified function handles directory creation during initialization and is not part of the inference or tokenization pipeline. Performance metrics for critical functions (common_token_to_piece, common_detokenize) show 5-11 ns improvements unrelated to this PR. No inference functions (llama_decode, llama_encode, llama_tokenize) were modified. Power consumption changes across all binaries are negligible (<0.008%). The code change eliminates compiler warnings without impacting tokens per second.

@loci-dev loci-dev force-pushed the main branch 26 times, most recently from 9612097 to c217e38 Compare December 6, 2025 08:10
@loci-dev loci-dev force-pushed the main branch 30 times, most recently from 1daebfe to 75a97fd Compare December 10, 2025 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants