Skip to content

Conversation

@jmoik
Copy link

@jmoik jmoik commented Sep 13, 2025

feature: add local translation using LLM with MLX

- add MLXTranslationService for on-device translation
- add MLXModelManager for model downloading and management
- add unified TranslationService interface supporting iOS/macOS
- integrate translation into ChatViewModel with translate/untranslate actions
- update ContentView with translation UI controls
- support almost any language (currently using gemma3-1B)
- Add Package.swift dependencies for MLX framework

Resolves need for privacy-preserving translation without internet dependency

Addresses issue #540

- add MLXTranslationService for on-device translation
- add MLXModelManager for model downloading and management
- add unified TranslationService interface supporting iOS/macOS
- integrate translation into ChatViewModel with translate/untranslate actions
- update ContentView with translation UI controls
- support almost any language (currently using gemma3-1B)
- Add Package.swift dependencies for MLX framework

Resolves need for privacy-preserving translation without internet dependency
@jmoik
Copy link
Author

jmoik commented Sep 13, 2025

This is WIP and not ready to be merged! If we agree we want to have this feature I will implement it more cleanly, currently there are issues with stop characters etc, but generally the translation works to almost any language on mac with ollama and on iOS with MLX. When pressing the translation button the first time, a 1B model is downloaded from huggingface, so that might take some time, then the translation is relatively fast. This was tested on iPhone 14. It uses about 1GB of memory at 4bit quantization, the phone has 6GB in total, but this feature would significantly increase memory and storage requirements and should definitely made optional.

@LagBug
Copy link

LagBug commented Sep 24, 2025

This is WIP and not ready to be merged! If we agree we want to have this feature I will implement it more cleanly, currently there are issues with stop characters etc, but generally the translation works to almost any language on mac with ollama and on iOS with MLX. When pressing the translation button the first time, a 1B model is downloaded from huggingface, so that might take some time, then the translation is relatively fast. This was tested on iPhone 14. It uses about 1GB of memory at 4bit quantization, the phone has 6GB in total, but this feature would significantly increase memory and storage requirements and should definitely made optional.

Have you tried it with the Foundation Models for iPhones 15 Pro and above?

@jmoik
Copy link
Author

jmoik commented Sep 26, 2025

This should work with foundation models as well, possibly have to adjust prompt and termination characters. I only have access to iPhone 14.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants