Skip to content

Fix various bugs for LoRA training #5161

Merged
oobabooga merged 4 commits intooobabooga:devfrom
AstrisCantCode:dev
Jan 3, 2024
Merged

Fix various bugs for LoRA training #5161
oobabooga merged 4 commits intooobabooga:devfrom
AstrisCantCode:dev

Conversation

@AstrisCantCode
Copy link
Copy Markdown
Contributor

  • Allow mistral models to have all modules targeted selectively, since the module names are the same as those for llama models
  • Fixed issue that occurred because prepare_model_for_kbit_training was being run on unquantized models that didn't need it (causing an OOM error)
  • Fixed issue in the training arguments that didn't allow BF16 training to take place even when the appropriate options were selected in the model menu
  • Fixed typo that erroneously (always) enabled IPEX

Checklist:

@oobabooga
Copy link
Copy Markdown
Owner

Thank you for the fixes

@oobabooga oobabooga merged commit b80e636 into oobabooga:dev Jan 3, 2024
@AstrisCantCode AstrisCantCode deleted the dev branch January 4, 2024 06:43
@AstrisCantCode AstrisCantCode restored the dev branch January 4, 2024 06:43
PoetOnTheRun pushed a commit to PoetOnTheRun/text-generation-webui that referenced this pull request Feb 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants