Skip to content

Conversation

@yizhe-skyfall
Copy link

No description provided.

@mmathew23
Copy link
Collaborator

Hi @yizhe-skyfall thank you for the contribution. I'm not able to replicate the issue on my side. For example if I put a debug statement in approximate_vllm_memory_usage in the current version of unsloth_zoo, I see that it print out true when loading in 4bit and fast_inference = True. Perhaps the issue was addressed since you first raised the PR. Could you verify if all is working as expected for you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants