Skip to content

Conversation

@younesbelkada
Copy link
Contributor

@younesbelkada younesbelkada commented Aug 24, 2023

What does this PR do?

Per the instructions of installing auto-gptq library we need to slightly update the Dockerfile otherwise it will install the 0.3.2 version which is not compatible with the integration

Will also update some expected values to make the slow tests pass

cc @ydshieh @SunMarc

Can confirm the Docker image is built successfully

Copy link
Collaborator

@ydshieh ydshieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works for me. Thank you for confirming the build is successful.

So basically we want to install the same version but with cu118 support, right?

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Aug 24, 2023

The documentation is not available anymore as the PR was closed or merged.

@younesbelkada younesbelkada marked this pull request as ready for review August 24, 2023 10:11
@younesbelkada
Copy link
Contributor Author

Can confirm the slow tests now pass in the docker image

So basically we want to install the same version but with cu118 support, right?

It is slightly trickier than that, auto-gptq listed in pypi does not contain the latest supported versions (>= 0.4.1), one needs to install that directly through the command that I have shared. Also that way you get the pre-built wheels instead of building auto-gptq at each install (which is the case right now if you do pip install auto-gptq)

@younesbelkada younesbelkada requested a review from SunMarc August 24, 2023 10:13
Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ! Thanks for checking that the image is built correctly :)

@younesbelkada younesbelkada merged commit 584eeb5 into huggingface:main Aug 24, 2023
@younesbelkada younesbelkada deleted the add-auto-gptq-dockerfile branch August 24, 2023 12:57
parambharat pushed a commit to parambharat/transformers that referenced this pull request Sep 26, 2023
huggingface#25713)

* add correct installation of GPTQ library

* update tests values
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants