Skip to content

Enable bf16 quantization of models, and fix resetting args.audio in the microservice between runs#832

Merged
Spycsh merged 82 commits intoopea-project:mainfrom
ctao456:ctao/opea
Oct 30, 2024
Merged

Enable bf16 quantization of models, and fix resetting args.audio in the microservice between runs#832
Spycsh merged 82 commits intoopea-project:mainfrom
ctao456:ctao/opea

Conversation

@ctao456
Copy link
Copy Markdown
Contributor

@ctao456 ctao456 commented Oct 29, 2024

Description

Enable bf16 quantization of models, and fix resetting args.audio in the microservice between runs

Issues

#775

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

https://docs.habana.ai/en/latest/PyTorch/PyTorch_Mixed_Precision/index.html

Tests

Tests on Xeon, Gaudi

ctao456 and others added 30 commits September 25, 2024 22:24
@ctao456
Update audioqna gateway to print text, in gateway.py
544f082 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'opea-project:main' into main
7dd0b8a
@ctao456
Merge branch 'opea-project:main' into main
254071d
@ctao456
updates needed for demo
f0c7a02 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'ctao/demo' of https://github.com/ctao456/GenAIComps int…
81a5943 
…o ctao/demo
@ctao456
original pr content
dbf8ad3 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'opea-project:main' into ctao/opea
98cbb1c
@ctao456
Merge branch 'opea-project:main' into main
9d7c4f9
@ctao456
Revert "updates needed for demo"
d85f33a 
This reverts commit f0c7a02.

Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
remove improper images
a8363b7 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Addressed some comments on previous pr
4c22b40 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Add Dockerfile for cpu support
dc59a77 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@hteeyeoh @ctao456
CODEOWNER: Update comp CODEOWNER (opea-project#757)
bf38204 
Signed-off-by: Yeoh, Hoong Tee <hoong.tee.yeoh@intel.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>
@XinyuYe-Intel @pre-commit-ci @ctao456
Add stable diffusion microservice (opea-project#729)
b932b94 
* add stable diffusion microservice.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* added test.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* changed output to images bytes data

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* unified inference and wrapper into one microservice.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

---------

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ZePan110 @pre-commit-ci @ctao456
Compatible with different platforms. (opea-project#766)
7fb8c6b 
* Compatible with different platforms.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix issue.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ZePan110 @ctao456
Optimize path and link validity check. (opea-project#745)
817c0f1 
Signed-off-by: ZePan110 <ze.pan@intel.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>
@chensuyue @ctao456
Add timeout for ut test (opea-project#773)
a14c76f 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>
@pre-commit-ci @ctao456
[pre-commit.ci] auto fixes from pre-commit.com hooks
8fae482 
for more information, see https://pre-commit.ci

Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'opea-project:main' into ctao/opea
b9420c7
@ctao456
test hyperlink
1879a1d 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
test hyperlink
cc938fb 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
test hyperlink issue
697fdbd 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
test hyperlink issue
120edae 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'opea-project:main' into main
e20ef65
@ctao456
put back hyperlinks in readme
4bfd1f3 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
remove possible error hyperlink
10b3b34 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
put hyperlink back
09ab934 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'opea-project:main' into ctao/opea
901adcc
@ctao456
major update to use FastAPI for wav2lip, and structure component format
303eb1f 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
d058496 
for more information, see https://pre-commit.ci
ctao456 and others added 19 commits October 19, 2024 16:51
@ctao456
update
38e4d46 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
update
7aa56f4 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
155ef8e 
for more information, see https://pre-commit.ci
@ctao456
Model download script moved from Dockerfiles to Docker entrypoint
0b7f385 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
update paths
f3b08ca 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Correct paths in readme
9b392bd 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'main' of https://github.com/ctao456/GenAIComps into cta…
c19d8f8 
…o/opea
@ctao456
Merge branch 'opea-project:main' into main
4181392
@ctao456
Merge branch 'main' of https://github.com/ctao456/GenAIComps into cta…
560a8fc 
…o/opea

Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
revert changes to audioqna gateway
8252bed 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
longer wait time after docker run
746778d 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
add mount volume in test scripts
fd8b026 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
add volume mount in test scripts
00ce8a8 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
udpate test script
c45acbf 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Merge branch 'opea-project:main' into ctao/opea
3d89eb9
@ctao456
udpate optimizations
8a93fdd 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
update
9ead8d8 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
update
d5cb94c 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
update
c995868 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
need outputs folder
e6cddc3 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
test
0647262 
Signed-off-by: Chun Tao <chun.tao@intel.com>
@ctao456
Copy link
Copy Markdown
Contributor Author

ctao456 commented Oct 29, 2024

@chensuyue @lvliang-intel Cleared all checks. Please review. Thanks.

@lvliang-intel lvliang-intel requested a review from Spycsh October 29, 2024 14:26
@Spycsh Spycsh merged commit 00abba2 into opea-project:main Oct 30, 2024
madison-evans pushed a commit to SAPD-Intel/GenAIComps that referenced this pull request May 12, 2025
@ctao456 @hteeyeoh
@XinyuYe-Intel @pre-commit-ci @ZePan110 @chensuyue
Enable bf16 quantization of models, and fix resetting args.audio in t…
b0f92c6 
…he microservice between runs (opea-project#832)

* Update audioqna gateway to print text, in gateway.py

Signed-off-by: Chun Tao <chun.tao@intel.com>

* updates needed for demo

Signed-off-by: Chun Tao <chun.tao@intel.com>

* original pr content

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Revert "updates needed for demo"

This reverts commit f0c7a02.

Signed-off-by: Chun Tao <chun.tao@intel.com>

* remove improper images

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Addressed some comments on previous pr

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Add Dockerfile for cpu support

Signed-off-by: Chun Tao <chun.tao@intel.com>

* CODEOWNER: Update comp CODEOWNER (opea-project#757)

Signed-off-by: Yeoh, Hoong Tee <hoong.tee.yeoh@intel.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>

* Add stable diffusion microservice (opea-project#729)

* add stable diffusion microservice.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* added test.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* changed output to images bytes data

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* unified inference and wrapper into one microservice.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

---------

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>

* Compatible with different platforms. (opea-project#766)

* Compatible with different platforms.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix issue.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>

* Optimize path and link validity check. (opea-project#745)

Signed-off-by: ZePan110 <ze.pan@intel.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>

* Add timeout for ut test (opea-project#773)

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Chun Tao <chun.tao@intel.com>

* test hyperlink

Signed-off-by: Chun Tao <chun.tao@intel.com>

* test hyperlink

Signed-off-by: Chun Tao <chun.tao@intel.com>

* test hyperlink issue

Signed-off-by: Chun Tao <chun.tao@intel.com>

* test hyperlink issue

Signed-off-by: Chun Tao <chun.tao@intel.com>

* put back hyperlinks in readme

Signed-off-by: Chun Tao <chun.tao@intel.com>

* remove possible error hyperlink

Signed-off-by: Chun Tao <chun.tao@intel.com>

* put hyperlink back

Signed-off-by: Chun Tao <chun.tao@intel.com>

* major update to use FastAPI for wav2lip, and structure component format

Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add dockerfiles in animation-compose-cd.yaml

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Fix end of file issue in animation-compose-cd.yaml

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Fix Docker deployment on Xeon

Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add versioning for all pip packages

Signed-off-by: Chun Tao <chun.tao@intel.com>

* e2e test script for animation

Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update e2e test script

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update e2e test script

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update readme

Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update gateway

Signed-off-by: Chun Tao <chun.tao@intel.com>

* udpate gateway

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Fix AVATAR_CHATBOT

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update gateway

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* test

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update gateway

Signed-off-by: Chun Tao <chun.tao@intel.com>

* fix max_tokens in AvatarChatbot gateway

Signed-off-by: Chun Tao <chun.tao@intel.com>

* test

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Model download script moved from Dockerfiles to Docker entrypoint

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update paths

Signed-off-by: Chun Tao <chun.tao@intel.com>

* Correct paths in readme

Signed-off-by: Chun Tao <chun.tao@intel.com>

* revert changes to audioqna gateway

Signed-off-by: Chun Tao <chun.tao@intel.com>

* longer wait time after docker run

Signed-off-by: Chun Tao <chun.tao@intel.com>

* add mount volume in test scripts

Signed-off-by: Chun Tao <chun.tao@intel.com>

* add volume mount in test scripts

Signed-off-by: Chun Tao <chun.tao@intel.com>

* udpate test script

Signed-off-by: Chun Tao <chun.tao@intel.com>

* udpate optimizations

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* update

Signed-off-by: Chun Tao <chun.tao@intel.com>

* need outputs folder

Signed-off-by: Chun Tao <chun.tao@intel.com>

* test

Signed-off-by: Chun Tao <chun.tao@intel.com>

---------

Signed-off-by: Chun Tao <chun.tao@intel.com>
Signed-off-by: Yeoh, Hoong Tee <hoong.tee.yeoh@intel.com>
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
Signed-off-by: ZePan110 <ze.pan@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
Co-authored-by: Hoong Tee, Yeoh <hoong.tee.yeoh@intel.com>
Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Spycsh Spycsh Spycsh approved these changes

@lvliang-intel lvliang-intel lvliang-intel approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@ctao456 @Spycsh @lvliang-intel @hteeyeoh @XinyuYe-Intel @ZePan110 @chensuyue