Skip to content

[PIR & Inference] Refactor fused_weight_only_linear_pass #59792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yuanlehome merged 12 commits into from
Dec 12, 2023
Merged

[PIR & Inference] Refactor fused_weight_only_linear_pass #59792

Changes from 4 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
d27ddf0
refactor: refactor fused_weight_only_linear_pass
Wanglongzhi2001 Dec 7, 2023
061ee4e
refactor: add else case for PADDLE_WITH_CUDA
Wanglongzhi2001 Dec 7, 2023
9a369a6
fix: fix typo
Wanglongzhi2001 Dec 7, 2023
6379b88
refactor: refactor pass
Wanglongzhi2001 Dec 8, 2023
5d07f9b
refactor: support sm 70, 75, 80 and 86 in pass
Wanglongzhi2001 Dec 8, 2023
cb98a32
refactor: refactor pass and test
Wanglongzhi2001 Dec 11, 2023
9647c7f
fix: fix typo
Wanglongzhi2001 Dec 11, 2023
3126e46
refactor: use xxOp::name() instead of pd_op.xx in pass
Wanglongzhi2001 Dec 11, 2023
c3c72d7
refactor: refactor error msg and fix typo
Wanglongzhi2001 Dec 11, 2023
201aab3
refactor: refactor pass and test
Wanglongzhi2001 Dec 11, 2023
b49e4f4
fix: fix typo
Wanglongzhi2001 Dec 11, 2023
32efabb
refactor: refactor IrDtype
Wanglongzhi2001 Dec 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions paddle/fluid/pir/transforms/fusion/fused_weight_only_linear_pass.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,15 @@

namespace {

inline int getSMVersion() {
int getSMVersion() {
int sm_version = 80;
#if defined(PADDLE_WITH_CUDA)
sm_version = paddle::platform::GetGPUComputeCapability(
paddle::platform::GetCurrentDeviceId());
#else
PADDLE_THROW(paddle::platform::errors::Unavailable(
"platform::GetGPUComputeCapability is not "
"supported in CPU only version."));
#endif
return sm_version;
}
Expand Down Expand Up @@ -74,15 +78,15 @@ class FusedWeightOnlyLinearPattern
res.Attr([](const pir::drr::MatchContext &match_ctx) -> std::any {
return "weight_only_int8";
});
// int arch = getSMVersion();
const auto &weight_quantize_arch_attr =
res.Attr([&](const pir::drr::MatchContext &match_ctx) -> std::any {
return 80;

const auto &arch_attr =
res.Attr([&](const pir::drr::MatchContext &match_ctx) -> int {
return getSMVersion();
});

const auto &weight_quantize = res.Op(
"pd_op.weight_quantize",
{{"algo", weight_only_int8_attr}, {"arch", weight_quantize_arch_attr}});
const auto &weight_quantize =
res.Op("pd_op.weight_quantize",
{{"algo", weight_only_int8_attr}, {"arch", arch_attr}});
weight_quantize({&res.Tensor("w")},
{&res.Tensor("quanted_weight_tensor"),
&res.Tensor("weight_scale_tensor")});
Expand All @@ -92,12 +96,9 @@ class FusedWeightOnlyLinearPattern
return "int8";
});

const auto &weight_only_linear_arch_attr = res.Attr(
[&](const pir::drr::MatchContext &match_ctx) -> int { return 80; });
const auto &weight_only_linear =
res.Op("pd_op.weight_only_linear",
{{"weight_dtype", weight_dtype_attr},
{"arch", weight_only_linear_arch_attr}});
{{"weight_dtype", weight_dtype_attr}, {"arch", arch_attr}});
weight_only_linear({&res.Tensor("x"),
&res.Tensor("quanted_weight_tensor"),
&res.Tensor("bias"),
Expand All @@ -119,8 +120,8 @@ class FusedWeightOnlyLinearPass : public pir::PatternRewritePass {

bool CanApplyOn(pir::Operation *op) const override {
int sm_vesion = getSMVersion();
if (sm_vesion != 70 && sm_vesion != 80 && sm_vesion != 86 &&
sm_vesion != 75) {
// TODO(Wanglongzhi2001): only support sm80 for now
if (sm_vesion != 80) {
Copy link
Contributor

@yuanlehome yuanlehome Dec 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

当前develop版本已经支持了70 75 80 86这四个架构

Copy link
Contributor Author

@Wanglongzhi2001 Wanglongzhi2001 Dec 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

return false;
}
return op->num_regions() > 0;
Expand Down