Skip to content

[Big Tensor] Fix big tensor problem for paddle.nn.functional.class_center_sample#488

Merged
cangtianhuang merged 5 commits intoPFCCLab:mainfrom
HeyDavid633:fix_class_center_sample
Aug 1, 2025
Merged

[Big Tensor] Fix big tensor problem for paddle.nn.functional.class_center_sample#488
cangtianhuang merged 5 commits intoPFCCLab:mainfrom
HeyDavid633:fix_class_center_sample

Conversation

@HeyDavid633
Copy link
Contributor

主要修复:

  1. label.numel() 为大Tensor时的错误抛出
  2. label.numel() 导致 MemoryBuffer 分配尺寸溢出的错误抛出
  3. 在 base_config.yaml 豁免了以上的错误抛出

原始的case尺寸,贴着 unint32 的上限设置;对于所有的case样例来说都是Torch error;即超过了 Torch 的限制
image

并在 tester 中增加了 case 以验证以上修复点2,尽管新增的case不是大Tensor

@paddle-bot
Copy link

paddle-bot bot commented Aug 1, 2025

Thanks for your contribution!

Copy link
Collaborator

@cangtianhuang cangtianhuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

torch error的case需要放到torch_error_skip中呀~

Comment on lines +1745 to +1747
paddle.nn.functional.class_center_sample(Tensor([356493278],"int32"), 10, 8, )
paddle.nn.functional.class_center_sample(Tensor([356493280],"int32"), 10, 8, )
paddle.nn.functional.class_center_sample(Tensor([2294967295],"int32"), 10, 8, )
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

新增配置放到 tester/api_config/8_big_tensor/big_tensor_1_8.txt

paddle.einsum: "(Unimplemented) cublas GEMM does not support N >"
paddle.nn.functional.class_center_sample:
- "(InvalidArgument) The total number of elements for 'label' should be less than"
- "(InvalidArgument) Illegal memory allocation, total allocated spacemust be greater than 0"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个报错看上去是溢出了,不需要在paddle中修复嘛?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paddle 修复的方式在这里 PaddlePaddle/Paddle#74364

Copy link
Collaborator

@cangtianhuang cangtianhuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cangtianhuang cangtianhuang merged commit 0bf0ff4 into PFCCLab:main Aug 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants