LinspaceKernel uses the dtype of 'self' as the type of 'step' when tensor is floating

[zrr1999]

PR Category

Operator Mechanism

PR Types

Performance

Description

import torch
import paddle

dtype = torch.float32
start = torch.tensor(1.0)
stop = torch.tensor(-1.0)

# paddle
dx = ((stop - start).to(dtype=torch.float64) / 28)
(start + dx*6).to(dtype).tolist()

# torch
dx = ((stop.to(dtype=dtype) - stop.to(dtype=dtype)) / 28)
(start + dx.to(dtype)*6).tolist()

受影响的 API：

• paddle.linspace 对齐全部case（共 85 个）

修改内容

• 计算 step 时，始终将 start和end转换为 double 再进行计算，以确保获取到最高精度的step。
• 改为先对stop和start进行cast在计算，可以避免溢出。
• 当 tensor的dtype为整数时且为GPU kernel时，将 step type 转换为float （原本为double）。
• 当 tensor的dtype为浮点数时，将 step type 转换为 tensor 的 dtype进行后续计算。

其他说明

GPU Kernel 在T为整数时，使用float比double性能更好，可以对齐PyTorch。

• paddle float32 forward 5.1293089389801025
• paddle float64 forward 8.464366674423218

性能测试代码

import paddle
import time

test_loop = 2**10

paddle.linspace(
    -1,
    1,
    2 ** 30,
    dtype="float32",
)
paddle.linspace(
    -1,
    1,
    2 ** 30,
    dtype="float64",
)
with paddle.no_grad():
    paddle.base.core._cuda_synchronize(paddle.CUDAPlace(0))
    start = time.time()
    for i in range(test_loop):
        paddle.linspace(
            -1,
            1,
            2 ** 30,
            dtype="float32",
        )
    start2 = time.time()
    paddle.base.core._cuda_synchronize(paddle.CUDAPlace(0))
    end = time.time()
    timeused = end - start
    print("paddle float32 forward", timeused)

with paddle.no_grad():
    paddle.base.core._cuda_synchronize(paddle.CUDAPlace(0))
    start = time.time()
    for i in range(test_loop):
        paddle.linspace(
            -1,
            1,
            2** 30,
            dtype="float64",
        )
    start2 = time.time()
    paddle.base.core._cuda_synchronize(paddle.CUDAPlace(0))
    end = time.time()
    timeused = end - start
    print("paddle float64 forward", timeused)
    print("end - start2", end - start2)

TODO:

• GPU kernel 有性能提升空间，一个线程可以计算多个值，降低调度开销。

Pcard-67164

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (develop@7cc27eb). Learn more about missing BASE report.

Additional details and impacted files

@@             Coverage Diff             @@
##             develop    #75238   +/-   ##
===========================================
  Coverage           ?   100.00%           
===========================================
  Files              ?         2           
  Lines              ?         5           
  Branches           ?         0           
===========================================
  Hits               ?         5           
  Misses             ?         0           
  Partials           ?         0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[A-nnonymous]

Need some modifications

[A-nnonymous]

在StepT为整形时，我们是不是应该添加一个检查来确认：stop-start后的值之间，是否有num-1个有效整数值？

[zrr1999]

这里的StepT不应该为整数，如果T是整数，应该通过 using StepT = std::conditional_t<std::is_integral_v, double, T>;转为double

[A-nnonymous]

不建议使用两个kernel名字来管理这个linspace功能，可以考虑使用C++的模版特化，使用统一的名称来管理两个kernel

[zrr1999]

done

[A-nnonymous]

确认一下paddle是否支持超长整型，比如INT128等

[zrr1999]

目前kernel还没有找到int128相关的内容，后面如果添加了再修改这里

[wanghuancoder]

LGTM

[A-nnonymous]

LGTM, except performance for now