Implement IEEE 754 rounding conditions for fp32 to fp16 conversion in host

[DongBaiYue]

PR Category

Operator Mechanism

PR Types

Bug fixes

Description

Problem Summary

When converting fp32 tensors to fp16, current implementation lacks IEEE 754-compliant rounding logic, leading to Precision Error.

Example

import numpy as np
import paddle
import torch

input_data = 1e-05
numpy_tensor = np.array(input_data).astype("float16")
torch_x = torch.tensor(numpy_tensor)
paddle_x = paddle.to_tensor(numpy_tensor)
print("numpy == 1e-5:", numpy_tensor == 1e-5 )              # True
print("torch == 1e-5:", (torch_x == 1e-5).cpu().numpy() )   # True
print("paddle == 1e-5:", (paddle_x == 1e-5 ) )              # False

Solution Approach

Implemented IEEE 754 round-to-nearest-even standard

// Rounding: round to nearest, ties to even
// https://en.wikipedia.org/wiki/Rounding#Rounding_half_to_even
const uint32_t lsb =
    (v.ui >> shift) & 0x1;               // Least significant retained bit
v.ui += (0xFFF + lsb) & -(v.ui < infN);  // Round with overflow protection

Validation

Input (fp32)	Previous Output	New Output
1e-5f	0x00a7	0x00a8
0.333f	0x3553	0x3554
6.10056e-5f	0x03ff	0x0400
5e-8f	0x0000	0x0001

Others

These codes are very difficult to understand, so I have added some comments to clarify them.

Pcard-67164

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[DongBaiYue]

/re-run coverage build

[DongBaiYue]

/re-run Infer

[DongBaiYue]

/re-run inference build

[DongBaiYue]

/re-run all-failed

[wanghuancoder]

LGTM

[DongBaiYue]

/re-run all-failed

[DongBaiYue]

/re-run all-failed

[DongBaiYue]

/re-run all-failed

[DongBaiYue]

/re-run all-failed

[wanghuancoder]

LGTM