[SYCL] Add sycl complex testing #1058

AidanBeltonS · 2022-06-15T09:25:25Z

This PR adds testing for the experimental sycl complex implementation.

It tests math functions, abs, acos, asin, atan, acosh, asinh, atanh, arg, conj, cos, cosh, exp, log, log10, norm, polar, pow, proj, sin, sinh, sqrt, tanh, and tanh. Additionally it tests the math operators +, -, /, * and their assign equivalents. It also tests stream operations with sycl::stream, std::istream, and std::ostream.

For simplicity the tests are broken into the test and a list of test cases. Each test is run for the complex type with double, float, and sycl::half.

Results are compared to values from std::complex. Due to differences in error handling for the pow function between, libstdc++ and libc++(which the implementation is based on), testing of NAN's and INF's of pow is not done through comparison with std::complex but instead using expected values.

Depends on: intel/llvm#6309

AerialMantis · 2022-07-05T10:14:25Z

@steffenlarsen now that intel/llvm#6309 is merged I was hoping we could merge this too, I see that the pre-ci jobs failed, but I can't see the results, could you send me the logs for these.

steffenlarsen · 2022-07-05T10:30:50Z

@steffenlarsen now that intel/llvm#6309 is merged I was hoping we could merge this too, I see that the pre-ci jobs failed, but I can't see the results, could you send me the logs for these.

@AerialMantis - Apologies, I will get to this ASAP. I believe most of the CI test failures are due to running prior to the corresponding intel/llvm changes being merged. If the failures persist after comments have been addressed I will supply the failure information.

AerialMantis · 2022-07-05T10:37:49Z

@steffenlarsen no problem, thanks very much!

SYCL/Complex/sycl_complex_stream_test.cpp

SYCL/Complex/sycl_complex_helper.hpp

SYCL/Complex/sycl_complex_math_test_cases.hpp

SYCL/Complex/sycl_complex_operator_test_cases.hpp

SYCL/Complex/sycl_complex_stream_test.cpp

AerialMantis · 2022-07-06T16:16:08Z

@steffenlarsen thanks for the feedback, these changes all sounds fine so happy to make them, though @AidanBeltonS is on holiday for a couple of weeks and I don't have write privileges for this, so we might just have to wait until he's back.

steffenlarsen · 2022-07-06T16:24:09Z

@steffenlarsen thanks for the feedback, these changes all sounds fine so happy to make them, though @AidanBeltonS is on holiday for a couple of weeks and I don't have write privileges for this, so we might just have to wait until he's back.

I am okay with waiting, if that is preferred. Alternatively, feel free to open a new PR with the changes and we can close this one.

AerialMantis · 2022-07-07T08:45:07Z

I am okay with waiting, if that is preferred. Alternatively, feel free to open a new PR with the changes and we can close this one.

That's okay I'm happy to wait till he's back.

Co-authored-by: Steffen Larsen <[email protected]>

AidanBeltonS · 2022-07-20T08:46:25Z

@steffenlarsen the tests are failing, I am not entirely sure why.
Would it be possible to obtain the test-log? Thanks

steffenlarsen · 2022-07-25T12:12:54Z

On Windows sycl_complex_math_test.cpp, sycl_complex_operator_test.cpp, sycl_complex_pow_test.cpp, and sycl_complex_stream_test.cpp fail with:

In file included from W:\jenkins-dir\workspace\LLVM-Test-Suite-CI-TMP\LLVM-Test-Suite-CI-WIN\llvm-test-suite\SYCL\Complex\sycl_complex_stream_test.cpp:7:
W:\jenkins-dir\workspace\LLVM-Test-Suite-CI-TMP\LLVM-Test-Suite-CI-WIN\llvm-test-suite\SYCL\Complex/sycl_complex_helper.hpp(13,18): error: constexpr variable 'NANd' must be initialized by a constant expression
constexpr double NANd(NAN);
                 ^    ~~~
W:\jenkins-dir\workspace\LLVM-Test-Suite-CI-TMP\LLVM-Test-Suite-CI-WIN\llvm-test-suite\SYCL\Complex/sycl_complex_helper.hpp(13,23): note: floating point arithmetic produces a NaN
constexpr double NANd(NAN);
                      ^
C:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\ucrt\corecrt_math.h(94,39): note: expanded from macro 'NAN'
#define NAN        (-(float)(INFINITY * 0.0F))

On Linux the tests fail due to different problems, though all seem to be with SYCL_DEVICE_FILTER=opencl:cpu,host:

sycl_complex_operator_test.cpp

terminate called after throwing an instance of 'cl::sycl::compile_program_error'
  what():  The program was built for 1 devices
Build program log for 'Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz':
Compilation started
Compilation done
Linking started
Linking done
Device build started
Options used by backend compiler: 
Failed to build device program
Error: unimplemented function(s) used:
_Z5isnanDh is undefined 
_Z5isinfDh is undefined 
_Z8copysignDhDh is undefined 
_Z4fabsDh is undefined 
_Z4fmaxDhDh is undefined 
_Z4logbDh is undefined 
_Z8isfiniteDh is undefined 
_Z5ldexpDhi is undefined 
CompilerException Failed to parse IR
 -11 (PI_ERROR_BUILD_PROGRAM_FAILURE)

error: command failed with exit status: -6

sycl_complex_math_test.cpp & sycl_complex_pow_test.cpp

error: command failed with exit status: -11

sycl_complex_stream_test.cpp

.../llvm-test-suite/SYCL/Complex/sycl_complex_stream_test.cpp:63:20: error: CHECK-COUNT: expected string not found in input (3 out of 3)
 // CHECK-COUNT-3: (1.5,-1)
                   ^
<stdin>:3:1: note: scanning from here
((inf,inf)
^
<stdin>:3:2: note: possible intended match here
((inf,inf)
 ^

Input file: <stdin>
Check file: .../llvm-test-suite/SYCL/Complex/sycl_complex_stream_test.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            1: (1.5,-1)
            2: (1.5,-1)
            3: ((inf,inf)
count:63'0     X~~~~~~~~~ error: no match found
count:63'1      ?         possible intended match
            4: (inf,inf)
count:63'0     ~~~~~~~~~
            5: (inf,inf)
count:63'0     ~~~~~~~~~
            6: (nan,nan)
count:63'0     ~~~~~~~~~
            7: (nan,nan)
count:63'0     ~~~~~~~~~
            8: (nan,nan)
count:63'0     ~~~~~~~~~
>>>>>>

error: command failed with exit status: 1

AidanBeltonS · 2022-07-28T10:54:09Z

@steffenlarsen I have the tests passing locally, would you be able to provide logs again? Many thanks

steffenlarsen · 2022-07-28T11:23:27Z

On Linux (PI_OpenCL):

Complex/sycl_complex_stream_test.cpp (SYCL_DEVICE_FILTER=host)

.../llvm-test-suite/SYCL/Complex/sycl_complex_stream_test.cpp:63:30: error: CHECK-ALL-TYPES-COUNT: expected string not found in input (2 out of 3)
 // CHECK-ALL-TYPES-COUNT-3: (1.5,-1)
                             ^
<stdin>:2:1: note: scanning from here
(inf,inf)
^

Input file: <stdin>
Check file: .../llvm-test-suite/SYCL/Complex/sycl_complex_stream_test.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
          1: (1.5,-1)
          2: (inf,inf)
count:63     X~~~~~~~~ error: no match found
          3: (nan,nan)
count:63     ~~~~~~~~~
>>>>>>

error: command failed with exit status: 1

NOTE: This seems to have been with SYCL_DEVICE_FILTER=host so it may be flaky as it doesn't show up on the other cases.

Regression/kernel_name_inside_sycl_namespace.cpp (SYCL_DEVICE_FILTER=opencl:cpu,host)

error: command failed with exit status: -11

On Windows (PI_LevelZero):

Complex/sycl_complex_math_test.cpp (SYCL_DEVICE_FILTER=ext_oneapi_level_zero:gpu,host)

Test failed with complex_type: double Computed on device Output: (1.5707963267948966,0) Reference: (nan,-nan(ind))
Test failed with complex_type: double Computed on host Output: (1.5707963267948966,0) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on device Output: (1.57079637,0) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (1.57079637,0) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on device Output: (1.5703,0) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on host Output: (1.5703,0) Reference: (nan,-nan(ind))
Test failed with complex_type: double Computed on device Output: (1.5707963267948966,0) Reference: (nan,-nan(ind))
Test failed with complex_type: double Computed on host Output: (1.5707963267948966,0) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on device Output: (1.57079637,0) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (1.57079637,0) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on device Output: (1.5703,0) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on host Output: (1.5703,0) Reference: (nan,-nan(ind))
atan test failed
Test failed with complex_type: double Computed on device Output: (0,1.5707963267948966) Reference: (nan,nan)
Test failed with complex_type: double Computed on host Output: (0,1.5707963267948966) Reference: (nan,nan)
Test failed with complex_type: float Computed on device Output: (0,1.57079637) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (0,1.57079637) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on device Output: (0,1.5703) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on host Output: (0,1.5703) Reference: (nan,nan)
Test failed with complex_type: double Computed on device Output: (0,1.5707963267948966) Reference: (nan,nan)
Test failed with complex_type: double Computed on host Output: (0,1.5707963267948966) Reference: (nan,nan)
Test failed with complex_type: float Computed on device Output: (0,1.57079637) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (0,1.57079637) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on device Output: (0,1.5703) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on host Output: (0,1.5703) Reference: (nan,nan)
atanh test failed
Test failed with complex_type: double Computed on device Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: double Computed on host Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: float Computed on device Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: float Computed on host Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: sycl::half Computed on device Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: sycl::half Computed on host Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: double Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: double Computed on host Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on host Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: double Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: double Computed on host Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: sycl::half Computed on host Output: (0,1) Reference: (nan,-nan(ind))
tan test failed
Test failed with complex_type: double Computed on device Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: double Computed on host Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: float Computed on device Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: float Computed on host Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: sycl::half Computed on device Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: sycl::half Computed on host Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: double Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: double Computed on host Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: float Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on host Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: double Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: double Computed on host Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: float Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: sycl::half Computed on host Output: (1,0) Reference: (nan,nan)
tanh test failed

error: command failed with exit status: 1

On Windows (PI_OpenCL):

Complex/sycl_complex_math_test.cpp (SYCL_DEVICE_FILTER=host)

Test failed with complex_type: float Computed on device Output: (1.57079637,0) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (1.57079637,0) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on device Output: (1.57079637,0) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (1.57079637,0) Reference: (nan,-nan(ind))
atan test failed
Test failed with complex_type: float Computed on device Output: (0,1.57079637) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (0,1.57079637) Reference: (nan,nan)
Test failed with complex_type: float Computed on device Output: (0,1.57079637) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (0,1.57079637) Reference: (nan,nan)
atanh test failed
Test failed with complex_type: float Computed on device Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: float Computed on host Output: (0,1) Reference: (-nan(ind),nan)
Test failed with complex_type: float Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on device Output: (0,1) Reference: (nan,-nan(ind))
Test failed with complex_type: float Computed on host Output: (0,1) Reference: (nan,-nan(ind))
tan test failed
Test failed with complex_type: float Computed on device Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: float Computed on host Output: (1,0) Reference: (-nan(ind),-nan(ind))
Test failed with complex_type: float Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: float Computed on device Output: (1,0) Reference: (nan,nan)
Test failed with complex_type: float Computed on host Output: (1,0) Reference: (nan,nan)
tanh test failed

error: command failed with exit status: 1

Complex/sycl_complex_stream_test.cpp (SYCL_DEVICE_FILTER=host)

FileCheck error: '<stdin>' is empty.
FileCheck command line:  filecheck.exe ...\llvm-test-suite\SYCL\Complex\sycl_complex_stream_test.cpp -check-prefix=CHECK-ALL-TYPES

error: command failed with exit status: 2

… into sycl_complex

…tanh

AidanBeltonS · 2022-08-23T11:53:54Z

@steffenlarsen I now have access to the logs. Looking at the current results they appear to be failing due to some DiscardEvents tests and are unrelated to the SYCL Complex tests.

steffenlarsen · 2022-08-23T11:57:15Z

@steffenlarsen I now have access to the logs. Looking at the current results they appear to be failing due to some DiscardEvents tests and are unrelated to the SYCL Complex tests.

Those tests should be fixed with intel/llvm@c0d639e but test-suite testing was likely out-of-sync at the time. I think we can disregard them.

This PR adds testing for the experimental sycl complex implementation. It tests math functions, `abs`, `acos`, `asin`, `atan`, `acosh`, `asinh`, `atanh`, `arg`, `conj`, `cos`, `cosh`, `exp`, `log`, `log10`, `norm`, `polar`, `pow`, `proj`, `sin`, `sinh`, `sqrt`, `tanh`, and `tanh`. Additionally it tests the math operators `+`, `-`, `/`, `*` and their assign equivalents. It also tests stream operations with `sycl::stream`, `std::istream`, and `std::ostream`. For simplicity the tests are broken into the test and a list of test cases. Each test is run for the complex type with `double`, `float`, and `sycl::half`. Results are compared to values from `std::complex`. Due to differences in error handling for the `pow` function between, libstdc++ and libc++(which the implementation is based on), testing of NAN's and INF's of pow is not done through comparison with `std::complex` but instead using expected values. Depends on: intel/llvm#6309

This PR adds testing for the experimental sycl complex implementation. It tests math functions, `abs`, `acos`, `asin`, `atan`, `acosh`, `asinh`, `atanh`, `arg`, `conj`, `cos`, `cosh`, `exp`, `log`, `log10`, `norm`, `polar`, `pow`, `proj`, `sin`, `sinh`, `sqrt`, `tanh`, and `tanh`. Additionally it tests the math operators `+`, `-`, `/`, `*` and their assign equivalents. It also tests stream operations with `sycl::stream`, `std::istream`, and `std::ostream`. For simplicity the tests are broken into the test and a list of test cases. Each test is run for the complex type with `double`, `float`, and `sycl::half`. Results are compared to values from `std::complex`. Due to differences in error handling for the `pow` function between, libstdc++ and libc++(which the implementation is based on), testing of NAN's and INF's of pow is not done through comparison with `std::complex` but instead using expected values. Depends on: intel#6309

Add sycl complex testing

8215a02

AidanBeltonS requested a review from a team as a code owner June 15, 2022 09:25

AidanBeltonS requested a review from steffenlarsen June 15, 2022 09:25

AidanBeltonS mentioned this pull request Jun 15, 2022

[SYCL] Add sycl complex implementation intel/llvm#6309

Merged

aidan.belton added 2 commits June 15, 2022 10:33

clang format

5e41d14

Change return value

5f4e9e0

steffenlarsen reviewed Jul 5, 2022

View reviewed changes

AidanBeltonS and others added 3 commits July 19, 2022 09:35

Apply suggestions from code review

212e525

Co-authored-by: Steffen Larsen <[email protected]>

Measure by complex magnitude

8547d54

Update complex tests based on feedback

f211892

aidan.belton added 2 commits July 27, 2022 10:29

Check type aspects

4aadef9

Use numeric limits

f01f6b4

Modify complex host stream check

c549590

aidan.belton added 4 commits August 10, 2022 15:56

Merge branch 'intel' of https://github.com/AidanBeltonS/llvm-test-suite…

ed1c484

… into sycl_complex

Merge branch 'intel' of https://github.com/AidanBeltonS/llvm-test-suite…

f5cdf1d

… into sycl_complex

Use reference values for some error codes with tan, tanh, atan, and a…

8e78ad8

…tanh

remove stream filecheck

4fcdc8f

steffenlarsen approved these changes Aug 23, 2022

View reviewed changes

steffenlarsen merged commit 3a47f25 into intel:intel Aug 23, 2022

AidanBeltonS deleted the sycl_complex branch August 31, 2022 15:06

[SYCL] Add sycl complex testing #1058

[SYCL] Add sycl complex testing #1058

Uh oh!

Conversation

AidanBeltonS commented Jun 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AerialMantis commented Jul 5, 2022

Uh oh!

steffenlarsen commented Jul 5, 2022

Uh oh!

AerialMantis commented Jul 5, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AerialMantis commented Jul 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steffenlarsen commented Jul 6, 2022

Uh oh!

AerialMantis commented Jul 7, 2022

Uh oh!

AidanBeltonS commented Jul 20, 2022

Uh oh!

steffenlarsen commented Jul 25, 2022

Uh oh!

AidanBeltonS commented Jul 28, 2022

Uh oh!

steffenlarsen commented Jul 28, 2022

Uh oh!

AidanBeltonS commented Aug 23, 2022

Uh oh!

steffenlarsen commented Aug 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AidanBeltonS commented Jun 15, 2022 •

edited

Loading

AerialMantis commented Jul 6, 2022 •

edited

Loading