Tests that "spoof" the CPU, in order to test all code paths?

[DilumAluthge]

If you look at the coverage for #6 (https://codecov.io/gh/JuliaLinearAlgebra/Octavian.jl/pull/6/diff), you'll see there are some code paths that are difficult to get coverage for.

For example, look at line 58 of src/block_sizes.jl. Line 58 is only executed if the condition 2_L2 * L2c > _L3 * L3c is true. If I understand correctly, all of these values are derived from VectorizationBase.CACHE_SIZE and VectorizationBase.CACHE_COUNT, which means that they are derived from the properties of the CPU.

Is there a way that we can "spoof" the CPU that is being used, so that we can make sure that we are testing all code paths?

[chriselrod]

It's true for one of my computers at home ;).

julia> using VectorizationBase

julia> _L1, _L2, __L3, _L4 = VectorizationBase.CACHE_SIZE
(32768, 1048576, 25952256, nothing)

julia> L1c, L2c, L3c, L4c = VectorizationBase.CACHE_COUNT
(18, 18, 1, 0)

julia> _L3 = something(__L3, 4194304)
25952256

julia> 2_L2 * L2c > _L3 * L3c
true

julia> versioninfo()
Julia Version 1.7.0-DEV.153
Commit 43f05ea984* (2020-12-24 14:32 UTC)
Platform Info:
  OS: Linux (x86_64-generic-linux)
  CPU: Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.0 (ORCJIT, cascadelake)
Environment:
  JULIA_NUM_THREADS = 18

That test is also kind of bad. Long tangent following:
It's meant to guess whether the L3 cache is inclusive of L2, i.e. does the L3 contain a copy of all memory in the L2?

For that cascadelake CPU, it doesn't. But for many other CPUs, it does.
But Hwloc.jl doesn't actually provide any information on that, so I was left trying to guess.

I used just two computers to try and guess at parameters: an old Haswell laptop, and that Cascadelake CPU.
One thing I noticed was that the Cascadelake liked to use a much larger percentage of its L3 cache size than the Haswell.
The Haswell's cache is inclusive, so much of the free space was eaten by the L2. The Cascadelake's L3 is non-inclusive, so all of its space was free. So it made sense to make those adjustments in determining block sizes.

But, this Tiger Lake's L3 cache is also non-inclusive:

julia> using VectorizationBase

julia> _L1, _L2, __L3, _L4 = VectorizationBase.CACHE_SIZE
(49152, 1310720, 12582912, nothing)

julia> L1c, L2c, L3c, L4c = VectorizationBase.CACHE_COUNT
(4, 4, 1, 0)

julia> _L3 = something(__L3, 4194304)
12582912

julia> 2_L2 * L2c > _L3 * L3c
false

julia> versioninfo()
Julia Version 1.7.0-DEV.168
Commit dedf290c99 (2020-12-25 20:12 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.0 (ORCJIT, tigerlake)

Yet my check returns false.
Additionally, block_sizes also assumes that the L2 is inclusive of L1, because it is on most CPUs, but it is not inclusive on this Tiger Lake (which also has relatively very large L1 and L2 caches, as you can see above).

I said earlier that Hwloc.jl doesn't help, but here is a copy/paste from a .xml file produced by hwloc (using hwloc-gather-topology /tmp/myhost):

      <object type="L3Cache" cpuset="0x000000ff" complete_cpuset="0x000000ff" nodeset="0x00000001" complete_nodeset="0x00000001" gp_index="8" cache_size="12582912" depth="3" cache_linesize="64" cache_associativity="12" cache_type="0">
        <info name="Inclusive" value="0"/>
        <object type="L2Cache" cpuset="0x00000011" complete_cpuset="0x00000011" nodeset="0x00000001" complete_nodeset="0x00000001" gp_index="7" cache_size="1310720" depth="2" cache_linesize="64" cache_associativity="20" cache_type="0">
          <info name="Inclusive" value="0"/>
          <object type="L1Cache" cpuset="0x00000011" complete_cpuset="0x00000011" nodeset="0x00000001" complete_nodeset="0x00000001" gp_index="5" cache_size="49152" depth="1" cache_linesize="64" cache_associativity="12" cache_type="1">
            <info name="Inclusive" value="0"/>
            <object type="L1iCache" cpuset="0x00000011" complete_cpuset="0x00000011" nodeset="0x00000001" complete_nodeset="0x00000001" gp_index="6" cache_size="32768" depth="1" cache_linesize="64" cache_associativity="8" cache_type="2">
              <info name="Inclusive" value="0"/>

See all the "Inclusive" value="0".
I've been looking through the code to try and figure out how to actually query this using Hwloc_jll.jl.
Still no idea because it doesn't actually seem to be a part of the API.

[chriselrod]

Maybe the easiest way would be just to figure out how to use cpu-id myself.

Related to the actual question, yes, it'd be great to spoof CPU features for testing purposes.

Maybe VectorizationBase should have an override option, where, for example, you can define an environmental variable for it to read a file you provided instead of one of them included in the actual repo.

[DilumAluthge]

Maybe VectorizationBase should have an override option, where, for example, you can define an environmental variable for it to read a file you provided instead of one of them included in the actual repo.

I think this could be useful. It would help us get 100% code coverage in Octavian.jl. But also, it could be helpful in cases where a user wants to really customize the CPU features that get passed to VectorizationBase.

It could also be helpful if e.g. someone wants to debug something, e.g. they want to reproduce a bug that only occurs on CPUs without AVX2, but all of the computers that they have access to have AVX2.

[chriselrod]

It could also be helpful if e.g. someone wants to debug something, e.g. they want to reproduce a bug that only occurs on CPUs without AVX2, but all of the computers that they have access to have AVX2.

Yes, that is something I want.

[DilumAluthge]

Fixed by JuliaSIMD/VectorizationBase.jl#46