Split ScalarQuantizer code into independent parts#4296

Closed

mdouze wants to merge 6 commits intofacebookresearch:mainfrom

mdouze:export-D73037185

Contributor

mdouze commented Apr 15, 2025

Summary: Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

mdouze and others added 6 commits

April 13, 2025 14:41


          SIMDConfig object

Differential Revision: D72937710


          fvec_madd simd impl

dbf7a41

Differential Revision: D72937708


          a few more functions in distances_simd.cpp

84fa742

Differential Revision: D72937711


          moved IndexIVFPQ and IndexPQ to dynamic dispatch

96f05e6

Differential Revision: D72937709


          Use simdlib abstraction in ScalarQuantizer implementation, split off …

24ad793

…training code, split quantizer code into headers, Make headers more independent

Summary:
Move the interface of SIMD functions to use the simdXfloat32 API to mutualize code.
Begin splitting the ScalarQuantizer.cpp
Continue splitting. Purely in header files for now.

Differential Revision: D72945865


          Split ScalarQuantizer code into independent parts

Summary: Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

facebook-github-bot added the CLA Signed label

Contributor

facebook-github-bot commented Apr 15, 2025

This pull request was exported from Phabricator. Differential Revision: D73037185

facebook-github-bot added the fb-exported label

Contributor

alexanderguzhva commented Apr 21, 2025 •

edited

Loading

https://github.com/facebookresearch/faiss/pull/4296/files#diff-6b8e2e1bd2570626453d39cccd202853ebf5306d63aa04bf0ab5ed1d4ab54e9c should not be included
Same with the other .o and .out files

Contributor

alexanderguzhva commented Apr 21, 2025 •

edited

Loading

it is worth having an API that controls SIMDConfig::level at the runtime, at least to evaluate and compare the performance of various SIMD implementations in benchmarks

mdouze added a commit to mdouze/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

3c52b71

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

Contributor Author

mdouze commented Apr 24, 2025

superseded by #4309

mdouze closed this

Contributor

facebook-github-bot commented May 5, 2025

This pull request was exported from Phabricator. Differential Revision: D73037185

mdouze added a commit to mdouze/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

8287edc

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

mdouze added a commit to mdouze/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

4a5963d

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

algoriddle pushed a commit to algoriddle/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

07c6534

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

Contributor

facebook-github-bot commented Jun 4, 2025

This pull request was exported from Phabricator. Differential Revision: D73037185

mdouze added a commit to mdouze/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

3844c89

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

02e1779

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

51f6741

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

0727f63

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

663ddec

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

795b147

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

0608fca

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

b7a3309

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

e4aab93

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

9ff10e5

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

9c24626

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

71cafa7

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

5eb4bae

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

a4b25df

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

f09b54b

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

6a95591

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

b111694

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

36aca6a

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

fb6b860

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

a86074e

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

371258e

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

81cb42b

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

9da30c2

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

b00d732

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

317a762

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

1f7d82d

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

9f4010d

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

adb1927

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

dd38801

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

d2b70f9

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

4d05ffd

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

c414374

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

bf55f3d

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

e4ac005

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

c7a35bc

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

8caa4fe

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4557

96f10a6

)

Summary:
Pull Request resolved: facebookresearch#4557

Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Reviewed By: mnorris11

Differential Revision: D73037185

subhadeepkaran pushed a commit to subhadeepkaran/faiss that referenced this pull request


          Split ScalarQuantizer code into independent parts (facebookresearch#4296

)

Summary:
Pull Request resolved: facebookresearch#4296

Splits the ScalarQuantizer code into parts so that the AVX2 and AVX512 can be compiled independently.

Differential Revision: D73037185

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed fb-exported