Allow user to choose which kernel to use for nbit-CPU-TBE through env variables#2299
Closed
excelle08 wants to merge 1 commit into
Closed
Allow user to choose which kernel to use for nbit-CPU-TBE through env variables#2299excelle08 wants to merge 1 commit into
excelle08 wants to merge 1 commit into
Conversation
✅ Deploy Preview for pytorch-fbgemm-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D53275825 |
excelle08
added a commit
to excelle08/FBGEMM
that referenced
this pull request
Jan 31, 2024
… variables (pytorch#2299) Summary: Allow user to choose which kernel (asmjit / autovec / ref) to use for nbit-CPU-TBE through env variables. Choosing which kernel (autovec/asmjit/ref) to use for nbit-CPU-TBE Available kernels: * ref: non-optimized, reference implementation that focuses on correctness, not performance * asmjit: hand-optimized kernel by having asmjit emit SIMD instructions during runtime. Only supports x86_64 CPUs with AVX2/AVX512 instruction sets * autovec: the kernel written in regular C++ code but in a way that makes compilers easier to generate vectorized SIMD instructions out of it. Supports both x86_64 and aarch64 CPUs. Currently only available on Linux. How to set environment variables: * No environment variables: on x86_64 we will default to asmjit kernel, and on aarch64 and linux we will default to autovec. On non-linux aarch64 we will fall back to ref. * FBGEMM_NO_AUTOVEC=1: on aarch64 linux we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1: on x86_64 we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1 AND FBGEMM_FORCE_AUTOVEC=1: on x86_64 we will use autovec if these two variables are set at the same time. No effect on other platforms. * FBGEMM_FORCE_AUTOVEC will override FBGEMM_NO_AUTOVEC if they are set at the same time. * These variables are considered set as long as they exist, even if their content is set to "0", "no" or "false". Differential Revision: D53275825
excelle08
force-pushed
the
export-D53275825
branch
from
5f85252 to
d4d7c91
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D53275825 |
excelle08
added a commit
to excelle08/FBGEMM
that referenced
this pull request
Jan 31, 2024
… variables (pytorch#2299) Summary: Allow user to choose which kernel (asmjit / autovec / ref) to use for nbit-CPU-TBE through env variables. Choosing which kernel (autovec/asmjit/ref) to use for nbit-CPU-TBE Available kernels: * ref: non-optimized, reference implementation that focuses on correctness, not performance * asmjit: hand-optimized kernel by having asmjit emit SIMD instructions during runtime. Only supports x86_64 CPUs with AVX2/AVX512 instruction sets * autovec: the kernel written in regular C++ code but in a way that makes compilers easier to generate vectorized SIMD instructions out of it. Supports both x86_64 and aarch64 CPUs. Currently only available on Linux. How to set environment variables: * No environment variables: on x86_64 we will default to asmjit kernel, and on aarch64 and linux we will default to autovec. On non-linux aarch64 we will fall back to ref. * FBGEMM_NO_AUTOVEC=1: on aarch64 linux we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1: on x86_64 we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1 AND FBGEMM_FORCE_AUTOVEC=1: on x86_64 we will use autovec if these two variables are set at the same time. No effect on other platforms. * FBGEMM_FORCE_AUTOVEC will override FBGEMM_NO_AUTOVEC if they are set at the same time. * These variables are considered set as long as they exist, even if their content is set to "0", "no" or "false". Reviewed By: jspark1105 Differential Revision: D53275825
excelle08
added a commit
to excelle08/FBGEMM
that referenced
this pull request
Jan 31, 2024
… variables (pytorch#2299) Summary: Allow user to choose which kernel (asmjit / autovec / ref) to use for nbit-CPU-TBE through env variables. Choosing which kernel (autovec/asmjit/ref) to use for nbit-CPU-TBE Available kernels: * ref: non-optimized, reference implementation that focuses on correctness, not performance * asmjit: hand-optimized kernel by having asmjit emit SIMD instructions during runtime. Only supports x86_64 CPUs with AVX2/AVX512 instruction sets * autovec: the kernel written in regular C++ code but in a way that makes compilers easier to generate vectorized SIMD instructions out of it. Supports both x86_64 and aarch64 CPUs. Currently only available on Linux. How to set environment variables: * No environment variables: on x86_64 we will default to asmjit kernel, and on aarch64 and linux we will default to autovec. On non-linux aarch64 we will fall back to ref. * FBGEMM_NO_AUTOVEC=1: on aarch64 linux we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1: on x86_64 we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1 AND FBGEMM_FORCE_AUTOVEC=1: on x86_64 we will use autovec if these two variables are set at the same time. No effect on other platforms. * FBGEMM_FORCE_AUTOVEC will override FBGEMM_NO_AUTOVEC if they are set at the same time. * These variables are considered set as long as they exist, even if their content is set to "0", "no" or "false". Reviewed By: jspark1105 Differential Revision: D53275825
excelle08
force-pushed
the
export-D53275825
branch
from
d4d7c91 to
8dbc7c2
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D53275825 |
… variables (pytorch#2299) Summary: Allow user to choose which kernel (asmjit / autovec / ref) to use for nbit-CPU-TBE through env variables. Choosing which kernel (autovec/asmjit/ref) to use for nbit-CPU-TBE Available kernels: * ref: non-optimized, reference implementation that focuses on correctness, not performance * asmjit: hand-optimized kernel by having asmjit emit SIMD instructions during runtime. Only supports x86_64 CPUs with AVX2/AVX512 instruction sets * autovec: the kernel written in regular C++ code but in a way that makes compilers easier to generate vectorized SIMD instructions out of it. Supports both x86_64 and aarch64 CPUs. Currently only available on Linux. How to set environment variables: * No environment variables: on x86_64 we will default to asmjit kernel, and on aarch64 and linux we will default to autovec. On non-linux aarch64 we will fall back to ref. * FBGEMM_NO_AUTOVEC=1: on aarch64 linux we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1: on x86_64 we will use ref. On other platforms this will have no effect. * FBGEMM_NO_ASMJIT=1 AND FBGEMM_FORCE_AUTOVEC=1: on x86_64 we will use autovec if these two variables are set at the same time. No effect on other platforms. * FBGEMM_FORCE_AUTOVEC will override FBGEMM_NO_AUTOVEC if they are set at the same time. * These variables are considered set as long as they exist, even if their content is set to "0", "no" or "false". Reviewed By: jspark1105, jasonjk-park Differential Revision: D53275825
excelle08
force-pushed
the
export-D53275825
branch
from
8dbc7c2 to
6547a86
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D53275825 |
Contributor
|
This pull request has been merged in 3d7af4a. |
meta-codesync Bot
pushed a commit
that referenced
this pull request
Jan 13, 2026
Summary: X-link: https://github.com/facebookresearch/FBGEMM/pull/2300 Pull Request resolved: #5308 total_prefetch_duration previously only timed `_prefetch()`, missing `bounds_check_indices_kernel_v2` launched in `prepare_inputs()`. moved timer to wrap both calls for accurate prefetch timing. Reviewed By: q10 Differential Revision: D90425962 fbshipit-source-id: 0d2fef51c84dc01ee81e54eaa43bfc42e7153c10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Allow user to choose which kernel (asmjit / autovec / ref) to use for nbit-CPU-TBE through env variables.
Choosing which kernel (autovec/asmjit/ref) to use for nbit-CPU-TBE
Available kernels:
correctness, not performance
instructions during runtime. Only supports x86_64 CPUs with
AVX2/AVX512 instruction sets
way that makes compilers easier to generate vectorized SIMD
instructions out of it. Supports both x86_64 and aarch64 CPUs.
Currently only available on Linux.
How to set environment variables:
kernel, and on aarch64 and linux we will default to autovec.
On non-linux aarch64 we will fall back to ref.
platforms this will have no effect.
platforms this will have no effect.
use autovec if these two variables are set at the same time.
No effect on other platforms.
are set at the same time.
if their content is set to "0", "no" or "false".
Differential Revision: D53275825