Skip to content

Navigation Menu

Appearance settings

View all features
- BY COMPANY SIZE
  Enterprises
  Small and medium teams
  Startups
  Nonprofits
- BY USE CASE
  App Modernization
  DevSecOps
  DevOps
  CI/CD
  View all use cases
- BY INDUSTRY
  Healthcare
  Financial services
  Manufacturing
  Government
  View all industries
View all solutions
- EXPLORE BY TOPIC
  AI
  Software Development
  DevOps
  Security
  View all topics
- EXPLORE BY TYPE
  Customer stories
  Events & webinars
  Ebooks & reports
  Business insights
  GitHub Skills
- SUPPORT & SERVICES
  Documentation
  Customer support
  Community forum
  Trust center
  Partners
View all resources
- COMMUNITY
  GitHub SponsorsFund open source developers
- PROGRAMS
  Security Lab
  Maintainer Community
  Accelerator
  GitHub Stars
  Archive Program
- REPOSITORIES
  Topics
  Trending
  Collections
- ENTERPRISE SOLUTIONS
  Enterprise platformAI-powered developer platform
- AVAILABLE ADD-ONS
  GitHub Advanced SecurityEnterprise-grade security features
  Copilot for BusinessEnterprise-grade AI features
  Premium SupportEnterprise-grade 24/7 support
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

sgl-project / sglang Public

Notifications You must be signed in to change notification settings
Fork 5.1k
Star 25.3k

Code
Issues 581
Pull requests 2k
Discussions
Actions
Security and quality
Insights

Additional navigation options

Code
Issues
Pull requests
Discussions
Actions
Security and quality
Insights

Add optimized native kernels in sgl-kernel#5150

Merged

zhyncs merged 39 commits intosgl-project:mainsgl-project/sglang:mainfrom

mingfeima:pr_native_kernels_for_cpumingfeima/sglang:pr_native_kernels_for_cpuCopy head branch name to clipboard

Apr 8, 2025

Conversation Commits39 (39)Checks Files changed

Merged

Add optimized native kernels in sgl-kernel#5150
zhyncs merged 39 commits intosgl-project:mainfrom
mingfeima:pr_native_kernels_for_cpu

Commits

Commits on Apr 8, 2025

Optimize `all_reduce` by porting the shared memory kernel of deepspeed (#6)

chunyuan-w
authored and
mingfeima
committed
add norm kernels for CPU
mingfeima
committed
add silu_and_mul kernels for CPU
mingfeima
committed
add grouped topk kernels for CPU
mingfeima
committed
add decode attention kernels for CPU
mingfeima
committed
add fused moe kernels for CPU
mingfeima
committed
decode attention: fix non-contiguous k_buffer and v_buffer
mingfeima
committed
add fused moe kernels for CPU: part 2
mingfeima
committed
decode attention: fix seq_len req_pool_indices dtypes using int64_t
mingfeima
committed
add extend attention kernel for CPU
mingfeima
committed
extend attention: fix bug in MLA when k_extend and k_buffer have different head number
mingfeima
committed
fused moe: fix when w13 has OC not multiples of 64
mingfeima
committed
add weight packed linear for bfloat16/float16 on CPU
mingfeima
committed
weight_packed_linear: remove out as input parameter
mingfeima
committed
convert_weight_packed: use int64_t for stride to avoid overflow
mingfeima
committed
add int8_scaled_mm for int8 W8A8 on CPU
mingfeima
committed
add biased_grouped_topk for CPU
mingfeima
committed
Add record_function for profiling (#14)

yanbing-j
authored and
mingfeima
committed
moe: apply avx512-bf16 tinygemm when M is small
mingfeima
committed
grouped_topk: add support for num_experts = 160, config from DeepSeekV2
mingfeima
committed
moe: change indexing from int32 to int64 to avoid overflow
mingfeima
committed
int8_scaled_mm: move dequant to per_token_quant_int8
mingfeima
committed
Add fused_moe int8 w8a8 support for CPU
mingfeima
committed
fused_add_rmsnorm: replace at::zeros with at::empty
mingfeima
committed
mv cpu source files from src/sgl-kernel/csrc/cpu to csrc/cpu
mingfeima
committed
biased_grouped_topk: fix correction_bias dtype, should be bfloat16 instead of float32
mingfeima
committed
Add bmm AMX and avx512-bf16 kernels on CPU
mingfeima
committed
Add RECORD_FUNCTION in bmm_cpu, int8 mm, per token quant (#22)

chunyuan-w
authored and
mingfeima
committed
Add rope.cpp and torch_extension_cpu.cpp from 47bc8df
mingfeima
committed
Add shared_expert for intel AMX
mingfeima
committed
Move empty from python to C++ (#25)

yanbing-j
authored and
mingfeima
committed
int8_scaled_mm: fuse quant A to reduce python overhead.
mingfeima
committed
decode_attention: use int64_t for indexing to avoid overflow in stride
mingfeima
committed
use setup_cpu.py for now
mingfeima
committed
remove debug print
mingfeima
committed
Add qkv_proj_with_rope which fused qkv projection with segment gemm and rope and layernorm on Intel AMX
mingfeima
committed
optimize all_gather (#33)

blzheng
authored and
mingfeima
committed
apply pre-commit format changes
mingfeima
committed
Merge branch 'main' into pr_native_kernels_for_cpu
zhyncs
authored

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Community
Docs
Contact

You can’t perform that action at this time.