UPSTREAM PR #17943: feat(cuda): Add highly optimized CUDA kernel for HardSwish activation

[loci-dev]

Mirrored from ggml-org/llama.cpp#17943

What this PR does: Implements a highly optimized, custom CUDA kernel for the GGML_UNARY_OP_HARDSWISH activation function.

Why it is valuable: This enables faster inference for any model using the HardSwish activation when running on NVIDIA GPUs via the GGML CUDA backend.

Testing: Verified functionality on NVIDIA GeForce MX250. Test passes (OK/Mismatch) when memory-support logic is bypassed.

[loci-review]

Explore the complete analysis inside the Version Insights

Performance Analysis Summary: PR #525 - CUDA HardSwish Kernel Optimization

Condition Assessment: Condition 2 applies - Minor changes with limited scope affecting a single non-core activation function.

Summary

This PR replaces the template-based HardSwish activation implementation with a specialized CUDA kernel in ggml/src/ggml-cuda/unary.cu. The changes affect only the HardSwish activation path within the CUDA backend's unary operations. No modifications were made to core inference functions (llama_decode, llama_encode, llama_tokenize) or tokenization paths. The implementation removes F16 type support, handling only F32 tensors. An unrelated modification in ggml-cuda.cu removes the explicit return statement for GGML_OP_LOG, causing it to fall through to GGML_OP_SSM_SCAN's conditional logic. Power consumption analysis shows zero measurable change across all 16 binaries, with differences below 1 nanojoule. The CMakeLists.txt change removes a deprecated build option check without performance impact.

Inference Impact: No effect on tokens per second. HardSwish is an activation function used in specific model architectures (MobileNetV3, EfficientNet) but not in the tokenization or primary inference pipeline. The llama_decode, llama_encode, and llama_tokenize functions remain unmodified.