[Feature]: Add ALiBi, Soft Cap, and Optimized Sliding Window Support to Spyre Attention

[jvlunteren]

Feature description

Extend Spyre attention with support for:

• ALiBi (Attention with Linear Biases)
• Soft Cap
• Optimized sliding-window attention: improving performance by skipping pages that fall outside the attention window

Motivation and context

Several widely used modern LLMs, including Mistral and Gemma models, depend on ALiBi, Soft Cap, or sliding-window attention. Implementing these features in Spyre Attention is necessary to support these models and expand backend coverage.

Proposed solution

No response

Checklist

• I have searched for similar feature requests