docs: add k-NN AVX512 support report for v2.18.0

docs/features/index.md

@@ -267,6 +267,7 @@
 ## k-nn
 
 - [Disk-Based Vector Search](k-nn/disk-based-vector-search.md)
+- [k-NN AVX512 SIMD Support](k-nn/k-nn-avx512-support.md)
 - [k-NN Byte Vector Support](k-nn/k-nn-byte-vector-support.md)
 - [k-NN Performance & Engine](k-nn/k-nn-performance-engine.md)
 - [Vector Search (k-NN)](k-nn/vector-search-k-nn.md)

docs/features/k-nn/k-nn-avx512-support.md

@@ -0,0 +1,146 @@
+# k-NN AVX512 SIMD Support
+
+## Summary
+
+The k-NN plugin supports AVX512 SIMD (Single Instruction Multiple Data) instructions for the Faiss engine, enabling hardware-accelerated vector operations on compatible x64 processors. This feature significantly improves vector search and indexing performance by leveraging modern CPU capabilities to process multiple data points simultaneously.
+
+## Details
+
+### Architecture
+
+```mermaid
+graph TB
+    subgraph "k-NN Plugin"
+        A[k-NN Plugin] --> B[JNI Layer]
+        B --> C{SIMD Library Selector}
+    end
+
+    subgraph "Native Libraries - x64"
+        C -->|AVX512 CPU| D[libopensearchknn_faiss_avx512.so]
+        C -->|AVX2 CPU| E[libopensearchknn_faiss_avx2.so]
+        C -->|No SIMD| F[libopensearchknn_faiss.so]
+    end
+
+    subgraph "Native Libraries - ARM64"
+        C -->|ARM64| G[libopensearchknn_faiss.so with NEON]
+    end
+
+    subgraph "Faiss Operations"
+        D --> H[Vector Distance Calculations]
+        E --> H
+        F --> H
+        G --> H
+        H --> I[Index Building]
+        H --> J[k-NN Search]
+    end
+```
+
+### Data Flow
+
+```mermaid
+flowchart LR
+    A[Query Vector] --> B[k-NN Plugin]
+    B --> C[JNI Bridge]
+    C --> D[SIMD-Optimized Faiss]
+    D --> E[Distance Calculations]
+    E --> F[Top-K Results]
+    F --> G[Search Response]
+```
+
+### Components
+
+| Component | Description |
+|-----------|-------------|
+| `libopensearchknn_faiss_avx512.so` | Faiss library with AVX512 SIMD instructions for maximum performance |
+| `libopensearchknn_faiss_avx2.so` | Faiss library with AVX2 SIMD instructions |
+| `libopensearchknn_faiss.so` | Non-optimized Faiss library (fallback) |
+| SIMD Library Selector | Runtime component that detects CPU capabilities and loads appropriate library |
+
+### Configuration
+
+| Setting | Type | Description | Default |
+|---------|------|-------------|---------|
+| `knn.faiss.avx512.disabled` | Static | Disable AVX512 library and fall back to AVX2 or non-optimized | `false` |
+| `knn.faiss.avx2.disabled` | Static | Disable AVX2 library and fall back to non-optimized | `false` |
+
+### Supported Platforms
+
+| Platform | Architecture | SIMD Support |
+|----------|--------------|--------------|
+| Linux | x64 | AVX512, AVX2, or none (auto-detected) |
+| Linux | ARM64 | NEON (always enabled) |
+| macOS | x64 | AVX2 only (AVX512 not available on Mac hardware) |
+| Windows | x64 | Not supported |
+
+### Usage Example
+
+AVX512 is automatically enabled when the hardware supports it. No configuration is required.
+
+To verify SIMD support, check the OpenSearch logs at startup:
+
+```
+[INFO ][o.o.k.j.JNIService] Loading native library: libopensearchknn_faiss_avx512.so
+```
+
+To disable AVX512 (e.g., for testing or compatibility):
+
+```yaml
+# opensearch.yml
+knn.faiss.avx512.disabled: true
+```
+
+### Building with SIMD Support
+
+```bash
+# Build OpenSearch k-NN with SIMD
+./gradlew build -Davx2.enabled=true -Davx512.enabled=true
+
+# Run with SIMD
+./gradlew run -Davx2.enabled=true -Davx512.enabled=true
+
+# Build JNI libraries directly
+cd jni
+cmake . -DAVX2_ENABLED=true -DAVX512_ENABLED=true
+make -j4
+```
+
+### Performance Characteristics
+
+| SIMD Level | Bits per Operation | Floats per Operation | Relative Performance |
+|------------|-------------------|---------------------|---------------------|
+| AVX512 | 512 | 16 | Highest |
+| AVX2 | 256 | 8 | Medium |
+| None | 32-64 | 1-2 | Baseline |
+
+SIMD optimization is most effective when:
+- Vector dimensions are multiples of 8
+- Large datasets require many distance calculations
+- Workloads are CPU-bound rather than I/O-bound
+
+## Limitations
+
+- AVX512 is only available on Linux x64 platforms
+- Windows does not support SIMD optimization for Faiss
+- macOS does not support AVX512 due to hardware limitations
+- Vector dimensions should be multiples of 8 for optimal SIMD utilization
+- Settings are static and require cluster restart to change
+- ARM64 platforms use NEON instructions (cannot be disabled)
+
+## Related PRs
+
+| Version | PR | Description |
+|---------|-----|-------------|
+| v2.18.0 | [#2110](https://github.com/opensearch-project/k-NN/pull/2110) | Add AVX512 support to k-NN for FAISS library |
+| v2.13.0 | - | Initial SIMD support with AVX2 and NEON |
+
+## References
+
+- [Issue #2056](https://github.com/opensearch-project/k-NN/issues/2056): Feature request for FAISS AVX512 support
+- [k-NN Index Documentation](https://docs.opensearch.org/2.18/search-plugins/knn/knn-index/#simd-optimization-for-the-faiss-engine): Official SIMD documentation
+- [k-NN Settings](https://docs.opensearch.org/2.18/search-plugins/knn/settings/): Configuration reference
+- [Blog: Boosting k-NN exact search performance](https://opensearch.org/blog/boosting-k-nn-exact-search/): Performance analysis with SIMD
+
+## Change History
+
+- **v2.18.0** (2024-10-22): Added AVX512 SIMD support for Faiss engine on x64 Linux
+- **v2.13.0**: Initial SIMD support with AVX2 (x64) and NEON (ARM64)

docs/releases/v2.18.0/features/k-nn/k-nn-avx512-support.md

@@ -0,0 +1,115 @@
+# k-NN AVX512 Support
+
+## Summary
+
+OpenSearch v2.18.0 adds AVX512 SIMD (Single Instruction Multiple Data) support to the k-NN plugin for the Faiss engine. This enhancement accelerates vector search and indexing operations on x64 processors that support AVX512 instructions, providing significant performance improvements over AVX2 for compatible hardware.
+
+## Details
+
+### What's New in v2.18.0
+
+- Added AVX512 SIMD instruction support for the Faiss library
+- Introduced a new native library `libopensearchknn_faiss_avx512.so` optimized for AVX512
+- Added automatic runtime detection and loading of the appropriate SIMD-optimized library
+- New cluster setting `knn.faiss.avx512.disabled` to control AVX512 usage
+- Updated build system to support AVX512 compilation flags
+
+### Technical Changes
+
+#### Architecture Changes
+
+```mermaid
+graph TB
+    subgraph "k-NN Plugin Startup"
+        A[Plugin Initialization] --> B{Check CPU Capabilities}
+        B -->|AVX512 Supported| C[Load libopensearchknn_faiss_avx512.so]
+        B -->|AVX2 Only| D[Load libopensearchknn_faiss_avx2.so]
+        B -->|No SIMD| E[Load libopensearchknn_faiss.so]
+    end
+
+    subgraph "Vector Operations"
+        C --> F[AVX512 Distance Calculations]
+        D --> G[AVX2 Distance Calculations]
+        E --> H[Standard Distance Calculations]
+    end
+```
+
+#### New Components
+
+| Component | Description |
+|-----------|-------------|
+| `libopensearchknn_faiss_avx512.so` | Faiss library compiled with AVX512 SIMD instructions |
+| AVX512 CPU detection | Runtime detection of AVX512 instruction set support |
+| Build flags | New `-Davx512.enabled` and `-DAVX512_ENABLED` cmake flags |
+
+#### New Configuration
+
+| Setting | Description | Default |
+|---------|-------------|---------|
+| `knn.faiss.avx512.disabled` | Static setting to disable AVX512 library loading | `false` |
+| `knn.faiss.avx2.disabled` | Static setting to disable AVX2 library loading | `false` |
+
+### Library Selection Logic
+
+The k-NN plugin automatically selects the best available library at runtime:
+
+1. If AVX512 is supported and not disabled → `libopensearchknn_faiss_avx512.so`
+2. If AVX2 is supported and AVX512 unavailable/disabled → `libopensearchknn_faiss_avx2.so`
+3. Otherwise → `libopensearchknn_faiss.so` (non-optimized)
+
+### Usage Example
+
+AVX512 support is enabled by default. To disable it, add to `opensearch.yml`:
+
+```yaml
+knn.faiss.avx512.disabled: true
+```
+
+For building with AVX512 support:
+
+```bash
+# Build with AVX512 enabled
+./gradlew build -Davx2.enabled=true -Davx512.enabled=true
+
+# Build JNI libraries with AVX512
+cd jni
+cmake . -DAVX2_ENABLED=true -DAVX512_ENABLED=true
+```
+
+### Performance Benefits
+
+SIMD optimization is applicable when the vector dimension is a multiple of 8. Performance ranking:
+
+**AVX512 > AVX2 > No optimization**
+
+The AVX512 instructions process 512 bits (16 floats) per operation compared to AVX2's 256 bits (8 floats), potentially doubling throughput for distance calculations.
+
+### Migration Notes
+
+- No migration required - AVX512 is automatically used when available
+- Existing indexes work without modification
+- To disable AVX512 on specific nodes, set `knn.faiss.avx512.disabled: true` in `opensearch.yml` before starting the cluster
+
+## Limitations
+
+- AVX512 support is only available on Linux (not Windows or macOS)
+- Requires x64 processors with AVX512 instruction set support
+- SIMD optimization only applies when vector dimensions are multiples of 8
+- Static setting requires cluster restart to change
+
+## Related PRs
+
+| PR | Description |
+|----|-------------|
+| [#2110](https://github.com/opensearch-project/k-NN/pull/2110) | Add changes for AVX-512 support in k-NN |
+
+## References
+
+- [Issue #2056](https://github.com/opensearch-project/k-NN/issues/2056): Feature request for FAISS AVX512 support
+- [k-NN Index Documentation](https://docs.opensearch.org/2.18/search-plugins/knn/knn-index/): SIMD optimization for the Faiss engine
+- [k-NN Settings Documentation](https://docs.opensearch.org/2.18/search-plugins/knn/settings/): AVX512 configuration settings
+- [Blog: Boosting k-NN exact search performance](https://opensearch.org/blog/boosting-k-nn-exact-search/): Performance improvements with SIMD
+
+## Related Feature Report
+
+- [Full feature documentation](../../../../features/k-nn/k-nn-avx512-support.md)

docs/releases/v2.18.0/index.md

@@ -113,6 +113,7 @@ This page contains feature reports for OpenSearch v2.18.0.
 
 ### k-NN
 
+- [k-NN AVX512 Support](features/k-nn/k-nn-avx512-support.md) - AVX512 SIMD support for Faiss engine, accelerating vector search and indexing on compatible x64 processors
 - [k-NN Performance & Engine](features/k-nn/k-nn-performance-engine.md) - Default engine changed to FAISS, approximate threshold updated to 15K, rescoring improvements, memory management enhancements
 - [k-NN Documentation](features/k-nn/k-nn-documentation.md) - JavaDoc cleanup for RescoreContext class
 - [k-NN Maintenance](features/k-nn/k-nn-maintenance.md) - Lucene 9.12 codec compatibility, force merge performance optimization, benchmark folder removal, code refactoring