Skip to content

Cache NPUKernel objects#2611

Merged
mawad-amd merged 17 commits intomainfrom
muhaawad/cache-objects
Sep 25, 2025
Merged

Cache NPUKernel objects#2611
mawad-amd merged 17 commits intomainfrom
muhaawad/cache-objects

Conversation

@mawad-amd
Copy link
Collaborator

@mawad-amd mawad-amd commented Sep 24, 2025

This PR implements function-level caching for JIT-compiled kernels using a global cache that stores compiled kernels based on function signatures, argument types and contents (for source code). This avoids redundant compilation by reusing cached kernels when the same function is called with identical argument types and shapes.

Example:

(ironenv) ➜  00_memcpy git:(muhaawad/cache-objects) ✗ rm -rf ~/.iron && python memcpy.py

Latency: 0.002165 seconds (2165.37 µs)
Effective Bandwidth: 61.98 GB/s

PASS!

(ironenv) ➜  00_memcpy git:(muhaawad/cache-objects) ✗ python memcpy.py 
Latency: 0.002128 seconds (2127.71 µs)
Effective Bandwidth: 63.08 GB/s

PASS!

Signed-off-by: Muhammad Awad <[email protected]>
Signed-off-by: Muhammad Awad <[email protected]>
Signed-off-by: Muhammad Awad <[email protected]>
Signed-off-by: Muhammad Awad <[email protected]>
Signed-off-by: Muhammad Awad <[email protected]>
Signed-off-by: Muhammad Awad <[email protected]>
@mawad-amd mawad-amd added this pull request to the merge queue Sep 25, 2025
Merged via the queue into main with commit 69ffd7a Sep 25, 2025
53 checks passed
@mawad-amd mawad-amd deleted the muhaawad/cache-objects branch September 25, 2025 23:53
fifield pushed a commit to fifield/mlir-aie that referenced this pull request Nov 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants