UPSTREAM PR #18151: ggml-hexagon: gelu optimization

[loci-dev]

Mirrored from ggml-org/llama.cpp#18151

Following the discussion regarding the refactor idea here, I have re-evaluated the approach. Upon reviewing the current implementations for activation, unary, and binary functions, I observed that the existing code heavily relies on L2 prefetching while under-utilizing the VTCM and DMA.

While L2 prefetching offers some benefits, it is limited by two main factors:

1. The L2 cache is significantly smaller than the VTCM.
2. The L2 cache is shared between L1 program instructions and the L1 data cache.

This PR optimizes the code (using GELU as the initial implementation) by shifting the workload to heavily utilize the VTCM and DMA, thereby freeing up the L2 cache for L1 instruction and data cache

Optimization Strategy

Instead of relying on L2 prefetching, this implementation employs a DMA ping-pong buffering approach:

1. Fetch: DMA moves data from DDR to VTCM (buffer A).
2. Compute: DSP processes data in VTCM (buffer B) while DMA fetches the next chunk into buffer A.
3. Write back :DMA moves processed data from VTCM back to DDR.

This allows for overlapping computation and memory transfer, resulting in significantly higher throughput.

Performance Benchmarks

The performance improvements are significant. Below is a comparison between the existing implementation and the new DMA/VTCM approach:

Dimensions	Old Implementation (L2 Prefetch)	New Implementation (DMA/VTCM)
4096 x 4096	~5000 µs	~3800 µs
4096 x 4304	~5300 µs	~3700 µs

NOTE: I used the GELU as an example, but this approach can easily extend to other operations.

Unaligned Load Resolution:
This approach inherently solves the unaligned load issues encountered previously. Since data is fetched from DDR via DMA, the DMA engine ensures that data is stored into aligned addresses within the VTCM, even if the source data in DDR is unaligned.

@max-krasnyansky