hsakmt: Expose and use CWSR and Control stack sizes

[kentrussell]

This will allow VGPR size to be determined dynamically, instead of relying on hardcoding

Motivation

Technical Details

JIRA ID

Test Plan

Test Result

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[superm1]

Mostly to connect dots for anyone who looks at this later: this matches this kernel change.

[dayatsin-amd]

In thunk, when CwsrSize and CtlStackSize are not zero, we need to add code to use these values instead of trying to calculate them.

This would be in queues.c:update_ctx_save_restore_size

[kentrussell]

Right. That felt like it should be a separate change, and I wasn't confident enough in knowing where all those calculations had to happen

[jokim-amd]

"This will allow VGPR size to be determined dynamically, instead of relying on hardcoding"
If the purpose of this change is just to expose the KFD authority save area size internally to Thunk with the idea that a follow on patch will actually use mentioned authority, then the description should mention that (also just mention the overall size, no one really needs to know about VGPRs).

But I don't see a need to do this unless we're going to use actually use it during queue creation.

[jokim-amd]

"In thunk, when CwsrSize and CtlStackSize are not zero, we need to add code to use these values instead of trying to calculate them."

I think we should still calculate this in Thunk for existing products.
The KFD is also broken for GFX1151 in reporting the correct size (patch in flight to correct this so have no version control because it's a bug).
Also user space should be allowed to over allocate if it wants.
I think for any existing chips which have been relying on thunk should max(kfd_cwsr_size, thunk_cwsr_size).
Future chips currently in npi can just rely on kfd_cwsr_size.

[kentrussell]

"This will allow VGPR size to be determined dynamically, instead of relying on hardcoding" If the purpose of this change is just to expose the KFD authority save area size internally to Thunk with the idea that a follow on patch will actually use mentioned authority, then the description should mention that (also just mention the overall size, no one really needs to know about VGPRs).

But I don't see a need to do this unless we're going to use actually use it during queue creation.

Fair enough. I didn't know who would know which part needed to be done next, but I can reword the commit message accordingly. This isn't my area of expertise. But I can try to throw something together