Implement Stack-Overflow-Detection

[MabezDev]

The ESP32 specifically is a big problem here as it has its ROM function data in near the middle of the memory address space, meaning its quite easy to overwrite ROM function data on there.

[bjoernQ]

Maybe we already honor it on some chips - not sure. Definitely not on ESP32

Do you have any reference how esp-idf handles it?

[MabezDev]

Actually on a second look, maybe we do? I think this section stops us allocating .bss or .data in that ROM data region.

I think the issue me & @georgik were seeing was because of allocating a larger buffer on the stack, and the stack was overwriting the ROM function data. The question is how do we stop this? It's quite hard imo because of the dynamic nature of the size of the stack at any given point in the program.

[bjoernQ]

Oh yes - I also thought about such problems before. On some chips we have Debug Assist which apparently can monitor SP but not on ESP32

Also, the Xtensa Debug Option doesn't seem to offer something for that

[igrr]

On Xtensa chips, we can use one debug watchpoint to set a guard region at the end of the stack, covering e.g. 64 bytes. It's not perfect, but it has some probability of catching stack overflow.

On RISC-V chips, we can use the stack pointer monitor feature of debug assist, as @bjoernQ has suggested. (No IDF support to refer to, yet, but I can give some tips as I have tested this feature on FPGAs.)

[MabezDev]

We now have stack monitor hardware support, along with rust/llvm stack guarding for chips that don't support it. Closing this now.