How to define SYSCALL?

[Godzil]

I'm currently reversing an OS based on 8086/80186 that use it's own syscall convention. I have all the information about them as it is documented, and it would be nice if there where a way to explain to ghidra how they are working.

Basically, it work a bit like DOS do, there are some INT which are dedicated to some functions, and it use registers as parameter.

I'm quite sure that Ghidra have such facility, to automatically document syscall, and take them correctly in the decompiler (instead of using the pseudo function "swi")

Let's take an example, this function is calling one of the syscall:

                             void __cdecl16near _display_control(uint flags)
             void              <VOID>         <RETURN>
             uint              Stack[0x2]:2   flags 
                             _display_control 
megStart:e000:005f(c),             
       e000:a672 55              PUSH       BP
       e000:a673 8b ec           MOV        BP,SP
       e000:a675 8b 5e 04        MOV        BX,word ptr [BP + flags]
       e000:a678 b4 00           MOV        AH,0x0
       e000:a67a cd 12           INT        0x12
       e000:a67c 5d              POP        BP
       e000:a67d c3              RET

The function is decompiled into:

void __cdecl16near _display_control(uint flags)
{
  code *pcVar1;
  pcVar1 = (code *)swi(0x12);
  (*pcVar1)();
  return;
}

Which is almost useless as, the flags parameter is lost, and the swi construction is somewhat incorrect and some crucial values in registers (here AH which store which function to actually call) is also lost

Is there is a way to tell Ghidra more about that?
Is that is only possible for disassembly by using some script?
Is there is a way for the decompiled code to explain Ghidra that this is really a syscall and so display properly something about?

In that case I can ignore the function as I know what it is doing, but in some cases the syscall was inlined in another function call, and Ghidra get somewhat confused by it.

[nihilus]

Did you try disabling the dead code removal?

[xyzz]

Looking at how ia.sinc is implemented I think ghidra wants you to override function pointer signature for the pcVar1. If you do so and add proper arguments to the function, then it should decompile correctly and propagate the flags argument as well as the syscall number into the pseudo-call.

[Godzil]

Ok sorry for late answer, "eliminate unreachable code" is already disabled.

@xyzz I'm not sure to understand, it does not accept code to be a function pointer, and I don't see how to define a function pointer with __regcall anyway.

[AntiBlueQuirk]

Yeah, I have to agree. There doesn't seem to be any way to apply a function pointer type to the pcVar1 variable. I think this is mainly a deficiency of the function type definition system.

You can only apply function prototypes from the Data Type Manager, but defining function types there is quite limited:

• There's no way to specify custom calling conventions; only the 5 generic calling conventions are supported. (__stdcall and so on)
• Critically, at least when using the "Real-mode" decompiler, the size of the code pointers don't even match, so Ghidra refuses to apply the type to the pcVar1 variable.

Custom calling conventions are basically required to properly instruct Ghidra about things like DOS's int 21h. It takes a subfunction in AH, and then the other parameters vary wildly in which registers they use.

I was able to get around the second issue by modifying the slaspecs, but then when applying the function type, Ghidra just gets upset about the type and throws a WARNING at the top of the decompile: "Unable to use type for symbol temp_1002a0331e2".

[russdill]

Is there at least a way to tie a register to the swi/svc instruction manually so that it isn't completely ignored?

[saruman9]

Ghidra v9.1 support system calls ("Created new overriding reference types, which improve and extend the ability to override calls, jumps, and callothers") and added syscall-related exercises to Advanced class. For this reason I think the issue can be close.

[russdill]

I'm not seeing how this functionality can be used on a swi/svc instruction.

[saruman9]

Maybe I don't understand your question. ${GHIDRA_HOME}/docs/GhidraClass/Advanced/improvingDisassemblyAndDecompilation.pdf, P. 54, Exercise: System Calls doesn't answer your question?

[russdill]

Thanks, that is working. There's only one oddity, it always adds the svc number to the function call even though it isn't part of the signature.

[Godzil]

I don't think it is working for 16bit like 8086 INTs. (like DOS INT)

With Guidra 9.1, I've followed the tutorial, I can't apply the "syscall" calling convention as there is no such thing, and I don't really know where to do the reference. All I get from this code for example:

       e000:a66d b4 00           MOV        AH,KEY_PRESS_CHECK
       e000:a66f cd 11           INT        INT_KEY                                          
       e000:a671 c3              RET

(in this OS, there are multiples INTs used for various functions, and AH is the function number from that INT called)

Is this warning:

WARNING: Output of swi destroyed by override!

And this error in the decompiler window:

Exception while decompiling e000:a66d: Decompiler process died

What is the way to do here?