1

I pruned everything to the min to show this problem

In Compaq Visual Fortran I write

    Do 1 I =1,N
    CALL SUBNAM(A,B,100)
1   CONTINUE

If "SUBNAM" is any routine written in Fortran , the DO loop count N can be as big as you like. N=100000 is no problem.

If SUBNAM is an assembler (Goasm) routine, even as simple as

_SUBNAM@12:  Push Ecx
             Pop  Ecx
             Ret

then N greater than about 86000 gives stack overflow. Even though the stack pointer is NOT corrupted!

If the assembler routine is only

_SUBNAM@12: RET

I do not get stack overflow with N as high as 100000

WTF???

Information: Fortran "decorates" subroutine names with a leading underscore and @mm at the end where mm is 4 times the number of arguments. This is the amount of stack in bytes it uses for pushing argument addresses. It is the calling program's responsibility to clear up the stack upon being returned to. I have peeked at the stack before and after the call to SUBNAM and the stack pointer does return to the pre-Call value.

I have disabled all traceback compiler options in case debugging info was being crested at every call. It may be, and maybe a Fortran SUBNAM includes code to erase the traceback info that the call to it created?

Does the Pentium processor have another stack that could be overflowing, other than ESP?

Improve this question
4
  • 3
    stdcall is a callee-pops convention. The @12 means you should be using ret 12 because that's how much the caller expects this function to move ESP by.
    – Peter Cordes
    Commented Jan 24, 2022 at 7:56
  • 1
    BTW, if you look at the compiler-generated asm for the function that doesn't break, you'll see the ret 12 there.
    – Peter Cordes
    Commented Jan 24, 2022 at 20:33
  • You are dead right! Thanks. I have mixed assembler and Fortran for years without knowing that, and never had problems until now, due to playing with very large do loop count. Ret 12 indeed solved the probblem. Don't know why , when I peeked the stackpounter value, that I didn't see it growing though!
    – Paul
    Commented Jan 24, 2022 at 21:36
  • I'm really surprised your code hasn't broken before; returning with ESP pointing to the wrong place will break the caller's access to its local vars, unless you compile with -fno-omit-frame-pointer or equivalent so those accesses are relative to EBP. Maybe for 32-bit Windows, that's the default even with optimization enabled? (Unlike for Linux).
    – Peter Cordes
    Commented Jan 24, 2022 at 21:45
Related questions
5
Help on VGA and putpixel intel x86 asm AT&T syntax
7
Stack overflow on subroutine call only when compiled with Intel Visual Fortran and fine when compiled by Compaq Visual Fortran
1
can GOTO create an overflow in stack memory? StackOverflow?
Load 5 more related questions Show fewer related questions

0

Reset to default