I'm just starting to learn a bit of c and assembly to learn about buffer overflows (so go easy on me, this is my first dive into programming
At this point I'm really mostly following along with some training videos online and trying to get a grasp. The videos use an x32 system, what I'm doing it trying to learn it in both 32 and 64 bit at the same time so I have two virtual machines I compile similar code on (I replace syscall codes as needed for x64, etc). I'm learning about buffer overflows right now.
This tiny script is to force a return value of 20 after the program runs.
Anyway, so there is this code that on x32 I'm supposed to compile this way:
gcc -ggdb -mpreferred-stack-boundary=2 -o Code Code.c
This compiles fine. This is the code:
char shellcode = "\xbb\x14\x00\x00\x00"
ret = (int *)&ret +2;
(*ret) = (int)shellcode;
on x32 it works the same as the video, which is after it runs, if I check the return code, it shows as 20:
# echo $?
On the 64-bit machine, if I try to compile with the exact same command, I get error:
$ gcc -ggdb -mpreferred-stack-boundary=2 -o ShellCode ShellCode.c
ShellCode.c:1: error: -mpreferred-stack-boundary=2 is not between 4 and 12
Instead of -mpreferred-stack-boundary=2, I tested compiling with a setting of 4, and also all even numbers up to 12, and I also tested leaving that switch out completely I don't get an error at compile time when I test 4-12 or leaving this switch out, but I'm not getting a return code of 20 that I'm supposed to get if this script is working right. In fact I seem to get random return codes each time I run it.
On the 32-bit machine, I get a 20 every time.
I don't yet have a good handle of the math involved, so this could have to do with the +2 in the script too, I'm not sure.
Does anyone have any ideas about this error and what the significance of this switch actually is?