Stack overflow! Increasing the stack size in OSX (beyond ulimit's 64Mb and up to 1Gb)

Some C programmers instead of allocating memory for their arrays

  {
  float *all = malloc(N*sizeof(float));
  ...
  free(all);
  }

like to use variable length arrays 

  {
  float all[N];
  ...
  }

the resulting code is less verbose and there is no need for freeing the arrays since it is returned at the end of the block. These variable length arrays are stored in the stack.
The issue with the stack is that it has a determined maximum size for an execution of the program, and when the stack runs out of memory the program crashes with a SEGAULT exeption.

When this happens the solution is to increase the stack size with ulimit -s, and run the program again. Or add the ulimit command to the .bash_profile, so that all the applications run with a big enough stack.

The problem occurs when we what to use, let's say, 100Mb of the stack, in Linux this is not an issue since the stack can be set to unlimited with ulimit -s 0 and it can grow to be as large as the phisical memory. But in OSX the maximum value for ulimit is harcoded in the kernel, and for a 32bit system it is 64MB. In practice it is even less, the maximum value of the stack I'm allowed to set is:

  ulimit -s 52800

There are a couple of methods to overcome the stack size limit in OSX.
The first one involves recompiling the kernel and I'm not going to discuss it here (see the reference).
The second one is simpler but involves recompiling the the application. It consists in indicating the linker (ld) the size and location of the stack for a particular application.
For instance to use a 256Mb stack compile the program with:

  gcc -o test256M big_array.c -Wl,-stack_size,0x10000000,-stack_addr,0xc0000000

The flag -stack_size,0x40000000 indicates the size of the stack, in this case 1Gb.
And -stack_addr,0xf0000000 sets the base address (4Gb which is the maximum for a 32bit system), also remember that the stack grows backwards.

For a 1Gb stack compile with:

  gcc -o test1G huge_array.c -Wl,-stack_size,0x40000000,-stack_addr,0xf0000000

I could not manage to use use that 1.5Gb of stack memory in a system with 4Gb. Anyway this should be sufficient to satisfy these stack hungry programs under OSX. On the same system a single malloc can grow well beyond 3Gb. 

Source:  How to increase the stack size on MAC OS X