In SYS V Release 3, you just go to /etc/master.d/kernel and change the value of the ULIMIT variable (It is in terms of blocks, not bytes), and then cd /boot; mkboot -k KERNEL; touch /etc/system; and reboot the machine. The kernel gets rebuilt for you and you're done.