threads without kernel mods

[Simon Brown]

In article <13598 at dog.ee.lbl.gov> torek at elf.ee.lbl.gov (Chris Torek) writes:
>In article <4815 at skye.ed.ac.uk> richard at aiai.UUCP (Richard Tobin) writes:
>>... Once you have n stacks, you may be able to longjmp() between them.
>
>And then again, you may not....  Longjmp is only defined for going `up'
>the stack, and can (and does) abort programs that attempt to use it in
>the `wrong' direction.  When you have two separate stacks and attempt
>to jump from one to the other, you are going neither up nor down, but
>the `<' comparison done internally gives an answer anyway (which answer
>it gives depends on the relative addresses of the two separate stacks)
>and thus implementations that check (such as 4.xBSD-on-a-VAX) may stop
>your code dead, rather than changing stacks.

Indeed. However, all is not lost (except your lunch :-)).

The problem is that you can't jump down the stack, so the obvious answer is
to make sure that you always jump up. So, to jump from one state to another,
you have to arrange to temporarily execute on some stack that is lower than
*any* of the thread stacks, and you have to get there without using longjmp.
And this can be done by a signal trap (with sigstack set to use this lowest
stack), from which one does the longjmp, rather than jumping directly. 

It gets a bit nasty, though. In particular, you have to find some way of 
ensuring that this trampoline stack is lower than any other stack. I did this
(on an IBM RS/6000, which also does longjmp sanity checking) by arranging for
all the the thread stacks to be created by malloc, whereas the trampoline
stack is declared static - but of course this isn't particularly portable.
You also have to remember to clear the sigstack bit in the signal context,
so that the signal isn't blocked while you're executing in the thread.

Some example code:

	static Thread lwp_schedjmp; /* scheduler master state */
	static Thread *lwp_curjmp;  /* ptr to place we're trying to jump to */
	static char spring_stack[SPRING_STACK_SIZE]; /* trampoline stack */

	static void springboard (int);

	/*
 	 * reschedule some thread
	 */
	void thread_reschedule (Thread *thread)
	{
    	    struct sigstack sstk;
    	    struct sigvec svec;

            if (setjmp(lwp_schedjmp.thread) == 0) {
	        /*
 	         * we want to jump down to the appropriate stack, but longjmp 
		 * won't let us do this directly. we can do it indirectly by
	         * going via a signal handler which is being handled on a 
		 * stack which is even lower in memory than the required 
		 * lightweight stack. thus, once we're in the signal handler, 
		 * we'll be allowed to longjmp to where we want to be. 
	         */

		/* take a non-auto copy of where we want to go */
		lwp_curjmp = thread;

		/* arrange to trap USR1 on a very low stack */
		svec.sv_handler = springboard;
		svec.sv_mask = 0;
		svec.sv_flags = SV_ONSTACK;
		sstk.ss_sp = spring_stack + SPRING_STACK_SIZE;
		sstk.ss_onstack = 0;
		sigstack(&sstk, (struct sigstack *)0);
		sigvec(SIGUSR1, &svec, (struct sigvec *)0);

		/* raise USR1 which will cause a jump to the thread */
		kill(getpid(), SIGUSR1);
	    }
        }

	/*
	 * signal handler for indirect thread reschedule
	 */
	static void springboard (int signo)
	{
    	    struct sigstack state;

    	    /* get rid of sigstack context */
    	    if (sigstack((struct sigstack *)0, &state) == 0) {
		state.ss_onstack = 0;
		sigstack(&state, (struct sigstack *)0);
    	    }

	    /* jump */
    	    longjmp(lwp_curjmp->thread, 1);
	}

But this is pretty slow, as it needs to handle a signal on each reschedule.
And pretty disgusting too, come to think of it.

>It is probably best to continue to avoid setjmp/longjmp as their specifications
>have changed in the past, suggesting that they may change again in the 
>future.

Such as a change which causes longjmp to destructively unwind the stack, say.
(I seem to have a vague clouded memory that the V7 longjmp did this? Or maybe
it was 4.1BSD)

-------------------------------------------------------------------------------
Simon Brown                                                   simon at meiko.co.uk
Meiko Scientific Ltd.                                         simon at ed.ac.uk