Out-of-bounds pointers

[Chris Torek]

[original question was about code similar to

	int array[K];
	p = &array[0];	/* ok */
	p--;		/* invalid */
]

>In article <217 at bbxsda.UUCP>, scott at bbxsda.UUCP (Scott Amspoker) writes:
>>We just got through an *extremely* long thread in comp.lang.c regarding
>>this issue ...

Actually, that thread was arguing about the invalid pointer value produced
by freeing a valid malloc()-derived pointer.  These are in fact entirely
different things: the freed pointer became invalid by the freeing action,
even though its bit pattern did not change, while the pointer above became
invalid by a pointer-arithmetic operation (which presumably changed its
bit pattern).

In article <2322 at munnari.oz.au> ok at cs.mu.oz.au (Richard O'Keefe) writes:
>I participated in that thread.  Basically it fizzled out because it was
>quite clear that the restriction isn't going to go away, so there wasn't
>any point in talking further.  But it also became clear that the 80286
>and 80386 are *NOT* examples of the problem; you wouldn't do pointer
>comparisons or pointer arithmetic in a segment register on an 80286
>because you _can't_.

Actually, in huge model, p-- really does do arithmetic on a value that
might be automatically reloaded into a segment register.  For instance,

	void f(void) {
		extern char foo[];
		extern int g(int);
		register char *p = foo + g(0);

		while (p != foo + g(1)) p--; /* strange timing loop, perhaps */
		*p++ = 'a'; *p++ = 'b'; *p = 'c';
	}

is as likely to use `es:di' for p as any other register pair (after
all, it is *a* pointer, and is the *only* pointer, and is used *as a*
pointer).  The comparison and decrement are in fact better done from
memory or from si:di, but some compilers are likely to simply say
`this looks like a pointer; we are in huge model; use es:di'.

>You can, if necessary, make your array one element longer than you need.

This is in fact the only certain solution for

	for (p = &array[K]; --p >= array;)

(i.e., change it to

	some_type array_[K+1];		/* dummy element at 0 */
	#define array (array_ + 1)	/* subscript range [-1..K-1] */
	for (p = &array[K]; --p >= array;)

or something equivalent).  Otherwise, if some_type is, e.g.,

	typedef struct enormous { char a[60000]; } sometype;

the operation `--p' will subtract 60,000 bytes from p on a VAX or Sun;
this could easily produce an underflow (32k of text, e.g.).  On a VAX,
it is possible to trap on integer underflow, including pointer
underflow, so you could rig a system to produce a runtime trap here.
-- 
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 454 7163)
Domain:	chris at cs.umd.edu	Path:	uunet!mimsy!chris