Fast file scan (long)

[Georg Nikodym]

In article <299 at lysator.liu.se> pen at lysator.liu.se (Peter Eriksson) writes:
>I`d like to know how to scan all files in a directory (and it's sub-
>directories) for a specific string (without regular expressions) as fast
>as possible with standard Unix tools and/or some special programs.
>
>(I've written such a program, and would like to compare my implementation
>of it with others.)
>
>(Using the find+fgrep combination is slooooow....)
>
>Any ideas?

Well if there aren't to many files (ie, less the 200-300) you can do
the following:

	fgrep "SEARCH_STRING" `find . -type f -print`

This will save the over head of restarting fgrep for each file.

Another way is:
	FILES=`find . -type f -print`
	for FILENAME in $FILES
	do
	  fgrep "SEARCH_STRING" $FILENAME
	done

This method is immune to list size limitations (because there is no list),
but fork/exec's a new fgrep for each file.

Another way is:
	find . -type f -exec fgrep "SEARCH_STRING" {} \;

This method is probably the one you first tried.  It's similar to the
example above and probably takes just as long (but I won't bet money
on either ;-) .

But the best method (ie. the one I used to use):

	dirlist=`find . -type d -print`

	for dir in $dirlist
	do
	  fgrep "SEARCH_STRING" $dir/*
	done

This method having a speed advantage over the second two examples,
without the list size limitation of the first (of course this limitation
really depends on your implementation of UNIX).

Hope that satisfied your curiousity,

Georg S. Nikodym  --  (416) 922-1937	|
SCO Canada, Inc.			| "Language is virus from outer space"
Toronto, Ontario			|             -William S. Burroughs
georgn at sco.COM				|

None of this information has been tested at all, USE CAUTION!!!!   ;-)