fgrep execution time

[David Bailey]

To those involved in the maintenance of the grep utility,

Noticed a major difference in the execution times of gnu grep and xpg4
grep under Solaris 5.8.

The problem occurs with grep -F -f <searchFile> when searchFile is
empty. 

Here are some examples comparing gnu grep version 2.5.1 with the
standard Solaris 5.8 utilities.

==> Using /bin/fgrep

time yes "a very long file" | \
        head -10000 | fgrep -f /dev/null
0.02u 0.00s 0:00.08 25.0%

time yes "a very long file" | \
        head -100000 | fgrep -f /dev/null
0.03u 0.02s 0:00.66 7.5%

time yes "a very long file" | \
        head -1000000 | fgrep -f /dev/null
0.40u 0.07s 0:06.70 7.0%

About 7 seconds for a 17MB file.


==> Using Solaris /usr/xpg4/bin/grep 

time yes "a very long file" | \
        head -10000 | /usr/xpg4/bin/grep -F -f /dev/null
0.01u 0.00s 0:00.02 50.0%

time yes "a very long file" | \
        head -100000 | /usr/xpg4/bin/grep -F -f /dev/null
0.00u 0.01s 0:00.02 50.0%

time yes "a very long file" | \
        head -1000000 | /usr/xpg4/bin/grep -F -f /dev/null
0.00u 0.01s 0:00.02 50.0%

Looks like the XPG4 version is file size independent.


==> Using GNU grep

time yes "a very long file" | \
        head -10000 | /usr/local/bin/fgrep -f /dev/null
0.00u 0.01s 0:07.95 0.1%

time yes "a very long file" | \
        head -100000 | /usr/local/bin/fgrep -f /dev/null
0.05u 0.03s 1:27.24 0.0%

time yes "a very long file" | \
        head -1000000 | /usr/local/bin/fgrep -f /dev/null
0.27u 0.14s 14:35.35 0.0%

Over 14 minutes for a 17MB file.

A 500MB file takes over an hour.

If searchFile is non-empty, runtimes are similar.

Hope this helps.

Thanks for keeping things GNU.

David M. Bailey
Hitachi ULSI Systems
Fukuoka, Japan

bailey-david.vcf
Description: David Bailey ÌJ[h