Re: [Chicken-hackers] need irregex benchmark

[matt welland]

Hi Felix,

It isn't a particularly complex benchmark but my logpro app relies
heavily on regexes and I'm seeing some somewhat slow performance. You
can get it here http://www.kiatoa.com/fossils/logpro. We process some
very large log files and have many waivers, ignores and error patterns.
The procedure using 50% of the cycles, misc:line-match-regexs, merely
applies a list of regexes to a line of text looking for the first match.
I suspect there is a better way (suggestions welcome).

Anyhow, I can't share our logpro and log files but a quick and dirty way
to exercise it would be the following:

cat `find /var/log/ -name \*.log` > /tmp/example.log

wc example.log
27039  170297 1757848 example.log

time logpro ~/data/logpro/example.logpro < /tmp/example.log > /dev/null
10.53s user 0.07s system 99% cpu 10.605 total

>From the profile
procedure                calls  seconds  average  percent
---------------------------------------------------------
analyze-logfile              1   12.352   12.352  100.000
misc:line-match-regexs  112641    6.420    0.000   51.975
html-print               27231    0.328    0.000    2.655
expect:get-type-info        37    0.000    0.000    0.000
trigger                      5    0.000    0.000    0.000
expect                       4    0.000    0.000    0.000
section                      3    0.000    0.000    0.000
adj-active-sections          2    0.000    0.000    0.000
process-log-file             1    0.000    0.000    0.000
print-results                1    0.000    0.000    0.000
expect:required              1    0.000    0.000    0.000
expect:ignore                1    0.000    0.000    0.000
expect:error                 1    0.000    0.000    0.000
setup-logpro                 0    0.000    0.000    0.000
trigger-with-limit           0    0.000    0.000    0.000
expect:warning               0    0.000    0.000    0.000

On a separate note I'd like to turn logpro and megatest into egg apps
someday. I read the distributed egg system docs and will give it a go
one of these days....

Thanks,

Matt
-=-
On Sun, 2011-05-22 at 16:53 +0200, Felix wrote:
> Hello!
> 
> 
> I'm looking for code that stresses irregex heavily, because I'd like
> to test optimizer-improvements and because irregex (being very
> portable and generic code without lowlevel hacks) needs every help the
> compiler can give.
> 
> So far I've used the test-suite, but it executes to fast and doesn't
> give any real indication about performance of irregex core operations.
> 
> Any pointers to code, or heavy-duty regexen that need long to execute
> would be very welcome.
> 
> 
> cheers,
> felix
> 
> _______________________________________________
> Chicken-hackers mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/chicken-hackers