slow indexdb

Gabor,

Gabor Kiss writes:
> 
> 'indexdb -s RIPE' runs 8 hours on a 4 processor SparCenter 2000.
> Is this normal? What are your experiences?

That is indeed pretty slow:

I get:

$ time indexdb -c ripe.db
real  6:16:50.4
user  1:45:19.1
sys     38:31.7

on a pretty much unloaded SPARC 20 running Solaris. Note that I am also
using the '-c' option which slows down the process considerably.

My experience indicates that the slowness most likely has to do with slow
disk I/O or a slow dbm interface in perl.

As long as index and db files are as big as your internal memory, things
tend to be pretty fast, using all CPU available on the machine. Also,
using the Berkely DB package makes the system much slower (then the
default perl dbm package) while according to the documentation it is
supposed to be one of the fastest dbm packages ... you might be able to
gain some speed there by fiddling with the caching and other parameters
for the dbm package.

I would thus recommend to use a standard perl with a very small
OVERFLOWSIZE=256 setting in your configuration file. This setting will
cause a lot of small files but is actually faster, safer (with (n)dbm)
and more space efficient then a big setting.

Another approach is to disable to some of the indexed values that you
don't use often like some of the '-i' values in the ATTR definitions or
direct lookup values like e-mail addresses in the object definitions or
mirror the db only once a month (or after a failure) and get updates by
using the real-time mirroring feature.

David K.
---