Re: memory leak in syncdb

[davidk@localhost]

Wojtek,

Wojtek Sylwestrzak writes:
>  
> > Syncdb -- isn't that a Perl script?  What version of Perl are you running?
> > Any Perl5 version before 5.004_04 contains memory leaks of some sort, and
> > even 5.004_04 probably still contains some.  Perl 4.036 seemed not too
> > bad, but that's an ancient version now, so I wouldn't bet on it.
> 
> Yes, I experienced various sort of memory leaks in earlier versions
> of perl 5, but currently I'm running 004_04/solaris
> and most of the memory problems with other programs vanished.
> 
> still syncdb is 0.5GB in size which seems much for an innocent 
> script like it is. But probably you're right blaming in on perl.
> 
> I'm curious if anyone else experienced this with syncdb and found
> any solution to it ?

I have been running several versions of perl5 on Solaris, but have not
experienced the problems with syncdb that you describe. All versions of
perl have some problems with memory leaks though (And Solaris in general
is a problem too, some interesting interaction between an easy to
misconfigure perl and Solaris would not surprise me). It is recommended
to call syncdb from a cronjob as a one time job or with parameters that
will query the server for less then 1000 times from a shell script that
restarts syncdb automatically after it has exited.

There might be another reason for your problem though. You indicate that
you are running the script every day succesfully with enormous memory
consumption. I am suspecting something else if the script doesn't
complete at all. The previous version (2.x) of the software had for sure
a bug that theoretically could cause a lot of memory consumption. Another
reason might be that you are querying for a too large set of updates
which could be caused by a configuration error at your end. The NCC
should be able, using the database logs, to assist you in debugging such
a problem.

David K.
---