Re: newbie: log rotation, compression and deletion

[David E. Nelson]

Hi Philippe,

On Fri, 10 Jun 2005, Philippe de Rochambeau wrote:

> Hello,
>
> I need to rotate, compress and delete old copies of logs written to, by 
> on-going perl scripts.
>
> When cfengine rotates logs, how does it know which applications/scripts 
> created the logs?

It doesn't know nor does it need to care.

> Assuming that it somehow figures out the name of the log-writing 
> applications/scripts (via a cfengine configuration file, perhaps), how does 
> it tell them to temporarily stop writing to the logs while cfengine rotates 
> them, compresses them, and deletes the old logs, and to resume writing 
> afterwards?

When CFEngine rotates files, I believe it preserves the original location of 
the log file by copying the data to another file then truncating the original 
log file.  Since log files alway appended to, the application should then 
seek() to the end of the file prior to writing any data.

> Do you need to write a HUP signal handler in your log-writing scripts for 
> cfengine to send signals to?

I don't believe so unless the application isn't properly written to handle 
truncated log files.  Which, IMHO, is not good practice.  I guess an 
application would be written to rotate its own log files via signals (HUP, 
USR1, USR2, etc) but that would have to be documented.

> If so, could someone please provide examples of such HUP-signal handlers?
>
> How do you tell cfengine which compression program to use (compress, gzip, 
> etc.)?

I've been using the following w/ good success.

disable:

  solaris.Sunday.Hr00.Min00_05::

    "/var/log/syslog" rotate=5

    "/var/adm/messages" rotate=5

shellcommands:

  any::

    "/usr/local/bin/gzip -q -f /var/log/*.[0-9] > /dev/null 2>&1"
      inform=false

  solaris::

    "/usr/local/bin/gzip -q -f /var/adm/*.[0-9] /var/cron/*.[0-9] > /dev/null 
2>&1"
      inform=false

I check for and compress log file each time CFEngine is run as it is common for 
an admin to uncompress and examine a log file but fail to compress it when 
done.

> I also need to compress and delete Apache logs which are created with 
> cronolog. Cronolog basically creates a new log file each day whose name 
> contains the current date.
>
> How can I tell cfengine to compress and delete those logs without rotating 
> them?

Don't specify the log files in a 'disable:' section and only use a 
'shellcomands:' section as shown above.  You might also include a filter that 
returns files whose mtime is greater than 1 day (don't want to compress an 
active log file).

> How can I tell it to only compress log files whose names contain dates 
> greater than 7 days ago and less than 15 days ago? For instance,

Well, this can be simple or complex.

Simple: Use a filter (see the CFEngine reference) to make a decision based upon 
the mtime of the file.  Problem with this is that the filename is not taken 
into account as to the age of the file.

Complex: Write a CFEngine module that uses Perl w/ the Date::Calc module (or 
similar calender calculator) to analyze the filename and make decisions based 
upon that to either compress or delete.

Good luck,
         /\/elson

> Should be deleted:
>
> log20050527.log
> log20050528.log
>
> Should be gzipped:
>
> log20050529.log
> log20050530.log
> log20050531.log
>
> Should be left as-is:
>
> log20050601.log
> log20050602.log
> log20050603.log
> log20050604.log
> ...
>
> (cronolog renamed this file)
>
> log20050610.log
>
>
>
>
> Many thanks.
>
> Philippe
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Help-cfengine mailing list
> Help-cfengine@gnu.org
> http://lists.gnu.org/mailman/listinfo/help-cfengine

--
~~ ** ~~ 4 out of 3 people have problems with fractions ~~ ** ~~