How to rotate a text file

[jmiturbe]

Hello,

We are redirecting the standard output of an application to a text file. We would like to rotate this "log" file.

Actually we copy the file content to another file and then execute a cat /dev/null >> NAME_OF_THE_LOG_FILE

The problem is that the application dosn't "start" writing on the first line. It continues writing after the last line number wroten before the execution of cat /dev/null.

Any ideas on how to solve this issue.

Thanks in advance,

Joseba M. Iturbe

 

[KenCunningham]

It sounds like an application problem, if indeed it is a problem. Presumably the application was written so that the logs are sequentially numbered, so that a trace is possible in case of errors. Or perhaps I'm misunderstanding the question?

 

[hoinz]

Hi,

Not sure what you mean with 'It continues writing after the last line number'.
A solution will depend on how exactly your application is working.
You may try an mv instead of a cp.
If you are lucky, your application will create a new file with the old name.
If your application continues to write to the old file (in spite of the new name) you might consider restarting it. (Restarting often is not an option though.)

hope this helps

 

[hoinz]

ust spotted a strange thing:
/dev/null >> NAME_OF_THE_LOG_FILE
One > should be enough, shouldn't it?

 

[jmiturbe]

Thanks for answering,

We are not talking about the own logs of the application.

We also need the standard output of the app to be written into a file, so we just redirected its output to a file.

The app "displays" a lot of info, so we need to rotate this redirected output because of the size of the generated files.

For this purpose we created a shell script that is executed periodically (cron) and that basically copies the content of the file into another file and then tries to delete the copied info performin a

cat /dev/null >> NAME_OF_THE_FILE

This way we get a new file with thousands of blank lines. And this is what we exactly want to correct. We like to optimize this script or find another way to rotate this test file. That is, to imitate the operation of any application log rotate.

Thanks,

Joseba

 

[jmiturbe]

Upss,

Yes,one > is enough!

Joseba

 

[hoinz]

... and it should do what you want?

 

[jmiturbe]

Hi,

We used only one >, and it doesn't work the way we need.

The cat /dev/null doesn't reset the line number, so it the output redirection continues to write on the WXYZ line instead of begin writing on the first line, even if the first WXYZ are blank after the cat /dev/null.

Thanks,

Joseba

 

[hoinz]

Hm, ...
the idea that your whole problem could be caused by this double > seemed so convincing ...
I've never seen that an application behaved like that. (If I understood what you described ...)
As Ken suggested, it seems that the application is to blame. Perhaps it always writes to given offset from the beginning of the file?
If so, your only chance is to change the application.
Did you try mv instead of cp? I have not much hope though...

 

[hoinz]

a rather weird idea:
Could you verify that /dev/null really is what you think it is?
E.g. ls -l /dev/null
Maybe it has somehow been replaced by a plain file consisting of thousands of empty lines.

 

[jmiturbe]

Hello,

Consider that this is not an application problem, because we are redirecting the unix standard output.

The application doesn't know anything about the text file.

Logically, the Unix redirection process doesn't know anything about the script that truncates the destination text file, so it doesn't change the line number on which it has to write the next line (at a lower level it can be a pointer + offset issue).

We are looking for the best aproach to rotate standard unix output redirection files.

Joseba

 

[jmiturbe]

The /dev/null seems to be OK:

lrwxrwxrwx 1 root other 27 dic 11 2002 /dev/null -> ../devices/pseudo/mm@0:null

 

[PHV]

You may consider starting your application piping its standard output to a script responsible for the log rotate.

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ222-2244

 

[mrregan]

you have to stop your application, create a new log file and then restart your application.

 

[Clairvoyant1332]

> you have to stop your application, create a new log file and then restart your application.

I agree. I've seen this same issue with some of our apps. We got around it by doing just that.

 

[jmiturbe]

Thank you very much,

Stopping the application is the first approach we thought about, but it's a 24x7 application, so this option was discarded.

We also thought about creating an intermediate script like PHV suggested, so we decided to post on this forum searching for this script or an alternate approach.

Did anybody out there "developed" a similar solution?

Thanks,

Joseba M. Iturbe

 

[PHV]

A starting point:
yourApplication | awk '
BEGIN{out="/path/to/NAME_OF_THE_LOG_FILE";i=0}
{print > out"."i}
NR%1000==0{close(out"."i++)}
'


Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ222-2244

 

[SamBones]

I rotate some logs that are being generated like that with the following...

cp name_of_the.log name_of_the.log.$(date '+%Y%m%d_%H%M%S')
:> name_of_the.log

That makes a date stamped copy then zeroes the log.

But I have seen some that act like you describe, where it doesn't seem to want to zero the file.

Hope this helps.

 

[Ygor]

This should change the name of the log file every hour...

#!/usr/bin/ksh

function fn_stamper {
   while read LINE
   do
      LOGFILE=/path/to/yourApp_$(date '+%Y%m%d%H').log
      echo "$LINE" >> $LOGFILE
   done
}

yourApplication | fn_stamper

 

[coffeysm]

Why not just try echo > /your/path/to/filename