audit map

[jinkys]

Hi there

I want to write a map that runs for every log file and produces a summary file removing all the log files. I have written a map that does this but I am wondering if anyone has a btter solution to getting all the log files onto input. I tried using the file watcher (source evnt) on the event server but this can't handle the number of log files there is so I now use a unix script to run the map for every log file it finds.

Does anyone have any other ways of doing this?

 

[BocaBurger]

Don't use a file output. If you run the command server (UNIX script) in the same system (user ID, etc,) as the Event Server, it will eventually corrupt the .mershmid file in the /tmp directory and cause a core dump. You may have too may maps running in one system. You might want to install another Event Server with the one audit log reading map, but have the output as a Database, this way it can multi-thread and keep up with the main Event Server.

You might also want to think about making this an error handling system, rather than just storing the data. You should not need audit logs for every single map in a system (unless every map is independant of the others). You may want to log the start and end of a process, and if you get a start log and no end log, then have your error handling system start working.



BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[jinkys]

we currently run maps using command server and event server under same user without problems. Are you saying we need multiple installations of the mercator software or just run under different users so the shared memory is shared between event server and command server?

 

[BocaBurger]

You would need to make sure the /tmp directory for each user is different. Having a common directory can result in the .merschid file becoming corrupt, since you would have two applications using (reading & writing) the same file. You don't want the shared memory shared.

The user who runs the script (CRON?) needs to have a different /tmp directory defined in the environment.

This is not an easily reproducible problem, if it was, we would have fixed it. What will eventually happen is the Event Server will crash for no apparent reason, and will not restart unless the .mer files are deleted and ipcs and ipcrm are run.

Event Server is a multi-threaded application, command server is not.



BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!