how does syslog-ng connect to mysql 3

[jhala]

I'm running syslog-ng v 1.6.11 on RHEL 4. I can log to local files with no problem. I want to also log to a mysql database. I have a mysql database server with a syslog database configured to receive logs.

On the syslog-ng server, do I need mysql installed in order to make the mysql pipe work? From what I read, you need a script scheduled in cron to run every minute to make the mysql connection to send the data. So the script goes on the syslog-ng server, and therefore I need mysql installed on the syslog-ng server to initiate the connection? I thought maybe mysql was somehow built into syslog-ng and it can make the connection itself. If mysql is needed on the syslog-ng server, are there any parameters to pass with reference to syslog-ng while installing mysql?

Here's my syslog-ng.conf, let me know if I'm missing anything:[/color green]

source net { udp(); };

destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ('$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG');\n") template-escape(yes)); };

filter f_server { host("server"); };

log { source(net); filter(f_server); destination(d_mysql); };

To create the pipe I did:

mkfifo /tmp/mysql.pipe

Here's a script I found for sending data through the pipe:

#
# Created by Tadghe Patrick Danu
#
#!/bin/bash

if [ -e /tmp/mysql.pipe ]; then
while [ -e /tmp/mysql.pipe ]
do
mysql -u theuserid --password=thepassword syslogdb < /tmp/mysql.pipe
done
else
mkfifo /tmp/mysql.pipe
fi

Thanks in advance, your help is greatly appreciated.

 

[thedaver]

Is your server doing anything that would generate log activity while you're monitoring?

I also would NOT expect you to see two (2) instances of syslog2mysql.sh running. That would be nonsense. Kill them both off and start over.

D.E.R. Management - IT Project Management Consulting

http://www.dermanagement.com/

 

[jhala]

In addition to logging to the mysqldb server, I am logging locally as well. And I see those logs being generated. I have also verified with a tcpdump.

I don't understand why the second instance of the shell script is being generated. I killed them both off, and just ran

./syslog2mysql.sh &

And the second instance keeps appearing...

Thanks

 

[thedaver]

Um, are you sure syslog-ng permits multiple destinations for the same log data? Perhaps that's the problem. Local logging may trump an external logging.


D.E.R. Management - IT Project Management Consulting

http://www.dermanagement.com/

 

[jhala]

damn, you're MONEY, thanks!

I thought I read a configuration that had both...

 

[jhala]

Here's an update...

So I took out one of the destinations in the log line of the syslog-ng config.
Restarted syslog-ng and it started working... and I only had one instance of the shell script in the process list...

Then I went back and added the second destination again,

log { source(net); filter(f_server); destination(d_mysql); destination(local_server); };

restarted everything, and that also worked!

So I'm logging to both the db and the local server.

Go figure...

Thanks again.

 

[geirendre]

Syslog-ng can indeed have multiple destinations for the
same log data.
If you look at this examples

http://www.softpanorama.org/Logs/Syslog_ng/configuration_examples.shtml

you se how syslog-ng.conf is set up.

Basicaly it is made up of 3 parts:

1- sources: where to get log entries from (localy, process, IP-addr, TCP-port, UDP-port etc)

2- filters: what messages shall be let trough.

3- destinations: where to send the log datas.

You can have as many instances of these tre parts as you wanth.

Then comes the clever part. You tie these parts together in
a log-line, which is made up of source, filter(optional) and destination.

You can reuse any source/filter/destination as many times
as you like.
So you can have 2 log-lines with the same source and filter,
but with different destinations.

http://sial.org/howto/logging/syslog-ng/#s2