Back to the main page
Calling Nagios
Okay, say you travel, company didn't pay you roaming (cell or Blackberry doesn't work), you don't have Internet/VPN access but you still have burning desire to check Nagios messages. What are you going to do?
Why not using just regular phone, maybe find nearest pay phone on the street.
What a joke!?
Well, not really. Keep reading, first little introduction.
HTML is markup language used to create web pages.
VoiceXML is also markup language but used to create voice application that run on the phone.
This is how it works?
1. Get a phone and call VoiceXML platform (server that has VoiceXML interpreter, speech recognizer and Text-to-speech engine).
I am using http://developer.voicegenie.com on this site, you'll find phone number and learn how to associate extension to URL.
2. The VoiceXML platform will look URL page associated with number you dial so you need public web site to host this page.
3. You page (.vxml file) is read by VoiceXML interpreter, you can provide input by speaking or pressing phone keys and output is done as text-to-speech.
4. Simple, right?
Let's go back to checking Nagios now.
Idea is to filter "messages of interest" from Nagios log file, and create .vxml file that has these messages.
The .vxml file is then uploaded to web site, this can be done, say, using cron job every 15 minutes.
And this is it. Once you are on the street and think about Nagios, just call the number and extension and nice female voice will tell you, say, errors for hosts and services.
Note: you'll probably need to create account on mentioned VoiceXML platform in order to associate phone extension to your URL, just go there and check info, it is very intuitive and simple.
I have borrowed the logtail utility from logcheck application (written by Craig Rowland).
For more info about this utility see page .
Basically logtail remembers position of last reading of logfile, so you always have fresh/new messages uploaded to the voice application.
Filtering of "messages of interest" are done with help of two files, I call them nagios.issue and nagios.ignore.
Just to give you some ideas how these files look like:
# cat nagios.issue
\[[0-9]*\] SERVICE ALERT: [a-zA-Z0-9]*\;PING\;WARNING\;
\[[0-9]*\] SERVICE ALERT: [a-zA-Z0-9]*\;PING\;CRITICAL\;
\[[0-9]*\] SERVICE ALERT: [a-zA-Z0-9]*\;LOAD\;WARNING\;
\[[0-9]*\] SERVICE ALERT: [a-zA-Z0-9]*\;LOAD\;CRITICAL\;
\[[0-9]*\] HOST ALERT: [a-zA-Z0-9]*-[a-zA-Z0-9]*\;DOWN\;
|
# cat nagios.ignore
\[[0-9]*\] Caught SIGHUP, restarting...
\[[0-9]*\] LOG ROTATION: DAILY
\[[0-9]*\] CURRENT HOST STATE: localhost\;
\[[0-9]*\] CURRENT SERVICE STATE: localhost\;
\[[0-9]*\] Auto-save of retention data completed successfully.
\[[0-9]*\] Warning: aggregate_status
\[[0-9]*\] Warning: Host
\[[0-9]*\] EXTERNAL COMMAND:
\[[0-9]*\] Warning: Duplicate definition
|
Note: these files must not have empty lines, so I am using sed to check this and remove them.
Also note that my script doesn't automatically upload .vxml file to web server (but I am sure you can "solve" this by yourself).
Here is the script, I guess commented well so you can understand how it works.
# cat nagios2vxml.sh
#!/bin/sh
#set -x
HOSTNAME=`hostname`
DATE=`date +'%Y-%m-%dT%H:%M'`
SYSADMIN="name@company.com"
# The path to logtail program
LOGTAIL=/usr/local/bin/logtail
# Dir for temp files
TMPDIR=/usr/local/etc/callnagios/tmp
GREP=`which egrep`
MAIL=`which mailx`
DIFF=`which diff`
ECHO=`which echo`
AWK=`which awk`
# issue/problems we are looking for and want to hear about
ISSUE=/usr/local/etc/callnagios/nagios.issue
# what we don't want to hear about
IGNORE=/usr/local/etc/callnagios/nagios.ignore
# ----- check for empty lines in issue/ignore files
# ------ if there are some, remove them and send email as warning
# NEL - Number of Empty Lines
OURFILES="${ISSUE} ${IGNORE}"
for FILE in ${OURFILES}
do
NEL=`grep "^$" ${FILE} | wc -l`
if [ ${NEL} -eq 0 ]
then
${ECHO} "There are NO empty lines in ${FILE}" > /dev/null
else
${ECHO} "There are ${NEL} empty lines in ${FILE} and they will be removed - be careful next time" | \
${MAIL} -s "${HOSTNAME} - Empty lines in ${FILE}" ${SYSADMIN}
/usr/bin/cat ${FILE} | /usr/bin/sed '/^$/d' > ${FILE}.out; mv ${FILE}.out ${FILE}
fi
done
# ---- end of checking for empty lines ------------
# logtail is checking log file
${LOGTAIL} /usr/local/etc/callnagios/nagios.log > ${TMPDIR}/feed.$$
# Set the flag variables
FOUND=0
# --------- Removing ignore lines from logs -----------
if [ -f "${IGNORE}" ]
then
${GREP} -v -f ${IGNORE} ${TMPDIR}/feed.$$ > ${TMPDIR}/newfeed.$$
fi
${DIFF} ${TMPDIR}/newfeed.$$ ${TMPDIR}/feed.$$ > /dev/null
if [ "$?" -eq 0 ]
then
# No ignore lines in new log messages, ${TMPDIR}/feed.$$ stays intact!
/usr/bin/rm ${TMPDIR}/newfeed.$$
else
# ${TMPDIR}/newfeed.$$ replaces ${TMPDIR}/feed.$$
# ${TMPDIR}/feed.$$ doesn't have ignore lines any more
/usr/bin/mv ${TMPDIR}/newfeed.$$ ${TMPDIR}/feed.$$
fi
# ----- search for issue and create vxml file
if [ -f "${ISSUE}" ]; then
if ${GREP} -i -f ${ISSUE} ${TMPDIR}/feed.$$ > ${TMPDIR}/feedoutput.$$
then
${ECHO} "< ?xml version=\"1.0\"? >" > ${TMPDIR}/nagios.vxml
${ECHO} "< vxml version=\"2.0\" xmlns=\"http://www.w3.org/2001/vxml\" >" >> ${TMPDIR}/nagios.vxml
${ECHO} "< form >" >> ${TMPDIR}/nagios.vxml
${ECHO} "< block >" >> ${TMPDIR}/nagios.vxml
${ECHO} "< prompt >" >> ${TMPDIR}/nagios.vxml
${ECHO} "Welcome to Nagios in your company. " >> ${TMPDIR}/nagios.vxml
${ECHO} "Host Messages " >> ${TMPDIR}/nagios.vxml
cat ${TMPDIR}/feedoutput.$$ | ${AWK} '$2 ~ /HOST/ && $3 ~ /ALERT/ {print $4}' \
| cut -d";" -f1-3 | ${AWK} '{print $0 " "}' >> ${TMPDIR}/nagios.vxml
${ECHO} "Service Messages " >> ${TMPDIR}/nagios.vxml
cat ${TMPDIR}/feedoutput.$$ | ${AWK} '$2 ~ /SERVICE/ && $3 ~ /ALERT/ {print $4}' \
| cut -d";" -f1-3 | ${AWK} '{print $0 " "}' >> ${TMPDIR}/nagios.vxml
${ECHO} "Goodbye and call me again!" >> ${TMPDIR}/nagios.vxml
${ECHO} "< /prompt >" >> ${TMPDIR}/nagios.vxml
${ECHO} "< /block >" >> ${TMPDIR}/nagios.vxml
${ECHO} "< /form >" >> ${TMPDIR}/nagios.vxml
${ECHO} "< /vxml >" >> ${TMPDIR}/nagios.vxml
FOUND=1
fi
fi
# ------------- if FOUND = 1 upload to web server
#if [ "$FOUND" -eq 1 ]; then
# For now this is done manually
#fi
# -- this emailing can be avoided, up to you
${ECHO} "Call 1.905.305.0293 or 1.905.305.0410 ext 408881 to hear new issues Nagios reports" | \
${MAIL} -s "${HOSTNAME} - FYI: Call Nagios to hear new issues on ${DATE} " ${SYSADMIN}
# Clean Up of temp files
rm -f ${TMPDIR}/feed.$$ ${TMPDIR}/feedoutput.$$
exit 0
|
There is the second script for generating more complex xvml file that gives more life to voice application.
The application talks with you and guides you if you don't talk or say something she is not supposed to hear.
Feel free to check it.
See the end of the script for phone numbers and extension to call.
# cat nagios2vxml-interactive.sh
#!/bin/sh
#set -x
HOSTNAME=`hostname`
DATE=`date +'%Y-%m-%dT%H:%M'`
SYSADMIN="name@company.com"
# The path to logtail program
LOGTAIL=/usr/local/bin/logtail
# Dir for temp files
TMPDIR=/usr/local/etc/callnagios/tmp
GREP=`which egrep`
MAIL=`which mailx`
DIFF=`which diff`
ECHO=`which echo`
AWK=`which awk`
# issue/problems we are looking for and want to hear about
ISSUE=/usr/local/etc/callnagios/nagios.issue
# what we don't want to hear about
IGNORE=/usr/local/etc/callnagios/nagios.ignore
# ----- check for empty lines in issue/ignore files
# ------ if there are some, remove them and send email as warning
# NEL - Number of Empty Lines
OURFILES="${ISSUE} ${IGNORE}"
for FILE in ${OURFILES}
do
NEL=`grep "^$" ${FILE} | wc -l`
if [ ${NEL} -eq 0 ]
then
${ECHO} "There are NO empty lines in ${FILE}" > /dev/null
else
${ECHO} "There are ${NEL} empty lines in ${FILE} and they will be removed - be careful next time" | \
${MAIL} -s "${HOSTNAME} - Empty lines in ${FILE}" ${SYSADMIN}
/usr/bin/cat ${FILE} | /usr/bin/sed '/^$/d' > ${FILE}.out; mv ${FILE}.out ${FILE}
fi
done
# ---- end of checking for empty lines ------------
# logtail is checking log file
${LOGTAIL} /usr/local/etc/callnagios/nagios.log > ${TMPDIR}/feed.$$
# Set the flag variables
FOUND=0
# --------- Removing ignore lines from logs -----------
if [ -f "${IGNORE}" ]
then
${GREP} -v -f ${IGNORE} ${TMPDIR}/feed.$$ > ${TMPDIR}/newfeed.$$
fi
${DIFF} ${TMPDIR}/newfeed.$$ ${TMPDIR}/feed.$$ > /dev/null
if [ "$?" -eq 0 ]
then
# No ignore lines in new log messages, ${TMPDIR}/feed.$$ stays intact!
/usr/bin/rm ${TMPDIR}/newfeed.$$
else
# ${TMPDIR}/newfeed.$$ replaces ${TMPDIR}/feed.$$
# ${TMPDIR}/feed.$$ doesn't have ignore lines any more
/usr/bin/mv ${TMPDIR}/newfeed.$$ ${TMPDIR}/feed.$$
fi
# ----- search for issue and create vxml file
if [ -f "${ISSUE}" ]; then
if ${GREP} -i -f ${ISSUE} ${TMPDIR}/feed.$$ > ${TMPDIR}/feedoutput.$$
then
${ECHO} "< ?xml version=\"1.0\"? >" > ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< vxml version=\"2.0\" >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< menu >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< prompt >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "Welcome to Nagios in your company." >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "What messages you want to hear, say Host or Service." >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /prompt >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< choice next = \"#getHost\" >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "host" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /choice >" >> ${TMPDIR}/nagios.vxml
${ECHO} "< choice next = \"#getService\" >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "service" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /choice >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< noinput >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "Are you shy to talk with me, say Host or Service." >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /noinput >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< nomatch >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "You are not saying what I need to hear, say Host or Service." >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /nomatch >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /menu >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< form id = \"#getHost\" >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< block >" >> ${TMPDIR}/nagiosinteract.vxml
cat ${TMPDIR}/feedoutput.$$ | ${AWK} '$2 ~ /HOST/ && $3 ~ /ALERT/ {print $4}' \
| cut -d";" -f1-3 | ${AWK} '{print $0}' >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "That is all for now, thanks for chating with me." >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< disconnect / >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /block >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /form >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< form id = \"#getService\" >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< block >" >> ${TMPDIR}/nagiosinteract.vxml
cat ${TMPDIR}/feedoutput.$$ | ${AWK} '$2 ~ /SERVICE/ && $3 ~ /ALERT/ {print $4}' \
| cut -d";" -f1-3 | ${AWK} '{print $0}' >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "That is all for now, thanks for chating with me." >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< disconnect / >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /block >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /form >" >> ${TMPDIR}/nagiosinteract.vxml
${ECHO} "< /vxml >" >> ${TMPDIR}/nagiosinteract.vxml
FOUND=1
fi
fi
# ------------- if FOUND = 1 upload to web server
#if [ "$FOUND" -eq 1 ]; then
# For now this is done manually
#fi
# -- this emailing can be avoided, up to you
${ECHO} "Call 1.905.305.0293 or 1.905.305.0410 ext 408882 to hear new issues Nagios reports" | \
${MAIL} -s "${HOSTNAME} - FYI: Call Nagios to hear new issues on ${DATE} " ${SYSADMIN}
# Clean Up
rm -f ${TMPDIR}/feed.$$ ${TMPDIR}/feedoutput.$$
exit 0
|
Back to the main page