Contents

Overview
Formatting your monitoring station script for retained data tracking Changing the check_temp script for retained data
Adding Custom Service Monitor with Retained Data to Uptime Infrastructure Monitor

Overview

This article is part of a series:
Part 1 - Creating Custom Service Monitors in Uptime Infrastructure Monitor Part 2 - Creating Custom Service Monitors with Retained Data Collection Part 3 - Creating Plugin Service Monitors in Uptime Infrastructure Monitor
Custom service monitors with retained data tracking expand on the basic Custom service monitor by allowing you to retain and graph historical trending information returned from your custom script. This enables you to store up to 10 custom application or business metrics per monitor within Uptime Infrastructure Monitor just like system performance metrics returned by the Uptime Infrastructure Monitor agent. This article builds on the scripts and knowledge that were developed in a previous article. Take some time to review the previous article before continuing.

Example graph produced using a custom service monitor with retained data.

Formatting your monitoring station script for retained data tracking

To use a custom script with retained data with Uptime Infrastructure Monitor, you must change the output format produced by your script. All of the rules found in a regular custom service monitor still apply with a few slight modifications, as detailed below:

1. The monitoring station script must return a single number (decimal or integer) per line – one line for each metric that you want to Uptime Infrastructure Monitor to retain. You can have a maximum of 10 lines retained. The example script output shown below retains the values '10.5' and '99' as two distinct graphable trends within Uptime Infrastructure Monitor:
2. ** check_temp.sh 10.5

99

1. The output must be in numeric format; you cannot have text output. This enables the service monitor to perform all threshold checking within Uptime Infrastructure Monitor instead of having your script determine thresholds and return an outage message in text format.
2. The script must exit with a success status (0), unless there has been a problem when the script is run or if you want to force a status for the service monitor.
3. The script must accept the hostname of the agent system as the first argument. Uptime Infrastructure Monitor will automatically add this argument to the arguments passed to your script.
4. In general, you do not need to change to your existing agent side script or configuration in order for your service monitor to retain performance metrics.

Changing the check_temp script for retained data

Using the check_temp.sh script as a basis for this example, you can easily change the script to fit within the context of a custom service monitor with retained data. To do this, you must make the following changes to the script:

1. Remove the logic included within the script to check the temperature and humidity thresholds.
2. The output that is produced by the script will be changed to output the current temperature level on a single line followed by the current humidity. Here is an example of how our output and command format will change.

Previous Format - For a regular custom service monitor.

• • ./check_temp.sh test-agent 9998 temp 60 80 WARNING - temperature is 64.5 on test-agent
  • ./check_temp.sh test-agent 9998 rh 25 30 CRITICAL - humidity is 32.8 on test-agent

New Format - For a custom service monitor with retained performance data.

• • ./check_temp.sh test-agent 9998 64.5

32.8
To produce the output listed above, you must edit the script so that it looks like the following example:
#!/bin/sh

1. This script takes the following arguments:
2. check_temp.sh hostname port # Example execution:
3. ./check_temp.sh my-agent 9998
4. This script can be placed anywhere on the monitoring station system as long as it is # executable by the uptime user.
   #First, collect our arguments AGENT=$1
   PORT=$2
   TMPFILE=/tmp/$$.temp
5. now use the info above to contact our agent, store the output in a file for parsing
   `echo -n rexec secretpassword /opt/uptime-agent/my-scripts/show_temp.sh my-arguments | /usr/local/uptime/bin
   /netcat $AGENT $PORT > $TMPFILE` Note
   If you are using agentcmd instead of netcat, replace netcat with agentcmd in the command above. For example:`echo -n /opt/uptime-agent/my-scripts/show_temp.sh my-arguments | /usr/local/uptime/scripts/agentcmd -p $PORT
   $AGENT rexec secretpassword > $TMPFILE`
   
   
   For more information see Using the agentcmd utility.

6. we have the output from the agent. If it is ERR that means there was a problem running the script on the agent
   `grep ERR $TMPFILE` if [ $? -eq 0 ]
   then
   echo "Could not execute agent side script!"

7. by exiting with a 2 we are forcing a CRIT service outage exit 2
   fi
8. in this script we don't need to check thresholds or determine which information to check
9. we just need to reformat the agent side script output slightly so that only numerical info is displayed # we do this by trimming off the first word returned on each line from the agent, leaving just the numbers # and printing that to screen, Uptime Infrastructure Monitor will handle the rest
   awk '{print $2}' $TMPFILE exit 0
   
   
   
   

   Adding Custom Service Monitor with Retained Data to Uptime Infrastructure Monitor

   Next, add your custom service monitor with retained performance metrics to the Uptime Infrastructure Monitor Web interface using the same process that you would use to add a standard custom service monitor to Uptime Infrastructure Monitor. The Custom with Retained Data monitor option is found in the List Other Monitors section of the Add New Service Instance page.
   The Custom with Retained Data service monitor template has the following monitor-specific settings:
   

   Optio n Name	Description	Example
   Script Name	The script name is the path to your monitoring station script, this is the script that Uptime Infrastructure Monitor will execute when running this service monitor. Be sure to use the complete path wherever possible and that the path is to a locally mounted volume. For Windows script paths you must use UNIX style directory separators (/ instead of ) and also place double quotes around the entire script name	UNIX /Linux Example:
   		/usr/local /uptime /check_t emp.sh
   		Window s Example:
   		"C:/my scripts /check_t emp.bat"
   Argu ments	These are the arguments that you would like Uptime Infrastructure Monitor to pass into your monitoring station script. No arguments are required but please be aware that Uptime Infrastructure Monitor will automatically include the selected hostname as the first argument to your script.	temp 60 80
   Varia	This is the warning threshold used against the output returned from your monitoring station script. This is a numeric comparison.	Output
   ble 1-	You must select both a comparison method and a threshold value to enable the warning level threshold	contains
   10		:
   Warni		"warning"
   ng
   Varia	This is the critical threshold used against the output returned from your monitoring station script. This is a numeric comparison. You	Output
   ble 1-	must select both a comparison method and a threshold value to enable the critical level threshold	contains
   10		:
   Critic		"critical"
   al
   Retai	This check box determines if Uptime Infrastructure Monitor will not only check the variable for threshold violations but will also retain	N/A
   ned	the returned values for graphing at a later time. In most cases you should check this box, without it data for the indicated variable
   Data	will not be retained for graphing.
   Tracki
   ng

   Based on the settings used in the example monitoring station script, configure the monitor with the following setting: Enter a name and description for the monitor.
   Select a host from the dropdown menu. Be sure to select the same host that your agent side script is on
   In the Script Name field, enter the path to the custom script on your monitoring station. On Windows systems be sure to use UNIX style / instead of and put quotation marks around your path. For example: "C:/my files/check_temp.bat"
   In the Arguments field, enter the arguments for the script. Uptime Infrastructure Monitor adds the agent name as the first argument automatically so do not include it.
   Select contains from the Warning dropdown and enter WARNING as the search text. Select contains from the Critical drop down and enter CRITICAL as the search text. Complete the remainder of the monitor template as you would for a normal service monitor.
   Example monitor configuration
   The image below illustrates a sample monitor configuration. This service monitor will indicate a WARN or CRIT whenever the monitoring station custom script returns WARNING or CRITICAL in its output.