ProCKSI

h1. Monit: Monitoring of Services

This installation guide will describe how to set up _independent_ instances of "Monit":http://www.tildeslash.com/monit/ on the master node and each slave.[[br]]

In the future, "M|Monit":http://www.tildeslash.com/mmonit/ should be considered, which allows easy single point administration and monitoring (from the master node).

h2. Installation

* Add the DAG repository on the _master node_ and _slave nodes_. Enter at the command line as _root_:

<pre>

 wget http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm

 rpm -Uvh rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm

</pre>

* Install Monit on the _master node_ and _slave nodes_. Enter at the command line as _root_:

<pre>

 yum install monit

</pre>

h2. Configuration

h3. Master node

On the master node, the following services will be monitore:[[br]]

_apache_, _cron_, _devices_ (/ & /home), _mysql_, _nfs_ (/home_nfs), _ntp_, _pbs_mom_, _pbs_sched_, _pbs_server_, _postfix_, _ssh_, _system_, _ypbind_, _yppasswd_, _ypserv_, [[br]]

and if all _slaves_ are reachable (_ping_)[[br]]

Currently, the monitoring of _pbs_maui_ is switched off in favour of _pbs_sched_.[[br]]

* Download the [source:Externals/Cluster/procksi_monit.tgz configuration files] from the repository and extract the files. Enter at the command line:

<pre>

tar -xvzf procksi_monit.tgz

</pre>

* Copy the files in _./monit/master_ to the appropriate directories (_/etc/_, _/etc/monit.d/_, _/home/procksi/monit/_).

* Change permissions of the monit token file. Enter at the command line:

<pre>

chown -R procksi.procksi_dev /home/procksi/monit/token

</pre>

* Edit the Apache configuration file _/etc/httpd/conf/httpd.conf_:

<pre>

#General Aliases for Monitoring and Testing

Alias /monit/    "/home/procksi/monit/"

Alias /ganglia/  "/usr/local/ganglia/html/"

Alias /trees/    "/home/procksi/trees/"

#Conditional Logging: Don't log Ganglia and Monit requests

[[SetEnvIf]] Request_URI "ganglia" dontlog

[[SetEnvIf]] Request_URI "^\/monit\/token$" dontlog

</pre>

* Restart the Apache server. Enter at the command line as _root_:

<pre>

/sbin/service httpd restart

</pre>

* Make the Monit daemon start at bootup. Enter at the command line as _root_:

<pre>

/sbin/chkconfig  monit  on

</pre>

* Start the Monit daemon. Enter at the command line as _root_:

<pre>

/sbin/service  monit  start

</pre>

h3. Slave nodes

On the master node, the following services will be monitore:[[br]]

_devices_ (/ and /scratch), _nfs_ (/home), _ntp_, _pbs_mom_, _ssh_, _system_, _ypbind_

* Download the [source:Externals/Cluster/procksi_monit.tgz configuration files] from the repository and extract the files. Enter at the command line:

<pre>

tar -xvzf procksi_monit.tgz

</pre>

* Copy a the files in _./monit/slave_ to the appropriate directories (_/etc/_, _/etc/monit.d/_).

* Edit _/etc/monit.d/system_ and set the correct host name for each slave node.

* Make the Monit daemon start at bootup. Enter at the command line as _root_:

<pre>

/sbin/chkconfig  monit  on

</pre>

* Start the Monit daemon. Enter at the command line as _root_:

<pre>

/sbin/service  monit  start

</pre>

h2. Online Monitoring

The status of each monitored service, process, file, etc. is available with the Monit's integrated webserver at port 2812 from _localhost_ and selected machines. Username and password can be found at the secret [[[secretAuthentication|authentication]]] page.

 || master01 || "|| slave01  || [http://procksi1.cs.nott.ac.uk:2812":http://procksi0.cs.nott.ac.uk:2812]

 || slave02  || "|| slave03  || [http://procksi3.cs.nott.ac.uk:2812":http://procksi2.cs.nott.ac.uk:2812]

 || slave04  || [http://procksi4.cs.nott.ac.uk:2812]

h2. Offline Monitoring

Monit sends alerts to "procksi@cs.nott.ac.uk" if services are unavailable, have been restarted, or similar events.