Introduction

When Are Host Checks Performed?

Hosts are checked by the Shinken Enterprise daemon:

  • At regular intervals, as defined by the check interval and retry interval options in your host definition.
  • On-demand when a check associated with the host changes state.
  • On-demand as needed as part of the host reachability logic (dependencies).

Regularly scheduled host checks are optional.

If you set the check_interval option in your host definition to zero (0), Shinken Enterprise will not perform checks of the hosts on a regular basis.

It will, however, still perform on-demand checks of the host as needed for other parts of the monitoring logic.

On-demand checks are made when a service associated with the host changes state because Shinken Enterprise needs to know if the host has also changed state. Services that change state are often an indicator that the host may have also changed state.

For example, if Shinken Enterprise detects that the "HTTP" check associated with a host just changed from a CRITICAL to an OK state, it may indicate that the host just recovered from a reboot and is now back up and running.

On-demand checks of hosts are also made as part of the dependencies logic. Shinken Enterprise is designed to detect network outages as quickly as possible, and distinguish between DOWN and UNREACHABLE host states. These are very different states and can help an administrator to quickly locate the cause of a network outage.

 
  

Dependencies and Checks

You can define parents that prevent Shinken Enterprise from checking the status of a host depending on the state of one or more other hosts. More information on dependencies can be found on the depdency logic page. 
  

Parallelization of Host Checks

All checks are run in parallel. 
  

Host States

Hosts that are checked can be in one of three different states:

  • UP
  • DOWN
  • UNREACHABLE
 
  

Host State Determination

Host checks are performed by commands, which can return a state of OK, WARNING, UNKNOWN, or CRITICAL. Shinken Enterprise does translate these plugin return codes into host states of UP, DOWN, or UNREACHABLE. The table below shows how plugin return codes correspond with preliminary host states. Some post-processing (which is described later) is done which may then alter the final host state.
Command resultHost state
OKUP
WARNINGDOWN
UNKNOWNDOWN
CRITICALDOWN

If the preliminary host state is DOWN, Shinken Enterprise will attempt to see if the host is really DOWN or if it is UNREACHABLE. The distinction between DOWN and UNREACHABLE host states is important, as it allows admins to determine root cause of network outages faster. The following table shows how Shinken Enterprise makes a final state determination based on the state of the hosts parent(s). A host's parents are defined in the parents directive in host definition.
Preliminary Host stateParent host stateFinal host state
DOWNAt least one parent is UPDOWN
DOWNAll parents are eitheir DOWN or UNREACHEABLEUNREACHABLE

More information on how Shinken Enterprise distinguishes between DOWN and UNREACHABLE states can be found on the dependency logic page.		 
  

Host State Changes

As you are probably well aware, hosts don't always stay in one state. Things break, patches get applied, and servers need to be rebooted. When Shinken Enterprise checks the status of hosts, it will be able to detect when a host changes between UP, DOWN, and UNREACHABLE states and take appropriate action.

These state changes result in different state types (HARD or SOFT), which can trigger event handlers to be run and notifications to be sent out. Detecting and dealing with state changes is what Shinken Enterprise Enterprise is all about.

When hosts change state too frequently they are considered to be flapping. A good example of a flapping host would be a server that keeps spontaneously rebooting as soon as the operating system loads. That's always a fun scenario to have to deal with.

Shinken Enterprise can detect when hosts start flapping, and can cancel notifications until flapping stops and the host's state stabilizes. More information on the flap detection logic can be found on the flapping page.