Nagios Email Notifications: Setup, Tuning & Troubleshooting

How the notification pipeline works

Every Nagios email is the end of a chain, and debugging is easiest when you know the links:

1. A check fails, then keeps failing through its retries until the object enters a hard problem state. Soft states never notify — that's by design, and it's the number-one source of "why didn't I get an email?" confusion. (Refresher: soft vs. hard states.)
2. Nagios evaluates notification logic: are notifications enabled globally and on the object? Does the state match notification_options? Is the current time inside notification_period? Is the object acknowledged or in downtime?
3. Each eligible contact is matched against their own notification options and timeperiod.
4. The contact's notification command runs — for email, a shell pipeline that formats a message and hands it to the local mail command.
5. Your MTA (Postfix, or a relay like msmtp) actually delivers it.

Step 1: a working mail path

Before touching Nagios config, prove the server can send mail as the nagios user:

sudo -u nagios /bin/mail -s "Nagios mail test" you@example.com <<< "It works."

If that doesn't arrive, no amount of Nagios configuration will help. On a modern network you'll usually relay through an authenticated smarthost (your provider's SMTP, or a transactional mail service) rather than delivering straight to the internet — configure Postfix's relayhost or use msmtp as a lightweight sendmail replacement, and check the spam folder before blaming the config.

Step 2: contacts

A minimal contact and contact group in contacts.cfg:

define contact {
    contact_name                    rvetter
    alias                           Richard
    email                           you@example.com
    host_notification_period        24x7
    service_notification_period     24x7
    host_notification_options       d,u,r
    service_notification_options    w,c,r
    host_notification_commands      notify-host-by-email
    service_notification_commands   notify-service-by-email
}

define contactgroup {
    contactgroup_name   admins
    alias               Nagios Administrators
    members             rvetter
}

The notification_options letters decide which transitions email you: d down, u unreachable (hosts) / unknown (services), w warning, c critical, r recovery, f flapping, s scheduled downtime, n none. Then reference the contact group from your hosts and services (typically via a template's contact_groups admins).

Step 3: the notification command

Nagios Core's sample config ships working notify-host-by-email and notify-service-by-email commands in commands.cfg — a printf pipeline into /bin/mail expanding macros like $HOSTNAME$, $SERVICEDESC$, $SERVICESTATE$, and $SERVICEOUTPUT$. Two upgrades worth making:

• Put the state and object in the subject line. ** PROBLEM: web01/HTTP is CRITICAL ** lets you triage from the lock screen without opening the message.
• Include the plugin output. The output line usually contains the actual error; an email without it just sends you hunting.

Finally, make sure the master switch is on in nagios.cfg: enable_notifications=1, and that hosts/services have notifications_enabled 1 (the default).

Tuning out the noise

An inbox you've learned to ignore is worse than no alerting. The levers, in order of impact:

• Notify on what's actionable. If nobody will act on Warnings at 3 a.m., don't send them at 3 a.m. — use a business-hours notification_period for warnings and 24×7 for criticals (via two service templates, or per-contact timeperiods).
• notification_interval — minutes between repeat notifications while a problem persists. 0 means "tell me once." Repeats every 15 minutes for a known, acknowledged issue are pure noise — which is what acknowledgements are for.
• first_notification_delay — wait N minutes after a hard failure before the first email; self-healing blips never page anyone.
• Retries before hard state — max_check_attempts and retry_interval decide how long a problem must persist before it's real. Flapping checks should be fixed or have flap detection do its job, not fill inboxes.
• Scheduled downtime for planned work — suppress notifications for the maintenance window instead of ignoring your phone. You can schedule downtime from NagMon in a few taps.

Troubleshooting "Nagios isn't sending emails"

Work down the pipeline — nagios.log tells you exactly how far a notification got:

grep 'NOTIFICATION' /usr/local/nagios/var/nagios.log | tail

• No NOTIFICATION entries at all → Nagios decided not to notify. Check: is the state hard yet? enable_notifications=1? Object notifications_enabled? Do the notification_options include this state? Is the time inside the notification period? Is the object acknowledged or in downtime?
• NOTIFICATION logged but no email → the command or mail path failed. Re-run the mail test as the nagios user, check the MTA log (/var/log/mail.log), and check spam filtering on the receiving end.
• Some contacts get it, others don't → compare the contacts' own notification options and timeperiods; they filter independently of the object's settings.

From email to fixed

The email is the starting gun, not the finish line. A workflow that works well for on-call admins: the notification lands on your phone with state and host in the subject; you open a mobile client like NagMon to see the blast radius — is it one service, or is the whole rack unreachable? — read the live plugin output, and acknowledge so repeat notifications stop and colleagues see it's handled. Then fix the thing.

Triage Nagios alerts from your pocket

NagMon shows live scope, plugin output, and lets you acknowledge or schedule downtime — the perfect follow-through to a well-tuned email alert.

Download NagMon