Mastering Alertmanager Configuration for Effective Prometheus Alerting

./alertmanager--config.file=alertmanager.yml

global:# The default SMTP From header field.[smtp_from:<tmpl_string>]# The default SMTP smarthost used for sending emails, including port number.# Port number usually is 25, or 587 for SMTP over TLS (sometimes referred to as STARTTLS).# Example: smtp.example.org:587[smtp_smarthost:<string>]# The default hostname to identify to the SMTP server.[smtp_hello:<string>| default = "localhost"]# SMTP Auth using CRAM-MD5, LOGIN and PLAIN. If empty, Alertmanager doesn't authenticate to the SMTP server.# PLAIN is only supported when using TLS.[smtp_auth_username:<string>]# SMTP Auth using LOGIN and PLAIN.[smtp_auth_password:<secret>]# SMTP Auth using LOGIN and PLAIN.[smtp_auth_password_file:<string>]# SMTP Auth using PLAIN.[smtp_auth_identity:<string>]# SMTP Auth using CRAM-MD5.[smtp_auth_secret:<secret>]# SMTP Auth using CRAM-MD5.[smtp_auth_secret_file:<string>]# The default SMTP TLS requirement.# Note that Go does not support unencrypted connections to remote SMTP endpoints.[smtp_require_tls:<bool> | default = true]# The default TLS configuration for SMTP receivers[smtp_tls_config:<tls_config>]# Force implicit TLS regardless of SMTP port[smtp_force_implicit_tls:<bool>]# Default settings for the JIRA integration.[jira_api_url:<string>]# The API URL to use for Slack notifications.[slack_api_url:<secret>][slack_api_url_file:<filepath>][slack_app_token:<secret>][slack_app_token_file:<filepath>][slack_app_url:<string>][victorops_api_key:<secret>][victorops_api_key_file:<filepath>][victorops_api_url:<string>| default = "https://alert.victorops.com/integrations/generic/20131114/alert/"][pagerduty_url:<string>| default = "https://events.pagerduty.com/v2/enqueue"][opsgenie_api_key:<secret>][opsgenie_api_key_file:<filepath>][opsgenie_api_url:<string>| default = "https://api.opsgenie.com/"][rocketchat_api_url:<string>| default = "https://open.rocket.chat/"][rocketchat_token:<secret>][rocketchat_token_file:<filepath>][rocketchat_token_id:<secret>][rocketchat_token_id_file:<filepath>][wechat_api_url:<string>| default = "https://qyapi.weixin.qq.com/cgi-bin/"][wechat_api_secret:<secret>][wechat_api_secret_file:<string>][wechat_api_corp_id:<string>][telegram_api_url:<string>| default = "https://api.telegram.org"]# The default Telegram bot token. It is mutually exclusive with `telegram_bot_token_file`.[telegram_bot_token:<secret>]# The default configuration to read the Telegram bot token from a file. It is mutually exclusive with `telegram_bot_token`.[telegram_bot_token_file:<string>][webex_api_url:<string>| default = "https://webexapis.com/v1/messages"][mattermost_webhook_url:<secret>][mattermost_webhook_url_file:<string>]# The default HTTP client configuration[http_config:<http_config>]# ResolveTimeout is the default value used by alertmanager if the alert does# not include EndsAt, after this time passes it can declare the alert as resolved if it has not been updated.# This has no impact on alerts from Prometheus, as they always include EndsAt.[resolve_timeout:<duration>| default = 5m]# Files from which custom notification template definitions are read.# The last component may use a wildcard matcher, e.g. 'templates/*.tmpl'.templates:[ -<filepath>...]# The root node of the routing tree.route:<route># A list of notification receivers.receivers:-<receiver>...# A list of inhibition rules.inhibit_rules:[ -<inhibit_rule>...]# DEPRECATED: use time_intervals below.# A list of mute time intervals for muting routes.mute_time_intervals:[ -<time_interval>...]# A list of time intervals for muting/activating routes.time_intervals:[ -<time_interval>...]

[receiver:<string>]# The labels by which incoming alerts are grouped together. For example,# multiple alerts coming in for cluster=A and alertname=LatencyHigh would# be batched into a single group.## To aggregate by all possible labels use the special value '...' as the sole label name, for example:# group_by: ['...']# This effectively disables aggregation entirely, passing through all# alerts as-is. This is unlikely to be what you want, unless you have# a very low alert volume or your upstream notification system performs# its own grouping.[group_by:'['<labelname>,... ']' ]# Whether an alert should continue matching subsequent sibling nodes.[ continue:<boolean>| default = false ]# DEPRECATED: Use matchers below.# A set of equality matchers an alert has to fulfill to match the node.match:[<labelname>:<labelvalue>, ... ]# DEPRECATED: Use matchers below.# A set of regex-matchers an alert has to fulfill to match the node.match_re:[<labelname>:<regex>, ... ]# A list of matchers that an alert has to fulfill to match the node.matchers:[ -<matcher>... ]# How long to wait before sending the first notification for a new group of# alerts. Allows to wait for alerts to arrive from other rule groups or# Prometheus servers, and for one or more inhibiting alerts to arrive and mute# any target alerts before the first notification.## A short group_wait will reduce the time to wait before sending the first# notification for a new group of alerts. However, if group_wait is too short# then the first notification might not contain the complete set of expected# alerts, and alerts that should be inhibited might not be inhibited if the# inhibiting alerts have not arrived in time.## A long group_wait will increase the time to wait before sending the first# notification for a new group of alerts. However, if group_wait is too long# then notifications for firing alerts might not be sent within a reasonable# time.## If an alert is resolved before group_wait has elapsed, no notification will# be sent for that alert. This reduces noise of flapping alerts.# A notification for any alerts that missed the initial group_wait will be# sent at the next group_interval instead.## If omitted, child routes inherit the group_wait of the parent route.[ group_wait:<duration>| default = 30s ]# How long to wait before sending subsequent notifications for an existing# group of alerts after group_wait.## The group_interval is a recurring timer that starts as soon as group_wait# has elapsed. At each group_interval, Alertmanager checks if any new alerts# have fired or any firing alerts have resolved since the last group_interval,# and if they have a notification is sent. If they haven't, Alertmanager checks# if the repeat_interval has elapsed instead.