Datadog Concepts Monitors and Alerting: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
Line 19: Line 19:
An anomaly detection alert uses past behavior to detect when a metric is behaving abnormally. For more details see [[#Anomaly_Monitor|Anomaly Monitor]]
An anomaly detection alert uses past behavior to detect when a metric is behaving abnormally. For more details see [[#Anomaly_Monitor|Anomaly Monitor]]
====Outliers====
====Outliers====
An outlier alert notifies when a member of a group (host, availability zone, partition, etc) is behaving unusually compared to the rest. For more details see [[#Outlier_Monitor|outlier monitors]].
====Forecast====
====Forecast====



Revision as of 22:19, 25 May 2022

External

Internal

Overview

When something goes wrong, a computer tells you about it. This is what a monitor is: a Datadog feature that actively checks metrics, integration availability, network endpoints, etc. and communicates when an alerting condition occurs. The monitor has a query and alert conditions. There are different monitor types.

Monitor Types

Metric Monitor

https://docs.datadoghq.com/monitors/create/types/metric/

Metric monitors watch a continuous stream of data. The metrics are collected via the Datadog Agent or the API and can be alerted upon if they cross a threshold (for example) over a given period of time. Other alert detection methods are available.

Any metric currently reporting to Datadog is available for monitors.

Alert Detection Method

Threshold

A threshold alert compares metric values to a static threshold. This is the standard alert case. On each alert evaluation, Datadog calculates average/min/max/sum over the selected period and checks if it is above or below the threshold. The distribution metric type offers additional threshold options of calculating percentiles over the selected period.

Change

A change alert compares the absolute or relative (%) change in value between N minutes ago and now, and against a given threshold. The compared data points are not single points but are computed using the parameters in the alert conditions section. On each alert evaluation, Datadog calculates the raw difference (a positive or negative value) between the series now and N minutes ago, then computes the average/minimum/maximum/sum over the selected period. An alert is triggered when this computed series crosses the threshold. This type of alert is useful to track spikes, drops, or slow changes in a metric when there is not an unexpected threshold.

Anomaly

An anomaly detection alert uses past behavior to detect when a metric is behaving abnormally. For more details see Anomaly Monitor

Outliers

An outlier alert notifies when a member of a group (host, availability zone, partition, etc) is behaving unusually compared to the rest. For more details see outlier monitors.

Forecast

Host Monitor

https://docs.datadoghq.com/monitors/create/types/host

A host monitor listens to the Datadog Agent heartbeats and notifies on the status of the heartbeat. This could give an indication whether the hosts the Agents run on are responsive. Every Datadog Agent reports a service check called datadog.agent.up with the status OK. The Host monitor has two kind of alert conditions: Check Alert and Cluster Alert.

Anomaly Monitor

https://docs.datadoghq.com/monitors/create/types/anomaly/

Outlier Monitor

https://docs.datadoghq.com/monitors/create/types/outlier/

Outlier Monitor

Triggered Monitor

Downtime

Incident

SLO

Alert

Alert Conditions

Check Alert

Cluster Alert

Notification

Operations