awseksPractitioner

Streamline EKS Operations: Cut MTTR with AWS DevOps Agent and Kubernetes Operator

5 min read AWS DevOps BlogAug 31, 2026Reviewed for accuracy

Practitioner — Hands-on experience recommended

In today's fast-paced cloud environments, downtime can be costly. The AWS DevOps Agent and its Kubernetes Operator are designed to optimize operations in Amazon EKS by autonomously analyzing root causes of failures. This means you can detect issues faster and respond more effectively, ultimately reducing your MTTR.

The DevOps Agent Operator plays a crucial role by detecting failures within your EKS cluster and sending the necessary context to the AWS DevOps Agent for investigation. To make this work, you need an external source to trigger the DevOps Agent through a webhook. Two conditions must be met: immediate failure detection and sufficient context. This setup allows you to narrow down which failures trigger an investigation by configuring parameters like WEBHOOK_MIN_SEVERITY and WEBHOOK_SKIP_CATEGORIES, although specific defaults for these parameters are not provided.

In production, you should be aware that if data collection or uploads to Amazon S3 or CloudWatch Logs fail, the system will requeue the pod with exponential backoff rather than dropping it. This ensures that your investigations continue without losing critical data. Additionally, be cautious with IAM permissions; while the example policy allows broad access, it's best to restrict it to your cluster’s nodes to enhance security. The AWS DevOps Agent is available in multiple regions, but ensure you are using Amazon EKS managed node groups or self-managed EC2 nodes for node-level log collection to function correctly.

Key takeaways

→Configure the DevOps Agent Operator to detect failures and trigger investigations automatically.
→Use webhooks to send context to the AWS DevOps Agent for effective failure analysis.
→Limit IAM permissions to your cluster’s nodes for enhanced security.
→Monitor data collection failures to avoid dropped pods and ensure continuous investigation.
→Utilize the AWS DevOps Agent across multiple AWS regions for optimal performance.

Why it matters

In production, reducing MTTR can significantly improve service reliability and customer satisfaction. Automating failure detection and investigation allows teams to focus on resolution rather than manual monitoring.

Code examples

JSON

1cat >devops-agent-operator-permission.json <<EOF
2{
3  "Version": "2012-10-17",
4  "Statement": [
5    {
6      "Sid": "SSMCommandExecution",
7      "Effect": "Allow",
8      "Action": [
9        "ssm:SendCommand",
10        "ssm:GetCommandInvocation"
11      ],
12      "Resource": [
13        "arn:aws:ec2:<aws-region>:*:instance/*",
14        "arn:aws:ssm:<aws-region>:*:*"
15      ]
16    },
17    {
18      "Sid": "S3LogStorage",
19      "Effect": "Allow",
20      "Action": [
21        "s3:PutObject"
22      ],
23      "Resource": "arn:aws:s3:::<s3-bucket-name>/*"
24    },
25    {
26      "Sid": "S3BucketAccess",
27      "Effect": "Allow",
28      "Action": [
29        "s3:ListBucket"
30      ],
31      "Resource": "arn:aws:s3:::<s3-bucket-name>"
32    },
33    {
34      "Sid": "CloudWatchLogsIncidentStorage",
35      "Effect": "Allow",
36      "Action": [
37        "logs:CreateLogStream",
38        "logs:PutLogEvents"
39      ],
40      "Resource": "arn:aws:logs:<aws-region>:*:log-group:/<cloudwatch-log-group-name>:*"
41    }
42  ]
43}
44EOF