Sentinels

Prove every infrastructure change is safe

Stop squinting at dashboards and working through a kubectl checklist after every change. Sentinels package AI-assisted analysis and deep integration tests into one automated, auditable check you can run across any set of clusters.

Book a demo Read the docs

sentinel · k8s-upgrade · run #57● 48 clusters · tier: dev

console-logsLOG3 errors found, all match known-noise rules✓ pass

ingress-healthKUBERNETESingress-nginx healthy on 48/48 clusters✓ pass

integration-testsINTEGRATION_TEST47/48 clusters passed✗ fail

Sentinel failed · promotion to staging blocked

AI ANALYSIS

dev-euw1 › PVCs bind and mount

Likely cause: the EBS CSI driver add-on is still on a version incompatible with 1.31, so volumes never attach. Upgrade aws-ebs-csi-driver to v1.35 and re-run.

Re-run on dev-euw1

Triggered by pipelineRules from git@mainJUnit export

Three kinds of checks

Everything a senior SRE would check, every time

Combine checks into one sentinel to confirm everything from container scheduling to storage and networking is working as expected.

AI resource validation

Deep-dive any resource, graded against your rules

Point a sentinel at a Deployment, StatefulSet, or any other resource. It investigates everything associated with it, including logs and related components, and returns a pass or fail.

Natural-language rule files, versioned in Git, tell the AI what to ignore and what counts as failure, so your team’s tribal knowledge is applied the same way every time.

rules/logrule.md

# logrule.md

Ignore "leader election lost" from cert-manager

if it recovers within 30s.

Fail on any x509 or certificate expiry error.

Fail if external-dns reports throttling for

more than 2 minutes.

Intelligent log tailing

Catch the failures that only show up over time

Watch error logs in chosen namespaces for a set window after a change, and analyze them with the same rule files. It’s the automated version of running kubectl logs -f and waiting to see if anything breaks.

Tailing errors · 5m window● live

kube-systemcoredns: i/o timeout 10.0.0.2:53ignored

cert-managerleader election lostignored

external-dnsThrottling: rate exceededwatching

Deep integration testing

Exercise the cluster, don’t just observe it

A default suite starts StatefulSets, binds storage volumes, and confirms pod-to-pod networking, with no setup required. Bring your own Terratest suites to validate the paths that matter to you.

Integration suite · JUnit

pods schedule and start✓

PVCs bind and mount✓

pod-to-pod networking✓

service DNS resolution✓

custom: payments smoke test✓

Built for

The changes that keep platform teams up at night

Kubernetes upgrades

Prove a version bump introduced no regressions before it reaches production.

Operator changes

Validate cross-cutting upgrades like Istio or cert-manager across every cluster they touch.

Network reconfigurations

Confirm CNI, ingress, and DNS changes are safe while there is still time to roll back.

GitOps native

Infrastructure tests, versioned like code

A sentinel isn’t a feature that only lives in a UI. Its checks, rules, and tests are a Kubernetes custom resource in Git, and the whole system is API-driven.

Plural consoleRun any sentinel on demand from AI → Sentinels.

CI pipelinesCall the API from GitHub Actions, GitLab, or any CI system.

Plural pipelinesGate promotions between environments on a passing run.

Set up your first sentinel

bootstrap/sentinels/k8s-upgrade.yaml

apiVersion: deployments.plural.sh/v1alpha1

kind: Sentinel

metadata:

name: k8s-upgrade

spec:

description: Baseline health after a Kubernetes upgrade

repositoryRef:

name: infra

namespace: infra

git:

ref: main

folder: rules

checks:

- name: console-logs

type: LOG

ruleFile: logrule.md

configuration:

log:

query: error

duration: 5m

namespaces: [cert-manager, external-dns, kube-system]

- name: integration-tests

type: INTEGRATION_TEST

configuration:

integrationTest:

format: JUNIT

tags:

tier: dev

Compliance

“Where’s the test from four months ago?” Right here.

Change management often requires proof that testing happened after every significant infrastructure change. Sentinels produce it automatically: pull structured JUnit results from the API and forward them to your compliance system.

Results are stored and retrievable, so audits stop being a search through old Slack threads and Confluence pages.

Read the announcement

GET /v1/sentinels/k8s-upgrade/runs/57

<testsuite name="integration-tests"

tests="240" failures="1">

...

</testsuites>

Questions

Why use AI for infrastructure testing?

Confirming infrastructure health usually means reading several text sources, like logs and resource status, and applying judgment. That can’t be done deterministically, so Sentinels use governed AI guided by rule files you write and version in Git. Integration tests cover the deterministic paths.

Can I bring my own tests?

Yes. Sentinels ship with a default suite that exercises scheduling, storage, and networking, and the runner can execute your own suites, such as Terratest, across any set of clusters.

How do results reach our compliance tooling?

Every run produces structured output in formats like JUnit that you can pull from the API and forward to whatever system your change-management process uses. Results stay stored and retrievable later.