CI/CD

Babysit Fly deployment

On every Fly app rollout, confirm machines moved cleanly and check post-deploy health, opening an incident with a rollback recommendation when degraded.

Triggered when Fly machine churn detects a new image ref rolling out across an app. The agent lists the app's machines and confirms every machine moved to the new image and is in the started state. If any machine is stuck on the old image or in a failed state, it opens an incident immediately with the affected machine IDs. For successful rollouts it compares post-deploy error rate and latency against a 2-hour baseline using Fly Prometheus metrics plus any connected APM, and opens an incident when the new image is degraded. The incident names the app, image ref diff, failing metric, and the previous image ref to roll back to. No automatic rollback — use this when you want a human to make the call.

Install

Create this automation from the CLI, or pick it in the console under Automations → New → From template.

Terminal
polylane automation from-template babysit-fly-deployment

Trigger

The event that starts this automation.

Fly Deploy10min delay

Fires when a Fly app rolls out a new image across one or more machines.

Compatible providers

1

Integrations this automation investigates and uses tools from. Connect them to unlock the full workflow.

Fly.io

Skills

3

These skills are installed automatically when you create this automation.

/deployment-validation
/investigate-errors
/investigate-latency

Actions

1

Platform-executed side effects the agent can request. These are applied when you create this automation.

Open Incident smart

Agent instructions

The prompt the agent follows on each execution. Customize it to fit your team's playbook.

## Role You are Fly Deploy Investigator. Validate the rollout that just landed; if machines are stuck or the new image is degraded, open an incident with evidence and a rollback recommendation. You do not roll back yourself. ## Sources 1. Trigger payload: app ID, image ref, machines updated, regions. 2. Fly machine listing: machine state, current image, region. 3. Fly Prometheus metrics + connected APM scoped to the app. ## Workflow 1. List machines and confirm the rollout completed across all regions. 2. If any machine is stuck on the old image or in a failed state, open an incident with severity 'high' and the affected machine IDs. 3. For successful rollouts: compare error rate and latency against the prior 2-hour baseline. If degraded, open an incident. 4. If healthy, post a brief HEALTHY report and stop. ## Output Incident body should include: app ID, image ref diff, stuck machine IDs (if any), evidence (logs or metric comparisons), recommended rollback (redeploy previous image ref). ## Operating rules - Cite metrics with provider, query, and time range. - Do not open an incident for healthy deploys. - Use severity 'critical' for full-outage signals, 'high' for stuck machines or elevated error rate, 'medium' for latency-only degradation.

Parallel passes

Each execution runs multiple analysis passes with different perspectives, then aggregates the results.2 passes

How it works

When a fly deploy event occurs, an agent runs the instructions above, investigating with your connected Fly.io tools, and sends results to your configured destinations.

Destinations

When you create this automation your email is added as a destination, so every run lands in your inbox. Add chat or webhook destinations afterward.

Related templates