How to send LLM alerts to Slack

When you finish, a Slack channel gets a message when your agent's error rate or spend crosses a line, and another when it recovers. You need a self-hosted mcpd that receives your runs, a Slack app with an incoming webhook for the channel, and somewhere to run a 30-line relay: a Cloudflare Worker or any small server.

1. Create a Slack incoming webhook

In your Slack app's settings, turn on incoming webhooks and add one for the channel that should get the alerts. Slack gives you a URL that starts with https://hooks.slack.com/services/. Keep it secret: anyone with the URL can post to the channel.

2. Deploy the relay

mcpd posts one generic JSON body to a webhook, the same for every receiver. Slack's incoming webhooks want a text field instead, so the relay reads the alert and posts a message. The body mcpd sends:

Field	Example
`monitor_id`, `name`, `project`	`support-error-rate`, its name, `support`
`condition`	`tripped`, or `nominal` when it recovers
`watch`, `threshold`, `value`, `samples`	`error_rate`, `0.05`, `0.5`, `40`
`detail`	One sentence for a person: the name, the value, the threshold and the number of runs
`at`	When it was measured, in RFC 3339

Both relays check a token in the query string, so nobody else can post through them. mcpd never shows a webhook's full URL in its API, only the host.

the relay

// A Cloudflare Worker that turns an mcpd alert into a Slack message.
// Secrets: SLACK_WEBHOOK_URL, RELAY_TOKEN. Variable: MCPD_URL.
export default {
  async fetch(request, env) {
    if (request.method !== 'POST') return new Response('POST only', { status: 405 })
    if (new URL(request.url).searchParams.get('token') !== env.RELAY_TOKEN) {
      return new Response('unknown token', { status: 401 })
    }
    const alert = await request.json()
    const state = { tripped: 'Tripped', nominal: 'Recovered' }[alert.condition] ?? alert.condition
    const res = await fetch(env.SLACK_WEBHOOK_URL, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({
        text: `${state}: ${alert.detail ?? alert.name} <${env.MCPD_URL}|Open mcpd>`,
      }),
    })
    return new Response(null, { status: res.ok ? 200 : 502 })
  },
}

# relay.py: turns an mcpd alert into a Slack message. Standard library only.
import json
import os
import urllib.request
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import parse_qs, urlparse

SLACK_WEBHOOK_URL = os.environ["SLACK_WEBHOOK_URL"]
RELAY_TOKEN = os.environ["RELAY_TOKEN"]
MCPD_URL = os.environ.get("MCPD_URL", "http://localhost:4317")


class Relay(BaseHTTPRequestHandler):
    def do_POST(self):
        token = parse_qs(urlparse(self.path).query).get("token", [""])[0]
        if token != RELAY_TOKEN:
            self.send_response(401)
            self.end_headers()
            return
        alert = json.loads(self.rfile.read(int(self.headers["content-length"])))
        state = {"tripped": "Tripped", "nominal": "Recovered"}.get(alert["condition"], alert["condition"])
        text = f"{state}: {alert.get('detail') or alert['name']} <{MCPD_URL}|Open mcpd>"
        req = urllib.request.Request(
            SLACK_WEBHOOK_URL,
            data=json.dumps({"text": text}).encode(),
            headers={"content-type": "application/json"},
        )
        with urllib.request.urlopen(req, timeout=10) as res:
            self.send_response(200 if res.status == 200 else 502)
        self.end_headers()


HTTPServer(("0.0.0.0", int(os.environ.get("PORT", "8080"))), Relay).serve_forever()

3. Add the monitors

Monitors live in the daemon's JSON configuration file. Point each one's webhook at the relay, with the token.

mcpd.json

{
  "monitor_interval_seconds": 60,
  "monitors": [
    {
      "id": "support-error-rate",
      "name": "Support agent error rate above 5%",
      "project": "support",
      "watch": "error_rate",
      "compare": "above",
      "threshold": 0.05,
      "window_seconds": 900,
      "min_samples": 20,
      "webhook": "https://relay.example.com/?token=YOUR_RELAY_TOKEN"
    },
    {
      "id": "support-daily-spend",
      "name": "Support agent spent more than $50 in a day",
      "project": "support",
      "watch": "cost",
      "compare": "above",
      "threshold": 50,
      "window_seconds": 86400,
      "min_samples": 1,
      "webhook": "https://relay.example.com/?token=YOUR_RELAY_TOKEN"
    }
  ]
}

Key	What it sets
`watch`	`error_rate` (0 to 1), `latency_p95` (milliseconds), `cost` (in the price book's currency), `trace_count`, or `score_mean` with a `scorer_id`
`compare`, `threshold`	`above` or `below` the number
`window_seconds`	How far back each reading looks. Default 300.
`min_samples`	The fewest runs in the window before the monitor can trip. Default 20.
`monitor_interval_seconds`	How often every monitor is read. The floor is 5.

Monitors and alerts lists every watch, including the two that read durable runs.

4. Start mcpd with the file

mcpd refuses to start on an unknown key, so a misspelled threshold fails loudly instead of watching nothing.

start

mcpd --config-file ./mcpd.json

# Check what the daemon read, without starting it.
mcpd config --config-file ./mcpd.json | grep monitors

mcpd config prints a line such as monitors 2, every 60s.

5. Read and acknowledge alerts

Monitors evaluated every 60 s

error_rate > 5% resolved
latency_p95 > 4 s ok
cost > $5 / h ok
score_mean no_tool_loop < 0.9 ok

It fires when the error rate crosses the line, stays firing while there are too few runs to judge, and resolves only on a real reading under the line.

A monitor latches. Once it trips it stays tripped until the number recovers, and a quiet window with no runs does not clear it. Each change of state is posted once: one message when it trips, one when it recovers. Acknowledging silences it while it still breaches.

read and acknowledge

# What each monitor reads now, and since when it has been tripped.
curl -s http://localhost:4317/v1/api/monitors | jq -c '.[] | {id: .monitor.id, condition, value: .last.value}'

# Acknowledge an alert. It stays acknowledged until the number recovers.
curl -s -X POST http://localhost:4317/v1/api/monitors/support-error-rate/ack

The Monitors screen in the interface shows the same state.

Troubleshooting

What you see	Why, and what to do
The monitor never trips, and its condition is `insufficient`	The window holds fewer runs than `min_samples`. Lower it, or widen `window_seconds`.
The log says `could not reach the alert webhook`	mcpd cannot connect to the relay. Check the URL from the machine mcpd runs on.
The log says `the alert webhook refused the notification`	The relay answered with an error: a wrong token, or Slack refused the post. The log line has the status.
One alert was never posted	mcpd does not retry a failed delivery. It logs it and moves on. The monitor's state is still correct in `GET /v1/api/monitors`.

mcpd is free and open source, one binary with its storage built in. Run it with the Quickstart and point your agent at it.