[PR #205] feat(webhook): Vercel receiver + MongoDB queue (changeStream consumer) #205

New issue

Open

opened 2026-10-27 21:10:36 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:10:36 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/Comfy-PR/pull/205
Author: @snomiao
Created: 4/28/2026
Status: 🔄 Open

Base: main ← Head: sno-bot

📝 Commits (7)

07eec2a feat(webhook): MongoDB-backed webhook queue + changeStream consumer
f905926 feat(webhook): Vercel Slack receiver → MongoDB webhook_queue
f502258 feat(webhook): extend queue to github + notion sources
a0eab6c fix: corrupt .claude.json crashed every agent spawn (no useful log)
79bc584 fix(agent-spawn): three bugs that all surfaced as "exit code 1"
5b711e4 fix(bot): stop 3000+ restart loop from poison-pill task resume
ea73861 chore(daemon): migrate comfy-pr-bot from pm2 to oxmgr

📊 Changes

12 files changed (+1457 additions, -54 deletions)

View changed files

📝 CLAUDE.md (+20 -7)
📝 app/api/webhook/github/route.ts (+25 -1)
➕ app/api/webhook/notion/route.ts (+142 -0)
➕ app/api/webhook/slack/route.ts (+137 -0)
📝 bot/error-collector.ts (+8 -0)
📝 bot/index.ts (+99 -0)
📝 bot/slack-bot.ts (+163 -22)
📝 bot/spawn-as-user.ts (+39 -2)
📝 bot/task-user.ts (+33 -0)
📝 bot/up.sh (+25 -22)
➕ bot/webhook-queue.ts (+224 -0)
➕ docs/cloudrun-vs-gke-vs-hybrid.md (+542 -0)

📄 Description

Summary

Move the Slack webhook ingest off the local VM bot and onto a serverless Vercel Function backed by a MongoDB queue. The bot tails the queue via changeStream, so Slack retries no longer get dropped while the bot is restarting/down.

What's new

`bot/webhook-queue.ts` — MongoDB-backed queue

enqueueWebhook() inserts a doc into webhook_queue (Atlas, replica set ⇒ changeStream supported)
startWebhookConsumer() drains any unprocessed backlog at boot, then tails the changeStream filtered by source
markWebhookProcessed() flips processed=true (and stashes truncated error stack on failure) so resume after restart doesn't reprocess
TTL index (expireAfterSeconds: 86400) on createdAt gives a 24h time-cap. Capped collections cap on bytes and don't allow per-doc updates, so we use a regular collection + TTL index instead.
Edge dedup collection (webhook_edge_dedup) with 1h TTL — used by the Vercel route to suppress retry storms before they reach the queue.

`bot/slack-bot.ts` — wire bot consumer

/slack/events HTTP handler now just verifies signature, dedups, and enqueueWebhook() — no inline dispatch.
startSlackBot() calls startWebhookConsumer({ sources: ["slack"] }) so the existing handleSlackEvent runs against queue docs instead of inbound HTTP.

`app/api/webhook/slack/route.ts` — Vercel receiver

Slack HMAC-SHA256 sig verification (5-min replay window)
URL verification challenge passthrough
event_callback → edge dedup → enqueueWebhook({ source: "slack" }) → 200
runtime: "nodejs" so the mongodb driver works (Edge runtime can't open Atlas connections)
Deployed at https://comfy-pr.vercel.app/api/webhook/slack

`docs/cloudrun-vs-gke-vs-hybrid.md`

Background analysis comparing GKE vs Cloud Run vs Hybrid deployment options that led to this lighter-weight queue design.

Cutover

Slack app A078498JA5T Event Subscriptions Request URL has been moved:

prbot.stukivx.xyz/slack/events  →  comfy-pr.vercel.app/api/webhook/slack

Slack verified the new URL successfully. The local /slack/events endpoint remains as a fallback during the cutover; safe to remove after a few days of clean Vercel traffic.

Test plan

bunx tsc --noEmit
bun test bot/taskInputFlow.spec.ts (4/4 pass)
Local: signed event_callback → bot HTTP /slack/events → webhook_queue insert → consumer dispatch → processed=true (~70ms)
Vercel: signed URL verification challenge → 200 {challenge:...} after SLACK_SIGNING_SECRET env var set
Live Slack DM "vercel webhook routing test" → comfy-pr.vercel.app → webhook_queue (source=slack) → VM consumer → processed=true within ~26s
Slack app config: Request URL updated to Vercel, Verified ✓
Run for a few days and confirm no dropped events / no auth issues / retry-storm dedup behaves
Then remove the local /slack/events HTTP handler and stop redirecting prbot.stukivx.xyz to the VM

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/Comfy-PR/pull/205 **Author:** [@snomiao](https://github.com/snomiao) **Created:** 4/28/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `sno-bot` --- ### 📝 Commits (7) - [`07eec2a`](https://github.com/Comfy-Org/Comfy-PR/commit/07eec2ad095fb7f2f257b51061552ccf8da65718) feat(webhook): MongoDB-backed webhook queue + changeStream consumer - [`f905926`](https://github.com/Comfy-Org/Comfy-PR/commit/f90592635451e791b9cfb7107143210d407deb37) feat(webhook): Vercel Slack receiver → MongoDB webhook_queue - [`f502258`](https://github.com/Comfy-Org/Comfy-PR/commit/f502258877b197e15326ce7abbb69a6872efdf97) feat(webhook): extend queue to github + notion sources - [`a0eab6c`](https://github.com/Comfy-Org/Comfy-PR/commit/a0eab6ceac355f525101d5575791abe84ce43222) fix: corrupt .claude.json crashed every agent spawn (no useful log) - [`79bc584`](https://github.com/Comfy-Org/Comfy-PR/commit/79bc5840f5e1d15663ce0c96adb67f3f563423cd) fix(agent-spawn): three bugs that all surfaced as "exit code 1" - [`5b711e4`](https://github.com/Comfy-Org/Comfy-PR/commit/5b711e4ac89b3233940a78c1062b28d9c67940d2) fix(bot): stop 3000+ restart loop from poison-pill task resume - [`ea73861`](https://github.com/Comfy-Org/Comfy-PR/commit/ea73861797c6f7528fc27b5fda7364eb815b51bc) chore(daemon): migrate comfy-pr-bot from pm2 to oxmgr ### 📊 Changes **12 files changed** (+1457 additions, -54 deletions) <details> <summary>View changed files</summary> 📝 `CLAUDE.md` (+20 -7) 📝 `app/api/webhook/github/route.ts` (+25 -1) ➕ `app/api/webhook/notion/route.ts` (+142 -0) ➕ `app/api/webhook/slack/route.ts` (+137 -0) 📝 `bot/error-collector.ts` (+8 -0) 📝 `bot/index.ts` (+99 -0) 📝 `bot/slack-bot.ts` (+163 -22) 📝 `bot/spawn-as-user.ts` (+39 -2) 📝 `bot/task-user.ts` (+33 -0) 📝 `bot/up.sh` (+25 -22) ➕ `bot/webhook-queue.ts` (+224 -0) ➕ `docs/cloudrun-vs-gke-vs-hybrid.md` (+542 -0) </details> ### 📄 Description ## Summary Move the Slack webhook ingest off the local VM bot and onto a serverless Vercel Function backed by a MongoDB queue. The bot tails the queue via changeStream, so Slack retries no longer get dropped while the bot is restarting/down. ## What's new ### `bot/webhook-queue.ts` — MongoDB-backed queue - `enqueueWebhook()` inserts a doc into `webhook_queue` (Atlas, replica set ⇒ changeStream supported) - `startWebhookConsumer()` drains any unprocessed backlog at boot, then tails the changeStream filtered by source - `markWebhookProcessed()` flips `processed=true` (and stashes truncated error stack on failure) so resume after restart doesn't reprocess - TTL index (`expireAfterSeconds: 86400`) on `createdAt` gives a 24h time-cap. Capped collections cap on bytes and don't allow per-doc updates, so we use a regular collection + TTL index instead. - Edge dedup collection (`webhook_edge_dedup`) with 1h TTL — used by the Vercel route to suppress retry storms before they reach the queue. ### `bot/slack-bot.ts` — wire bot consumer - `/slack/events` HTTP handler now just verifies signature, dedups, and `enqueueWebhook()` — no inline dispatch. - `startSlackBot()` calls `startWebhookConsumer({ sources: ["slack"] })` so the existing `handleSlackEvent` runs against queue docs instead of inbound HTTP. ### `app/api/webhook/slack/route.ts` — Vercel receiver - Slack HMAC-SHA256 sig verification (5-min replay window) - URL verification challenge passthrough - `event_callback` → edge dedup → `enqueueWebhook({ source: "slack" })` → 200 - `runtime: "nodejs"` so the mongodb driver works (Edge runtime can't open Atlas connections) - Deployed at `https://comfy-pr.vercel.app/api/webhook/slack` ### `docs/cloudrun-vs-gke-vs-hybrid.md` Background analysis comparing GKE vs Cloud Run vs Hybrid deployment options that led to this lighter-weight queue design. ## Cutover Slack app **A078498JA5T** Event Subscriptions Request URL has been moved: ``` prbot.stukivx.xyz/slack/events → comfy-pr.vercel.app/api/webhook/slack ``` Slack verified the new URL successfully. The local `/slack/events` endpoint remains as a fallback during the cutover; safe to remove after a few days of clean Vercel traffic. ## Test plan - [x] `bunx tsc --noEmit` - [x] `bun test bot/taskInputFlow.spec.ts` (4/4 pass) - [x] Local: signed `event_callback` → bot HTTP /slack/events → `webhook_queue` insert → consumer dispatch → `processed=true` (~70ms) - [x] Vercel: signed URL verification challenge → `200 {challenge:...}` after `SLACK_SIGNING_SECRET` env var set - [x] Live Slack DM "vercel webhook routing test" → comfy-pr.vercel.app → `webhook_queue` (source=slack) → VM consumer → `processed=true` within ~26s - [x] Slack app config: Request URL updated to Vercel, **Verified ✓** - [ ] Run for a few days and confirm no dropped events / no auth issues / retry-storm dedup behaves - [ ] Then remove the local `/slack/events` HTTP handler and stop redirecting `prbot.stukivx.xyz` to the VM 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-27 21:10:36 +00:00