Knowledge transfer · Nov 2026
Handover: keeping @ai running
I'm away from 12 to 24 November. This page is for whoever covers: where @ai lives, how to ship a fix, and what to do when someone says it's broken. For how it all works, see the KT.
Where it lives
- Production:
ai.bhaskarapp.com. Staging:dev.agents.bhaskarapp.com, a Vercel preview alias. - Vercel project:
agents. Functions and sandboxes run insin1. - Repo:
dbdigital-vercel-programmatic-sandbox/agents. Changes reachmainthrough PRs. - Data: Neon Postgres (
DATABASE_URL) and Upstash Redis (memory, spend reports, caches). - Slack connects through Vercel Connect:
slack/aiin production,slack/appbuilderon preview. There are no Slack env vars.
One gotcha: schedules and triggers only run in production. Vercel Cron doesn't fire on preview deployments, so a schedule made on staging never runs.
Shipping a fix
- Open a PR to
main. Before merging, runpnpm typecheck,pnpm buildandpnpm build:eve. - The build doesn't run migrations. If the PR adds a file in
drizzle/, apply it withpnpm db:migrate(or from/cms/migrations) before the new code serves traffic./cms/migrationsonly opens for the emails inappconfig.json→migrationAdminEmails, so add yours. - Rolling back: promote the previous production deployment in Vercel. Migrations don't roll back with it (the revert route only deletes the bookkeeping row), so keep schema changes additive.
- Don't upgrade eve. It's pinned at 0.87.0 with our patch in
patches/, and going back below 0.83 breaks sessions created after it.
When someone says it's broken
| They see | Where to look | What to do |
|---|---|---|
| "Your rolling AI budget is exhausted" | Their row in /cms/users | Raise their AI budget. |
| "Budget verification is temporarily unavailable" | Vercel logs, Upstash and Neon | The spend lookup failed (Redis, the database or the Gateway report). It clears once they're back. |
| They can't use @ai at all | /cms/users | Tick their AI access. |
| @ai answers messages meant for people, or ignores ones meant for it | Observability → Jev → slack-follow-up | Relax the mention screen first, not the 0.8 threshold. |
| A new chat can't get a sandbox | Logs: "Vercel Sandbox create failed" | The log has the Sandbox API's error code. snapshot_region_mismatch means the template snapshot is in another region: scripts/snapshots/README.md. |
| A schedule didn't run | The schedule's page shows its last error | Production only. A failed start is retried after 5 minutes. |
| A trigger didn't run | The trigger's page shows its last error, including Jev's skip reason | No error means the message never matched: check its channel, author and filter. |
| A CloudWatch query is blocked | Their CloudWatch budget in /cms/users | Raise it. CLOUDWATCH_GUARDRAIL_ENABLED=false turns blocking off; spend is still recorded. |
| Errors or slow replies from the model | Logs: "Model call failed", "AI Gateway used a fallback route" | Every call has a fallback model, so a provider outage shows up here first. |
| A bug report | Observability → Bugs | Filed from the web form or the Gluon MCP's report_bug tool. |
Where to look
- Vercel logs. Each line is a fixed message plus JSON, so search for the message, or for a session ID (
wrun_…). - Observability (
/observability, admins only): overview, users, reliability, events, Jev and bugs. - Tables, read-only:
analytics_events(every turn, model call and tool call),jev_evaluations,bug_reports,trigger_fires,scheduled_task_runs,background_commands. - No error tracker. The app has no Sentry SDK, so server errors only reach Vercel logs.
EVE_LOG_LEVEL=debuglogs how long each part of a turn's setup takes.
Emergency levers
- Cut off a user: untick their AI access in
/cms/users, or set their budget to $0. - Stop an automation: the owner can pause it on its Schedules or Triggers page. For someone else's, set its
statustopausedin the database. - Turn off an integration: blank its token env var and redeploy.
- Turn off the Gluon MCP: block
/mcp.
Accounts it depends on
- Vercel: hosting, Sandbox, Cron, Workflow, AI Gateway, and Connect for Slack and the MCP connectors.
- Neon and Upstash.
- GitHub: read, push and PR tokens for each org. If one expires, clones, pushes or PRs fail. The push token for
dbdigital-vercel-programmatic-sandboxis Naman's, and it also merges PRs there. - AWS: S3 for artifacts, and the role the AWS MCP assumes.
@ai KT