112 KiB
Discord Bot
Immich Discord bot built with NestJS, discordx, and PostgreSQL (Kysely ORM).
Tech Stack
- Runtime: Node.js (24.x), TypeScript, CommonJS
- Framework: NestJS with
@nestjs/schedulefor cron jobs - Discord: discord.js + discordx (decorator-based slash commands, events, modals, buttons)
- Database: PostgreSQL via Kysely (type-safe query builder), file-based migrations
- Testing: Vitest with manual mocks;
database.repository.spec.tsalone runs against a real database, and only whenTEST_DB_URLnames one migrated to the latest schema (it deletes that database's mirror link, identity and Zulip expander rows, and checks the expander migration's seed by running itsdownandupin a transaction it rolls back); otherwise it is skipped - Build:
nest build(SWC compiler),eslint,prettier
Architecture
Layers
- Discord layer (
src/discord/) - Slash commands, events, help-desk, context menus. These are@Discord()+@Injectable()NestJS classes that use discordx decorators (@Slash,@On,@ModalComponent,@ButtonComponent). - Service layer (
src/services/) - Business logic. Injected into discord layer. Services use@Inject(ITokenName)for repository dependencies. - Repository layer (
src/repositories/) - External integrations (database, Discord API, GitHub, Zulip, RSS, etc). Each has an interface insrc/interfaces/with a string token (export const IFoo = 'IFoo'). - Interface layer (
src/interfaces/) - Defines repository contracts and Kysely table types. TheDatabasetype indatabase.interface.tsmaps table names to their column types. - Renderer layer (
src/renderers/) - Pure functions, no DI, one module per chat platform. Each turns a platform-neutralNotificationinto that platform's wire shape (toDiscordMessage,toMattermostBlock,toZulipMessage). Consumed only byNotificationService; services never import a renderer, and no renderer imports another.
Dependency Injection
Repositories are provided via NestJS DI tokens in src/repositories/index.ts:
{ provide: IDatabaseRepository, useClass: DatabaseRepository }
Services inject them with @Inject(IDatabaseRepository).
Registration
- Services: Listed in
src/services/index.ts→ imported intoAppModule - Repositories/Providers: Listed in
src/repositories/index.ts→ imported intoAppModule - Discord classes: Listed directly in
AppModule(DiscordCommands,DiscordEvents,DiscordHelpDesk,DiscordContextMenus,DiscordMirrorEvents,DiscordMirrorCommands) - Init order:
AppModule.onModuleInitcalls each service'sinit()by hand, in a fixed order.ZulipExpanderService.init()comes right after the migrations: it loads the streams with GitHub expansion that the expanders and the queue registration read, so it must precedeZulipService.init(). A service that subscribes to Zulip events (ZulipService.onMessage,onMessageUpdate,onMessagesDeleted,onQueueRegistered) must have itsinit()called there, and beforeZulipService.init(), which starts the event loop (see Zulip event queue). Today that isMirrorService(the Discord-Zulip mirror, first, right after the migrations, since it reads its links from the database; its handlers only enqueue),ChatService(the expanders) andZulipCommandService(the commands). Last, once Zulip is up,ChatService.loginToDiscordlogs in to Discord (not with thedevtoken) and awaits it, so the HTTP server listens only once Discord is ready: a login that hangs holds back the webhooks and the@Cronjobs, while the Zulip event loop, already started, keeps running.DiscordRepository.loginresolves atclientReady, once the guilds and so their channels are cached; discord.js's ownloginresolves at the gateway's READY, before them. Alongside the login,I'm alive, running <version>!goes toteam.botonce per process, never on a gateway reconnect: when Discord turns ready, at once with thedevtoken, or without Discord after 60s (DISCORD_READY_WAIT_MS) of a login still pending. A failed login is reported toteam.botinstead (Discord login failed: …, which reaches Zulip alone) and then fails the boot, so a restart retries it.
Database Migrations
Located in src/schema/migrations/ with naming pattern {timestamp}-{description}.ts. Each exports up() and down() functions running raw SQL through Kysely. Migrations run automatically on module init via DatabaseService.runMigrations(), so every migration must be safe on a populated table.
Tables are declared with @immich/sql-tools decorators in src/schema/tables/, and a migration is generated from the difference between those declarations and a database that is at the current schema: npm run build, then DB_URL=postgres://... npm run migrations:generate. The generator writes src/{timestamp}-Migration.ts; move it into src/schema/migrations/ under a descriptive name. To verify one, boot the compiled app once against a scratch database (uri in the environment) and check the schema.
Adding a New Database Table
- Declare the table in
src/schema/tables/{name}.table.tsand generate the migration (see above) - Add Selectable/Insertable/Updateable types in
src/schema/index.ts - Add table to the
Databaseinterface there - Add repository methods to
IDatabaseRepositoryinterface - Implement methods in
src/repositories/database.repository.ts
Adding a New Slash Command
Commands live in src/discord/commands.ts (the mirror's in src/discord/mirror-commands.ts). Use discordx decorators:
@Slash({ name, description })on method@SlashOption({...})for parameters@SlashChoice(...)for enum choices- Autocomplete: set
autocomplete: trueon option, checkinteraction.isAutocomplete()in handler
Auth Guard
Legacy commands use authGuard() to restrict to allowed channels (BotSpam, SupportCrew, QQ). New commands should NOT use authGuard — permissions will be configured via Discord's built-in command permissions UI instead.
Discord handler errors
discordx drops whatever an @On/@Once handler throws: no log, and no client error event. The client therefore has one global guard, reportErrors (src/discord/guards.ts), which runs around every discordx handler (events, slash commands, context menus, buttons, modals, simple commands) and hands what it throws to the handler set with DiscordRepository.onHandlerError. ChatService.init sets that to ChatService.onError, the same method the client error event reaches through DiscordEvents.onError: DiscordAPIError[10008] is ignored, anything else goes through logError as Discord bot error: … to team.bot. A throwing interaction handler is caught by the guard as well, so it no longer reaches the client error event and is reported once. The guard never throws: a failed report is logged. It calls the handler directly instead of emitting error, so a failing error handler cannot recurse.
Cron Jobs
Use @Cron(expression) decorator from @nestjs/schedule. Cron expressions stored in Constants.Cron.
Notifications
Anything posted to a chat channel as a card (GitHub events, GitHub status incidents, purchases, reports, release alerts) goes through one seam. A service never names a platform in a notification path.
- The service builds a
Notification(src/interfaces/notification.interface.ts): a requiredkind(feed,release,incident,purchase,report,alert,rss,log), an optional domain-namespacedaccent(pr.merged,issue.closed,order.cancelled, ...), plusauthor,title,url,body,fieldsandtimestamp(an ISO 8601 string that only thersskind renders). - The service calls
NotificationService.notify(destination, notification)with a logical destination such ascommunity.releasesorteam.purchases. Destinations are audience-scoped:community.*is public,team.*is internal. Business rules like "a private repo skips the community" are expressed by choosing destinations, not platforms. NotificationRoutesinsrc/constants.tsmaps every destination to the platforms and channels it reaches, including per-routesilent(Mattermost),crosspost(Discord) andtopic(Zulip). A destination with no route for a platform simply does not post there. The whole matrix is reviewable in that one table. Today everyteam.*destination except the FHS ones reaches Zulip (streamImmichThirdParties, one topic per subject;team.release-alertsandteam.botgo toImmichAlerts);community.*destinations are Discord only.NotificationService(src/services/notification.service.ts) renders the notification once per routed platform with that platform's renderer and sends it, Discord first, then Mattermost, then Zulip, one platform at a time.- A notification whose channel is chosen per row rather than by a route (an RSS feed) goes through
NotificationService.notifyTarget(target, notification)instead.NotificationTarget(notification.interface.ts) is one explicit address:{ platform: 'discord', channelId },{ platform: 'mattermost', channelId }or{ platform: 'zulip', stream, topic }, andtoNotificationTarget(exported fromnotification.service.ts) turns a stored row'sservice,channelIdandtopicinto one, so the caller never branches on the platform. It uses the same renderers and the samedeliverasnotify, with the target as the log label (Could not notify channel 123 on discord: …,… stream 107, topic "blog" on zulip: …), and differs from it in one way: it resolves to whether the post went through (falsefor a failed send, for a Discord target while Discord is not ready, and for a Zulip target while Zulip is not initialised, each skipped without rendering or logging), so the caller can retry what was not delivered.notifystill resolves to nothing: nothing acts on its result, and its callers post one event to several destinations in turn.
Delivery policy, covered by notification.service.spec.ts:
- Unconfigured platforms are skipped. A Zulip route is attempted only when
zulip.isInitialised()(the same notionZulipRepositorythrows on:initnever ran because thedevsentinel keys skipped it), and a Discord route only whendiscord.isReady()(discord.js'sisReady: never with thedevtoken or after a failed login, and not beforeclientReady, when a send would find no guild channel and post nothing). Local dev therefore never throws on team notifications. Mattermost is always configured. - A platform outage never rejects. A platform that fails does not stop the ones after it, and
notifyresolves even when every attempted platform failed. Each failure is logged as anerrorwith the destination and platform; when no platform took the notification, onefatalline (Could not notify <destination> on any platform: notification dropped) says so.src/main.tsenables thefatallevel for that reason.- The trade-off: a webhook now answers success even if every chat post failed, and the log is the only place a dropped notification shows.
- The alternative, rejecting on a total failure, silences Zulip whenever Discord is down. The handlers post one event to several destinations in sequence (
await notify('community.pull-requests'), thenawait notify('team.pull-requests')), andcommunity.*destinations route to Discord alone, so the first call's rejection would skip the team post and Zulip with it. That is worse now that Zulip is the team's primary platform, and it cannot be fixed in the services, which this seam keeps unchanged.
- Rendering is not a platform failure. Each platform's payload is rendered immediately before that platform's own send, in platform order and outside the isolation above. A renderer bug propagates to the caller as the programming error it is instead of being logged as an outage, and a bug in a later platform's renderer cannot undo the posts already made before it. Nothing is rendered for a skipped platform.
Rules that keep the seam clean:
- Renderers derive every layout decision (title size, whether a body slot exists, truncation, fields layout, author style, whether the title links) from
kind, never from which keys a notification has or what its values are: a feed title links even when itsurlis''. Services never pass render options. Only whether an existing slot is filled depends on the data: afeedalways has a body slot, which renders empty when there is no body.- One exception is
rss. A feed post's title, link and date are optional content thattoRSSNotificationdrops when they are missing or invalid, so each renderer shows those slots only when they are filled: the title links only when there is aurl, an untitled post is labelled with itsurl, and the Zulip date appears only with atimestamp. The DiscordtoRSSEmbedsets only the keys the post has, which keeps its embed identical to the oneEmbedBuilderbuilt before the seam. The rest ofrss's layout still comes from its kind (optionalTitleAndLinkandtimestampSlotin the renderers' tables). - The other exception is
log(below): every renderer shows its detail, the: bodyon Discord and the quote on Zulip, only when there is a non-emptybody. Every other kind follows the rule above.
- One exception is
logis not a card: it is the bot's own startup line and every errorlogErrorandwithErrorLogging(src/util.ts) report, all posted toteam.bot(Discord #bot-spam, ZulipImmichAlertstopicbot).toDiscordMessagesends it as plain text,title: body(the title alone without a body);toZulipMessageposts the title as a line and quotes the body (an error's text, which can hold anything), both mention-neutralised. Mattermost has no route for it. A failed post is logged byNotificationServiceand never goes back throughlogError, so reporting an error cannot recurse.- Truncation that applies on every platform is content and belongs in the service (feed bodies are shortened to 500 before rendering). Truncation that applies on one platform is presentation and belongs in that renderer (release descriptions are shortened to 500 on Mattermost only).
- An accent token names the event at its call site, never a colour.
src/renderers/palette.tsmaps tokens to RGB numbers; the Discord and Mattermost renderers read it. Zulip has no colours, sosrc/renderers/zulip.renderer.tskeeps its own token-to-emoji table (Emoji) chosen by what the token means, not by the colour it shares. Several tokens sharing a colour or an emoji is expected. webhook.service.ts,schedule.service.tsandrss.service.tsnever calldiscord.sendMessage,mattermost.sendorzulip.sendMessagefor a notification. The Zulip release announcement inhandleReleaseNotificationand the pull request topic messages inhandlePullRequestZulipTopicare bespoke plain-text messages, notNotifications, and stay direct calls, exactly as the Discord forum-thread messages inhandlePullRequestTeamUpdatedo.
Adding a New Notification Platform
- Add
src/renderers/{platform}.renderer.ts: a pureto{Platform}Message(notification: Notification)that switches onkindfor layout and mapsaccentto the platform's affordance (readPalettefor a colour, or keep a token-to-emoji table for a platform without colours). Do not import another renderer ordiscord.js, directly or throughsrc/util(which depends on it); string helpers such asshortenandasHexColorcome fromsrc/format.ts, as do the Zulip markdown guards (neutraliseZulipMentions,neutraliseZulipLabel,toZulipQuote), which live there rather than in the renderer because the bespoke PR-topic messages inwebhook.service.tsand thesimilarcommand need them too and a service never imports a renderer. - Add an optional
{platform}entry toNotificationRouteand fill in the routes inNotificationRoutes(src/constants.ts). Destinations that share a channel today (issues and discussions, purchases and reports) are separate on purpose so they can land in different places; on Zulip they already are. - Inject the platform's repository interface into
NotificationService, add ato{Platform}method that renders and sends throughdeliver, and call it innotify, after the platforms already there, when the destination has a route for it (and the platform is configured, if it can be unconfigured).delivertakes a render thunk and a send and handles the lazy rendering, logging and failure isolation. Add the platform's address toNotificationTarget, a case tonotifyTargetand, if rows can store it, a branch totoNotificationTarget.
Nothing in webhook.service.ts or schedule.service.ts should change.
Zulip
The bot talks to Zulip through a typed openapi-fetch client, not an SDK.
- Generated types:
src/generated/zulip.tsis generated from Zulip's OpenAPI spec and must never be edited by hand; it is excluded from prettier and eslint. Regenerate it withnpm run zulip:types. That script inpackage.jsonis the only place the Zulip release tag is pinned; bump it there when the server is upgraded, rerun the script and commit the output. - Transport:
src/repositories/zulip.client.tsbuilds aClient<paths>per identity (createZulipClient). Its rules, all covered byzulip.client.spec.ts:- Base URL is
${ZULIP_DOMAIN}/api/v1; a realm ending in/or/apiis normalised. - Every request body is sent
application/x-www-form-urlencoded, which is what the spec declares for every endpoint we call (POST /messages, laterPATCH /messages/{id}andPOST /register). Strings go as they are;number,boolean, arrays and objects areJSON.stringify'd;undefinedis omitted. Query strings follow the same rule, so an array such asnarrowbecomes one JSON value, never repeated keys. A parameter the spec declares as a JSON-encoded string (narrowandmessage_idsonGET /messagesare typedstring) is passed already stringified. - Multipart (
POST /realm/emoji/{emoji_name}): spreadmultipart({ field: file })into the call. Every part is aFile, so it carries a filename with an extension and a content type;fetchsets the boundary. Do not setContent-Typeyourself. - Every call rejects with
ZulipApiError(status,code,msg) on a non-2xx response or aresult: "error"body, sodatais always set when a call resolves. Network errors and timeouts reject too; every request has a timeout. - A
429is retried after the body'sretry-after(falling back to theRetry-Afterheader), with a bounded attempt count and a bounded maximum wait; the retried request re-sends its body. Nothing else is retried: a 429 was not processed, but retrying a 5xx onPOST /messagescould double-post. - Zulip counts requests per user and says in every answer what is left (
X-RateLimit-Remaining) and when the whole budget is back (X-RateLimit-Reset, in seconds, read against the answer'sDateso the bot's clock does not matter). Once an answer says0is left, every request of that identity waits for the reset, at most a minute (Zulip's default rule is 200 a minute), with onewarn, instead of meeting a 429; a wait ends with its request when the caller's signal aborts. The budget (ZulipRateLimit) is per identity, not per client:ZulipRepositorygives the bot's three clients one and the user's another. On a 12.3 server the reset of a spent budget is a minute off (a 429 would allow the next request a fraction of a second later, since Zulip refills gradually), so the wait is conservative but costs no throughput over a minute; 230 requests in a row took 63s there with no 429. - Credentials and the
Authorizationheader are never logged.
- Base URL is
- Two identities, four clients:
ZulipRepositoryholds abotclient (posts messages) and auserclient (uploads emoji) because Zulip only lets human accounts upload emoji (This endpoint does not accept bot requests). Config keepszulip.botandzulip.userfor that reason. Anuploadsclient is the bot identity with a 120s timeout, for the mirror'sPOST /user_uploads. A fourth client,events, is the bot identity again with a long timeout, used forGET /eventsalone: the server holds that request open on purpose, for up to theevent_queue_longpoll_timeout_secondsit returns fromPOST /register(90 by default), so it cannot share the 30s that every other request should fail within. The server answers a quiet queue with a heartbeat about that many seconds after it received the poll, so the client's timeout is that value plus a 30s margin (longpollTimeoutMs, 120s by default): a client timeout equal to the server's would lose the race on every quiet poll and make an idle channel look like a dead connection.registerQueuerebuilds theeventsclient with the server's value plus the margin. Every request also carries the caller's ownsignalwhen one is passed (getEventspasses the event loop's): the client composes it with the attempt's timeout ininit(AbortSignal.any), sincefetch(request, init)replaces the request's signal withinit.signalrather than adding to it. All are created once, inZulipService.init(skipped with thedevsentinel keys); calling a repository method before that throwsZulip client not initialised. - Endpoints:
ZulipRepositoryexposes only what the bot uses today:sendMessage(resolves to the new message's{ id }),getMessage(the message's ID, stream, sender name and current topic, asked withallow_empty_topic_nameso the empty "general chat" topic comes back as''rather than as the realm's translated display name),updateMessage(PATCH /messages/{id}: a content edit or a topic move with apropagateMode, never both in one call, which Zulip rejects),createEmote,listEmoji(GET /realm/emoji, deactivated ones included),getSubscriptions(the bot's streams), the event queue'sgetOwnUser(the bot's ID and full name; throwsZulip returned no user ID for the botrather than resolve without the ID the loop filters its own messages by, while a missing name comes back as'', since only the command router needs it),getMessages(GET /messages: thenumBeforenewest messages of one stream and topic,anchor: newest, as raw markdown, with the narrow passed as one JSON string; thesimilarcommand reads the topic with it),registerQueue(the queue plus the streams it will carry, each with whether it is private),getEvents(with the loop's abort signal) anddeleteQueue(see below),isInitialised, whichNotificationServicechecks before routing to Zulip,getUser(a user's name and role, which the mirror commands authorise by),getStream(a stream's name and privacy),sendDirectMessage(the answer to a direct message command), and for the mirrordeleteMessage,uploadFile(POST /user_uploads, multipartfilename),downloadUpload(a/user_uploads/path, see Discord-Zulip dev mirror),getStreamMessagesBefore(a page of one stream before a message ID or from the newest, optionally leaving one sender out with a negatedsendernarrow),getEmojiCodes(the realm's staticemoji_codes.json:unicode, name to Unicode, andnames, code point sequence to name; the emote sync reads the names, the mirror both) andaddReaction/removeReaction(as the bot, the emoji given by name, code and type; Zulip answersREACTION_ALREADY_EXISTSorREACTION_DOES_NOT_EXISTwhen there is nothing to do).getMessagealso returns the message's reactions with who gave each, andlistEmojieach realm emoji's ID. Each phase adds only the endpoints it needs, a few lines each thanks to the generated types; do not add unused methods. - Streams:
Constants.Zulip.Streamsholds the channels the bot posts to by numeric ID, named after the channel (ImmichThirdParties: 111carries every team notification,ImmichPullRequests: 112one topic per pull request,ImmichAlerts: 113the release workflow alerts and the bot's own startup and error lines).Constants.Zulip.TeamStreamsholds the ones it listens in: every privateimmich-*stream, 107 to 113; the two are separate maps becauseStreamsis pinned by a characterization test.Constants.Zulip.Commands(where commands are taken) is that set; where the expanders run is configured at runtime (see Zulip message expanders). IDs survive a rename; the dev server mirrors the names but not the IDs. Topic strings live inNotificationRoutes, not here. - Startup subscription check: those three are private streams, and Zulip lets an unsubscribed bot post to public streams only. The bot is a plain member and cannot subscribe itself to a private stream, so
ZulipService.initfetches its subscriptions right after the clients exist and logs onewarnper stream inConstants.Zulip.RequiredSubscriptionsit is missing, naming the stream. It never throws and never self-subscribes: a missing subscription is visible at deploy time instead of failing the first post at 3am, and it must not stop the bot from booting.RequiredSubscriptionslists those three and notFUTOStaff(2), where the holiday notice has posted since before the check existed: whether that stream is public or the bot was subscribed by hand, its subscription is not one the deploy needs to prove, so it is not asserted on a guess. The streams the bot listens in are checked separately, by the event loop at every queue registration (see below): they are not inRequiredSubscriptions, whose three entries and warn text are pinned by characterization tests. - Renderer:
toZulipMessage(src/renderers/zulip.renderer.ts) flattens aNotificationinto one message of Zulip markdown.zulip.renderer.spec.tspins all of the following.- Shape:
{emoji} **[title](url)** — [author](url), then the body, then the fields. Alinefield (every kind butincident) is one**name:** valueline; ablockfield (incident) is a bold name line with the value quoted beneath it. - Feed bodies are quoted: a GitHub markdown body goes inside a tilde quote fence so its headings and lists stay subordinate to the title. Zulip closes a fence on a line equal to the opening one, so the fence is always one tilde longer than the longest tilde run inside it and no line can close it; a backtick fence or a shorter tilde run just opens a nested block inside the quote. An incident field value is multi-line prose and gets the same quote.
- Release bodies are inline: a release body is one of the one-line
ReleaseMessagesslogans or nothing, never the release notes. Its 500-character cap mirrors Mattermost's and is insurance that never fires today. - Zulip markdown only: Zulip renders only
*/**emphasis (no_forms), treats a single newline as a line break and shows an unknown:name:literally, so the renderer emits Unicode emoji characters and nothing Discord-only. - Nothing in a notification was written for Zulip: titles and bodies come from any GitHub user, order messages from any buyer, incident text from GitHub Status, and Zulip has no backslash escaping (its Markdown disables Python-Markdown's
escapepattern, so\]is a backslash and a]). So the renderer neutralises instead of escaping, with a zero-width space or a character reference where a character must be broken up and with containment where a value must stay in its slot:neutraliseMentionsputs a zero-width space after the sigil of every@**user**,@_**user**,@*group*and#**stream**in every interpolated string, quoted body included. Nobody can ping the channel through a notification.neutraliseLabel(the renderer's name forneutraliseZulipLabelinsrc/format.ts, shared with thesimilarcommand) also runs over the title and author name, which are interpolated into the heading's[label](url). Python-Markdown ends a label where its count of[and]returns to zero, so a single unbalanced bracket either closes the label early (an issue titledClick here ](https://evil.example) thankswould make evil.example the heading's link) or keeps it from closing (A lone ] bracketandlone [ bracketlose the title link). It therefore writes every[as[and every]as]: Zulip passes a character reference through as HTML, so it renders, copies and searches as the bracket, but the link pattern never counts it. Tested against a Zulip 12.3 server; the one blemish is inside inline code, where Zulip shows the reference as written (a titleFix `arr[0]`showsarr[0]). Bodies and field values are not link labels and keep their brackets, so an alert body's own[text](url)still links.- A field value cannot leave its slot. A
linevalue shares a line with its name, so its line breaks become spaces (a field name's too, in either layout): nothing in a merch order message ever starts a line, so it can open no fence, heading or list and cannot pose as the next field. Ablockvalue is quoted like a feed body, with the same fence rule. - Other markup in those strings renders as markdown, which can only garble a heading or a line, never notify anyone, forge a link or escape a slot.
- Shape:
Zulip pull request topics
The Zulip analogue of the #team-pull-requests Discord forum: one topic per public immich-app/immich PR in stream 112, kept by WebhookService.handlePullRequestZulipTopic. It mirrors handlePullRequestTeamUpdate, runs right after it whether or not it succeeded, and never changes it: the Discord path is pinned by its characterization tests and stays byte-identical. handlePullRequestTeamPlatforms settles the Discord path, runs the Zulip path, then rethrows the Discord rejection as the same object, so a Discord outage or rate limit cannot leave a topic uncreated or out of sync, and the webhook still answers exactly as it did (Discord's error, Discord's status). Both paths persist on the same pull_request row (discordThreadId, zulipMessageId); a PR may have either, both or neither. The backfill (backfillPullRequests, below under Zulip commands) does not go through handlePullRequestTeamPlatforms: it runs each path on its own, and only for the platform whose side the row lacks, because an opened replay through the Discord path of a PR that has a thread rewrites the thread's name and starter message.
- Model: the topic is named
#{number}: {title}, cut to 58 code points, Zulip'sMAX_TOPIC_NAME_LENGTHof 60 less the two of the resolved prefix, so that resolving a topic the bot named never overflows the limit (shortenCodePoints, never the Discord 100), and trimmed, since Zulip strips a name before storing it. Onopened(not by a bot, as on Discord) the bot posts one message with the PR's full title linked to the PR (**[title](url)**, throughneutraliseZulipLabel), since the topic name may have cut it, and the body in a quote fence, mentions neutralised, and stores the returned message ID. Zulip has no pin, so the link Discord pins as a second message is folded into the first. Later events post plain notices:closedposts merged/closed by whom, then resolves the topic (rename to✔ {topic}withpropagate_mode: change_all, Zulip's archiving; the resolved name is sent untruncated: a human may have renamed the topic to the full 60, and then the prefix takes it to 62 and Zulip truncates the stored name itself, because Zulip only recognises a resolve when the name it was sent, before its own...truncation, is✔plus the current name, and a pre-truncated name stores the same string as a plain move: no resolved notice, and checked against the move permission and time limit instead ofcan_resolve_topics_group, which is exempt from the limit);reopenedposts the notice, then unresolves (strips the prefix);converted_to_draftposts the notice. Reviews and comments are not echoed there, as on Discord. - The message ID is the key, never a topic string. Topics are mutable strings with no ID: a human may rename or resolve one at any time, and a reply to a stale name silently opens a new, empty topic beside the conversation. So
pull_requeststores onlyzulipMessageId, every event first callsgetMessagefor the topic as it is now, and every post and rename is derived from that: the close notice lands in the renamed topic, resolving keeps the human's name (✔+ current), and a topic a human already resolved is not resolved again. A#{number}: {title}name is only ever computed for anopenedpost or a title change. - Title and body sync follows the
pull_request.editedpayload'schanges:changes.titlerenames the topic to the new name (keeping it resolved if it was), and either change rewrites the first message, which carries both the full title and the body. The rename and the edit are separateupdateMessagecalls, because Zulip refuses a content edit and a topic move in one request. Aneditedreview or review comment carrieschangesabout itself, not the PR, and is ignored before the topic is read, as is a PR edit that changed neither title nor body (touchesZulipTopic): those events cost no Zulip call. This is a deliberate departure from the Discord thread, which is renamed to#{number}: {title}on every PR and review event: on Zulip the current topic name is the only record of what a human did to it, so a rename on every event would clobber a human rename or resolve on the next label or push, and "a human rename is respected" wins. The cost is that a lost or refusedpull_request.editedleaves the name stale until the title is edited again; the topic still works, since every post finds it through the message ID, and a human can rename it by hand. Any other event on a PR with a topic reads nothing and posts nothing. - Degrading without permissions. Realm settings can refuse any of this:
message_content_edit_limit_seconds(default ten minutes) blocks an old message's edit,move_messages_within_stream_limit_secondsblocks an old topic's rename,can_resolve_topics_groupandcan_move_messages_between_topics_groupcan block the bot outright. PRs live for weeks, so these will be hit. Every edit, rename and resolve is therefore best effort, andisZulipRefusalinzulip.client.ts(shared with the mirror, next toisZulipMessageGoneandisZulipFailure) tells a refusal from an outage onPATCH /messages/{id}:MOVE_MESSAGES_TIME_LIMIT_EXCEEDED(achange_allmove whose older messages are past the limit) is a refusal by code alone; every other refusal is a400 BAD_REQUEST, Zulip's catch-all code, and is told apart bymsg. The check is a denylist, not an allowlist: a400 BAD_REQUESTis a refusal unless itsmsgis one of the documented answers that are not (Nothing to changeandTopic can't be emptyare programming errors,Invalid message(s)is a deleted message; all three are outages). The 12.3 spec'smsgenum for the endpoint cannot be used as an allowlist: it lists the content-edit refusals only, with a stale typo (has pastwhere the 12.3 server sendsThe time limit for editing this message has passed), and none of the move refusals the server raises (You don't have permission to resolve topics in this channel.,The time limit for editing this message's topic has passed.,You don't have permission to move this message). So an unknown or translated refusal degrades to the fallback message, never to a lost notice, and the strings are not pinned to the Zulip version. The empty topic name (Zulip's "general chat") cannot be resolved, so a PR topic a human moved there gets its notices, posted withtopic: ''(valid since Zulip 10), and no rename. A refused rename or resolve is logged once as awarnand answered with a plain message in the topic that could not be moved (Pull request has been renamed to: …with the full title, not the cut topic name,The topic could not be resolved automatically: <Zulip's reason>, likewise for unresolving), so the information is not lost; a fallback post that fails in its turn is logged as anerrorand cannot reject either. A refused edit of the first message is logged and nothing more: the title and body are on GitHub. Any other failure (a 5xx, a timeout, a rate limit that outlasted the client's retries, a topic that cannot be read) is an outage, logged as anerrorwithout a fallback message, since that post would fail too. One read failure is not an outage:GET /messages/{id}answering400 BAD_REQUESTwith themsgInvalid message(s)means the stored first message was deleted or is no longer visible to the bot, and retrying that ID on every later event would lose every notice forever.isZulipMessageGonerequires both:BAD_REQUESTis Zulip's catch-all and it has no distinct code for a missing message, so the documented text is the only thing that tells it from a transient or permission failure, and the failure direction is deliberate: if Zulip rewords it, the read counts as an outage and the update is skipped (safe, the next event retries the same ID) rather than opening a second topic and overwriting the key (a permanent split).readOrRebuildZulipTopicthen logs awarn, posts the first message again under the PR's current#{number}: {title}, stores the new ID and carries on with the event in that topic (the close notice and the resolve land there). If the topic still exists with other messages under another name, the rebuild opens a new topic beside it: the key is gone, so there is nothing better to do, and a human can move the messages together. Nothing in the Zulip path can reject:handlePullRequestZulipTopiccatches everything, so a blocked edit never fails the GitHub webhook, and the Discord path has already run by then; a Discord rejection is held and rethrown only after the Zulip path, so it cannot stop it either. Its catch tells a Zulip failure (isZulipFailure: aZulipApiError, undici'sfetch failed, the client's timeout; logged asZulip failed while updating the topic of pull request #N) from anything else (Unexpected error while updating the Zulip topic of pull request #N), so an outage and a bug are not triaged from the same line. Zulip not being initialised (local dev) skips the path entirely.
Zulip event queue
Zulip has no push transport: a client registers an event queue (POST /register) and long-polls it (GET /events with queue_id and last_event_id), which the server holds open until an event or its heartbeat. ZulipService runs that loop; ZulipRepository only wraps the four endpoints. The loop is where zulip-js failed: its loop spun at one request per second against a dead queue, forever, without a log line, and every rule below exists so that this one cannot.
- Registration:
event_types: ["message", "update_message", "delete_message", "reaction"],apply_markdown: false, so handlers see the markdown the sender typed rather than rendered HTML, and thebulk_message_deletionclient capability, so a deletion arrives as one event with every ID. Message events go to theonMessagehandlers; update events go only to theonMessageUpdatehandlers, and never when they are rendering-only (a link preview), made by the server (user_idnull) or made by the bot itself; deletion events go only to theonMessagesDeletedhandlers; reaction events, but the bot's own, go only to theonReactionhandlers; after every registration, the first and each re-registration, theonQueueRegisteredhandlers get the subscribed stream IDs. Today the mirror is the only one to use the last four. The bot receives messages from the streams it is subscribed to, which for private streams is the only way to see them;all_public_streamsis deliberately not set, since it would add every public channel of the FUTO realm, and the nitter mirror, which runs in every stream the bot hears, with it. That makes the bot's subscriptions the only thing that puts a stream's messages on the queue, so the register call also asks for them (fetch_event_types: ["subscription", "realm"], which adds thesubscriptionslist and the realm settings to the answer and changes nothing about the events the queue receives; of the realm settings onlyrealm_empty_topic_display_nameis kept, asZulipService.emptyTopicName: the queue does not declareempty_topic_name, so events andGET /messagesname the empty topic by it, "general chat"), andZulipService.registerQueuelogs onewarnper listening stream the queue cannot see (The Zulip bot is not subscribed to stream 107 (ImmichGeneral): its event queue carries no messages from it, so nothing is expanded there until an admin subscribes it). The set it checks is every stream with GitHub expansion on (ZulipExpanderService, as configured at that registration) plusConstants.Zulip.Commands, notTeamStreams, so a stream with GitHub expansion alone, or on the command list alone, is checked too; the nitter mirror runs everywhere and adds none (named by its bare ID if no constant map names it). Without it a bot never subscribed toimmich-generalwould register, poll and receive nothing, deaf in the main team channel with no line in the log. It is checked at every registration, not once at boot, so a subscription added while the bot runs is confirmed at the next re-registration; the loop carries on with the streams it can see. The loop also readsgetOwnUseronce, before its first poll, and drops every message whosesenderIdis the bot's own, or it would answer its own replies; the account is exposed asZulipService.ownUser(undefineduntil then, so always set when a handler runs), which is where the command router gets the name it matches a mention against. - Other bots are dropped too, by email. Discord skips every bot (
message.author.bot) and Mattermost skips every bot (post.props.from_bot); a message event on Zulip carries nois_botflag, soisBotSenderinzulip.service.tsgoes by the sender's API email instead: Zulip creates every bot as{short_name}-bot@{realm host}and its own asnotification-bot@…andwelcome-bot@…, and a bot's API email is never replaced by theuser{id}@…placeholder that the realm's email visibility setting puts in a human's. So-bot@before the domain is a bot. Without it a GitHub integration, a CI notifier or another team's bot posting intoimmich-pull-requestswould get the#1234references of every post expanded under it, the noise the allowlist keeps out ofimmich-third-parties. The failure direction is a human whose address ends in-bot, who gets no expansions; a bot cannot be created on any other address, so the other direction does not exist. The filter is in the loop'sdispatch, next to the own-user one, not in the expanders: the command router must not take commands from a bot either, and does not, without a check of its own. A handler registered withonMessage(handler, { withBots: true })(the mirror's) is the one exception: it gets other bots' messages too, never the bot's own. - Polling: every event's ID raises the queue's
lastEventId, heartbeats included, before that event's handlers run, and the next poll sends the raised cursor, which acknowledges everything up to it. Events are not guaranteed consecutive; the cursor never moves backwards. - A dead queue re-registers, at once. Zulip garbage-collects an idle queue and drops every queue on restart, so
BAD_EVENT_QUEUE_IDis routine, not a failure: the loop clears its queue, logs it atloglevel, registers a new one and polls that, and it never asks about the dead ID again. The one immediate retry is exactly one: if the fresh queue is reported dead too, or registering it fails, the failure count carries into the backoff below, so even a server that answersBAD_EVENT_QUEUE_IDto everything is polled a handful of times in ten minutes. - Every other error backs off. A rejected
getOwnUser,registerQueueorgetEvents(a 5xx,fetch failed, a rate limit that outlasted the client's retries) is logged as anerrorevery time, with the pause, and the pause doubles from 1s to a cap of 60s across consecutive failures; a successful poll resets it.zulip.service.spec.tsasserts the count: at most 16 polls in a ten-minute outage, against the 600 that a one-second loop would make. - A poll that times out is polled again straight away. The client aborts a
GET /eventsthat outlives the server's own long-poll timeout, which means the connection died silently, not that the queue did; the loop logs it atdebugand polls again on a fresh connection. That is not a spin either: each such poll took the whole long-poll timeout to fail. - No two polls start within a second of each other. The loop starts a one-second floor with every poll and waits for it before the next one. A real long poll takes far longer, so this never delays one; it only bounds the loop against a server that answers at once, which the rules above do not cover: long polling disabled or a proxy that does not hold requests (every poll answers
[]immediately; without the floor that was measured at ~5,800 polls a second), or requests balanced across servers of which only one holds the queue (every other poll isBAD_EVENT_QUEUE_ID, and every success in between resets the backoff, so the immediate re-registration is taken every time; measured at ~1,300 registrations a second). With the floor those are one poll a second and one registration every two, each logged. The floor and the backoff overlap rather than add: the first backoff pause is the same second. - A loop stuck at that rate says so. Everything the floor bounds is routine on its own: a dead queue is logged at
logand an instant empty answer not at all, so a server that has lost its queues for good, or a proxy that never holds a request, would leave the bot at a steady request a second with nothing in the log but routine lines. The loop therefore counts the rounds since the server last held a poll open past the floor (the one thing that proves the queue and the connection work) and, at everyUNHEALTHY_STREAK(10) of them, logs awarnsaying so and what it is doing (The Zulip event loop is not healthy: the server has not held a poll open for the last 10 rounds (0 dead queues re-registered, 10 polls answered at once with nothing); it keeps trying, at most once a second, but receives nothing in the meantime). A round counts when the queue is reported dead or when a poll is answered within the floor with no events; a poll held open resets the count. A poll answered at once with events counts for nothing either way: a busy queue answers at once and is healthy, so a flood cannot trigger it, and a server restart costs one dead queue, not a warning. Errors and timeouts are already logged as they happen and leave the count alone. It changes nothing about what the loop does; it only makes the existing state visible, andzulip.service.spec.ts(health) pins that a healthy loop never logs it. - Handlers cannot kill the loop, nor stall it silently.
ZulipService.onMessage(handler)subscribes a handler to every message no bot sent; handlers run in registration order, one message at a time, and a handler that throws is logged (A Zulip message handler failed on message N) and skipped for that message only. The catch covers a rejection; it does not cover a promise that never settles, and no poll goes out while a message is being handled (the expanders await GitHub in that chain, outside the Zulip client's timeout), so a hung handler would leave every stream unheard with no line in the log until Zulip garbage-collected the idle queue ten minutes later.runHandlertherefore waits on each handler for at most 30s (HANDLER_TIMEOUT_MS), then logsA Zulip message handler has not finished message N after 30000ms; the loop is moving on without itas anerrorand goes on to the next handler and the next poll. The handler cannot be cancelled, so it runs on, never awaited again, and its own outcome is logged when it settles: awarnif it finishes late, the failure line withafter the loop had stopped waiting for itif it fails late. A slow but healthy expansion costs one spurious error line and a late reply, never deafness; the timer is cleared as soon as the handler settles, so a handler that finishes in time leaves nothing pending.ChatService.initregisters the expanders this way andZulipCommandService.initthe command router, one moreonMessagecall from its own service, bounded the same way; both run beforeZulipService.initstarts the loop, so nothing is missed. A registration must happen in that service'sinit(), andAppModule.onModuleInitmust call thatinit()beforeZulipService.init(), exactly as it callschatService.init()andzulipCommandService.init()beforezulipService.init()today.onModuleInitcalls each service'sinit()by hand, in the order written there; nothing else runs them. A handler registered in aninit()thatAppModulenever calls never fires, and one registered afterZulipService.init()misses every message received before it, both silently:onMessageonly appends to a list, and the loop cannot tell a handler that was never registered from one that has nothing to say. - Shutdown cancels the poll and deletes the queue.
ZulipService.onModuleDestroyaborts the loop, waits for a registration in flight if there is one, deletes the queue (DELETE /events), then waits for the loop to exit. The loop's signal travels into everyGET /events, so the abort cancels the poll in flight itself and the loop returns at once, without registering another; it does not depend on the server ending the poll when the queue is deleted (a server that has lost connectivity, or a failedDELETE, would otherwise hold the process on the orphaned socket for the whole long-poll timeout, until the orchestrator's SIGKILL). A registration in flight (POST /register, right after a dead queue or at boot) is the one request the abort does not cancel: the server creates the queue whether or not the client reads the answer, so cancelling would leave an unknown queue behind, and giving up at the grace would leave a known one.registerQueuekeeps its promise inregistrationand stores the queue as soon as the answer arrives,onModuleDestroyawaits it (bounded by that request's own 30s timeout, and settled long ago in the common case) and then deletes what it registered; a registration that fails leaves nothing to delete. The wait for the loop itself is bounded (5s) all the same, for a handler mid-message or a poll that ignores the cancel; a queue that was not deleted is garbage-collected by the server after its idle timeout. A backoff pause and the poll floor end at once on abort, with their timers cleared.src/main.tscallsapp.enableShutdownHooks()because Nest does not runonModuleDestroyon SIGTERM without it. In local dev thedevsentinel keys skipinitentirely, so no loop runs andonModuleDestroyis a no-op. - Tests (
zulip.service.spec.ts,event loop): fake timers, with agetEventsmock whose every call stays pending until the test settles it, so the poll in flight is under the test's control and no real time passes during a backoff. A settled poll alone never starts the next one under fake timers: the floor has to elapse first (nextPoll()advances it), which is what pins the floor; a poll settled before the floor elapsed is one the server answered at once, which is what thehealthtests use, and a healthy long poll is one settled afteradvance(90_000). The pre-existinginittests park the loop in such a poll and end it fromdeleteQueue, as a real server would, so their assertions are untouched; theirregisterQueuemock reports every listening stream as private, so the registration checks stay quiet there.
Zulip message expanders
ChatService.onZulipMessage mirrors onMessageCreate on Discord and onMattermostPosted on Mattermost, reusing the same GitHub expansion (handleGithubThreadReferences: #1234, owner/repo#1234 and issue, PR and discussion URLs to titles and links; handleGithubFileReferences: file permalinks to code snippets; the two halves of handleGithubReferences, called separately here so that only the first is neutralised, see below) and handleTwitterReferences (an x.com link to its nitter.net mirror). That logic is platform-neutral and returns plain strings; it is shared, never forked. What differs on Zulip:
- The reply is a new message, not an edit. Mattermost appends the expansion to the user's own post; a Zulip bot cannot edit another user's message, so the expansion is posted into the same stream and topic, where the topic is the conversation and needs no backlink. All of one message's expansions go in one reply, GitHub first; there is no silent flag on Zulip. The expanders see only
messageevents: updates and deletions reach only the mirror's registries, so editing the source message later does not expand it again; do not route those events to the expanders. - The GitHub expander runs only in the streams it is turned on in; the nitter mirror runs in every stream. The
zulip_expandertable holds one row per stream with GitHub expansion (streamIdis its primary key). Its migration seeds what used to be hardcoded, theImmichstream (54) and everyimmich-*stream (107 to 113).ZulipExpanderServiceloads the table once at init and answersisEnabledfrom that cache, so no message costs a query; it is the only writer. Theexpanderscommand (below) changes one stream at a time, runs a stream's changes one after the other (a command the loop stopped waiting for may still be writing) and reads the stream back from the table after each, falling back to what the write reported when that read fails. A message in any other stream gets the nitter mirror alone, without a GitHub call; a direct message gets nothing. The bot must also be subscribed to the stream, which the command warns about and the event loop checks at every registration (see above). - Privileged in every stream it is on in. Private-repository details are gated behind
isPrivilegedin the GitHub repository; the GitHub expander passestrue, as Mattermost does, because the FUTO realm is not open to the public, so every stream of the realm,Immichincluded, is readable by realm members only. Stream privacy is not checked: turning GitHub expansion on in a stream is the decision to show private repository titles and code there. - The reply is neutralised where that protects something. GitHub titles and links are written by anyone, so
neutraliseZulipMentionsruns over each of them, as over a notification, and over the nitter mirror, which is built from whatever the sender typed: nobody can ping the stream through an issue title. A code snippet is left exactly as GitHub has it: it sits inside a code fence, where Zulip renders no mention, so neutralising it protects nothing and the zero-width space would silently corrupt the code (${file#*.}in a shell script reads as a mention to the neutraliser). That is why the handler calls the two halves ofhandleGithubReferencesseparately rather than the combined method. - Bots are filtered before the handler, by email, not by a flag.
onMessageCreatechecksmessage.author.botandonMattermostPostedcheckspost.props.from_botin the handler itself;onZulipMessagechecks nothing, because the event loop has already dropped the bot's own messages and every other bot's (isBotSender, the-bot@rule under Zulip event queue above) before any handler sees them. A handler therefore never needs its own bot check, and a message from a sender the server gave no email for is treated as a human's.
Discord-Zulip dev mirror
MirrorService (src/services/mirror.service.ts) mirrors the Discord channels an administrator links (the development channels, #dev, #dev-off-topic and the #dev-focus-topic forum, to begin with) with Zulip streams, both ways. Contributors on Discord are outside FUTO and the realm is internal, so: nothing pings unless that is meant, every link is announced on both sides, identity comes from IDs, and each side keeps its own moderation. The pure pieces live in src/mirror/; DiscordMirrorEvents (src/discord/mirror.ts) feeds it Discord events, with no error handling of its own (the global guard reports what a handler throws; shardReady alone logs the failure of the check it starts without awaiting), and IDiscordMirrorInterface is the Discord repository again under its own token (useExisting).
- Links (
mirror_link, one row per pair, keyed by the Discord channel ID, which is also the pair's key in the log) are made at runtime by a handshake that takes an administrator on each side, with no other guardrail: any text channel or forum, any stream, public or internal. A Zulip realm administrator or owner runsmirror-link [topic=<main topic>]in the stream; that changes nothing yet and answers with/mirror-link id:<link id>, which a Discord member with the Administrator permission runs in the channel to link (for a forum, in any of its posts). A link ID is six characters, valid an hour and once; a stream has one pending request, the newest; IDs live in memory, so a restart voids pending ones. A refused completion (wrong kind of channel, missing permission) keeps the ID for another try. Unlinking ismirror-unlinkin the stream or/mirror-unlinkin the channel, again for administrators. What is checked is technical only: the channel is a text channel or a forum, neither side is in another link, the bot has View Channel and Manage Webhooks there and is subscribed to the stream; anything else the bot lacks is said in the reply. A text channel has a main topic, the empty topic (Zulip's "general chat", stored as'') unless the request named one (topic=general chatmeans the empty topic too), a forum none. Instead of guardrails, every link and unlink is announced on both sides, naming both ends by name and ID and who did it (for a link, both administrators) (Zulip: in the main topic, or topicmirrorfor a forum; Discord: a message in the channel, or a post in the forum), and the link announcement is pinned when the bot may pin (its ID is kept to unpin it on unlink).MirrorLinkService(src/services/mirror-link.service.ts) does the checks, rows and announcements;MirrorService.enableturns the pair on at once (webhook, channel check, catch-up) anddisablestops queueing anything for it at once, while what is queued finishes. Unlinking keeps themirror_conversationandmirror_messagerows, which are found by the Discord channel ID, so linking the same channel again carries on its conversations. At everyshardReady, and every ten minutes because a resumed session brings none and a webhook posts whatever the bot's permissions, a pair is turned off while its channel is gone, is not of its kind, or the bot lacks View Channel or Manage Webhooks there; it comes back on, with a catch-up, once it passes. - Identities (
mirror_identity, Zulip user ID to Discord user ID, one each way) are self-service:/zulip-linkon Discord answers privately with a code, valid ten minutes and once, which the Zulip user sends the bot in a direct message (link <code>); that Zulip account is then linked with the Discord account that asked, replacing any link either had. Codes live in memory, so a restart voids the pending ones./zulip-unlinkon Discord anddiscord-unlink(orunlinkin a direct message) on Zulip remove the caller's own link. A link takes effect at once (MirrorService.refreshIdentities) and counts only while the Discord member holds the Team or Immich role, checked again once ten minutes old; otherwise the sender appears asName (Zulip). - Webhooks: the runtime bot creates and owns one per channel. Its token is never logged or stored, and it runs through a
WebhookClientof its own, because the bot's REST debug lines, whichDiscordEventslogs, name a webhook route by its token. A webhook deleted on Discord is recreated at most once an hour; a failed recreation, or a lookup that failed atshardReady, does not count. - Conversations: a
mirror_conversationrow joins one Discord location to one Zulip topic: the channel to its fixed main topic (text pairs), a public thread or forum post to a topic of its own. A thread conversation is found throughzulipAnchorMessageId, never through the topic string, which is a cache kept current byupdate_messageevents and read again from the anchor once per queue registration and before every Discord rename. A conversation goes where its anchor goes; messages moved away without it lose theirconversationId. The empty topic is''everywhere in the mirror: posted as'', stored as''with the key'', and every topic Zulip hands in (message and update events, catch-up pages) is mapped from the registered display name to''first (fromZulipTopic);getMessageasks for''itself. Zulip stores a topic sent as exactlygeneral chatin the empty topic, sotopicKeykeys that spelling as''too, while a topic really namedGeneral Chatstays a topic of its own. - Messages: one
mirror_messagerow per Discord message, so one per part of a split Zulip message;origindecides which way an edit or deletion flows. A deleted message keeps its row withdeletedAtset: edits, replies and anchors skip it, while the create dedupe and the catch-up high-water marks still count it, so catch-up never brings back what one side's moderation removed. - Entry points only enqueue onto one
SerialQueueper pair (src/mirror/queue.ts; the discordx handlers have priority 0, so they run before the other handlers of the event), so the Zulip loop never waits on the mirror. A Discord edit whose message comes partial (one discord.js had not cached) is queued as it comes and read (message.fetch()) only when its turn comes, so it keeps its place among the pair's changes, before a deletion that follows it; one that cannot be read (deleted, hidden from the bot) is dropped with adebugline. Deletions and reactions need nothing a partial lacks. Nothing is dropped, and the next op starts only once the current one settles, since discord.js can wait out a long rate limit inside one request: a 180s watchdog logs an op still running, again every 180s, and never moves on without it. The files of one message share a 120s transfer deadline, so a slow file becomes anattachment not mirrorednote rather than outliving the watchdog. - Catch-up runs after every
shardReady,shardResumeand Zulip queue registration. From the moment the gateway connection drops (shardReconnecting,shardDisconnect) until it has run, live creates are turned away with a throttledwarn, because a new session can deliver messages beforeshardReadyand mirroring one first would move the high-water mark past what was missed. It reads back from the newest message to the high-water mark (on Discord the newest Discord-origin row of the channel and of each thread active in the last week, on Zulip the newest Zulip-origin row of the stream) or to the start of the six-hour window or the moment the link was made, whichever is later (a re-link never copies what was posted while unlinked), 100 at a time and at most five pages, and also reads every place it turned a create away since its last complete run, which the marks would miss (a new thread or forum post, a thread idle for over a week, a stream with no Zulip-origin row yet, a Zulip message moved since). Threads and forum posts made while the bot was away have neither a mark nor a turned-away create, so it also lists each linked channel's public threads (listMirrorThreads: the active ones and the 50 most recently archived, by when each was made, since a thread started from an old message has that message's ID) and reads, from the start, the newest 20 made within the window that have no conversation; any beyond those are named in awarnand not mirrored. What it finds goes through the normal create path, oldest first. Edits and deletions made while the bot was away bring no event, so a complete catch-up is followed by a recheck of the rows made in the last 24 hours (RecheckMaxAgeHours, never before the link; the newest 200, and it logs when it stops there), on the side whose events may have gone unheard since the last recheck (Discord after a new gateway session, Zulip after a queue registration, both for a pair turned on): each Discord location with such Discord-origin rows is read back, at most five pages, to its oldest one, and each Zulip-origin message is read by ID (getMessagesByIds,message_ids, 100 at a time). A Discord source that is gone is deleted on Zulip and one whose hash changed is edited there, a Zulip source that is gone has its Discord copies deleted (the 7-day guard included) and one whose content changed is edited, all through the live edit and deletion paths, so Zulip's edit limits and every retry apply. A row older than what the pages reached is left alone, and so is every row of a location or stream that shows none of its messages at all, which is likelier lost access than everything deleted. A recheck during which the mirror lost track again drops what it read, which may be out of date, and leaves both sides to the next one. A Zulip deletion of a contributor's copy, or a Discord deletion of a team member's copy, while the bot was away is not noticed. - Backfill (
MirrorService.backfill, Discord to Zulip only):/mirror-backfillon Discord, in a thread or forum post (that thread) or a linked text channel (its own messages, not its threads, each of which is backfilled on its own), andmirror-backfillon Zulip, in a topic of a linked stream (the channel for the main topic, the thread for a thread's topic; any other topic is refused), both for administrators, copy every Discord message of that location that has nomirror_messagerow, oldest first, up to a snowflake of the moment it started, through the live create path (mirrorDiscordMessage, so identities, files, replies, emotes and rows are the same), with every mention silent and the late time marker. It streams forwards (fetchMirrorMessagesAfter, 100 at a time from the location's own ID), keeping only the unmirrored part of one page (and, within a 20s budget, the messages its replies answer, so an old reply keeps its author), and runs one queue op per page read or message, pushed behind whatever is queued and half a second apart after each copy (BACKFILL_PACE_MS, about 7,000 messages an hour, well inside Zulip's per-user rate limit), so the pair's other conversations keep flowing and no op nears the watchdog. While it runs, creates for that location (live and catch-up) wait and run after it, in order, each read again first (fetchMirrorMessage: a deleted one is dropped) and followed by a reaction sync, as is every copy that has reactions. Before the first copy the bot posts📜 History from Discord from before the mirror follows (from <time:…>)in the topic (a thread without one claims it first, and the notice holds it until the first copy opens the conversation), after the last📜 End of the history from Discord: N messages(with how many could not be copied); with nothing missing it posts nothing and answersNothing to backfill: …. It waits while a side is not ready or not caught up, retries what the create path retries, and stops when the pair goes off (📜 The history from Discord stops here for now, after N messages: …), is unlinked, the location is gone, a read fails for good, or the bot shuts down; running it again carries on, since mirrored messages are skipped. One backfill per location at a time. An edit of a message older than the link, which only a backfill copies, keeps its mentions silent: Zulip notifies a mention an edit adds. The Discord answer is a private acknowledgement, edited with the count at the end while the interaction's 15-minute token lasts; on Zulip it is an acknowledgement in the topic, and the end notice is the report. Logs name IDs and counts only. - Never post twice. A create is repeated only when the other side provably never took it: a failure before the send, a Zulip 429 the client's retries did not outlast, or a connection never made (
DiscordMirrorError('unreachable')). That message is turned away and another catch-up scheduled, 30s later and doubling to 10 minutes; one that fails three times for reasons other than an outage is given up with anerror. A send that may have gone through (a 5xx, a timeout, a reset connection;'unavailable') is never sent again. Once a send went through, what it posted is stored (themirror_messagerows, and the conversation it opens) even when the database fails for now (a lost connection, a restart, a deadlock;isTransientDatabaseFailure): the failed write is tried again as a queue op of its own after 5, 15, 30, 60 and 90s, with the writes after it (later parts) behind it, and an insert tried again that finds its row counts as stored. Meanwhile the creates, edits, deletions, moves and reaction syncs of that message, and the creates in the thread or topic it opened, wait and run once it is in. Any other failure, the last retry, an unlink or a shutdown gives up with anerrornaming the IDs; the message is then never sent again (in memory), and what waited runs and finds no row. Edits, deletions and renames wait, in order, while the side they go to is not ready, and ones Discord fails for now are tried again every 30s, up to twenty times, since both are safe to repeat. - Reactions go both ways, but each platform lets the bot react only as itself, so a count never crosses: the bot's reaction on one side stands for however many people react with that emoji on the other, added once one does and taken back once none does. A sync reads both sides afresh (
getMirrorReactionsthrough the bot's REST client,getMessageon Zulip) and adds or removes only the bot's own reactions, so it is idempotent, any change to a message is the same sync, and one queued covers the changes that arrive before it runs. Discord to Zulip counts the reactions on every part of a split Zulip message; Zulip to Discord reacts on part 0. Unicode goes by code points (Zulip's table has no variation selectors or skin tones, so a Discord emoji is looked up without them; Discord is asked for a Zulip emoji as it is and, if it answersUnknown Emoji, once more with a variation selector after each character that takes one), a Discord emote by the realm emoji the emote sync made of it, found with the sync's own naming (zulipEmojiNames, suffixes included) and only if that realm emoji exists, and a realm emoji only when the guild has the emote it came from; anything else is skipped, as are Zulip's own extra emoji. While the emotes, the realm emoji or Zulip's table cannot be read, a sync changes nothing, since it could not tell an emote nobody uses from one it cannot name. Text uses the same mapping, both ways. The bot's own reactions never count: the Zulip loop drops them,DiscordMirrorEventsdrops the bot user's, and a sync leaves out the bot's side itself. A Zulip reaction event names no stream, so it is queued on every pair, which finds the message among its own rows or does nothing. Discord's removal events for users it has not cached needPartials.User. - Deletions and moves: a Zulip deletion of a team message deletes its Discord copies, also in the stream a mirrored message was moved to, but not after 7 days (
DeleteSyncMaxAgeDays: retention purges send the same events); a Zulip deletion of a contributor's copy never touches Discord, and a Discord moderator deleting a team member's copy only forgets it. Zulip reports a topic moved to a stream the bot cannot read as a deletion of its messages, so the copies go either way and a thread losing its anchor and every other mirrored message is detached: the bot must be subscribed to every stream a mirrored topic may be moved into. A move it can see (update_messagewithnew_stream_id) detaches the conversation and keeps the copies. Resolving a topic renames its thread to the resolved name and then archives it, and unresolving it takes the thread out of the archive; archiving on Discord alone changes nothing on Zulip, since Discord archives every quiet thread on its own, and a post in an archived thread unarchives it as before. - Forum tags: the applied tags of a forum post head the first message of its topic as a
**Tags:** bug, mobileline, part of that message's stored lead, when the post's own first message made the topic. A change of tags (threadUpdate) rewrites that line, editing the message with the post as it reads now (fetchMirrorMessage); when the topic was started on Zulip, the post's first message was never mirrored or Zulip refuses the edit (its edit limit), the bot postsTags changed: …in the topic instead. - Rendering: whatever a Discord message quotes or hides (a
>>>quote, a spoiler, the start of the message a reply answers) is a fence closed before anything that follows, so nothing a contributor writes, nor a cut, can pull the rest of the Zulip message into it. A Discord thread started from a message (in a text channel the thread has that message's ID, whichfetchMirrorMessagereads from the channel whoever sent it) opens its Zulip topic with↪ Thread started from a message by **name**, linked to the Zulip copy of that message (zulipNarrowLink) or, when it was never mirrored, to the Discord message, and the start of it quoted; the line is part of the first message's stored lead, so an edit keeps it. A forum post starts with its own first message and has none. The silent pill of a verified team member is the header's alone: in a silent message a mention of one is the name as text, so no line can pass for a message of theirs. A Zulip channel or topic reference (#**channel**,#**channel>topic**,#**channel>topic@id**, a narrow link or realm URL, labelled#channel > topicor not) becomes the jump link of the Discord copy when it names a mirrored message, else the<#id>mention of the linked Discord channel (the stream itself, or its main topic) or of the thread of a mirrored topic, and(Zulip link)for anything else; a stream named in#**…**is found by its name, read withgetStreamonce per queue registration. The other way, a Discord<#id>of a linked channel becomes#**stream>main topic**(#**stream**for a forum) and one of a mirrored thread#**stream>topic**, or a narrow link labelled#stream > topicwhen a name cannot go in that syntax; any other channel stays escaped text. - Uploads: Zulip to Discord re-hosts only the
/user_uploads/links in the message itself, in either storage layout (local or S3).downloadUploadrefuses any path that URL normalisation or the server could steer elsewhere before sending the bot's credentials, follows a redirect only to another HTTPS origin (the S3 backend) and without credentials, and refuses a login page. An upload inside a Zulip spoiler is attached asSPOILER_<name>, the only way Discord hides a file. A Zulip edit that adds/user_uploads/links (the edit event'sorig_contentsays which are new) has the new files attached to part 0 of the Discord copy, which keeps the attachments it had (the webhook edit names them, since an edit that sends files keeps only those), within the same limits (ten files counting the ones the message had, the size caps, the transfer deadline) and with the same notes for what cannot go, also when Discord finds them too large. Adding files is not safe to repeat, so a retry of the edit sends them again only when the edit that carried them never reached Discord; once it went through, or may have (a 5xx, a timeout), they are not sent again, and an edit Discord took without answering can leave them out rather than risk a second copy. An edit found by catch-up has no content before it, and an edit that takes an upload away leaves the file on Discord; both are logged. Discord to Zulip downloads from Discord's CDN only and uploads withuploadFile; a Discord edit cannot add attachments, so there is no such case that way. - Bots cross too, as bots. On Discord every other bot and every webhook but the mirror's own is a mirror candidate (
isMirrorCandidate), and so are the bot's own replies (the GitHub and Twitter expansions), never its other messages (the link announcements and their forum posts); the Zulip header names them**Name** (bot), and the embeds a bot wrote (rich, never link previews) follow its message as a quote, counted in the source hash only when there are some. The mirror's own webhooks are every webhook the bot owns in a linked channel, found when it looks for its webhook and added when it makes one (isOwnMirrorWebhook); a copy posted through one that is gone since is still never mirrored back, because its message ID is a Zulip-origin row. On Zulip the mirror registers its message handlerwithBots, the only one that gets other bots' messages, and mirrors every sender but Zulip's Notification Bot (its move and resolve notices say what the mirror carries over itself), asName (Zulip bot), the email gateway (emailgateway@zulip.com, the one system bot without-bot@in its address) included; a bot is never looked up as a linked team member. - Loop prevention: the event loop drops the bot's own Zulip messages, edits and reactions whatever the handler, and
DiscordMirrorEventsthe bot user's Discord reactions; the mirror's webhook copies are never candidates, and a Discord message of the bot that is mirrored lands on Zulip as the bot's own, which the loop drops, so every notice, echo and bot message crosses once at most. - Logging: pair keys (Discord channel IDs) and IDs only, never content, names, URLs or webhook tokens, and never
logError, which posts to BotSpam. A create Discord refuses for good is answered in its Zulip topic with a throttled⚠ Not mirrored to Discord: <reason>.
Zulip commands
ZulipCommandService (src/services/zulip-command.service.ts) is how the team drives the bot from Zulip. Zulip has no registrable slash commands, no modals, no buttons, no autocomplete and no ephemeral replies; the whole interaction model is "mention the bot, it answers in the topic", chosen over keeping these commands on Discord. The Discord slash commands are untouched and keep working.
- A command is a message that starts with a mention of the bot,
@**Name**or the silent@_**Name**, either with Zulip's|user_idsuffix, matched by the name fromZulipService.ownUser(which the loop read before its first poll) without regard to case. Only newlines may come before the mention: a first line indented by four spaces or a tab is a Markdown code block, which Zulip renders as code and notifies nobody of, so a command in one is ignored exactly as one in a fence, a quote block or inline code is (the mention regex is anchored on^[\r\n]*, not^\s*, andparseCommand's spec pins each of those forms). A mention anywhere else (thanks @**Immich**) is not a command and gets no reply, so the bot cannot be summoned by accident mid-sentence; the cost is that@**Immich** thanksis answered with the unknown-command line. A mention alone is answered with the help. Zulip's own "Quote and reply" is not a command either: it starts the reply with a silent mention of the quoted author and[said](<link>):before thequotefence, so a reply that quotes the bot starts with a mention of it;parseCommandignores a mention followed by[said]((theQUOTE_AND_REPLYregex), or the bot would answer the most natural way of replying to it withUnknown command …noise in the topic. A command typed after such a quote is not seen; mention the bot in a message of its own. TheparseCommand,splitArgumentsandtokenizefunctions live insrc/zulip-command-parser.ts, which the mirror also imports to leave commands unmirrored, and are pinned byzulip-command.service.spec.ts. - Parsing: after the mention, the first token is the command (case-insensitive) and the rest its arguments (
parseCommandyields the name and the tokens). A double-quoted run, straight or curly (phone keyboards curl them), keeps its spaces anywhere in a token, sotext="two words"and"two words"are one token each. Options are sorted once the command is known (splitArguments): a tokenkey=value(the key a word) is a named option only when the command declares that key; any otherword=valueis a positional argument, in its place, sosimilar the upload fails when CORS=strict on nginxcompares the whole sentence (word=valueis everywhere in the error textsimilarexists to match:LOG_LEVEL=debug,uid=1000,error=ENOENT), whilebackfill-pull-requests pr=1234still reaches that command as an argument it cannot read. A quote that is never closed is answered with what went wrong, an unknown command with a pointer tohelp, a command with the wrong arguments with its usage line; never a stack trace. An argument the command does not take is answered, never dropped: each entry in the command table declares how many positionals it takes and which options it reads, and the dispatcher answers more positionals than that with the usage line before the command runs, and the command answers a positional it cannot read the same way (emote-sync now,backfill-pull-requests pr=1234,fourthwall update ORD-1 id=ORD-2,similar text="a" b). With no autocomplete and no confirmation step, a typo innumber=must not turn a backfill of one pull request into a backfill of every one.helplists every command with its arguments, which matters more than on Discord, since there is no autocomplete to find them with. - Every reply is public, in the same stream and topic as the command (
sendMessagewith the message'sstreamIdandtopic, the empty topic included). There is no ephemeral equivalent on Zulip: several of these commands reply ephemerally on Discord (/fourthwall,/backfill-pull-requests, the private "Find similar issues"), and on Zulip the whole team sees the reply. Replies are kept short for that reason, and every interpolated string a human wrote (a command name, an order ID, the messagesimilarcompared, GitHub titles, emote names, an error message) goes throughneutraliseZulipMentions, so a reply can ping nobody; the ones that are echoed as typed (a command name, an order ID, the textsimilarcompared, and an error's message, which some service wrote) go throughcode(), inline code with every backtick stripped and every whitespace run collapsed to a space, so nothing in them can close the span early or start a line as a heading or a fence in the bot's voice; an unknown command name and the textsimilarcompared are also cut to 80 characters (ECHO_LENGTH), an error's message to 300, so a pasted wall of text is not posted back whole. GitHub titles in asimilarline also go throughneutraliseZulipLabel, the renderer's link-label containment, so a title cannot add a link of its own next to the bot's. Every reply also goes through the onereply()seam, which cuts it to Zulip'smax_message_length(10000,shortenCodePoints,fit): a reply the server refuses is one the topic never sees, and a long list of hits would otherwise make thesimilarresult unpostable. A cut that lands inside a code span would leave the rest of the reply rendered as code, so a cut reply with an odd number of backticks (the bot's own spans hold none) is cut one code point shorter and, if that did not close the span, given a closing backtick, within the limit. The listhelpposts ends with a blank line, or Markdown would render the line after it as a continuation of the last item. - Authorisation is the stream list. Zulip has no per-channel bot permissions, so commands are taken only in
Constants.Zulip.Commands(everyimmich-*stream, 107 to 113) and a command anywhere else,Immichincluded, is ignored without a reply. Whether a stream is private is not checked; the list is kept to team streams because these commands act (backfill-pull-requestscreates threads and topics,emote-syncwrites realm emoji through the user account,fourthwall updatemutates orders). Themirror-*commands andexpandersare the exception: the stream they act on is usually not a team one, so they are taken in any stream and authorised by the sender's role instead (getUser: owner 100 or administrator 200), and anyone else is told so (each command'sadministratorsnames what only they can do, for the refusal);helpmarks them(administrators). Direct messages takelink <code>andunlinkonly (alsodiscord-link/discord-unlink, with or without the mention), exactly that many words, answered to the sender alone, so nothing else said to the bot gets an answer. The event loop has already dropped every bot's messages (isBotSender) before the handler sees one, so no bot can drive a command. - A failure never reaches the loop. Every command runs inside a catch: a handler that throws is logged as an
error(The Zulip command <name> failed on message N) and answered with`<name>` failed: <the error's message, shortened>; a reply that cannot be posted is logged (Could not reply to the Zulip command in message N) and the handler resolves all the same. The loop's own catch and 30s handler timeout stay as the backstop. - Slow commands run detached from the loop, because the loop polls nothing while a handler runs and stops waiting after 30s:
emote-sync,backfill-pull-requests allandfourthwall update allpost an acknowledgement in the topic at once, do the work in the background and post the outcome, or the failure line, when it is done; the handler returns after the acknowledgement.inBackgroundtakes the acknowledgement and the work, and posts the acknowledgement before any of the work starts: the backfill lists the open pull requests inside the work, not before the acknowledgement (Going through every open pull request, creating the Discord thread and the Zulip topic each one lacks; this can take a while…), because a rate-limited GitHub client waits out the limit (an hour, seen in testing) before it answers, which left no acknowledgement and the handler stuck at the loop's 30s cap. A listing that fails is posted as the job's failure line; the count is in the outcome. The wide form is asked for by name. There is no confirmation step and no way to stop a run once it has started (the acknowledgement arrives after the work is committed to), and the wide form is the expensive one:backfill-pull-requests allreads every open PR's row and creates a forum thread and a Zulip topic for every one that lacks them, which after a fresh Zulip rollout is one topic per open PR, hundreds of posts;fourthwall update allfetches every order from Fourthwall again. So neither is the bare command:backfill-pull-requestsandfourthwall updatealone get the usage line, the narrow repair (backfill-pull-requests 1234,fourthwall update ORD-1) is one word away, and the fan-out takes the explicitall(any case, positional ornumber=all/id=all). One run of each command at a time, inline or in the background: the narrow forms take the same lock (underLock) as the wide ones, sobackfill-pull-requests 1234during a runningbackfill-pull-requests allis answered with`backfill-pull-requests` is already running; wait for it to finish.and does nothing, rather than reachinghandlePullRequestTeamPlatformsfor #1234 concurrently with the background run, where both would read no thread and no topic ID, both would create, and only one ID would survive in the row, an orphaned forum thread or topic that no later event touches. An acknowledgement that cannot be posted does not start the work, and an outcome that cannot be posted is logged (Could not post the outcome of the Zulip command <name>); neither leaves the command locked. - Commands (community commands,
/link*,/messages*, the help desk,/prune,/age,/release-notes, stay Discord-only):-
help: the list above. -
emote-sync:ChatService.syncEmotes(guildId)for one fixed server,Constants.Discord.EmoteSyncServer({ id: '979116623879368755', name: 'Immich' }; the Discord command syncs the guild it is run in, and the second guild inConstants.Discord.Serversis not reachable from Zulip). Because the target is not where the command is run, the acknowledgement, the outcome and thehelpline all name it (Syncing the emotes of the Immich Discord server (979116623879368755) to Zulip and Mattermost…). The sync is platform-neutral and returns anEmoteSyncReport;formatEmoteSyncReport(report, subject?)turns it into the one report line both platforms post (Done syncing: 3 emotes, 1 uploaded to Zulip, 2 uploaded to Mattermost, 1 failed: …, 1 renamed: …, 1 already on Zulip: …, orDone syncing the emotes of the Immich Discord server (979116623879368755): …with the subject the Zulip command passes), which the Discord command caps at 2000 characters and the Zulip one at 10000, Zulip'smax_message_length, afterneutraliseZulipMentions, since emote names are Discord's. The Discord reply text is pinned by the Phase 0 characterization assertions inchat.service.spec.ts, which drive the sync throughDiscordCommands.handleEmoteSync. -
backfill-pull-requests <number|all>:WebhookService.backfillPullRequests(pullRequests, { discord: true, zulip: true }), which creates, for each PR given, the Discord forum thread and the Zulip topic it lacks, and touches nothing that exists. Until this branch the command, on Discord, was a silent no-op:GithubService.getOpenPullRequestsoverwrote the GraphQL nodeidwith the numericfullDatabaseIdand never setnode_id, sogetPullRequestById(pull_request.node_id)missed every row, both team paths returned early, and/backfill-pull-requestsansweredSuccessfully backfilled pull requestshaving created nothing.toPullRequestEventingithub.service.tsnow keeps the node ID asnode_id(pinned bygithub.service.spec.ts), which means the command really creates threads and topics now: one thread per open, human-opened PR in thepull_requesttable that has none, and one topic per such PR that has none, which after the Zulip rollout is every open PR. It is still admin-only and manually invoked, and there is no auto-run; do not add one. The backfill reads each PR's row first (node_id): a PR that is not in the table (itsopenedwebhook never arrived) or was opened by a bot is skipped without a call, since neither path would create for it, and so is one that already has what every platform asked would create; for the rest, each platform missing its side gets the PR as anopenedevent through its own path (handlePullRequestTeamUpdate,handlePullRequestZulipTopic), never throughhandlePullRequestTeamPlatforms, since anopenedreplay through the Discord path of a PR that has a thread would rewrite the thread's name and starter message (hundreds of them, for the PRs that only lack a topic). The row is read again afterwards and what was created is what it now holds: a platform that was asked and stored no ID counts as failed, whether its path rejected (logged asCould not backfill pull request #N), resolved without a thread (…: Discord created no thread) or, as the Zulip path does, logged its own failure. As in the webhook, a Discord rejection does not stop the Zulip side of the same PR, and one PR failing does not stop the rest. The result is aBackfillReport(threadsandtopicscreated, by number, eachundefinedfor a platform not asked, or Zulip not initialised;skippedwith a reason,not tracked,opened by a botoralready complete;failed), andformatBackfillReportis the one line both commands post:Backfill of 187 open pull requests done: created 0 Discord threads and 150 Zulip topics; skipped 35 (33 already complete, 2 opened by a bot); failed 2 (#123, #456), see the log.Every count is what happened; neither command posts a blanket success. Withallthe Zulip command acknowledges, then lists the openimmich-app/immichPRs (GithubService.getOpenPullRequests) and backfills them in the background. With a number (1234,#1234ornumber=1234), the common repair, that one PR is fetched directly (GithubService.getOpenPullRequest, one GraphQL call, never a page through every open PR to find it) and done inline, under the same lock, and answered with the same report (Backfill of pull request #1234 done: created 1 Discord thread and 1 Zulip topic; skipped 0; failed 0., orPull request #1234 is not open in immich-app/immich, so there is nothing to backfill.when it is closed, merged or does not exist); no argument, a number that is not one, or one given twice, gets the usage. Discord's/backfill-pull-requestsruns the same loop asked for Discord alone ({ discord: true, zulip: false }): it creates the forum thread each open PR lacks, never a Zulip topic, carries on past a PR that fails, and answers ephemerally with the report, cut to 2000 characters; a failure to list the PRs still rejects as before (pinned by the/backfill-pull-requests on Discordtests inwebhook.service.spec.ts). The Zulip command is the one that catches both platforms up. -
fourthwall update <id|all>:ChatService.updateFourthwallOrders(id). With an ID the order is refreshed inline, under the command's lock, and answered; withallevery order is, in the background; with neither, the usage.FourthwallRepository.getOrderreturns whatever JSON Fourthwall answered, so an answer that is not an order (a wrong ID, refused credentials, an outage) fails withFourthwall did not return order <id>: …and writes nothing, rather than aTypeErrorabout readingvalue. -
similar [text]: the "Find similar issues" message context menu, which Zulip does not have. With text it compares that; without, it reads the topic back (getMessages, the ten newest) and takes the last message a human wrote, skipping the command itself, the bot's own messages, other bots' and other commands, and says when there is none. The reply echoes what it compared, on one line and shortened, then the hits (ChatService.handleFindSimilarIssuesOrDiscussions, shared with Discord; the Zulip command passes itneutraliseZulipLabelas the title neutraliser, which Discord's callers leave out, so their output is unchanged), or that nothing was found. Discord's ephemeral "Find similar issues (private)" variant has no Zulip equivalent and stays Discord-only. -
schedule-add <name> cron=<expression> message=<text> [topic=<topic>] [suppress-embeds=<true|false>],schedule-list,schedule-edit <name> [cron=…] [message=…] [topic=…] [suppress-embeds=…]andschedule-remove <name>: the Zulip scheduled messages (service: 'zulip'rows, see Scheduled messages). A message is posted in the stream the command is given in, in thetopic=given or the command's own topic (the empty "general chat" topic included);schedule-edittakeskey=valuearguments where Discord opens a modal, needs at least one ofcron,messageandtopic, and reschedules the running job at once. Each looks a name up among the Zulip rows only, so a Discord or Mattermost message can be neither edited nor removed from Zulip; names are unique across every platform (scheduled_message_name_uq), so a name taken elsewhere is refused with the database's error.suppress-embedsis accepted, validated astrueorfalse, ignored and said to be ignored, inhelpand in the reply: Zulip link previews are a realm setting, and the per-message flag Zulip has is per recipient.schedule-listlists every Zulip scheduled message with its stream, topic, schedule and the start of its text, mentions neutralised. A message's own text is posted as the team wrote it, mentions included; that is its point. -
mirror-link [topic=<main topic>](starts a link of this stream and answers with the Discord command that completes it),mirror-unlink(this stream's link),mirror-backfill(copies the Discord history of the channel or thread this topic mirrors into it) andmirror-list(every link with both names, IDs, kind, main topic and whether it is on, and every linked account), administrators only, in any stream;discord-unlinkremoves the sender's own linked account. See Discord-Zulip dev mirror. On Discord,/mirror-link id:<link id>completes a link and/mirror-unlinkremoves one, run in the channel (for a forum, in any of its posts),/mirror-backfillbackfills the thread, post or text channel it is run in, and/mirror-listlists them, all for the Administrator permission (the command default, checked again at runtime) and answered privately; a Zulipmirror-unlinkreply given in the topic of the announcement leaves out what the announcement says. -
expanders on,expanders offandexpanders list, administrators only, in any stream, no other argument: turn GitHub expansion on or off in the stream the command is given in (Turned on GitHub expansion in this stream.,Nothing changed: GitHub expansion was already off in this stream.), with a warning when turning it on in a stream the bot is not subscribed to (read fromgetSubscriptionsbefore the change; a failed read changes nothing).listnames every stream it is on in (getStream, the bare ID when it cannot be read) and marks the ones the bot is not subscribed to. The nitter mirror has no command: it runs everywhere. See Zulip message expanders. -
rss-subscribe <url> [topic=<topic>],rss-unsubscribe <url>andrss-list: the Zulip RSS feeds of the stream the command is given in (service: 'zulip'rows, see RSS).rss-subscribeanswers a feed the stream already has with the topic it goes to; otherwise it fetches the feed in the background (a feed's server can take longer than the loop waits for a handler), acknowledged at once and answered with the outcome: the newest post is posted to thetopic=given or the command's topic, and a feed that cannot be fetched or posted leaves no row.rss-listexists because Zulip has no autocomplete to find a feed's URL with, which is what Discord's/rss-unsubscribeoffers. One feed per stream: the topic is an attribute of the row, not part of its key.
-
Scheduled messages
ScheduledMessageService (src/services/scheduled-message.service.ts) posts each scheduled_message row on its cron expression, one CronJob per row, registered in init for every row and whenever one is created or edited. A scheduled message is plain text with a per-platform preview flag, not a rendered card, so it does not go through the notification seam: the send is the senders table in the service, one entry per service (discord sends content with SuppressEmbeds when suppressEmbeds, mattermost sends message with remove_link_preview, zulip sends content to stream: Number(channelId) and topic, '' when the row has none). Adding a platform is one entry there, not another branch. A failed send is logged (Failed to send scheduled message <id>: …) and the job carries on.
- Columns:
service(discord,mattermostorzulip, defaultdiscord) and a nullabletopic, which only a Zulip row sets. A Zulip row keeps the numeric stream ID inchannelId. Every row that existed before thetopiccolumn has itNULLand keeps its platform. suppressEmbedsis Discord and Mattermost only. Zulip cannot turn off link previews for one message (previews are a realm setting, and the per-message flag is per recipient), so the Zulip sender ignores it; the Zulip commands accept it and say so.- Every platform manages its own rows: the Discord slash commands the
discordrows, the Mattermost commands themattermostrows, the Zulip commands (above) thezuliprows. - An edit reschedules on every platform.
updateScheduledMessage(name, service, changes)validates a new cron expression before anything is written, updates the row only if it belongs to that platform, then stops the running job and registers the updated row (reschedule), so the next tick sends the new text on the new schedule. The Zulipschedule-editand the Mattermost dialog use it; the Mattermost dialog stores only the fields it edits (cronExpression,message,suppressEmbeds), not the rest of the dialog response, and a failure there is logged (Failed to edit scheduled message <name>: …) rather than left as an unhandled rejection. The Discord modal keeps its own path and reschedules the same way.
RSS
RSSService (src/services/rss.service.ts) polls every rss_feed row every 15 minutes and posts each new post as an rss notification through notifyTarget, to the row's own target (toNotificationTarget): a Discord channel, or a Zulip stream and topic. It never names a platform when it posts.
- Columns:
service(discordorzulip, defaultdiscord) and a nullabletopic; the primary key is (url,channelId,service). A Zulip row keeps the numeric stream ID inchannelId. Every row that existed before these columns is adiscordrow with no topic and is polled and posted exactly as before. The repository addresses a row by its whole key,serviceincluded:getRSSFeeds({ channelId, service }),removeRSSFeed(url, channelId, service)(which resolves to whether a row was removed) andupdateRSSFeed({ url, channelId, service, … });getRSSFeeds()returns every feed on every service. - The Discord embed is what it always was, byte for byte, key order included:
toRSSEmbedin the Discord renderer sets only the keys the post has, always in the same order (author, title, description, timestamp, URL). The feed is the author, linked to the subscribed feed URL, with its image as the icon.rss.service.spec.tspins the JSON. On Zulip the post is one message: the title linked, the feed linked as the author, the date as a<time:…>that every reader sees in their own time zone, and the summary in a quote fence. A post without a title is labelled with its own link, so it still links to itself, and a post whose link was dropped has its title unlinked. Link targets in a Zulip heading have(,)and whitespace percent-encoded, so a post's link, which anyone can write, cannot end its link early and mention the stream. - A bad post does not jam the feed.
toRSSNotificationbuilds the notification from sanitised data: a title or feed title over Discord's 256 is shortened, the summary is shortened to 4096 (on every platform, so it is content), and a link that is not absolutehttp(s), a feed image that is nothttp(s)orattachment, or a date that does not parse is dropped; an empty title is left out. The post is delivered without the bad field instead of throwing on every poll. A post left with no title, link, summary or date, from a feed with no title, has nothing to post (Discord refuses an empty embed), so it is skipped with awarn(Skipping <id> of the RSS feed <url>: it has nothing to post) and counts as delivered. A post with only a feed author or only a date is still posted: on Discord as that author or timestamp alone, exactly as before, and on Zulip as a heading without a title. - Delivery is at least once, never lossy, never duplicated. A feed's new posts are posted oldest first; at the first one that is not delivered (
notifyTargetresolvedfalse), the feed stops,lastIdis stored as the last post that was, and awarnsays the post is retried on the next poll. Nothing before it is posted again, and it is not skipped. - One feed cannot stop the others. Each feed is polled inside its own catch: a feed that cannot be fetched or stored is logged with its URL (
Could not update the RSS feed <url> in <service> channel <id>: …) and the poll moves on to the next one. - No orphan rows. Subscribing inserts the row, then fetches the feed and posts its newest post; if the feed has no posts, cannot be fetched or the post is not delivered (
Could not post the newest post of <url>), the row it inserted is removed and the error rethrown, so the command reports it. A row that could not be inserted is never removed. - Discord's
/rss-subscribeand/rss-unsubscribemanage thediscordrows of the channel they are run in, the Zulip commands (above) thezuliprows of their stream.searchRSSFeeds, the/rss-unsubscribeautocomplete, returns at most 25 feeds, the most Discord accepts in an autocomplete response; before, a channel with 26 or more feeds got no suggestions at all.
Emote sync
/emote-sync on Discord and emote-sync on Zulip (ChatService.syncEmotes(guildId), which returns an EmoteSyncReport that formatEmoteSyncReport renders for both, with a subject naming the server on Zulip) upload every Discord emote to Zulip and Mattermost. The report counts what was read from Discord and what each platform took (3 emotes, 1 uploaded to Zulip, 2 uploaded to Mattermost) and says so plainly when the server has no emotes (the Discord server has no emotes, so nothing was uploaded). A server the bot cannot see (getEmotes answers undefined: not logged in to Discord, or not a member) is an error, Cannot read the emotes of Discord server <id>: …, which the Zulip command posts as its failure line, never a report of nothing done. The Mattermost side takes the Discord name as it is, and is idempotent the same way as the Zulip side below: it pages through the realm's custom emoji first (MattermostRepository.listEmoji, GET /emoji 200 at a time) and skips a name that is already there, listed as N already on Mattermost: …; an upload Mattermost refuses as a duplicate (api.emoji.create.duplicate.app_error, a name taken since the listing) counts the same, not as a failure. Nothing is ever overwritten or deleted. If the listing fails, it is logged once and every emote is uploaded, a duplicate still counting as already there. The Zulip side does not take the name as it is:
- Names: the 12.3 spec for
POST /realm/emoji/{emoji_name}says a name can only contain letters, numbers, dashes and spaces, that upper and lower case are treated the same and that underscores are treated the same as spaces.toZulipEmojiNameinchat.service.tsimplements that documented rule: lowercase (case is one name to Zulip, so the sync must settle on one spelling to compare withlistEmoji()), every other character becomes_(theidentifierfallback for a nameless emote carries a:), a trailing run of_/-is dropped because the server refuses a name ending in one (Emoji names must end with either a letter or digit., which the spec's description leaves out), and an empty result falls back toemote. A dot is outside the documented set and is not relied on. - Collisions: two Discord emotes can normalise to one name (
catJAMandCatJam).claimZulipEmojiNameappends2,3, … in Discord order, deciding the suffix from the run and Zulip's built-in names (below) alone, so it comes out the same on every sync. The reply lists renames (nameless:3 → nameless_3,CatJam → catjam2); a change of case only is not reported, since Zulip does not tell the two apart. - Built-in names: Zulip refuses a member's upload under the name of one of its built-in emoji (
Only administrators can override default emoji.), and from an administrator it would replace the built-in for the whole realm, which the sync must never do. So every name in the realm'semoji_codes.json(getEmojiCodes, 3339 names on 12.3) starts the run claimed, as doeszulip(Zulip's logo emoji is not in that table, yet Zulip lets even a member replace it realm-wide). An emote named like one takes the next free suffix the same way (fire → fire2, orfire3whenfire2is taken too), reported as a rename and found again on the next sync. A built-in name that an active realm emoji already holds (an administrator's earlier override) is not claimed, so the first emote normalising to it counts as already on Zulip rather than gaining a suffixed duplicate. If the table cannot be fetched, that is logged and the Zulip side of the run is skipped, with the reason in the reply, as when the realm emoji cannot be listed: without it an administrator's account would replace built-ins, and a case twin of a built-in-named emote would take the suffix that emote gets once the table is back. - Idempotency: before uploading, the sync reads
listEmoji()and skips any name that an active realm emoji already holds, counting it as already synced and listing it by Discord name (N already on Zulip: catJAM, CatJam → catjam2, …). That is what makes a second sync a no-op instead of re-uploadingcatjamascatjam2,catjam3, … on every run; a deactivated emoji frees its name. The corollary is that an emoji a human uploaded by hand under a name a Discord emote normalises to is taken to be that emote and is never duplicated or suffixed: the sync cannot tell its own uploads from a human's (that would take the uploading account's user ID, an endpoint it does not have), so it chooses idempotence and makes the shadowing visible in the reply instead. If the listing itself fails, the Zulip side of the whole run is skipped, logged once and said once in the reply (0 uploaded to Zulip (skipped: its emoji could not be listed)) rather than blamed on each emote; Mattermost still syncs and its failures are still listed. An upload Zulip answers with401(the user account's key refused,Malformed API keyin production while the bot account listed the emoji fine) would be refused for every emote, so it stops the Zulip side for the rest of the run instead: oneerrorline with Zulip's reason, never the key, namingZULIP_USER_USERNAMEandZULIP_USER_API_KEY, andN uploaded to Zulip (skipped: Zulip refused the credentials of the user account that uploads emoji)in the reply, counting what Zulip took before and blaming no emote.getConfigtrims everyZULIP_*value, since a key stored with its trailing newline is the likely cause of that answer. The reply stays within Discord's 2000 characters (10000 on Zulip).
Commands
npm run build # Build with nest
npm run check # TypeScript type check
npm run lint # ESLint
npm run format # Prettier check
npm run test # Vitest
npm run check:all # format + lint + check + test:cov
npm run zulip:types # Regenerate src/generated/zulip.ts from the pinned Zulip OpenAPI spec
Key Files
src/app.module.ts- Root NestJS modulesrc/main.ts- Bootstrap, Discord client initsrc/config.ts- Environment variable loadingsrc/constants.ts- Enums, channel IDs, role IDs, cron expressions, notification route table (NotificationRoutes)src/format.ts- String helpers with no platform imports (shorten,shortenCodePoints,plural,asHexColor, and the Zulip markdown guardsneutraliseZulipMentions,neutraliseZulipLabelandtoZulipQuote); the only helper module renderers may importsrc/util.ts- Discord-aware helpers (error reporting toteam.bot, hyperlinks, report field builders)src/discord/commands.ts- All slash commandssrc/discord/events.ts- Discord event handlerssrc/services/discord.service.ts- Core bot logicsrc/interfaces/database.interface.ts- DB schema types + repository interfacesrc/repositories/database.repository.ts- Kysely DB queriessrc/interfaces/notification.interface.ts- Platform-neutralNotificationmodel (kind,accent,author,title,url,body,fields)src/services/notification.service.ts- Destination-to-platform fan-out for notificationssrc/services/zulip.service.ts- Zulip event queue loop (onMessagehandlers, re-registration, backoff, shutdown) and the holiday noticesrc/services/zulip-command.service.ts- The Zulip commands: mention parsing, stream gating, the command table and its repliessrc/services/zulip-expander.service.ts- The cachedzulip_expandertable: the streams GitHub expansion runs insrc/services/scheduled-message.service.ts- Scheduled message jobs and the per-platformsenderstablesrc/services/rss.service.ts- RSS polling, post sanitising (toRSSNotification) and delivery throughnotifyTargetsrc/renderers/- Per-platformNotificationrenderers (discord,mattermost,zulip) and the shared accent palettesrc/generated/zulip.ts- Generated Zulip API types (npm run zulip:types), never edited by handsrc/repositories/zulip.client.ts- Typed Zulip transport: form/JSON encoding, multipart, errors, 429 retry, timeoutsrc/services/mirror.service.ts- The Discord-Zulip dev mirror: routing, edits, deletions, moves, catch-upsrc/services/mirror-link.service.ts- Mirror links and linked accounts: checks, rows, announcements, the identity codessrc/discord/mirror-commands.ts- The mirror's Discord slash commandssrc/zulip-command-parser.ts- The Zulip command parser (parseCommand,tokenize,splitArguments)src/mirror/- The mirror's pure pieces: pairs, names, both translators, the per-pair queue, the Discord message DTOsrc/discord/mirror.ts- The Discord events the mirror listens to