mirror of
https://github.com/immich-app/discord-bot.git
synced 2026-09-30 13:32:52 +08:00
feat: configure zulip github expansion per stream by command (#654)
This commit is contained in:
Zack Pollard
@@ -8,7 +8,7 @@ Immich Discord bot built with NestJS, discordx, and PostgreSQL (Kysely ORM).
|
||||
- **Framework**: NestJS with `@nestjs/schedule` for cron jobs
|
||||
- **Discord**: discord.js + discordx (decorator-based slash commands, events, modals, buttons)
|
||||
- **Database**: PostgreSQL via Kysely (type-safe query builder), file-based migrations
|
||||
- **Testing**: Vitest with manual mocks; `database.repository.spec.ts` alone runs against a real database, and only when `TEST_DB_URL` names one migrated to the latest schema (it deletes that database's mirror link and identity rows); otherwise it is skipped
|
||||
- **Testing**: Vitest with manual mocks; `database.repository.spec.ts` alone runs against a real database, and only when `TEST_DB_URL` names one migrated to the latest schema (it deletes that database's mirror link, identity and Zulip expander rows, and checks the expander migration's seed by running its `down` and `up` in a transaction it rolls back); otherwise it is skipped
|
||||
- **Build**: `nest build` (SWC compiler), `eslint`, `prettier`
|
||||
|
||||
## Architecture
|
||||
@@ -34,7 +34,7 @@ Services inject them with `@Inject(IDatabaseRepository)`.
|
||||
- **Services**: Listed in `src/services/index.ts` → imported into `AppModule`
|
||||
- **Repositories/Providers**: Listed in `src/repositories/index.ts` → imported into `AppModule`
|
||||
- **Discord classes**: Listed directly in `AppModule` (`DiscordCommands`, `DiscordEvents`, `DiscordHelpDesk`, `DiscordContextMenus`, `DiscordMirrorEvents`, `DiscordMirrorCommands`)
|
||||
- **Init order**: `AppModule.onModuleInit` calls each service's `init()` by hand, in a fixed order. A service that subscribes to Zulip events (`ZulipService.onMessage`, `onMessageUpdate`, `onMessagesDeleted`, `onQueueRegistered`) must have its `init()` called there, and before `ZulipService.init()`, which starts the event loop (see *Zulip event queue*). Today that is `MirrorService` (the Discord-Zulip mirror, first, right after the migrations, since it reads its links from the database; its handlers only enqueue), `ChatService` (the expanders) and `ZulipCommandService` (the commands). Last, once Zulip is up, `ChatService.loginToDiscord` logs in to Discord (not with the `dev` token) and awaits it, so the HTTP server listens only once Discord is ready: a login that hangs holds back the webhooks and the `@Cron` jobs, while the Zulip event loop, already started, keeps running. `DiscordRepository.login` resolves at `clientReady`, once the guilds and so their channels are cached; discord.js's own `login` resolves at the gateway's READY, before them. Alongside the login, `I'm alive, running <version>!` goes to `team.bot` once per process, never on a gateway reconnect: when Discord turns ready, at once with the `dev` token, or without Discord after 60s (`DISCORD_READY_WAIT_MS`) of a login still pending. A failed login is reported to `team.bot` instead (`Discord login failed: …`, which reaches Zulip alone) and then fails the boot, so a restart retries it.
|
||||
- **Init order**: `AppModule.onModuleInit` calls each service's `init()` by hand, in a fixed order. `ZulipExpanderService.init()` comes right after the migrations: it loads the streams with GitHub expansion that the expanders and the queue registration read, so it must precede `ZulipService.init()`. A service that subscribes to Zulip events (`ZulipService.onMessage`, `onMessageUpdate`, `onMessagesDeleted`, `onQueueRegistered`) must have its `init()` called there, and before `ZulipService.init()`, which starts the event loop (see *Zulip event queue*). Today that is `MirrorService` (the Discord-Zulip mirror, first, right after the migrations, since it reads its links from the database; its handlers only enqueue), `ChatService` (the expanders) and `ZulipCommandService` (the commands). Last, once Zulip is up, `ChatService.loginToDiscord` logs in to Discord (not with the `dev` token) and awaits it, so the HTTP server listens only once Discord is ready: a login that hangs holds back the webhooks and the `@Cron` jobs, while the Zulip event loop, already started, keeps running. `DiscordRepository.login` resolves at `clientReady`, once the guilds and so their channels are cached; discord.js's own `login` resolves at the gateway's READY, before them. Alongside the login, `I'm alive, running <version>!` goes to `team.bot` once per process, never on a gateway reconnect: when Discord turns ready, at once with the `dev` token, or without Discord after 60s (`DISCORD_READY_WAIT_MS`) of a login still pending. A failed login is reported to `team.bot` instead (`Discord login failed: …`, which reaches Zulip alone) and then fails the boot, so a restart retries it.
|
||||
|
||||
### Database Migrations
|
||||
|
||||
@@ -121,7 +121,7 @@ The bot talks to Zulip through a typed `openapi-fetch` client, not an SDK.
|
||||
- Credentials and the `Authorization` header are never logged.
|
||||
- **Two identities, four clients**: `ZulipRepository` holds a `bot` client (posts messages) and a `user` client (uploads emoji) because Zulip only lets human accounts upload emoji (`This endpoint does not accept bot requests`). Config keeps `zulip.bot` and `zulip.user` for that reason. An `uploads` client is the bot identity with a 120s timeout, for the mirror's `POST /user_uploads`. A fourth client, `events`, is the bot identity again with a long timeout, used for `GET /events` alone: the server holds that request open on purpose, for up to the `event_queue_longpoll_timeout_seconds` it returns from `POST /register` (90 by default), so it cannot share the 30s that every other request should fail within. The server answers a quiet queue with a heartbeat about that many seconds after it *received* the poll, so the client's timeout is that value plus a 30s margin (`longpollTimeoutMs`, 120s by default): a client timeout equal to the server's would lose the race on every quiet poll and make an idle channel look like a dead connection. `registerQueue` rebuilds the `events` client with the server's value plus the margin. Every request also carries the caller's own `signal` when one is passed (`getEvents` passes the event loop's): the client composes it with the attempt's timeout in `init` (`AbortSignal.any`), since `fetch(request, init)` replaces the request's signal with `init.signal` rather than adding to it. All are created once, in `ZulipService.init` (skipped with the `dev` sentinel keys); calling a repository method before that throws `Zulip client not initialised`.
|
||||
- **Endpoints**: `ZulipRepository` exposes only what the bot uses today: `sendMessage` (resolves to the new message's `{ id }`), `getMessage` (the message's ID, stream, sender name and *current* topic, asked with `allow_empty_topic_name` so the empty "general chat" topic comes back as `''` rather than as the realm's translated display name), `updateMessage` (`PATCH /messages/{id}`: a content edit or a topic move with a `propagateMode`, never both in one call, which Zulip rejects), `createEmote`, `listEmoji` (`GET /realm/emoji`, deactivated ones included), `getSubscriptions` (the bot's streams), the event queue's `getOwnUser` (the bot's ID and full name; throws `Zulip returned no user ID for the bot` rather than resolve without the ID the loop filters its own messages by, while a missing name comes back as `''`, since only the command router needs it), `getMessages` (`GET /messages`: the `numBefore` newest messages of one stream and topic, `anchor: newest`, as raw markdown, with the narrow passed as one JSON string; the `similar` command reads the topic with it), `registerQueue` (the queue plus the streams it will carry, each with whether it is private), `getEvents` (with the loop's abort signal) and `deleteQueue` (see below), `isInitialised`, which `NotificationService` checks before routing to Zulip, `getUser` (a user's name and role, which the mirror commands authorise by), `getStream` (a stream's name and privacy), `sendDirectMessage` (the answer to a direct message command), and for the mirror `deleteMessage`, `uploadFile` (`POST /user_uploads`, multipart `filename`), `downloadUpload` (a `/user_uploads/` path, see *Discord-Zulip dev mirror*), `getStreamMessagesBefore` (a page of one stream before a message ID or from the newest, optionally leaving one sender out with a negated `sender` narrow), `getEmojiCodes` (the realm's static `emoji_codes.json`: `unicode`, name to Unicode, and `names`, code point sequence to name; the emote sync reads the names, the mirror both) and `addReaction`/`removeReaction` (as the bot, the emoji given by name, code and type; Zulip answers `REACTION_ALREADY_EXISTS` or `REACTION_DOES_NOT_EXIST` when there is nothing to do). `getMessage` also returns the message's reactions with who gave each, and `listEmoji` each realm emoji's ID. Each phase adds only the endpoints it needs, a few lines each thanks to the generated types; do not add unused methods.
|
||||
- **Streams**: `Constants.Zulip.Streams` holds the channels the bot posts to by numeric ID, named after the channel (`ImmichThirdParties: 111` carries every team notification, `ImmichPullRequests: 112` one topic per pull request, `ImmichAlerts: 113` the release workflow alerts and the bot's own startup and error lines). `Constants.Zulip.TeamStreams` holds the ones it listens in: every private `immich-*` stream, 107 to 113; the two are separate maps because `Streams` is pinned by a characterization test. `Constants.Zulip.Expanders` (one list per expander) and `Constants.Zulip.Commands` (where commands are taken) are each that set today. IDs survive a rename; the dev server mirrors the names but not the IDs. Topic strings live in `NotificationRoutes`, not here.
|
||||
- **Streams**: `Constants.Zulip.Streams` holds the channels the bot posts to by numeric ID, named after the channel (`ImmichThirdParties: 111` carries every team notification, `ImmichPullRequests: 112` one topic per pull request, `ImmichAlerts: 113` the release workflow alerts and the bot's own startup and error lines). `Constants.Zulip.TeamStreams` holds the ones it listens in: every private `immich-*` stream, 107 to 113; the two are separate maps because `Streams` is pinned by a characterization test. `Constants.Zulip.Commands` (where commands are taken) is that set; where the expanders run is configured at runtime (see *Zulip message expanders*). IDs survive a rename; the dev server mirrors the names but not the IDs. Topic strings live in `NotificationRoutes`, not here.
|
||||
- **Startup subscription check**: those three are private streams, and Zulip lets an unsubscribed bot post to public streams only. The bot is a plain member and cannot subscribe itself to a private stream, so `ZulipService.init` fetches its subscriptions right after the clients exist and logs one `warn` per stream in `Constants.Zulip.RequiredSubscriptions` it is missing, naming the stream. It never throws and never self-subscribes: a missing subscription is visible at deploy time instead of failing the first post at 3am, and it must not stop the bot from booting. `RequiredSubscriptions` lists those three and not `FUTOStaff` (2), where the holiday notice has posted since before the check existed: whether that stream is public or the bot was subscribed by hand, its subscription is not one the deploy needs to prove, so it is not asserted on a guess. The streams the bot *listens* in are checked separately, by the event loop at every queue registration (see below): they are not in `RequiredSubscriptions`, whose three entries and warn text are pinned by characterization tests.
|
||||
- **Renderer**: `toZulipMessage` (`src/renderers/zulip.renderer.ts`) flattens a `Notification` into one message of Zulip markdown. `zulip.renderer.spec.ts` pins all of the following.
|
||||
- *Shape*: `{emoji} **[title](url)** — [author](url)`, then the body, then the fields. A `line` field (every kind but `incident`) is one `**name:** value` line; a `block` field (`incident`) is a bold name line with the value quoted beneath it.
|
||||
@@ -147,7 +147,7 @@ The Zulip analogue of the `#team-pull-requests` Discord forum: one topic per pub
|
||||
|
||||
Zulip has no push transport: a client registers an event queue (`POST /register`) and long-polls it (`GET /events` with `queue_id` and `last_event_id`), which the server holds open until an event or its heartbeat. `ZulipService` runs that loop; `ZulipRepository` only wraps the four endpoints. The loop is where `zulip-js` failed: its loop spun at one request per second against a dead queue, forever, without a log line, and every rule below exists so that this one cannot.
|
||||
|
||||
- **Registration**: `event_types: ["message", "update_message", "delete_message", "reaction"]`, `apply_markdown: false`, so handlers see the markdown the sender typed rather than rendered HTML, and the `bulk_message_deletion` client capability, so a deletion arrives as one event with every ID. Message events go to the `onMessage` handlers; update events go only to the `onMessageUpdate` handlers, and never when they are rendering-only (a link preview), made by the server (`user_id` null) or made by the bot itself; deletion events go only to the `onMessagesDeleted` handlers; reaction events, but the bot's own, go only to the `onReaction` handlers; after every registration, the first and each re-registration, the `onQueueRegistered` handlers get the subscribed stream IDs. Today the mirror is the only one to use the last four. The bot receives messages from the streams it is subscribed to, which for private streams is the only way to see them; `all_public_streams` is deliberately not set, since it would add every public channel of the FUTO realm, which the allowlists below would discard anyway. That makes the bot's subscriptions the only thing that puts a stream's messages on the queue, so the register call also asks for them (`fetch_event_types: ["subscription", "realm"]`, which adds the `subscriptions` list and the realm settings to the answer and changes nothing about the events the queue receives; of the realm settings only `realm_empty_topic_display_name` is kept, as `ZulipService.emptyTopicName`: the queue does not declare `empty_topic_name`, so events and `GET /messages` name the empty topic by it, "general chat"), and `ZulipService.registerQueue` logs one `warn` per stream in any list of `Constants.Zulip.Expanders` the queue cannot see (`The Zulip bot is not subscribed to stream 107 (ImmichGeneral): its event queue carries no messages from it, so nothing is expanded there until an admin subscribes it`). The set it checks is the union of the expander lists and `Constants.Zulip.Commands`, not `TeamStreams`, so a stream added to one expander's list alone, or to the command list alone, is checked too (named by its bare ID if no constant map names it). Without it a bot never subscribed to `immich-general` would register, poll and receive nothing, deaf in the main team channel with no line in the log. It is checked at every registration, not once at boot, so a subscription added while the bot runs is confirmed at the next re-registration; the loop carries on with the streams it can see. The loop also reads `getOwnUser` once, before its first poll, and drops every message whose `senderId` is the bot's own, or it would answer its own replies; the account is exposed as `ZulipService.ownUser` (`undefined` until then, so always set when a handler runs), which is where the command router gets the name it matches a mention against.
|
||||
- **Registration**: `event_types: ["message", "update_message", "delete_message", "reaction"]`, `apply_markdown: false`, so handlers see the markdown the sender typed rather than rendered HTML, and the `bulk_message_deletion` client capability, so a deletion arrives as one event with every ID. Message events go to the `onMessage` handlers; update events go only to the `onMessageUpdate` handlers, and never when they are rendering-only (a link preview), made by the server (`user_id` null) or made by the bot itself; deletion events go only to the `onMessagesDeleted` handlers; reaction events, but the bot's own, go only to the `onReaction` handlers; after every registration, the first and each re-registration, the `onQueueRegistered` handlers get the subscribed stream IDs. Today the mirror is the only one to use the last four. The bot receives messages from the streams it is subscribed to, which for private streams is the only way to see them; `all_public_streams` is deliberately not set, since it would add every public channel of the FUTO realm, and the nitter mirror, which runs in every stream the bot hears, with it. That makes the bot's subscriptions the only thing that puts a stream's messages on the queue, so the register call also asks for them (`fetch_event_types: ["subscription", "realm"]`, which adds the `subscriptions` list and the realm settings to the answer and changes nothing about the events the queue receives; of the realm settings only `realm_empty_topic_display_name` is kept, as `ZulipService.emptyTopicName`: the queue does not declare `empty_topic_name`, so events and `GET /messages` name the empty topic by it, "general chat"), and `ZulipService.registerQueue` logs one `warn` per listening stream the queue cannot see (`The Zulip bot is not subscribed to stream 107 (ImmichGeneral): its event queue carries no messages from it, so nothing is expanded there until an admin subscribes it`). The set it checks is every stream with GitHub expansion on (`ZulipExpanderService`, as configured at that registration) plus `Constants.Zulip.Commands`, not `TeamStreams`, so a stream with GitHub expansion alone, or on the command list alone, is checked too; the nitter mirror runs everywhere and adds none (named by its bare ID if no constant map names it). Without it a bot never subscribed to `immich-general` would register, poll and receive nothing, deaf in the main team channel with no line in the log. It is checked at every registration, not once at boot, so a subscription added while the bot runs is confirmed at the next re-registration; the loop carries on with the streams it can see. The loop also reads `getOwnUser` once, before its first poll, and drops every message whose `senderId` is the bot's own, or it would answer its own replies; the account is exposed as `ZulipService.ownUser` (`undefined` until then, so always set when a handler runs), which is where the command router gets the name it matches a mention against.
|
||||
- **Other bots are dropped too, by email.** Discord skips every bot (`message.author.bot`) and Mattermost skips every bot (`post.props.from_bot`); a message event on Zulip carries no `is_bot` flag, so `isBotSender` in `zulip.service.ts` goes by the sender's API email instead: Zulip creates every bot as `{short_name}-bot@{realm host}` and its own as `notification-bot@…` and `welcome-bot@…`, and a bot's API email is never replaced by the `user{id}@…` placeholder that the realm's email visibility setting puts in a human's. So `-bot@` before the domain is a bot. Without it a GitHub integration, a CI notifier or another team's bot posting into `immich-pull-requests` would get the `#1234` references of every post expanded under it, the noise the allowlist keeps out of `immich-third-parties`. The failure direction is a human whose address ends in `-bot`, who gets no expansions; a bot cannot be created on any other address, so the other direction does not exist. The filter is in the loop's `dispatch`, next to the own-user one, not in the expanders: the command router must not take commands from a bot either, and does not, without a check of its own. A handler registered with `onMessage(handler, { withBots: true })` (the mirror's) is the one exception: it gets other bots' messages too, never the bot's own.
|
||||
- **Polling**: every event's ID raises the queue's `lastEventId`, heartbeats included, before that event's handlers run, and the next poll sends the raised cursor, which acknowledges everything up to it. Events are not guaranteed consecutive; the cursor never moves backwards.
|
||||
- **A dead queue re-registers, at once.** Zulip garbage-collects an idle queue and drops every queue on restart, so `BAD_EVENT_QUEUE_ID` is routine, not a failure: the loop clears its queue, logs it at `log` level, registers a new one and polls that, and it never asks about the dead ID again. The one immediate retry is exactly one: if the fresh queue is reported dead too, or registering it fails, the failure count carries into the backoff below, so even a server that answers `BAD_EVENT_QUEUE_ID` to everything is polled a handful of times in ten minutes.
|
||||
@@ -164,8 +164,8 @@ Zulip has no push transport: a client registers an event queue (`POST /register`
|
||||
`ChatService.onZulipMessage` mirrors `onMessageCreate` on Discord and `onMattermostPosted` on Mattermost, reusing the same GitHub expansion (`handleGithubThreadReferences`: `#1234`, `owner/repo#1234` and issue, PR and discussion URLs to titles and links; `handleGithubFileReferences`: file permalinks to code snippets; the two halves of `handleGithubReferences`, called separately here so that only the first is neutralised, see below) and `handleTwitterReferences` (an `x.com` link to its `nitter.net` mirror). That logic is platform-neutral and returns plain strings; it is shared, never forked. What differs on Zulip:
|
||||
|
||||
- **The reply is a new message, not an edit.** Mattermost appends the expansion to the user's own post; a Zulip bot cannot edit another user's message, so the expansion is posted into the same stream and topic, where the topic is the conversation and needs no backlink. All of one message's expansions go in one reply, GitHub first; there is no silent flag on Zulip. The expanders see only `message` events: updates and deletions reach only the mirror's registries, so editing the source message later does not expand it again; do not route those events to the expanders.
|
||||
- **Each expander runs only in an allowlisted stream.** `Constants.Zulip.Expanders` holds one list of stream IDs per expander (`GithubReferences`, `TwitterMirror`), both the `Immich` stream (54) plus `Constants.Zulip.TeamStreams` today (every `immich-*` stream, 107 to 113). A message in any other stream, and a direct message, is ignored by that expander without a GitHub call. Adding a stream is one line in `zulipTeamStreams` (or in one expander's list, to differ); the bot must also be subscribed to it, which the event loop checks for every list at every registration (see above). There is no runtime toggle.
|
||||
- **Privileged in every allowlisted stream.** Private-repository details are gated behind `isPrivileged` in the GitHub repository; the GitHub expander passes `true`, as Mattermost does, because the FUTO realm is not open to the public, so every stream on its list, `Immich` included, is readable by realm members only. Stream privacy is not checked: putting a stream on the list is the decision to show private repository titles and code there.
|
||||
- **The GitHub expander runs only in the streams it is turned on in; the nitter mirror runs in every stream.** The `zulip_expander` table holds one row per stream with GitHub expansion (`streamId` is its primary key). Its migration seeds what used to be hardcoded, the `Immich` stream (54) and every `immich-*` stream (107 to 113). `ZulipExpanderService` loads the table once at init and answers `isEnabled` from that cache, so no message costs a query; it is the only writer. The `expanders` command (below) changes one stream at a time, runs a stream's changes one after the other (a command the loop stopped waiting for may still be writing) and reads the stream back from the table after each, falling back to what the write reported when that read fails. A message in any other stream gets the nitter mirror alone, without a GitHub call; a direct message gets nothing. The bot must also be subscribed to the stream, which the command warns about and the event loop checks at every registration (see above).
|
||||
- **Privileged in every stream it is on in.** Private-repository details are gated behind `isPrivileged` in the GitHub repository; the GitHub expander passes `true`, as Mattermost does, because the FUTO realm is not open to the public, so every stream of the realm, `Immich` included, is readable by realm members only. Stream privacy is not checked: turning GitHub expansion on in a stream is the decision to show private repository titles and code there.
|
||||
- **The reply is neutralised where that protects something.** GitHub titles and links are written by anyone, so `neutraliseZulipMentions` runs over each of them, as over a notification, and over the nitter mirror, which is built from whatever the sender typed: nobody can ping the stream through an issue title. A code snippet is left exactly as GitHub has it: it sits inside a code fence, where Zulip renders no mention, so neutralising it protects nothing and the zero-width space would silently corrupt the code (`${file#*.}` in a shell script reads as a mention to the neutraliser). That is why the handler calls the two halves of `handleGithubReferences` separately rather than the combined method.
|
||||
- **Bots are filtered before the handler, by email, not by a flag.** `onMessageCreate` checks `message.author.bot` and `onMattermostPosted` checks `post.props.from_bot` in the handler itself; `onZulipMessage` checks nothing, because the event loop has already dropped the bot's own messages and every other bot's (`isBotSender`, the `-bot@` rule under *Zulip event queue* above) before any handler sees them. A handler therefore never needs its own bot check, and a message from a sender the server gave no email for is treated as a human's.
|
||||
|
||||
@@ -198,7 +198,7 @@ Zulip has no push transport: a client registers an event queue (`POST /register`
|
||||
- **A command is a message that starts with a mention of the bot**, `@**Name**` or the silent `@_**Name**`, either with Zulip's `|user_id` suffix, matched by the name from `ZulipService.ownUser` (which the loop read before its first poll) without regard to case. Only newlines may come before the mention: a first line indented by four spaces or a tab is a Markdown code block, which Zulip renders as code and notifies nobody of, so a command in one is ignored exactly as one in a fence, a quote block or inline code is (the mention regex is anchored on `^[\r\n]*`, not `^\s*`, and `parseCommand`'s spec pins each of those forms). A mention anywhere else (`thanks @**Immich**`) is not a command and gets no reply, so the bot cannot be summoned by accident mid-sentence; the cost is that `@**Immich** thanks` is answered with the unknown-command line. A mention alone is answered with the help. **Zulip's own "Quote and reply" is not a command either**: it starts the reply with a silent mention of the quoted author and `[said](<link>):` before the `quote` fence, so a reply that quotes the bot starts with a mention of it; `parseCommand` ignores a mention followed by `[said](` (the `QUOTE_AND_REPLY` regex), or the bot would answer the most natural way of replying to it with `Unknown command …` noise in the topic. A command typed after such a quote is not seen; mention the bot in a message of its own. The `parseCommand`, `splitArguments` and `tokenize` functions live in `src/zulip-command-parser.ts`, which the mirror also imports to leave commands unmirrored, and are pinned by `zulip-command.service.spec.ts`.
|
||||
- **Parsing**: after the mention, the first token is the command (case-insensitive) and the rest its arguments (`parseCommand` yields the name and the tokens). A double-quoted run, straight or curly (phone keyboards curl them), keeps its spaces anywhere in a token, so `text="two words"` and `"two words"` are one token each. **Options are sorted once the command is known** (`splitArguments`): a token `key=value` (the key a word) is a named option only when the command declares that key; any other `word=value` is a positional argument, in its place, so `similar the upload fails when CORS=strict on nginx` compares the whole sentence (`word=value` is everywhere in the error text `similar` exists to match: `LOG_LEVEL=debug`, `uid=1000`, `error=ENOENT`), while `backfill-pull-requests pr=1234` still reaches that command as an argument it cannot read. A quote that is never closed is answered with what went wrong, an unknown command with a pointer to `help`, a command with the wrong arguments with its usage line; never a stack trace. **An argument the command does not take is answered, never dropped**: each entry in the command table declares how many positionals it takes and which options it reads, and the dispatcher answers more positionals than that with the usage line before the command runs, and the command answers a positional it cannot read the same way (`emote-sync now`, `backfill-pull-requests pr=1234`, `fourthwall update ORD-1 id=ORD-2`, `similar text="a" b`). With no autocomplete and no confirmation step, a typo in `number=` must not turn a backfill of one pull request into a backfill of every one. `help` lists every command with its arguments, which matters more than on Discord, since there is no autocomplete to find them with.
|
||||
- **Every reply is public, in the same stream and topic** as the command (`sendMessage` with the message's `streamId` and `topic`, the empty topic included). There is no ephemeral equivalent on Zulip: several of these commands reply ephemerally on Discord (`/fourthwall`, `/backfill-pull-requests`, the private "Find similar issues"), and on Zulip the whole team sees the reply. Replies are kept short for that reason, and every interpolated string a human wrote (a command name, an order ID, the message `similar` compared, GitHub titles, emote names, an error message) goes through `neutraliseZulipMentions`, so a reply can ping nobody; the ones that are echoed as typed (a command name, an order ID, the text `similar` compared, and an error's message, which some service wrote) go through `code()`, inline code with every backtick stripped and every whitespace run collapsed to a space, so nothing in them can close the span early or start a line as a heading or a fence in the bot's voice; an unknown command name and the text `similar` compared are also cut to 80 characters (`ECHO_LENGTH`), an error's message to 300, so a pasted wall of text is not posted back whole. GitHub titles in a `similar` line also go through `neutraliseZulipLabel`, the renderer's link-label containment, so a title cannot add a link of its own next to the bot's. Every reply also goes through the one `reply()` seam, which cuts it to Zulip's `max_message_length` (10000, `shortenCodePoints`, `fit`): a reply the server refuses is one the topic never sees, and a long list of hits would otherwise make the `similar` result unpostable. A cut that lands inside a code span would leave the rest of the reply rendered as code, so a cut reply with an odd number of backticks (the bot's own spans hold none) is cut one code point shorter and, if that did not close the span, given a closing backtick, within the limit. The list `help` posts ends with a blank line, or Markdown would render the line after it as a continuation of the last item.
|
||||
- **Authorisation is the stream list.** Zulip has no per-channel bot permissions, so commands are taken only in `Constants.Zulip.Commands` (every `immich-*` stream, 107 to 113) and a command anywhere else, `Immich` included, is ignored without a reply. Whether a stream is private is not checked; the list is kept to team streams because these commands act (`backfill-pull-requests` creates threads and topics, `emote-sync` writes realm emoji through the user account, `fourthwall update` mutates orders). **The `mirror-*` commands are the exception**: the stream being linked is usually not a team one, so they are taken in any stream and authorised by the sender's role instead (`getUser`: owner 100 or administrator 200), and anyone else is told so. **Direct messages take `link <code>` and `unlink` only** (also `discord-link`/`discord-unlink`, with or without the mention), exactly that many words, answered to the sender alone, so nothing else said to the bot gets an answer. The event loop has already dropped every bot's messages (`isBotSender`) before the handler sees one, so no bot can drive a command.
|
||||
- **Authorisation is the stream list.** Zulip has no per-channel bot permissions, so commands are taken only in `Constants.Zulip.Commands` (every `immich-*` stream, 107 to 113) and a command anywhere else, `Immich` included, is ignored without a reply. Whether a stream is private is not checked; the list is kept to team streams because these commands act (`backfill-pull-requests` creates threads and topics, `emote-sync` writes realm emoji through the user account, `fourthwall update` mutates orders). **The `mirror-*` commands and `expanders` are the exception**: the stream they act on is usually not a team one, so they are taken in any stream and authorised by the sender's role instead (`getUser`: owner 100 or administrator 200), and anyone else is told so (each command's `administrators` names what only they can do, for the refusal); `help` marks them `(administrators)`. **Direct messages take `link <code>` and `unlink` only** (also `discord-link`/`discord-unlink`, with or without the mention), exactly that many words, answered to the sender alone, so nothing else said to the bot gets an answer. The event loop has already dropped every bot's messages (`isBotSender`) before the handler sees one, so no bot can drive a command.
|
||||
- **A failure never reaches the loop.** Every command runs inside a catch: a handler that throws is logged as an `error` (`The Zulip command <name> failed on message N`) and answered with `` `<name>` failed: <the error's message, shortened> ``; a reply that cannot be posted is logged (`Could not reply to the Zulip command in message N`) and the handler resolves all the same. The loop's own catch and 30s handler timeout stay as the backstop.
|
||||
- **Slow commands run detached from the loop**, because the loop polls nothing while a handler runs and stops waiting after 30s: `emote-sync`, `backfill-pull-requests all` and `fourthwall update all` post an acknowledgement in the topic at once, do the work in the background and post the outcome, or the failure line, when it is done; the handler returns after the acknowledgement. `inBackground` takes the acknowledgement and the work, and posts the acknowledgement before any of the work starts: the backfill lists the open pull requests inside the work, not before the acknowledgement (`Going through every open pull request, creating the Discord thread and the Zulip topic each one lacks; this can take a while…`), because a rate-limited GitHub client waits out the limit (an hour, seen in testing) before it answers, which left no acknowledgement and the handler stuck at the loop's 30s cap. A listing that fails is posted as the job's failure line; the count is in the outcome. **The wide form is asked for by name.** There is no confirmation step and no way to stop a run once it has started (the acknowledgement arrives after the work is committed to), and the wide form is the expensive one: `backfill-pull-requests all` reads every open PR's row and creates a forum thread and a Zulip topic for every one that lacks them, which after a fresh Zulip rollout is one topic per open PR, hundreds of posts; `fourthwall update all` fetches every order from Fourthwall again. So neither is the bare command: `backfill-pull-requests` and `fourthwall update` alone get the usage line, the narrow repair (`backfill-pull-requests 1234`, `fourthwall update ORD-1`) is one word away, and the fan-out takes the explicit `all` (any case, positional or `number=all` / `id=all`). **One run of each command at a time, inline or in the background**: the narrow forms take the same lock (`underLock`) as the wide ones, so `backfill-pull-requests 1234` during a running `backfill-pull-requests all` is answered with `` `backfill-pull-requests` is already running; wait for it to finish. `` and does nothing, rather than reaching `handlePullRequestTeamPlatforms` for #1234 concurrently with the background run, where both would read no thread and no topic ID, both would create, and only one ID would survive in the row, an orphaned forum thread or topic that no later event touches. An acknowledgement that cannot be posted does not start the work, and an outcome that cannot be posted is logged (`Could not post the outcome of the Zulip command <name>`); neither leaves the command locked.
|
||||
- **Commands** (community commands, `/link*`, `/messages*`, the help desk, `/prune`, `/age`, `/release-notes`, stay Discord-only):
|
||||
@@ -210,6 +210,7 @@ Zulip has no push transport: a client registers an event queue (`POST /register`
|
||||
|
||||
- `schedule-add <name> cron=<expression> message=<text> [topic=<topic>] [suppress-embeds=<true|false>]`, `schedule-list`, `schedule-edit <name> [cron=…] [message=…] [topic=…] [suppress-embeds=…]` and `schedule-remove <name>`: the Zulip scheduled messages (`service: 'zulip'` rows, see *Scheduled messages*). A message is posted in the stream the command is given in, in the `topic=` given or the command's own topic (the empty "general chat" topic included); `schedule-edit` takes `key=value` arguments where Discord opens a modal, needs at least one of `cron`, `message` and `topic`, and reschedules the running job at once. Each looks a name up among the Zulip rows only, so a Discord or Mattermost message can be neither edited nor removed from Zulip; names are unique across every platform (`scheduled_message_name_uq`), so a name taken elsewhere is refused with the database's error. `suppress-embeds` is accepted, validated as `true` or `false`, ignored and said to be ignored, in `help` and in the reply: Zulip link previews are a realm setting, and the per-message flag Zulip has is per recipient. `schedule-list` lists every Zulip scheduled message with its stream, topic, schedule and the start of its text, mentions neutralised. A message's own text is posted as the team wrote it, mentions included; that is its point.
|
||||
- `mirror-link [topic=<main topic>]` (starts a link of this stream and answers with the Discord command that completes it), `mirror-unlink` (this stream's link), `mirror-backfill` (copies the Discord history of the channel or thread this topic mirrors into it) and `mirror-list` (every link with both names, IDs, kind, main topic and whether it is on, and every linked account), administrators only, in any stream; `discord-unlink` removes the sender's own linked account. See *Discord-Zulip dev mirror*. On Discord, `/mirror-link id:<link id>` completes a link and `/mirror-unlink` removes one, run in the channel (for a forum, in any of its posts), `/mirror-backfill` backfills the thread, post or text channel it is run in, and `/mirror-list` lists them, all for the Administrator permission (the command default, checked again at runtime) and answered privately; a Zulip `mirror-unlink` reply given in the topic of the announcement leaves out what the announcement says.
|
||||
- `expanders on`, `expanders off` and `expanders list`, administrators only, in any stream, no other argument: turn GitHub expansion on or off in the stream the command is given in (`Turned on GitHub expansion in this stream.`, `Nothing changed: GitHub expansion was already off in this stream.`), with a warning when turning it on in a stream the bot is not subscribed to (read from `getSubscriptions` before the change; a failed read changes nothing). `list` names every stream it is on in (`getStream`, the bare ID when it cannot be read) and marks the ones the bot is not subscribed to. The nitter mirror has no command: it runs everywhere. See *Zulip message expanders*.
|
||||
- `rss-subscribe <url> [topic=<topic>]`, `rss-unsubscribe <url>` and `rss-list`: the Zulip RSS feeds of the stream the command is given in (`service: 'zulip'` rows, see *RSS*). `rss-subscribe` answers a feed the stream already has with the topic it goes to; otherwise it fetches the feed in the background (a feed's server can take longer than the loop waits for a handler), acknowledged at once and answered with the outcome: the newest post is posted to the `topic=` given or the command's topic, and a feed that cannot be fetched or posted leaves no row. `rss-list` exists because Zulip has no autocomplete to find a feed's URL with, which is what Discord's `/rss-unsubscribe` offers. One feed per stream: the topic is an attribute of the row, not part of its key.
|
||||
|
||||
### Scheduled messages
|
||||
@@ -271,6 +272,7 @@ npm run zulip:types # Regenerate src/generated/zulip.ts from the pinned Zulip O
|
||||
- `src/services/notification.service.ts` - Destination-to-platform fan-out for notifications
|
||||
- `src/services/zulip.service.ts` - Zulip event queue loop (`onMessage` handlers, re-registration, backoff, shutdown) and the holiday notice
|
||||
- `src/services/zulip-command.service.ts` - The Zulip commands: mention parsing, stream gating, the command table and its replies
|
||||
- `src/services/zulip-expander.service.ts` - The cached `zulip_expander` table: the streams GitHub expansion runs in
|
||||
- `src/services/scheduled-message.service.ts` - Scheduled message jobs and the per-platform `senders` table
|
||||
- `src/services/rss.service.ts` - RSS polling, post sanitising (`toRSSNotification`) and delivery through `notifyTarget`
|
||||
- `src/renderers/` - Per-platform `Notification` renderers (`discord`, `mattermost`, `zulip`) and the shared accent palette
|
||||
|
||||
@@ -16,6 +16,7 @@ import { GithubService } from 'src/services/github.service';
|
||||
import { MirrorService } from 'src/services/mirror.service';
|
||||
import { ScheduledMessageService } from 'src/services/scheduled-message.service';
|
||||
import { ZulipCommandService } from 'src/services/zulip-command.service';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { ZulipService } from 'src/services/zulip.service';
|
||||
|
||||
const middleware = [{ provide: APP_PIPE, useValue: new ValidationPipe({ transform: true, whitelist: true }) }];
|
||||
@@ -41,12 +42,15 @@ export class AppModule implements OnModuleInit {
|
||||
private mirrorService: MirrorService,
|
||||
private scheduledMessageService: ScheduledMessageService,
|
||||
private zulipCommandService: ZulipCommandService,
|
||||
private zulipExpanderService: ZulipExpanderService,
|
||||
private zulipService: ZulipService,
|
||||
) {}
|
||||
|
||||
async onModuleInit() {
|
||||
await this.githubService.init();
|
||||
await this.databaseService.runMigrations();
|
||||
// The expanders and the queue registration's subscription check read it, so it loads before ZulipService.init.
|
||||
await this.zulipExpanderService.init();
|
||||
// Every Zulip handler registers in its service's init, which must run before ZulipService.init starts the loop, or it silently misses messages.
|
||||
await this.mirrorService.init();
|
||||
await this.chatService.init();
|
||||
|
||||
@@ -262,10 +262,6 @@ export const Constants = {
|
||||
zulipStreams.ImmichPullRequests,
|
||||
zulipStreams.ImmichAlerts,
|
||||
],
|
||||
Expanders: {
|
||||
GithubReferences: [zulipStreams.Immich, ...Object.values(zulipTeamStreams)],
|
||||
TwitterMirror: [zulipStreams.Immich, ...Object.values(zulipTeamStreams)],
|
||||
},
|
||||
/** Zulip has no per-channel bot permissions, so this list is the only authorisation for commands. */
|
||||
Commands: Object.values(zulipTeamStreams),
|
||||
TeamStreams: zulipTeamStreams,
|
||||
|
||||
@@ -27,6 +27,7 @@ import {
|
||||
UpdateMirrorMessage,
|
||||
UpdateRSSFeed,
|
||||
UpdateScheduledMessage,
|
||||
ZulipExpander,
|
||||
} from 'src/schema';
|
||||
import { PullRequestTable } from 'src/schema/tables/pull-request.table';
|
||||
|
||||
@@ -117,4 +118,9 @@ export interface IDatabaseRepository {
|
||||
setMirrorIdentity(zulipUserId: number, discordUserId: string): Promise<MirrorIdentity[]>;
|
||||
/** Resolves to the removed identity, `undefined` when there was none. */
|
||||
removeMirrorIdentity(owner: MirrorIdentityOwner): Promise<MirrorIdentity | undefined>;
|
||||
getZulipExpanders(): Promise<ZulipExpander[]>;
|
||||
/** Resolves to whether the stream was added, `false` when it was there already. */
|
||||
addZulipExpander(streamId: number, createdBy: string): Promise<boolean>;
|
||||
/** Resolves to whether the stream was there to remove. */
|
||||
removeZulipExpander(streamId: number): Promise<boolean>;
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Kysely, sql } from 'kysely';
|
||||
import { DatabaseRepository } from 'src/repositories/database.repository';
|
||||
import { Database } from 'src/schema';
|
||||
import { down, up } from 'src/schema/migrations/1790263846796-ZulipExpanders';
|
||||
import { afterAll, beforeEach, describe, expect, it, vitest } from 'vitest';
|
||||
|
||||
const uri = process.env.TEST_DB_URL;
|
||||
@@ -10,7 +11,7 @@ vitest.mock('src/config', () => ({ getConfig: () => ({ database: { uri: process.
|
||||
const CHANNEL = '100000000000000001';
|
||||
const OTHER_CHANNEL = '100000000000000002';
|
||||
|
||||
// Needs a database migrated to the latest schema; its mirror_link and mirror_identity rows are deleted.
|
||||
// Needs a database migrated to the latest schema; its mirror_link, mirror_identity and zulip_expander rows are deleted.
|
||||
describe.skipIf(!uri)(DatabaseRepository.name, () => {
|
||||
const sut = new DatabaseRepository();
|
||||
const db = (sut as unknown as { db: Kysely<Database> }).db;
|
||||
@@ -18,6 +19,7 @@ describe.skipIf(!uri)(DatabaseRepository.name, () => {
|
||||
beforeEach(async () => {
|
||||
await db.deleteFrom('mirror_link').execute();
|
||||
await db.deleteFrom('mirror_identity').execute();
|
||||
await db.deleteFrom('zulip_expander').execute();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
@@ -86,6 +88,53 @@ describe.skipIf(!uri)(DatabaseRepository.name, () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('zulip expanders', () => {
|
||||
const streams = (rows: { streamId: number }[]) => rows.map(({ streamId }) => streamId);
|
||||
|
||||
it('should seed GitHub expansion in the Immich stream and every immich team stream', async () => {
|
||||
const rolledBack = new Error('rolled back');
|
||||
await expect(
|
||||
db.transaction().execute(async (trx) => {
|
||||
await down(trx);
|
||||
await up(trx);
|
||||
const rows = await trx.selectFrom('zulip_expander').selectAll().orderBy('streamId').execute();
|
||||
expect(streams(rows)).toEqual([54, 107, 108, 109, 110, 111, 112, 113]);
|
||||
expect(new Set(rows.map(({ createdBy }) => createdBy))).toEqual(new Set(['migration']));
|
||||
throw rolledBack;
|
||||
}),
|
||||
).rejects.toBe(rolledBack);
|
||||
});
|
||||
|
||||
it('should add a stream once, keep who added it first, and list by stream', async () => {
|
||||
expect(await sut.addZulipExpander(120, 'Alice on Zulip (user 12)')).toBe(true);
|
||||
expect(await sut.addZulipExpander(120, 'Bob on Zulip (user 13)')).toBe(false);
|
||||
expect(await sut.addZulipExpander(54, 'Bob on Zulip (user 13)')).toBe(true);
|
||||
|
||||
const rows = await sut.getZulipExpanders();
|
||||
expect(rows).toEqual([
|
||||
{ streamId: 54, createdBy: 'Bob on Zulip (user 13)', createdAt: expect.any(Date) },
|
||||
{ streamId: 120, createdBy: 'Alice on Zulip (user 12)', createdAt: expect.any(Date) },
|
||||
]);
|
||||
});
|
||||
|
||||
it('should remove only that stream, and resolve to whether it was there', async () => {
|
||||
await sut.addZulipExpander(120, 'Alice');
|
||||
await sut.addZulipExpander(121, 'Alice');
|
||||
|
||||
expect(await sut.removeZulipExpander(120)).toBe(true);
|
||||
expect(await sut.removeZulipExpander(120)).toBe(false);
|
||||
expect(streams(await sut.getZulipExpanders())).toEqual([121]);
|
||||
});
|
||||
|
||||
it('should refuse a second row of the same stream', async () => {
|
||||
await db.insertInto('zulip_expander').values({ streamId: 120, createdBy: 'Alice' }).execute();
|
||||
|
||||
await expect(
|
||||
db.insertInto('zulip_expander').values({ streamId: 120, createdBy: 'Bob' }).execute(),
|
||||
).rejects.toThrow('zulip_expander_pkey');
|
||||
});
|
||||
});
|
||||
|
||||
it('should list the recent rows of a channel and its threads, newest first, without deleted ones', async () => {
|
||||
const row = (discordMessageId: string, overrides: Record<string, unknown> = {}) => ({
|
||||
discordMessageId,
|
||||
|
||||
@@ -38,6 +38,7 @@ import {
|
||||
UpdateMirrorMessage,
|
||||
UpdateRSSFeed,
|
||||
UpdateScheduledMessage,
|
||||
ZulipExpander,
|
||||
} from 'src/schema';
|
||||
import { PullRequestTable } from 'src/schema/tables/pull-request.table';
|
||||
|
||||
@@ -547,4 +548,27 @@ export class DatabaseRepository implements IDatabaseRepository {
|
||||
: query.where('discordUserId', '=', owner.discordUserId);
|
||||
return matching.returningAll().executeTakeFirst();
|
||||
}
|
||||
|
||||
getZulipExpanders(): Promise<ZulipExpander[]> {
|
||||
return this.db.selectFrom('zulip_expander').selectAll().orderBy('streamId').execute();
|
||||
}
|
||||
|
||||
async addZulipExpander(streamId: number, createdBy: string): Promise<boolean> {
|
||||
const added = await this.db
|
||||
.insertInto('zulip_expander')
|
||||
.values({ streamId, createdBy })
|
||||
.onConflict((oc) => oc.column('streamId').doNothing())
|
||||
.returning('streamId')
|
||||
.executeTakeFirst();
|
||||
return added !== undefined;
|
||||
}
|
||||
|
||||
async removeZulipExpander(streamId: number): Promise<boolean> {
|
||||
const removed = await this.db
|
||||
.deleteFrom('zulip_expander')
|
||||
.where('streamId', '=', streamId)
|
||||
.returning('streamId')
|
||||
.executeTakeFirst();
|
||||
return removed !== undefined;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import { PullRequestTable } from 'src/schema/tables/pull-request.table';
|
||||
import { RSSFeedTable } from 'src/schema/tables/rss-feeds.table';
|
||||
import { ScheduledMessageTable } from 'src/schema/tables/scheduled-messages.table';
|
||||
import { SponsorTable } from 'src/schema/tables/sponsor.table';
|
||||
import { ZulipExpanderTable } from 'src/schema/tables/zulip-expander.table';
|
||||
|
||||
@Extensions(['uuid-ossp'])
|
||||
@Database({ name: 'discord-bot' })
|
||||
@@ -29,6 +30,7 @@ export class DiscordBotDatabase {
|
||||
MirrorMessageTable,
|
||||
MirrorLinkTable,
|
||||
MirrorIdentityTable,
|
||||
ZulipExpanderTable,
|
||||
];
|
||||
}
|
||||
|
||||
@@ -75,6 +77,8 @@ export type NewMirrorLink = Insertable<MirrorLinkTable>;
|
||||
export type MirrorIdentity = Selectable<MirrorIdentityTable>;
|
||||
export type NewMirrorIdentity = Insertable<MirrorIdentityTable>;
|
||||
|
||||
export type ZulipExpander = Selectable<ZulipExpanderTable>;
|
||||
|
||||
export interface Database {
|
||||
payment: PaymentTable;
|
||||
sponsor: SponsorTable;
|
||||
@@ -88,4 +92,5 @@ export interface Database {
|
||||
mirror_message: MirrorMessageTable;
|
||||
mirror_link: MirrorLinkTable;
|
||||
mirror_identity: MirrorIdentityTable;
|
||||
zulip_expander: ZulipExpanderTable;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
import { Kysely, sql } from 'kysely';
|
||||
|
||||
export async function up(db: Kysely<any>): Promise<void> {
|
||||
await sql`CREATE TABLE "zulip_expander" (
|
||||
"streamId" integer NOT NULL,
|
||||
"createdBy" character varying NOT NULL,
|
||||
"createdAt" timestamp with time zone NOT NULL DEFAULT now(),
|
||||
CONSTRAINT "zulip_expander_pkey" PRIMARY KEY ("streamId")
|
||||
);`.execute(db);
|
||||
await sql`INSERT INTO "zulip_expander" ("streamId", "createdBy")
|
||||
SELECT unnest(ARRAY[54, 107, 108, 109, 110, 111, 112, 113]), 'migration';`.execute(db);
|
||||
}
|
||||
|
||||
export async function down(db: Kysely<any>): Promise<void> {
|
||||
await sql`DROP TABLE "zulip_expander";`.execute(db);
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
import { Column, CreateDateColumn, Generated, PrimaryColumn, Table } from '@immich/sql-tools';
|
||||
|
||||
/** A stream the GitHub expander runs in. */
|
||||
@Table('zulip_expander')
|
||||
export class ZulipExpanderTable {
|
||||
@PrimaryColumn({ type: 'integer' })
|
||||
streamId!: number;
|
||||
|
||||
@Column()
|
||||
createdBy!: string;
|
||||
|
||||
@CreateDateColumn()
|
||||
createdAt!: Generated<Date>;
|
||||
}
|
||||
@@ -13,8 +13,10 @@ import { IMattermostInterface } from 'src/interfaces/mattermost.interface';
|
||||
import { IOutlineInterface } from 'src/interfaces/outline.interface';
|
||||
import { IZulipInterface, ZulipReceivedMessage } from 'src/interfaces/zulip.interface';
|
||||
import { ZulipApiError } from 'src/repositories/zulip.client';
|
||||
import { ZulipExpander } from 'src/schema';
|
||||
import { ChatService, formatEmoteSyncReport, hasBlacklistedUrl, toZulipEmojiName } from 'src/services/chat.service';
|
||||
import { NotificationService } from 'src/services/notification.service';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { ZulipMessageHandler, ZulipService } from 'src/services/zulip.service';
|
||||
import { MockInstance, Mocked, afterEach, beforeEach, describe, expect, it, vitest } from 'vitest';
|
||||
|
||||
@@ -135,6 +137,9 @@ const newDatabaseMockRepository = (): Mocked<IDatabaseRepository> => ({
|
||||
getMirrorIdentities: vitest.fn(),
|
||||
setMirrorIdentity: vitest.fn(),
|
||||
removeMirrorIdentity: vitest.fn(),
|
||||
getZulipExpanders: vitest.fn(),
|
||||
addZulipExpander: vitest.fn(),
|
||||
removeZulipExpander: vitest.fn(),
|
||||
});
|
||||
|
||||
const newMattermostMockRepository = (): Mocked<IMattermostInterface> => ({
|
||||
@@ -205,6 +210,7 @@ describe('Bot test', () => {
|
||||
let mattermostMock: Mocked<IMattermostInterface>;
|
||||
let zulipMock: Mocked<IZulipInterface>;
|
||||
let zulipServiceMock: ReturnType<typeof newZulipServiceMock>;
|
||||
let zulipExpanders: ZulipExpanderService;
|
||||
let fetchMock: ReturnType<typeof vitest.fn>;
|
||||
|
||||
beforeEach(() => {
|
||||
@@ -217,6 +223,7 @@ describe('Bot test', () => {
|
||||
mattermostMock = newMattermostMockRepository();
|
||||
zulipMock = newZulipMockRepository();
|
||||
zulipServiceMock = newZulipServiceMock();
|
||||
zulipExpanders = new ZulipExpanderService(databaseMock);
|
||||
// 7TV and BTTV lookups go through the global fetch.
|
||||
fetchMock = vitest.fn();
|
||||
vitest.stubGlobal('fetch', fetchMock);
|
||||
@@ -232,6 +239,7 @@ describe('Bot test', () => {
|
||||
zulipMock,
|
||||
zulipServiceMock as unknown as ZulipService,
|
||||
new NotificationService(discordMock, mattermostMock, zulipMock),
|
||||
zulipExpanders,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1748,6 +1756,10 @@ describe('Bot test', () => {
|
||||
});
|
||||
|
||||
it('should subscribe the Zulip expanders to the event loop', async () => {
|
||||
databaseMock.getZulipExpanders.mockResolvedValue([
|
||||
{ streamId: 107, createdBy: 'migration', createdAt: new Date(0) },
|
||||
]);
|
||||
await zulipExpanders.init();
|
||||
await sut.init();
|
||||
|
||||
expect(zulipServiceMock.onMessage).toHaveBeenCalledOnce();
|
||||
@@ -1784,13 +1796,58 @@ describe('Bot test', () => {
|
||||
});
|
||||
|
||||
describe('onZulipMessage', () => {
|
||||
beforeEach(() => {
|
||||
zulipMock.sendMessage.mockResolvedValue({ id: 901 });
|
||||
const expanderRow = (streamId: number): ZulipExpander => ({
|
||||
streamId,
|
||||
createdBy: 'migration',
|
||||
createdAt: new Date(0),
|
||||
});
|
||||
|
||||
it('should run in the Immich stream and every immich team stream', () => {
|
||||
expect(Constants.Zulip.Expanders.GithubReferences).toEqual([54, 107, 108, 109, 110, 111, 112, 113]);
|
||||
expect(Constants.Zulip.Expanders.TwitterMirror).toEqual([54, 107, 108, 109, 110, 111, 112, 113]);
|
||||
beforeEach(async () => {
|
||||
zulipMock.sendMessage.mockResolvedValue({ id: 901 });
|
||||
databaseMock.getZulipExpanders.mockResolvedValue(
|
||||
[54, 107, 108, 109, 110, 111, 112, 113, 120].map((streamId) => expanderRow(streamId)),
|
||||
);
|
||||
await zulipExpanders.init();
|
||||
});
|
||||
|
||||
it('should mirror x.com links but ask GitHub nothing in a stream without GitHub expansion', async () => {
|
||||
await sut.onZulipMessage(zulipMessage({ streamId: 121, content: 'https://x.com/immich/status/1 fixes #4242' }));
|
||||
|
||||
expect(githubMock.getIssueOrPrMessage).not.toHaveBeenCalled();
|
||||
expect(githubMock.getRepositoryFileContent).not.toHaveBeenCalled();
|
||||
expect(zulipMock.sendMessage).toHaveBeenCalledExactlyOnceWith({
|
||||
stream: 121,
|
||||
topic: 'thumbnails',
|
||||
content: 'https://nitter.net/immich/status/1',
|
||||
});
|
||||
});
|
||||
|
||||
it('should expand GitHub references in a stream turned on after the seed', async () => {
|
||||
await sut.onZulipMessage(zulipMessage({ streamId: 120, content: 'https://x.com/immich/status/1 fixes #4242' }));
|
||||
|
||||
expect(zulipMock.sendMessage).toHaveBeenCalledExactlyOnceWith({
|
||||
stream: 120,
|
||||
topic: 'thumbnails',
|
||||
content: 'https://github.com/immich-app/immich/pull/4242\nhttps://nitter.net/immich/status/1',
|
||||
});
|
||||
});
|
||||
|
||||
it('should follow a change to GitHub expansion at once, with no query per message', async () => {
|
||||
databaseMock.removeZulipExpander.mockResolvedValue(true);
|
||||
const remaining = (await databaseMock.getZulipExpanders()).filter(({ streamId }) => streamId !== 107);
|
||||
databaseMock.getZulipExpanders.mockResolvedValue(remaining);
|
||||
await zulipExpanders.disable(107);
|
||||
const reads = databaseMock.getZulipExpanders.mock.calls.length;
|
||||
|
||||
await sut.onZulipMessage(zulipMessage({ content: 'https://x.com/immich/status/1 fixes #4242' }));
|
||||
|
||||
expect(githubMock.getIssueOrPrMessage).not.toHaveBeenCalled();
|
||||
expect(zulipMock.sendMessage).toHaveBeenCalledExactlyOnceWith({
|
||||
stream: 107,
|
||||
topic: 'thumbnails',
|
||||
content: 'https://nitter.net/immich/status/1',
|
||||
});
|
||||
expect(databaseMock.getZulipExpanders).toHaveBeenCalledTimes(reads);
|
||||
});
|
||||
|
||||
it('should reply with the expanded GitHub references in the same stream and topic', async () => {
|
||||
@@ -1918,9 +1975,20 @@ describe('Bot test', () => {
|
||||
it.each([
|
||||
{ name: 'FUTO staff', streamId: Constants.Zulip.Streams.FUTOStaff },
|
||||
{ name: 'an unknown stream', streamId: 999 },
|
||||
])('should do nothing in $name', async ({ streamId }) => {
|
||||
])('should expand nothing from GitHub in $name, and still mirror x.com links there', async ({ streamId }) => {
|
||||
await sut.onZulipMessage(zulipMessage({ streamId, content: '#4242 https://x.com/immich/status/1' }));
|
||||
|
||||
expect(githubMock.getIssueOrPrMessage).not.toHaveBeenCalled();
|
||||
expect(zulipMock.sendMessage).toHaveBeenCalledExactlyOnceWith({
|
||||
stream: streamId,
|
||||
topic: 'thumbnails',
|
||||
content: 'https://nitter.net/immich/status/1',
|
||||
});
|
||||
});
|
||||
|
||||
it('should send nothing in a stream without GitHub expansion when there is no x.com link', async () => {
|
||||
await sut.onZulipMessage(zulipMessage({ streamId: 999, content: 'see #4242' }));
|
||||
|
||||
expect(githubMock.getIssueOrPrMessage).not.toHaveBeenCalled();
|
||||
expect(zulipMock.sendMessage).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -18,6 +18,7 @@ import { IOutlineInterface } from 'src/interfaces/outline.interface';
|
||||
import { IZulipInterface, ZulipEmojiCodes, ZulipReceivedMessage } from 'src/interfaces/zulip.interface';
|
||||
import { ZulipApiError } from 'src/repositories/zulip.client';
|
||||
import { NotificationService } from 'src/services/notification.service';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { ZulipService } from 'src/services/zulip.service';
|
||||
import { formatCommand, logError, makeIssueOrPRMessage, makeLink } from 'src/util';
|
||||
|
||||
@@ -196,6 +197,7 @@ export class ChatService {
|
||||
@Inject(IZulipInterface) private zulip: IZulipInterface,
|
||||
private zulipService: ZulipService,
|
||||
private notifications: NotificationService,
|
||||
private zulipExpanders: ZulipExpanderService,
|
||||
) {}
|
||||
|
||||
async init() {
|
||||
@@ -239,16 +241,14 @@ ${messageParts.join('\n')}`,
|
||||
}
|
||||
|
||||
const parts: string[] = [];
|
||||
if (Constants.Zulip.Expanders.GithubReferences.includes(streamId)) {
|
||||
if (this.zulipExpanders.isEnabled(streamId)) {
|
||||
// Snippets are not neutralised: Zulip renders no mention inside a code fence,
|
||||
// and a zero-width space would corrupt the code.
|
||||
const snippets = await this.handleGithubFileReferences(content, true);
|
||||
const links = await this.handleGithubThreadReferences({ content }, true);
|
||||
parts.push(...snippets, ...links.filter((link) => link !== undefined).map(neutraliseZulipMentions));
|
||||
}
|
||||
if (Constants.Zulip.Expanders.TwitterMirror.includes(streamId)) {
|
||||
parts.push(...(await this.handleTwitterReferences(content)).map(neutraliseZulipMentions));
|
||||
}
|
||||
parts.push(...(await this.handleTwitterReferences(content)).map(neutraliseZulipMentions));
|
||||
|
||||
if (parts.length !== 0) {
|
||||
await this.zulip.sendMessage({ stream: streamId, topic, content: parts.join('\n') });
|
||||
|
||||
@@ -10,6 +10,7 @@ import { ScheduleService } from 'src/services/schedule.service';
|
||||
import { ScheduledMessageService } from 'src/services/scheduled-message.service';
|
||||
import { WebhookService } from 'src/services/webhook.service';
|
||||
import { ZulipCommandService } from 'src/services/zulip-command.service';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { ZulipService } from 'src/services/zulip.service';
|
||||
|
||||
export const services = [
|
||||
@@ -26,5 +27,6 @@ export const services = [
|
||||
MirrorService,
|
||||
MirrorLinkService,
|
||||
ZulipCommandService,
|
||||
ZulipExpanderService,
|
||||
ZulipService,
|
||||
];
|
||||
|
||||
@@ -102,6 +102,9 @@ const newDatabaseMockRepository = (): Mocked<IDatabaseRepository> => ({
|
||||
getMirrorIdentities: vitest.fn(),
|
||||
setMirrorIdentity: vitest.fn(),
|
||||
removeMirrorIdentity: vitest.fn(),
|
||||
getZulipExpanders: vitest.fn(),
|
||||
addZulipExpander: vitest.fn(),
|
||||
removeZulipExpander: vitest.fn(),
|
||||
});
|
||||
|
||||
const newDiscordMockRepository = (): Mocked<IDiscordInterface> => ({
|
||||
|
||||
@@ -7,7 +7,7 @@ import { PullRequestBaseEvent } from 'src/interfaces/github.interface';
|
||||
import { IMattermostInterface } from 'src/interfaces/mattermost.interface';
|
||||
import { IRSSInterface } from 'src/interfaces/rss.interface';
|
||||
import { IZulipInterface, ZulipReceivedMessage, ZulipUser } from 'src/interfaces/zulip.interface';
|
||||
import { NewRSSFeed, NewScheduledMessage, RSSFeed, ScheduledMessage, UpdateRSSFeed } from 'src/schema';
|
||||
import { NewRSSFeed, NewScheduledMessage, RSSFeed, ScheduledMessage, UpdateRSSFeed, ZulipExpander } from 'src/schema';
|
||||
import { ChatService, EmoteSyncReport } from 'src/services/chat.service';
|
||||
import { GithubService } from 'src/services/github.service';
|
||||
import {
|
||||
@@ -24,6 +24,7 @@ import { RSSService } from 'src/services/rss.service';
|
||||
import { ScheduledMessageService } from 'src/services/scheduled-message.service';
|
||||
import { BackfillPlatforms, BackfillReport, WebhookService } from 'src/services/webhook.service';
|
||||
import { ZulipCommandService } from 'src/services/zulip-command.service';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { ZulipMessageHandler, ZulipService } from 'src/services/zulip.service';
|
||||
import { parseCommand, splitArguments, tokenize } from 'src/zulip-command-parser';
|
||||
import { Mocked, afterEach, beforeEach, describe, expect, it, vitest } from 'vitest';
|
||||
@@ -114,11 +115,28 @@ const definedOnly = <T extends object>(values: T) =>
|
||||
const newFakeDatabase = () => {
|
||||
const scheduled: ScheduledMessage[] = [];
|
||||
const feeds: RSSFeed[] = [];
|
||||
const expanders: ZulipExpander[] = [];
|
||||
const findFeed = (url: string, channelId: string, service: RSSFeed['service']) =>
|
||||
feeds.findIndex((feed) => feed.url === url && feed.channelId === channelId && feed.service === service);
|
||||
return {
|
||||
scheduled,
|
||||
feeds,
|
||||
expanders,
|
||||
getZulipExpanders: () => Promise.resolve(expanders.map((row) => ({ ...row }))),
|
||||
addZulipExpander: (streamId: number, createdBy: string) => {
|
||||
if (expanders.some((row) => row.streamId === streamId)) {
|
||||
return Promise.resolve(false);
|
||||
}
|
||||
expanders.push({ streamId, createdBy, createdAt: new Date() });
|
||||
return Promise.resolve(true);
|
||||
},
|
||||
removeZulipExpander: (streamId: number) => {
|
||||
const index = expanders.findIndex((row) => row.streamId === streamId);
|
||||
if (index !== -1) {
|
||||
expanders.splice(index, 1);
|
||||
}
|
||||
return Promise.resolve(index !== -1);
|
||||
},
|
||||
getScheduledMessages: (service?: ScheduledMessage['service']) =>
|
||||
Promise.resolve(scheduled.filter((row) => service === undefined || row.service === service)),
|
||||
getScheduledMessage: (name: string, service: ScheduledMessage['service']) =>
|
||||
@@ -221,15 +239,16 @@ const HELP = [
|
||||
'- `rss-subscribe <url> [topic=<topic>]`: post the newest post of that RSS feed now, and every new one after it (checked every 15 minutes), in this stream, in the topic given or this one',
|
||||
'- `rss-unsubscribe <url>`: stop posting that RSS feed in this stream',
|
||||
'- `rss-list`: list the RSS feeds this stream is subscribed to, with their topics',
|
||||
"- `mirror-link [topic=<main topic>]`: start mirroring this stream with a Discord text channel or forum, both ways: this answers with the `/mirror-link` command a Discord administrator then runs in that channel; the main topic (text channels only, general chat by default) holds the channel's own messages",
|
||||
'- `mirror-unlink`: stop mirroring this stream with its Discord channel, and announce it on both sides',
|
||||
'- `mirror-backfill`: copy the messages of the Discord channel or thread this topic mirrors that are not here yet into this topic, oldest first, between two notices; new Discord messages there wait until it is done',
|
||||
'- `mirror-list`: list the mirrored channels and streams, and the linked accounts',
|
||||
"- `mirror-link [topic=<main topic>]` (administrators): start mirroring this stream with a Discord text channel or forum, both ways: this answers with the `/mirror-link` command a Discord administrator then runs in that channel; the main topic (text channels only, general chat by default) holds the channel's own messages",
|
||||
'- `mirror-unlink` (administrators): stop mirroring this stream with its Discord channel, and announce it on both sides',
|
||||
'- `mirror-backfill` (administrators): copy the messages of the Discord channel or thread this topic mirrors that are not here yet into this topic, oldest first, between two notices; new Discord messages there wait until it is done',
|
||||
'- `mirror-list` (administrators): list the mirrored channels and streams, and the linked accounts',
|
||||
'- `expanders <on|off|list>` (administrators): turn GitHub expansion (issue, pull request and discussion links and `#1234` to their titles, file permalinks to code) on or off in this stream, or `list` the streams it is on in; x.com links are mirrored on nitter.net in every stream',
|
||||
'- `discord-unlink`: unlink your Zulip account from your Discord account, so that your messages appear on Discord as "Name (Zulip)"',
|
||||
'- `similar [text]`: list the immich-app/immich issues and discussions like the text, or without text like the last message a human wrote in this topic, looked for among its ten newest',
|
||||
'',
|
||||
'Arguments are positional or `key=value`; quote a value with spaces (`text="two words"`). Every reply is posted here, in the topic.',
|
||||
'The `mirror-*` commands are taken in any stream, from organization administrators and owners only. To link your Zulip account with your Discord account, run `/zulip-link` on Discord and send me the code it gives you in a direct message.',
|
||||
'The commands marked (administrators) are taken in any stream, from organization administrators and owners only. To link your Zulip account with your Discord account, run `/zulip-link` on Discord and send me the code it gives you in a direct message.',
|
||||
].join('\n');
|
||||
|
||||
describe('tokenize', () => {
|
||||
@@ -365,6 +384,7 @@ describe('ZulipCommandService', () => {
|
||||
let discordMock: Mocked<Pick<IDiscordInterface, 'sendMessage'>>;
|
||||
let rssMock: Mocked<IRSSInterface>;
|
||||
let mirrorLinksMock: ReturnType<typeof newMirrorLinkServiceMock>;
|
||||
let zulipExpanders: ZulipExpanderService;
|
||||
|
||||
const replies = () => zulipMock.sendMessage.mock.calls.map(([payload]) => payload);
|
||||
const send = (content: string, overrides: Partial<ZulipReceivedMessage> = {}) =>
|
||||
@@ -385,6 +405,7 @@ describe('ZulipCommandService', () => {
|
||||
const discord = discordMock as unknown as IDiscordInterface;
|
||||
const mattermost = {} as IMattermostInterface;
|
||||
const db = database as unknown as IDatabaseRepository;
|
||||
zulipExpanders = new ZulipExpanderService(db);
|
||||
sut = new ZulipCommandService(
|
||||
zulipMock,
|
||||
zulipServiceMock as unknown as ZulipService,
|
||||
@@ -394,6 +415,7 @@ describe('ZulipCommandService', () => {
|
||||
new ScheduledMessageService(db, discord, mattermost, zulipMock),
|
||||
new RSSService(db, new NotificationService(discord, mattermost, zulipMock), rssMock),
|
||||
mirrorLinksMock as unknown as MirrorLinkService,
|
||||
zulipExpanders,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1786,4 +1808,172 @@ describe('ZulipCommandService', () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('expanders', () => {
|
||||
const ADMIN = { userId: 12, fullName: 'Alice', role: 200 };
|
||||
const NOT_AN_ADMINISTRATOR =
|
||||
'Only Zulip organization administrators and owners can change or list GitHub expansion.';
|
||||
const NOT_SUBSCRIBED =
|
||||
'⚠ I am not subscribed to this stream, so none of its messages reach me and nothing is expanded here until an administrator subscribes me.';
|
||||
const HEADER =
|
||||
'GitHub expansion (issue, pull request and discussion links and `#1234` to their titles, file permalinks to code) is on in:';
|
||||
const USAGE = 'Usage: `expanders <on|off|list>`';
|
||||
const row = (streamId: number): ZulipExpander => ({ streamId, createdBy: 'migration', createdAt: new Date(0) });
|
||||
const contents = () => replies().map(({ content }) => content);
|
||||
const stored = () => database.expanders.map(({ streamId }) => streamId).sort((a, b) => a - b);
|
||||
|
||||
beforeEach(async () => {
|
||||
zulipMock.getUser.mockResolvedValue(ADMIN);
|
||||
zulipMock.getSubscriptions.mockResolvedValue([{ streamId: 107 }, { streamId: 120 }]);
|
||||
database.expanders.push(row(107));
|
||||
await zulipExpanders.init();
|
||||
});
|
||||
|
||||
it('should turn GitHub expansion on in this stream, in any stream, and store who did it', async () => {
|
||||
await send('@**Immich** expanders on', { streamId: 120, topic: 'setup' });
|
||||
|
||||
expect(replies()).toEqual([
|
||||
{ stream: 120, topic: 'setup', content: 'Turned on GitHub expansion in this stream.' },
|
||||
]);
|
||||
expect(database.expanders).toEqual([
|
||||
row(107),
|
||||
{ streamId: 120, createdBy: 'Alice on Zulip (user 12)', createdAt: expect.any(Date) },
|
||||
]);
|
||||
expect(zulipExpanders.isEnabled(120)).toBe(true);
|
||||
});
|
||||
|
||||
it('should say when it was already on or off, whatever the case of the command', async () => {
|
||||
await send('@**Immich** EXPANDERS ON');
|
||||
await send('@**Immich** expanders Off');
|
||||
await send('@**Immich** expanders off');
|
||||
|
||||
expect(contents()).toEqual([
|
||||
'Nothing changed: GitHub expansion was already on in this stream.',
|
||||
'Turned off GitHub expansion in this stream.',
|
||||
'Nothing changed: GitHub expansion was already off in this stream.',
|
||||
]);
|
||||
expect(stored()).toEqual([]);
|
||||
expect(zulipExpanders.isEnabled(107)).toBe(false);
|
||||
});
|
||||
|
||||
it('should change this stream only', async () => {
|
||||
await send('@**Immich** expanders on', { streamId: 120 });
|
||||
await send('@**Immich** expanders off', { streamId: 107 });
|
||||
|
||||
expect(zulipExpanders.list()).toEqual([120]);
|
||||
expect(stored()).toEqual([120]);
|
||||
});
|
||||
|
||||
it('should warn when it turns expansion on in a stream the bot is not subscribed to, and not when it turns it off', async () => {
|
||||
await send('@**Immich** expanders on', { streamId: 130 });
|
||||
await send('@**Immich** expanders on', { streamId: 130 });
|
||||
await send('@**Immich** expanders off', { streamId: 130 });
|
||||
|
||||
expect(contents()).toEqual([
|
||||
`Turned on GitHub expansion in this stream.\n${NOT_SUBSCRIBED}`,
|
||||
`Nothing changed: GitHub expansion was already on in this stream.\n${NOT_SUBSCRIBED}`,
|
||||
'Turned off GitHub expansion in this stream.',
|
||||
]);
|
||||
expect(zulipMock.getSubscriptions).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('should change nothing when the subscriptions cannot be read, and say so', async () => {
|
||||
zulipMock.getSubscriptions.mockRejectedValue(new Error('Zulip is down'));
|
||||
|
||||
await send('@**Immich** expanders on', { streamId: 120 });
|
||||
|
||||
expect(contents()).toEqual(['`expanders` failed: `Zulip is down`']);
|
||||
expect(stored()).toEqual([107]);
|
||||
expect(zulipExpanders.isEnabled(120)).toBe(false);
|
||||
});
|
||||
|
||||
it('should keep the cache as it was when the table refuses the change', async () => {
|
||||
vitest.spyOn(database, 'addZulipExpander').mockRejectedValue(new Error('connection terminated'));
|
||||
|
||||
await send('@**Immich** expanders on', { streamId: 120 });
|
||||
|
||||
expect(contents()).toEqual(['`expanders` failed: `connection terminated`']);
|
||||
expect(zulipExpanders.list()).toEqual([107]);
|
||||
});
|
||||
|
||||
it('should list every stream by name and ID, and mark one the bot is not subscribed to', async () => {
|
||||
database.expanders.push(row(54), row(130), row(140));
|
||||
await zulipExpanders.init();
|
||||
const names: Record<number, string> = { 54: 'Immich', 107: 'immich-general', 130: '@**all** news' };
|
||||
zulipMock.getStream.mockImplementation((streamId) =>
|
||||
streamId === 140
|
||||
? Promise.reject(new Error('Invalid channel ID'))
|
||||
: Promise.resolve({ streamId, name: names[streamId], inviteOnly: false }),
|
||||
);
|
||||
zulipMock.getSubscriptions.mockResolvedValue([{ streamId: 54 }, { streamId: 107 }]);
|
||||
|
||||
await send('@**Immich** expanders list', { streamId: 120 });
|
||||
|
||||
expect(contents()).toEqual([
|
||||
[
|
||||
HEADER,
|
||||
'- **#Immich** (54)',
|
||||
'- **#immich-general** (107)',
|
||||
'- **#@**all** news** (130) (⚠ I am not subscribed, so nothing reaches me there)',
|
||||
'- stream 140 (⚠ I am not subscribed, so nothing reaches me there)',
|
||||
].join('\n'),
|
||||
]);
|
||||
});
|
||||
|
||||
it('should list without the subscription marks when the subscriptions cannot be read', async () => {
|
||||
zulipMock.getStream.mockResolvedValue({ streamId: 107, name: 'immich-general', inviteOnly: true });
|
||||
zulipMock.getSubscriptions.mockRejectedValue(new Error('Zulip is down'));
|
||||
|
||||
await send('@**Immich** expanders list');
|
||||
|
||||
expect(contents()).toEqual([`${HEADER}\n- **#immich-general** (107)`]);
|
||||
});
|
||||
|
||||
it('should say when GitHub expansion is on in no stream', async () => {
|
||||
await send('@**Immich** expanders off');
|
||||
await send('@**Immich** expanders list');
|
||||
|
||||
expect(contents()[1]).toBe('GitHub expansion is on in no stream.');
|
||||
expect(zulipMock.getStream).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([
|
||||
['an owner', 100, true],
|
||||
['an administrator', 200, true],
|
||||
['a moderator', 300, false],
|
||||
['a member', 400, false],
|
||||
['a guest', 600, false],
|
||||
])('should take the expanders command from %s: %s', async (_, role, allowed) => {
|
||||
zulipMock.getUser.mockResolvedValue({ ...ADMIN, role });
|
||||
|
||||
await send('@**Immich** expanders on', { streamId: 120 });
|
||||
await send('@**Immich** expanders off', { streamId: 107 });
|
||||
await send('@**Immich** expanders list', { streamId: 54 });
|
||||
|
||||
expect(zulipMock.getUser).toHaveBeenCalledTimes(3);
|
||||
expect(zulipMock.getUser).toHaveBeenCalledWith(12);
|
||||
if (allowed) {
|
||||
expect(stored()).toEqual([120]);
|
||||
expect(contents()).toHaveLength(3);
|
||||
} else {
|
||||
expect(stored()).toEqual([107]);
|
||||
expect(contents()).toEqual(Array(3).fill(NOT_AN_ADMINISTRATOR));
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
'@**Immich** expanders',
|
||||
'@**Immich** expanders toggle',
|
||||
'@**Immich** expanders on github',
|
||||
'@**Immich** expanders off twitter',
|
||||
'@**Immich** expanders list all',
|
||||
'@**Immich** expanders on expander=github',
|
||||
])('should answer %j with the usage and change nothing', async (content) => {
|
||||
await send(content, { streamId: 120 });
|
||||
|
||||
expect(contents()).toEqual([USAGE]);
|
||||
expect(stored()).toEqual([107]);
|
||||
expect(zulipMock.getSubscriptions).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -15,6 +15,7 @@ import { MirrorActor, MirrorLinkReply, MirrorLinkService } from 'src/services/mi
|
||||
import { RSSService } from 'src/services/rss.service';
|
||||
import { ScheduledMessageService } from 'src/services/scheduled-message.service';
|
||||
import { BackfillPlatforms, WebhookService, formatBackfillReport } from 'src/services/webhook.service';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { ZulipService, describeZulipStream, isBotSender } from 'src/services/zulip.service';
|
||||
import { Arguments, ParseResult, parseCommand, splitArguments, tokenize } from 'src/zulip-command-parser';
|
||||
|
||||
@@ -31,8 +32,13 @@ const SUPPRESS_EMBEDS_IGNORED =
|
||||
/** Zulip's roles are ordered: 100 is an owner, 200 an administrator. */
|
||||
const ZULIP_ADMINISTRATOR_ROLE = 200;
|
||||
|
||||
const NOT_AN_ADMINISTRATOR =
|
||||
'Only Zulip organization administrators and owners can change or list the Discord-Zulip mirror.';
|
||||
const MIRROR = 'change or list the Discord-Zulip mirror';
|
||||
|
||||
const GITHUB_EXPANSION =
|
||||
'issue, pull request and discussion links and `#1234` to their titles, file permalinks to code';
|
||||
|
||||
const NOT_SUBSCRIBED =
|
||||
'⚠ I am not subscribed to this stream, so none of its messages reach me and nothing is expanded here until an administrator subscribes me.';
|
||||
|
||||
const EMOTE_SYNC_SERVER = `the ${Constants.Discord.EmoteSyncServer.name} Discord server (${Constants.Discord.EmoteSyncServer.id})`;
|
||||
|
||||
@@ -74,8 +80,8 @@ type Command = {
|
||||
description: string;
|
||||
positionals: number;
|
||||
options: string[];
|
||||
/** Taken in any stream, from organization administrators and owners only. */
|
||||
administrators?: true;
|
||||
/** Taken in any stream, from organization administrators and owners only; what they alone can do, for the refusal. */
|
||||
administrators?: string;
|
||||
run: (context: CommandContext) => Promise<string | undefined>;
|
||||
};
|
||||
|
||||
@@ -180,7 +186,7 @@ export class ZulipCommandService {
|
||||
"start mirroring this stream with a Discord text channel or forum, both ways: this answers with the `/mirror-link` command a Discord administrator then runs in that channel; the main topic (text channels only, general chat by default) holds the channel's own messages",
|
||||
positionals: 0,
|
||||
options: ['topic'],
|
||||
administrators: true,
|
||||
administrators: MIRROR,
|
||||
run: (context) => this.mirrorLink(context),
|
||||
},
|
||||
'mirror-unlink': {
|
||||
@@ -188,7 +194,7 @@ export class ZulipCommandService {
|
||||
description: 'stop mirroring this stream with its Discord channel, and announce it on both sides',
|
||||
positionals: 0,
|
||||
options: [],
|
||||
administrators: true,
|
||||
administrators: MIRROR,
|
||||
run: ({ message }) => this.mirrorUnlink(message),
|
||||
},
|
||||
'mirror-backfill': {
|
||||
@@ -197,7 +203,7 @@ export class ZulipCommandService {
|
||||
'copy the messages of the Discord channel or thread this topic mirrors that are not here yet into this topic, oldest first, between two notices; new Discord messages there wait until it is done',
|
||||
positionals: 0,
|
||||
options: [],
|
||||
administrators: true,
|
||||
administrators: MIRROR,
|
||||
run: ({ message }) => this.mirrorBackfill(message),
|
||||
},
|
||||
'mirror-list': {
|
||||
@@ -205,9 +211,17 @@ export class ZulipCommandService {
|
||||
description: 'list the mirrored channels and streams, and the linked accounts',
|
||||
positionals: 0,
|
||||
options: [],
|
||||
administrators: true,
|
||||
administrators: MIRROR,
|
||||
run: () => this.mirrorLinks.list('zulip'),
|
||||
},
|
||||
expanders: {
|
||||
usage: 'expanders <on|off|list>',
|
||||
description: `turn GitHub expansion (${GITHUB_EXPANSION}) on or off in this stream, or \`list\` the streams it is on in; x.com links are mirrored on nitter.net in every stream`,
|
||||
positionals: 1,
|
||||
options: [],
|
||||
administrators: 'change or list GitHub expansion',
|
||||
run: (context) => this.expanders(context),
|
||||
},
|
||||
'discord-unlink': {
|
||||
usage: 'discord-unlink',
|
||||
description:
|
||||
@@ -235,6 +249,7 @@ export class ZulipCommandService {
|
||||
private scheduledMessageService: ScheduledMessageService,
|
||||
private rssService: RSSService,
|
||||
private mirrorLinks: MirrorLinkService,
|
||||
private zulipExpanders: ZulipExpanderService,
|
||||
) {}
|
||||
|
||||
async init() {
|
||||
@@ -300,7 +315,7 @@ export class ZulipCommandService {
|
||||
}
|
||||
try {
|
||||
if (command.administrators && !(await this.isAdministrator(message.senderId))) {
|
||||
return NOT_AN_ADMINISTRATOR;
|
||||
return `Only Zulip organization administrators and owners can ${command.administrators}.`;
|
||||
}
|
||||
return await command.run({ message, args, options });
|
||||
} catch (error) {
|
||||
@@ -398,14 +413,17 @@ export class ZulipCommandService {
|
||||
}
|
||||
|
||||
private help() {
|
||||
const lines = Object.values(this.commands).map(({ usage, description }) => `- ${code(usage)}: ${description}`);
|
||||
const lines = Object.values(this.commands).map(
|
||||
({ usage, description, administrators }) =>
|
||||
`- ${code(usage)}${administrators ? ' (administrators)' : ''}: ${description}`,
|
||||
);
|
||||
return [
|
||||
'Mention me at the start of a message in a team stream, then one of:',
|
||||
...lines,
|
||||
// The blank line ends the list: without it, Markdown reads the next line as the last item's continuation.
|
||||
'',
|
||||
`Arguments are positional or ${code('key=value')}; quote a value with spaces (${code('text="two words"')}). Every reply is posted here, in the topic.`,
|
||||
`The ${code('mirror-*')} commands are taken in any stream, from organization administrators and owners only. To link your Zulip account with your Discord account, run ${code('/zulip-link')} on Discord and send me the code it gives you in a direct message.`,
|
||||
`The commands marked (administrators) are taken in any stream, from organization administrators and owners only. To link your Zulip account with your Discord account, run ${code('/zulip-link')} on Discord and send me the code it gives you in a direct message.`,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
@@ -618,6 +636,56 @@ export class ZulipCommandService {
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
private async expanders({ message, args }: CommandContext) {
|
||||
const action = args[0]?.toLowerCase();
|
||||
const { streamId } = message;
|
||||
if (action === 'list') {
|
||||
return this.expanderList();
|
||||
}
|
||||
if (action === 'off') {
|
||||
const removed = await this.zulipExpanders.disable(streamId);
|
||||
return removed
|
||||
? 'Turned off GitHub expansion in this stream.'
|
||||
: 'Nothing changed: GitHub expansion was already off in this stream.';
|
||||
}
|
||||
if (action !== 'on') {
|
||||
return this.usage('expanders');
|
||||
}
|
||||
const subscriptions = await this.zulip.getSubscriptions();
|
||||
const added = await this.zulipExpanders.enable(
|
||||
streamId,
|
||||
`${message.senderFullName} on Zulip (user ${message.senderId})`,
|
||||
);
|
||||
const reply = added
|
||||
? 'Turned on GitHub expansion in this stream.'
|
||||
: 'Nothing changed: GitHub expansion was already on in this stream.';
|
||||
return subscriptions.some((subscription) => subscription.streamId === streamId)
|
||||
? reply
|
||||
: `${reply}\n${NOT_SUBSCRIBED}`;
|
||||
}
|
||||
|
||||
private async expanderList() {
|
||||
const streams = this.zulipExpanders.list();
|
||||
if (streams.length === 0) {
|
||||
return 'GitHub expansion is on in no stream.';
|
||||
}
|
||||
const [subscriptions, names] = await Promise.all([
|
||||
this.zulip.getSubscriptions().catch(() => undefined),
|
||||
Promise.all(streams.map((streamId) => this.zulip.getStream(streamId).catch(() => undefined))),
|
||||
]);
|
||||
const subscribed = subscriptions && new Set(subscriptions.map(({ streamId }) => streamId));
|
||||
return [
|
||||
`GitHub expansion (${GITHUB_EXPANSION}) is on in:`,
|
||||
...streams.map((streamId, index) => {
|
||||
const stream = names[index];
|
||||
const name = stream ? `**#${neutraliseZulipMentions(stream.name)}** (${streamId})` : `stream ${streamId}`;
|
||||
const warning =
|
||||
subscribed && !subscribed.has(streamId) ? ' (⚠ I am not subscribed, so nothing reaches me there)' : '';
|
||||
return `- ${name}${warning}`;
|
||||
}),
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
private async similar({ message, args, options }: CommandContext) {
|
||||
if (options.text !== undefined && args.length > 0) {
|
||||
return this.usage('similar');
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
import { Logger } from '@nestjs/common';
|
||||
import { IDatabaseRepository } from 'src/interfaces/database.interface';
|
||||
import { ZulipExpander } from 'src/schema';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { beforeEach, describe, expect, it, Mocked, vitest } from 'vitest';
|
||||
|
||||
const row = (streamId: number): ZulipExpander => ({ streamId, createdBy: 'migration', createdAt: new Date(0) });
|
||||
|
||||
describe(ZulipExpanderService.name, () => {
|
||||
let database: Mocked<Pick<IDatabaseRepository, 'getZulipExpanders' | 'addZulipExpander' | 'removeZulipExpander'>>;
|
||||
let sut: ZulipExpanderService;
|
||||
|
||||
let table: ZulipExpander[];
|
||||
|
||||
beforeEach(async () => {
|
||||
table = [row(107), row(54)];
|
||||
database = {
|
||||
getZulipExpanders: vitest.fn(async () => [...table]),
|
||||
addZulipExpander: vitest.fn(async (streamId) => {
|
||||
if (table.some((existing) => existing.streamId === streamId)) {
|
||||
return false;
|
||||
}
|
||||
table.push(row(streamId));
|
||||
return true;
|
||||
}),
|
||||
removeZulipExpander: vitest.fn(async (streamId) => {
|
||||
const before = table.length;
|
||||
table = table.filter((existing) => existing.streamId !== streamId);
|
||||
return table.length !== before;
|
||||
}),
|
||||
};
|
||||
sut = new ZulipExpanderService(database as unknown as IDatabaseRepository);
|
||||
await sut.init();
|
||||
});
|
||||
|
||||
it('should know no stream before init', () => {
|
||||
const fresh = new ZulipExpanderService(database as unknown as IDatabaseRepository);
|
||||
|
||||
expect(fresh.list()).toEqual([]);
|
||||
expect(fresh.isEnabled(107)).toBe(false);
|
||||
});
|
||||
|
||||
it('should load the table once, at init, and answer from the cache', () => {
|
||||
expect(sut.isEnabled(107)).toBe(true);
|
||||
expect(sut.isEnabled(54)).toBe(true);
|
||||
expect(sut.isEnabled(999)).toBe(false);
|
||||
expect(database.getZulipExpanders).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('should list the streams by ID', () => {
|
||||
expect(sut.list()).toEqual([54, 107]);
|
||||
});
|
||||
|
||||
it('should enable, resolve to whether the table took it and cache it at once', async () => {
|
||||
expect(await sut.enable(120, 'Alice on Zulip (user 12)')).toBe(true);
|
||||
expect(await sut.enable(120, 'Alice on Zulip (user 12)')).toBe(false);
|
||||
|
||||
expect(database.addZulipExpander).toHaveBeenCalledWith(120, 'Alice on Zulip (user 12)');
|
||||
expect(sut.isEnabled(120)).toBe(true);
|
||||
expect(sut.list()).toEqual([54, 107, 120]);
|
||||
});
|
||||
|
||||
it('should disable, resolve to whether the table had it and drop it at once', async () => {
|
||||
expect(await sut.disable(107)).toBe(true);
|
||||
expect(await sut.disable(107)).toBe(false);
|
||||
|
||||
expect(database.removeZulipExpander).toHaveBeenLastCalledWith(107);
|
||||
expect(sut.isEnabled(107)).toBe(false);
|
||||
expect(sut.list()).toEqual([54]);
|
||||
});
|
||||
|
||||
it('should leave the cache as it was when the table refuses a change', async () => {
|
||||
database.addZulipExpander.mockRejectedValue(new Error('connection terminated'));
|
||||
|
||||
await expect(sut.enable(999, 'Alice')).rejects.toThrow('connection terminated');
|
||||
|
||||
expect(sut.isEnabled(999)).toBe(false);
|
||||
});
|
||||
|
||||
it('should run a change after one still writing, so the cache ends as the table does', async () => {
|
||||
let finishAdd = () => {};
|
||||
const add = database.addZulipExpander.getMockImplementation()!;
|
||||
database.addZulipExpander.mockImplementationOnce(
|
||||
(...args) => new Promise((resolve) => (finishAdd = () => resolve(add(...args)))),
|
||||
);
|
||||
|
||||
const enabling = sut.enable(120, 'Alice');
|
||||
const disabling = sut.disable(120);
|
||||
await Promise.resolve();
|
||||
expect(database.removeZulipExpander).not.toHaveBeenCalled();
|
||||
|
||||
finishAdd();
|
||||
expect(await enabling).toBe(true);
|
||||
expect(await disabling).toBe(true);
|
||||
|
||||
expect(table.some(({ streamId }) => streamId === 120)).toBe(false);
|
||||
expect(sut.isEnabled(120)).toBe(false);
|
||||
});
|
||||
|
||||
it('should cache a change as the table reported it when reading the table back fails', async () => {
|
||||
vitest.spyOn(Logger.prototype, 'warn').mockImplementation(() => {});
|
||||
database.getZulipExpanders.mockRejectedValueOnce(new Error('connection terminated'));
|
||||
expect(await sut.enable(120, 'Alice')).toBe(true);
|
||||
expect(sut.isEnabled(120)).toBe(true);
|
||||
|
||||
database.getZulipExpanders.mockRejectedValueOnce(new Error('connection terminated'));
|
||||
expect(await sut.disable(107)).toBe(true);
|
||||
expect(sut.isEnabled(107)).toBe(false);
|
||||
expect(sut.list()).toEqual([54, 120]);
|
||||
expect(Logger.prototype.warn).toHaveBeenCalledTimes(2);
|
||||
expect(Logger.prototype.warn).toHaveBeenCalledWith(
|
||||
'Could not read the Zulip expanders of stream 107 back, so the change is cached as the table reported it',
|
||||
expect.any(Error),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,85 @@
|
||||
import { Inject, Injectable, Logger } from '@nestjs/common';
|
||||
import { IDatabaseRepository } from 'src/interfaces/database.interface';
|
||||
|
||||
/** The streams of the `zulip_expander` table, cached so that no message costs a query; only this service writes the table. */
|
||||
@Injectable()
|
||||
export class ZulipExpanderService {
|
||||
private logger = new Logger(ZulipExpanderService.name);
|
||||
private streams = new Set<number>();
|
||||
private writes = new Map<number, Promise<unknown>>();
|
||||
|
||||
constructor(@Inject(IDatabaseRepository) private database: IDatabaseRepository) {}
|
||||
|
||||
async init() {
|
||||
const rows = await this.database.getZulipExpanders();
|
||||
this.streams = new Set(rows.map(({ streamId }) => streamId));
|
||||
}
|
||||
|
||||
isEnabled(streamId: number) {
|
||||
return this.streams.has(streamId);
|
||||
}
|
||||
|
||||
list() {
|
||||
return [...this.streams].sort((a, b) => a - b);
|
||||
}
|
||||
|
||||
/** Resolves to whether it was off and is now on. */
|
||||
enable(streamId: number, createdBy: string) {
|
||||
return this.write(streamId, () => this.database.addZulipExpander(streamId, createdBy), true);
|
||||
}
|
||||
|
||||
/** Resolves to whether it was on and is now off. */
|
||||
disable(streamId: number) {
|
||||
return this.write(streamId, () => this.database.removeZulipExpander(streamId), false);
|
||||
}
|
||||
|
||||
/**
|
||||
* A command the event loop stopped waiting for can still be writing when the next one runs, so a stream's writes
|
||||
* run one at a time and its cache is read back from the table after each.
|
||||
*/
|
||||
private write(streamId: number, change: () => Promise<boolean>, enabled: boolean): Promise<boolean> {
|
||||
const run = (this.writes.get(streamId) ?? Promise.resolve()).then(async () => {
|
||||
let changed: boolean;
|
||||
try {
|
||||
changed = await change();
|
||||
} catch (error) {
|
||||
await this.reload(streamId).catch(() => undefined);
|
||||
throw error;
|
||||
}
|
||||
try {
|
||||
await this.reload(streamId);
|
||||
} catch (error) {
|
||||
this.logger.warn(
|
||||
`Could not read the Zulip expanders of stream ${streamId} back, so the change is cached as the table reported it`,
|
||||
error,
|
||||
);
|
||||
this.cache(streamId, enabled);
|
||||
}
|
||||
return changed;
|
||||
});
|
||||
const settled = run.catch(() => undefined);
|
||||
this.writes.set(streamId, settled);
|
||||
void settled.then(() => {
|
||||
if (this.writes.get(streamId) === settled) {
|
||||
this.writes.delete(streamId);
|
||||
}
|
||||
});
|
||||
return run;
|
||||
}
|
||||
|
||||
private async reload(streamId: number) {
|
||||
const rows = await this.database.getZulipExpanders();
|
||||
this.cache(
|
||||
streamId,
|
||||
rows.some((row) => row.streamId === streamId),
|
||||
);
|
||||
}
|
||||
|
||||
private cache(streamId: number, enabled: boolean) {
|
||||
if (enabled) {
|
||||
this.streams.add(streamId);
|
||||
} else {
|
||||
this.streams.delete(streamId);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Logger } from '@nestjs/common';
|
||||
import { DateTime, Settings } from 'luxon';
|
||||
import { Constants } from 'src/constants';
|
||||
import { IDatabaseRepository } from 'src/interfaces/database.interface';
|
||||
import { HolidayDto, IHolidaysInterface } from 'src/interfaces/holidays.interface';
|
||||
import {
|
||||
IZulipInterface,
|
||||
@@ -12,6 +13,8 @@ import {
|
||||
ZulipReceivedMessage,
|
||||
} from 'src/interfaces/zulip.interface';
|
||||
import { ZulipApiError } from 'src/repositories/zulip.client';
|
||||
import { ZulipExpander } from 'src/schema';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
import { ZulipService } from 'src/services/zulip.service';
|
||||
import { afterEach, beforeEach, describe, expect, it, Mock, Mocked, vitest } from 'vitest';
|
||||
|
||||
@@ -83,17 +86,27 @@ const setNow = (iso: string) => {
|
||||
Settings.now = () => millis;
|
||||
};
|
||||
|
||||
const expanderRow = (streamId: number): ZulipExpander => ({
|
||||
streamId,
|
||||
createdBy: 'migration',
|
||||
createdAt: new Date(0),
|
||||
});
|
||||
|
||||
const SEEDED_EXPANDERS = [54, 107, 108, 109, 110, 111, 112, 113].map((streamId) => expanderRow(streamId));
|
||||
|
||||
const NOTICE = "Tomorrow is a federal holiday: Independence Day. There won't be any meetings tomorrow.";
|
||||
|
||||
describe('ZulipService', () => {
|
||||
let sut: ZulipService;
|
||||
let holidaysMock: Mocked<IHolidaysInterface>;
|
||||
let zulipMock: Mocked<IZulipInterface>;
|
||||
let expanderDatabase: Mocked<Pick<IDatabaseRepository, 'getZulipExpanders' | 'addZulipExpander'>>;
|
||||
let expanders: ZulipExpanderService;
|
||||
|
||||
const originalNow = Settings.now;
|
||||
const originalZone = Settings.defaultZone;
|
||||
|
||||
beforeEach(() => {
|
||||
beforeEach(async () => {
|
||||
// "Tomorrow" is resolved in the default zone, so pin it for deterministic dates.
|
||||
Settings.defaultZone = 'utc';
|
||||
// The cron fires at 22:00; this is the evening before Independence Day.
|
||||
@@ -101,7 +114,13 @@ describe('ZulipService', () => {
|
||||
|
||||
holidaysMock = newHolidaysMock();
|
||||
zulipMock = newZulipMock();
|
||||
sut = new ZulipService(holidaysMock, zulipMock);
|
||||
expanderDatabase = {
|
||||
getZulipExpanders: vitest.fn().mockResolvedValue(SEEDED_EXPANDERS),
|
||||
addZulipExpander: vitest.fn(),
|
||||
};
|
||||
expanders = new ZulipExpanderService(expanderDatabase as unknown as IDatabaseRepository);
|
||||
await expanders.init();
|
||||
sut = new ZulipService(holidaysMock, zulipMock, expanders);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
@@ -625,21 +644,50 @@ describe('ZulipService', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('should check every stream of every expander, not only the team streams, naming an unnamed one by its ID', async () => {
|
||||
const mirror = Constants.Zulip.Expanders.TwitterMirror;
|
||||
mirror.push(999);
|
||||
try {
|
||||
await sut.init();
|
||||
await flush();
|
||||
} finally {
|
||||
mirror.pop();
|
||||
}
|
||||
it('should check every stream GitHub expansion is on in, not only the team streams, naming an unnamed one by its ID', async () => {
|
||||
expanderDatabase.getZulipExpanders.mockResolvedValue([...SEEDED_EXPANDERS, expanderRow(999)]);
|
||||
await expanders.init();
|
||||
|
||||
expect(Logger.prototype.warn).toHaveBeenCalledOnce();
|
||||
expect(Logger.prototype.warn).toHaveBeenCalledWith(
|
||||
await sut.init();
|
||||
await flush();
|
||||
|
||||
expect(Logger.prototype.warn).toHaveBeenCalledExactlyOnceWith(
|
||||
'The Zulip bot is not subscribed to stream 999: its event queue carries no messages from it, so nothing is expanded there until an admin subscribes it',
|
||||
);
|
||||
});
|
||||
|
||||
it('should check a stream GitHub expansion was turned on in since the last registration at the next one', async () => {
|
||||
await sut.init();
|
||||
await flush();
|
||||
expect(Logger.prototype.warn).not.toHaveBeenCalled();
|
||||
|
||||
expanderDatabase.addZulipExpander.mockResolvedValue(true);
|
||||
expanderDatabase.getZulipExpanders.mockResolvedValue([...SEEDED_EXPANDERS, expanderRow(997)]);
|
||||
await expanders.enable(997, 'Alice on Zulip (user 12)');
|
||||
polls[0].reject(badQueue());
|
||||
await nextPoll();
|
||||
|
||||
expect(zulipMock.registerQueue).toHaveBeenCalledTimes(2);
|
||||
expect(Logger.prototype.warn).toHaveBeenCalledExactlyOnceWith(
|
||||
'The Zulip bot is not subscribed to stream 997: its event queue carries no messages from it, so nothing is expanded there until an admin subscribes it',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not check a stream once GitHub expansion is off there and it takes no commands', async () => {
|
||||
expanderDatabase.getZulipExpanders.mockResolvedValue(
|
||||
SEEDED_EXPANDERS.filter(({ streamId }) => streamId !== Constants.Zulip.Streams.Immich),
|
||||
);
|
||||
await expanders.init();
|
||||
zulipMock.registerQueue.mockResolvedValue({
|
||||
queue: { queueId: 'q1', lastEventId: -1 },
|
||||
subscribedStreamIds: [...Object.values(Constants.Zulip.TeamStreams)],
|
||||
});
|
||||
|
||||
await sut.init();
|
||||
await flush();
|
||||
|
||||
expect(Logger.prototype.warn).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('polling', () => {
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
ZulipUser,
|
||||
} from 'src/interfaces/zulip.interface';
|
||||
import { ZulipApiError } from 'src/repositories/zulip.client';
|
||||
import { ZulipExpanderService } from 'src/services/zulip-expander.service';
|
||||
|
||||
export type ZulipMessageHandler = (message: ZulipReceivedMessage) => Promise<void> | void;
|
||||
export type ZulipUpdateHandler = (update: ZulipMessageUpdated) => Promise<void> | void;
|
||||
@@ -61,9 +62,6 @@ export const describeZulipStream = (streamId: number) => {
|
||||
return name ? `${streamId} (${name})` : `${streamId}`;
|
||||
};
|
||||
|
||||
const listeningStreams = () =>
|
||||
new Set([...Object.values(Constants.Zulip.Expanders).flat(), ...Constants.Zulip.Commands]);
|
||||
|
||||
@Injectable()
|
||||
export class ZulipService implements OnModuleDestroy {
|
||||
private logger = new Logger(ZulipService.name);
|
||||
@@ -81,6 +79,7 @@ export class ZulipService implements OnModuleDestroy {
|
||||
constructor(
|
||||
@Inject(IHolidaysInterface) private holidays: IHolidaysInterface,
|
||||
@Inject(IZulipInterface) private zulip: IZulipInterface,
|
||||
private expanders: ZulipExpanderService,
|
||||
) {}
|
||||
|
||||
async init() {
|
||||
@@ -232,7 +231,7 @@ export class ZulipService implements OnModuleDestroy {
|
||||
this.emptyTopic = emptyTopicName ?? this.emptyTopic;
|
||||
this.logger.log(`Registered Zulip event queue ${queue.queueId}`);
|
||||
const subscribed = new Set(subscribedStreamIds);
|
||||
for (const streamId of listeningStreams()) {
|
||||
for (const streamId of this.listeningStreams()) {
|
||||
if (!subscribed.has(streamId)) {
|
||||
this.logger.warn(
|
||||
`The Zulip bot is not subscribed to stream ${describeZulipStream(streamId)}: its event queue carries no messages from it, so nothing is expanded there until an admin subscribes it`,
|
||||
@@ -249,6 +248,10 @@ export class ZulipService implements OnModuleDestroy {
|
||||
return queue;
|
||||
}
|
||||
|
||||
private listeningStreams() {
|
||||
return new Set([...this.expanders.list(), ...Constants.Zulip.Commands]);
|
||||
}
|
||||
|
||||
private async releaseQueue() {
|
||||
const queue = this.queue;
|
||||
this.queue = undefined;
|
||||
|
||||
Reference in New Issue
Block a user