Skip to content

Start message queues on demand - #18

Open
ryanwinchester wants to merge 2 commits into
mainfrom
lazy-message-queues
Open

ryanwinchester wants to merge 2 commits into
mainfrom
lazy-message-queues

Conversation

@ryanwinchester

Copy link
Copy Markdown
Owner

Supersedes #16, which came from @neilberkman's point that every bot has to list its conversations at startup, and needs channels:read, groups:read, im:read and mpim:read for that, even when it only sends DMs or handles interactions.

#16 added channels: false to turn the channel server and the message queues off. But the queues only exist to keep send_message/2 to Slack's rate limit of one message per second per channel, and they don't need to know the bot's channels in advance. This starts a channel's queue the first time the bot sends to it, which removes the startup listing for every bot without an option, and keeps send_message/2 working.

Changes

  • Slack.MessageServer.send/3 looks up the channel's queue, and starts it under Slack.DynamicSupervisor if there isn't one yet. Concurrent first sends share one queue. Slack.Supervisor keeps the bot in the queue registry's metadata, so a queue can be started from just the bot module that send_message/2 passes.
  • A new queue sends its first message right away. Queues used to start with a one-second timer, which didn't matter when they started at boot.
  • Slack.ChannelServer, its registry, the users.conversations listing and the member_joined_channel / channel_left handling in Slack.Socket are removed. Those events still go to handle_event/3.
  • send_message/2 logs an error and returns {:error, reason} when it can't queue the message (Slack.Supervisor isn't running, or the module isn't the bot), instead of dropping it silently.
  • The channels option is deprecated: it logs a warning and does nothing.
  • README: the example app needs only channels:history and chat:write, and message.channels.

Behavior changes

  • send_message/2 now sends to channels the bot didn't have a queue for, including DMs by user ID and channels joined after startup. Before, those messages were dropped without a trace.
  • Startup makes one fewer API call and needs fewer scopes.
  • Queues aren't stopped once started, so there's one per channel the bot has sent to since it started, instead of one per conversation it's in. Stopping idle queues would add a race between a send and the queue shutting down, so it's left out until it's needed.

This also removes some channel server bugs: a restarted channel server crashed on queues that were still running, leaving a channel didn't stop its queue (the supervisor restarted it) and crashed for channels it wasn't tracking, group_left wasn't handled, and a message sent from a member_joined_channel handler could arrive before the queue existed.

Testing

mix compile --warnings-as-errors (dev and test), mix format --check-formatted, mix credo, mix dialyzer and mix test --warnings-as-errors (42 tests, 3 runs) on Elixir 1.15 / OTP 26, and the tests on Elixir 1.20 / OTP 29. Reverting each change makes a test fail.

The per-channel message queues only exist to keep send_message/2 to
Slack's rate limit of one message per second per channel, so they don't
need to be started for every conversation at startup. Start a channel's
queue the first time the bot sends to it instead.

- The bot no longer lists its conversations at startup, so it doesn't
  need channels:read, groups:read, im:read or mpim:read for that, and
  the ChannelServer and its join/leave tracking are gone.
- send_message/2 now works for any channel, including DMs by user ID
  and channels joined after startup, which were silently dropped.
- A new queue sends its first message right away.
- send_message/2 logs an error and returns {:error, reason} when the
  message can't be queued, instead of dropping it silently.
- The channels option is deprecated: it logs a warning and does nothing.
- Return an error instead of crashing the caller when the queue's
  supervisor isn't running, for example while it restarts.
- Find a channel's queue by its registered name, which skips one that
  exited before the registry removed it, so the message isn't cast to
  a dead queue.
- Only treat an ArgumentError from the registry reads as the registry
  not running.
- Accept the bot struct in Slack.MessageServer.send/3 again.
- Log a queue starting once, from init/1, at debug level.
- Note that each channel or user ID has its own queue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant