Use queues for sending emails

[tobyzerner]

Sometimes submitting a post can be slow because you have to wait for Flarum to send notification emails on the backend before it sends back the 200 OK response.

We should look at using Laravel's Queue component (or similar) to send emails asynchronously. Of course it will be optional as many shared hosts won't have the ability to run a queue worker in the background.

[franzliedke]

Yes, please. Using the sync driver by default. We can improve this even more, by using one of my two experimental approaches from a long time ago: tardiqueue and threadqueue.

Both (but probably mostly the first one) are aimed at providing a more performant "poor man's queue" system for use with e.g. shared hosts.

[luceos]

@franzliedke something you might need to consider is the limitation by many shared hosts of the number of processes allowed per hosting plan/website. A common OS that implements this is CloudLinux.

[franzliedke]

@luceos Laravel's sync driver is basically just a dummy - it executes a job immediately instead of pushing it onto a queue. It's basically the poor man's queue adapter. So is my tardiqueue extension, it's just a little more advanced.

[luceos]

Yes I understand, I was referring to the threadqueue solution @franzliedke ;)

[franzliedke]

Gotcha, yeah, that should only be used where possible.

[franzliedke]

That's highly experimental anyhow.

[luceos]

Would like to take a shot at this if you're ok with it @flarum/core.

todo

• Add queue to ioc.
• Set queue driver to sync per default.
• Have all background tasks run through queue if possible, but at least the mail sending.

---

Would it be acceptable to simply call the QueueServiceProvider of the Illuminate\Queue package?

[franzliedke]

@luceos Please do. 👍

TODO list looks great.

Please create a stripped-down version of the service provider - the one from Laravel registers all the drivers in order to be as configurable as possible (and some console commands which we don't need for now). Ours should only set up the sync driver for now - at one point we will have a config flag that will be used to switch to another driver instead. The DatabaseServiceProvider might serve as inspiration.

[luceos]

First implementation already sends over sync. I've extended the illuminate service provider and the QueueManager. I'll see if I can make a non-extended provider.

[luceos]

Some thoughts I had:

• I've created a DispatchJobsTrait that mirrors the behavior seen in the DispathEventsTrait, but I'm not sure that's the best of solving sending jobs.
• I'm using the Bus Dispatcher of Laravel to send Jobs into the queue. On hindsight, this might not be the best way of doing so.
• I've created a generic MailJob which allows sending mails to the queue with body, subject and to. We might need to extend this further or even require adding a Message object as argument.
• As discussed above, the service provider needs cleaning and/or stripping.
• The Queue requires an Encrypter, I've copied the implementation Laravel uses for now, but there are two things here that we need to decide on:
  • The cipher is now hardcoded to AES-256-CBC that's what Laravel does too,
  • The app_key is now using an md5 of the url, I forgot whether Flarum has its own key somewhere, please advise.

Anything else?

[luceos]

@franzliedke could you share you thoughts on the above once you have the opportunity to do so? Greatly appreciated.

[franzliedke]

@luceos Do you still need anything here, or is the feedback in the PR enough?

[luceos]

No it's fine, I'll patch it up asap. I've self assigned this issue.

[luceos]

Wow I even looked into this. I think I have an idea how to move forward with this. Dispatch now in laravel is also pretty much like a sync driver.. Will give this a shot again soon, might make bokt/flarum-queue deprecated.