Aggregating message flows from different social platforms for users

from Academician Created on 3 years ago , last updated on 3 years ago Version # 1 2021 views 0 likes 0 collects

The theme of this sample tutorial is to aggregate messages from different social media and import them into a single customer support platform.

Each integrated message platform sends messages to us by calling Webhook. If a queue task is distributed for each received message, it cannot ensure that the message sequence after final processing is consistent with that when received (due to multiple reasons such as processing time, whether problems occur during processing, etc.).

To ensure the message delivery order, you can only send another message after one message is sent successfully.

To simplify the demonstration, we only receive messages from Telegram and send them to Intercom.

Secondary queue

To send a message after another message is sent successfully, you need to create a Secondary queue 。 This queue is driven by a data table. It stores all received messages. We can group them by users or sort them by time:

To find messages for a specific user in the queue, you can do this:

Distribute Message Task

We have already introduced that the task chain can be distributed through the API method provided by Laravel, and the tasks will be executed one by one. Obviously, this is also suitable for our scenario:

But we need to find a way to messages Query messages for each user in the table, and then push them to the corresponding task chain of the user for processing. To achieve this function, we can push a Messenger Task, which will monitor messages Table and push tasks to the task chain under its control:

This task class handle The method implementation code is as follows:

When all messages are pushed to the task chain Messenger Push itself to the queue. So that all messages can continue to be processed through Messenger Process message push.

If there is no message for the time being, the task will continue after a delay of 5 seconds.

To make Messenger It can be executed in an infinite loop and needs to tries The property value is set to 0. Otherwise, the task will fail after several retries and exit the execution of the task.

The whole process is as follows:

Each user has its own Messenger Task processing message receiving and sending.

send message

stay ProcessMessage In the task class, handle() The method is as follows:

This task must be fault-tolerant. Otherwise, if the task is marked as failed, the next task will not be executed and the user will not be able to receive messages. To do this, we need to enable the task to support unlimited retry and handle the failure itself:

take tries When the attribute value is set to 0, Larvel will never mark the queue task as failed. At the same time, we will count the number of task runs. If it exceeds 5 times, we will report an error, and then skip this task to run the next task.

Control payload size

Larave will store the load data of all tasks in the task chain into the load of the first task in the task chain, which means that if there are too many tasks in the task chain, the first task will have very large load data, which will be a hidden danger to the queue driver and system resources we rely on.

To solve this problem, you need to limit each time from messages Size of result set obtained in data table:

Cancel user account

Now we have a system that always handles message sending and receiving for each user Messager Queue task. If we want to revoke the specified user account, we need to remove this task from the queue in some way.

To implement this function, you can set a tag on the user model is_active = true , and then after each run Messager Check this flag during the task: