XXL-JOB is a lightweight distributed task scheduling platform. Its core design goal is rapid development, simple learning, lightweight and easy expansion. The source code has been opened and connected to the online product lines of many companies for out of the box use.
XXL-JOB Features:
1. Simple: support CRUD operations on tasks through Web pages, which is easy to operate and can be started in one minute;
2. Dynamic: supports dynamic modification of task status, start/stop of tasks, and termination of running tasks, with immediate effect;
3. HA (central type) of the dispatching center: the dispatching adopts a central design. The "dispatching center" develops dispatching components by itself and supports cluster deployment, which can ensure the HA of the dispatching center;
4. Executor HA (distributed): distributed task execution. The task "executor" supports cluster deployment and can ensure task execution HA;
5. Registration center: the executor will automatically register tasks periodically, and the scheduling center will automatically discover the registered tasks and trigger execution. At the same time, it also supports manual entry of actuator address;
6. Elastic capacity expansion and capacity reduction: once a new actuator machine goes online or offline, the task will be reassigned during the next scheduling;
7. Routing strategies: When deploying the actuator cluster, it provides a variety of routing strategies, including the first, last, polling, random, consistent HASH, least frequently used, most recently unused, failover, busy transfer, etc;
8. Failover: When the task routing policy is set to "Failover", if a machine in the executor cluster fails, Failover will automatically switch to a normal executor to send scheduling requests.
9. Blocking processing strategy: the processing strategy when the scheduling is too intensive for the executor to process. The strategies include: single machine serial (default), discarding subsequent scheduling, and scheduling before overwriting;
10. Task timeout control: supports user-defined task timeout. If the task runs overtime, it will actively interrupt the task;
11. Task failure retry: supports user-defined task failure retry times. When a task fails, it will actively retry according to the preset failure retry times; The sharding task supports the failure retry of sharding granularity;
12. Task failure alarm; By default, email failure alarm is provided. At the same time, an extension interface is reserved to facilitate the extension of SMS, nailing and other alarm methods;
13. Fragmentation broadcasting task: when the executor cluster is deployed and the task routing strategy is "Fragmentation broadcasting", the one-time task scheduling will broadcast and trigger all the executors in the cluster to execute a task, and the fragmentation task can be developed according to the fragmentation parameters;
14. Dynamic sharding: the sharding broadcast task is sharded with the actuator as the dimension, and supports dynamic expansion of the actuator cluster to dynamically increase the number of shards and cooperate with business processing; It can significantly improve the task processing ability and speed when conducting large data business operations.
15. Event triggering: In addition to "Cron Mode" and "Task Dependency Mode" to trigger task execution, event based task triggering mode is supported. The dispatch center provides API services that trigger single execution of tasks, which can be flexibly triggered according to business events.
16. Task progress monitoring: supports real-time monitoring of task progress;
17. Rolling real-time log: supports online viewing of scheduling results, and supports real-time viewing of the complete execution log output by the actuator in the rolling mode;
18. GLUE: Web IDE is provided to support online development of task logic code, dynamic publishing, real-time compilation and effectiveness, and the process of deployment and launch is omitted. It supports historical version backtracking of 30 versions.
19. Script tasks: support the development and running of script tasks in GLUE mode, including shell, Python, NodeJS, PHP, PowerShell and other types of scripts;
20. Command line task: a generic command line task handler (bean task, "CommandJobHandler") is provided natively; The business side only needs to provide the command line;
21. Task dependency: supports the configuration of sub task dependency. When the parent task is completed and successfully executed, it will actively trigger the execution of a sub task. Multiple sub tasks are separated by commas;
22. Consistency: The "Dispatch Center" guarantees the consistency of cluster distributed scheduling through the DB lock. One task scheduling will only trigger one execution;
23. User defined task parameters: support online configuration of scheduling task input parameters, which takes effect immediately;
24. Scheduling thread pool: scheduling system multithreads trigger scheduling operation to ensure accurate execution of scheduling without being blocked;
25. Data encryption: the communication between the dispatching center and the actuator is encrypted to improve the security of dispatching information;
26. Email alarm: email alarm is supported when the task fails, and multiple email addresses can be configured to send alarm emails in groups;
27. Push Maven central warehouse: the latest stable version will be pushed to Maven central warehouse to facilitate user access and use;
28. Running report: supports real-time viewing of running data, such as number of tasks, scheduling times, number of actuators, etc; And dispatching reports, such as dispatching date distribution chart, dispatching success distribution chart, etc;
29. Full asynchronism: the task scheduling process is designed and implemented in full asynchronism, such as asynchronous scheduling, asynchronous operation, asynchronous callback, etc., which effectively cuts the peak flow of intensive scheduling, and theoretically supports the operation of tasks of any duration;
30. Cross language: The dispatch center and the executor provide language independent RESTful API services, and any language of a third party can dock with the dispatch center or implement the executor accordingly. In addition, other cross language schemes such as "multi task mode" and "httpJobHandler" are also provided;
31. Internationalization: The dispatching center supports internationalization settings, and provides Chinese and English as the default languages;
32. Containerization: provide the official docker image, and update the pushed docker hub in real time to further realize the product's out of the box use;
33. Thread pool isolation: the scheduling thread pool is isolated and split, and slow tasks are automatically degraded into the "Slow" thread pool to avoid running out of scheduling threads and improve system stability;
34. User management: supports online management system users. There are two roles: administrator and ordinary user;
35. Permission control: the executor dimension controls permissions. Administrators have full permissions. Ordinary users need to assign executor permissions before allowing related operations;
Xxl job distributed task scheduling platform update log:
v2.3.1
1. [Fix] Fix the risk vulnerability and upgrade the low version project dependency: CVE-2021-2471, CVE-2022-22965, etc.
2. [Repair] Repair the fault alarm logic, and drop the mailbox verification logic to EmailJobAlarm to avoid interference with other alarm modes.
3. [Optimization] AccessToken is enabled by default for scheduling communication to improve system security (it is recommended that the production environment customize accessToken).
4. [Optimization] Consolidate multiple PRs, optimize the project code structure and robustness: PR-2833 PR-2812、PR-2541、PR-2537、PR-2514、PR-2509、PR-2591。
5. [Optimization] Optimize the task thread name to improve readability and problem location efficiency (ISSUE-2527).
v2.3.0
1. [Add] Scheduling expiration policy: the compensation processing policy for the scheduling center to miss the scheduling time, including: ignore, immediately compensate and trigger once;
2. [New] Trigger strategy: in addition to the conventional Cron, API, parent and child task trigger methods, the new trigger methods of "fixed interval trigger, (fixed delay trigger, in the experiment)" are provided;
3. [Add] New task auxiliary tool "XxlJobHelper": provides unified task auxiliary ability, including: task context information maintenance and acquisition (task parameters, task ID, sharding parameters), log output, task result settings, etc;
3.1. Discard the "ShardingUtil" component: use "XxlJobHelper. getShardIndex()/getShardTotal();" instead to obtain the sharding parameters;
3.2 Obsolete the "XxlJobLogger" component: use "XxlJobHelper. log" for log output instead;
4. [Optimization] The "execute" method of the task core class "IJobHandler" cancels the access parameter design. Instead, use "XxlJobHelper. getJobParam" to obtain the task parameters and replace the method input parameters, and use "XxlJobHelper. handleSuccess/handleFail" to set the task results and replace the method output parameters. The example code is as follows:;
@XxlJob("demoJobHandler")
public void execute() {
String param = XxlJobHelper.getJobParam(); //Get Parameters
XxlJobHelper.handleSuccess(); //Set Task Results
}
5. [Optimization] Cron editor enhancement: when Cron editor is modified, the latest running time can be viewed in real time;
6. [Optimization] Standardize the actuator example project;
7. [Optimize] Task scheduling lifecycle reconfiguration: schedule, trigger, handle, callback, and complete;
8. [Optimization] Actuator registration component optimization: the registration logic is adjusted to asynchronous mode to improve the registration performance;
9. [Optimization] Actuator authentication verification: when the actuator starts, it actively verifies the accessToken. If it is empty, it will give an active Warn alarm; (Planned security enhancement: AccessToken dynamic generation, dynamic start/stop, etc.)
10. [Optimization] Optimization of mailbox alarm configuration: separate the "spring. mail. from" and "spring. mail. username" attributes, and more flexibly support some password free mailbox services;
11. [Optimization] Multiple projects rely on upgrading to newer and stable versions, such as netty, groovy, spring, springboot, mybatis, etc;
12. [Optimization] Regular upgrade of UI components to improve component stability;
13. [Optimization] Interactive optimization of the dispatching center page: cancel the password column of the user management module; Autocomplete cancellation is expressed in multiple places; Executor management module XSS intercepts and verifies;
14. [Optimization] Optimize the slow SQL problem of task status detection in the dispatching center;
15. [Repair] GLUE Java mode task, init/destroy cannot perform problem repair;
16. [Fix] Cron editor problem fix: fix the problem that when a single cron field is modified in a small probability, other fields are reset;
17. [Repair] Optimization of the general HTTP task handler (httpJobHandler): fix the problem that "setDoOutput (true)" causes the GetMethod of the task request to fail;
18. [Repair] The example task of the executive Commandhandler is optimized to fix the problem of script process suspension in extreme cases;
19. [Repair] Optimize the scheduling communication component, and repair the heartbeat detection failure when calling the DotNet version actuator in RestFull mode;
20. [Repair] The remote execution log of the dispatching center is queried for garbled code;
21. [Repair] Optimize the loading sequence of components in the dispatching center, and repair the scheduling failure caused by the initial slow scheduling components in extreme cases;
22. [Repair] The actuator registers thread optimization to repair the NPE problem caused by initialization failure in extreme cases;
23. [Repair] Optimize the connection pool of the scheduling thread, and repair the connection validity verification timeout problem;
24. [Repair] Optimize the fields of the actuator registry, and solve the problem that too many actuator registration nodes lead to the failure of registration information storage and update;
25. [Repair] Optimize the rotation training routing strategy to repair the concurrency problem under small probability;
26. [Repair] Repair the problem that https changes to http after redirecting the page;
27. [Repair] Optimize the log cleaning of the actuator, and repair the abnormal cleaning problem caused by the empty log file under small probability;
